The Riverbed Blog (testing)

A blog in search of a tagline

Kicking the Tires in a Lab

Posted by riverbedtest on August 19, 2011

As engineers we enjoy getting under the hood of a new solution and getting our hands dirty. In some cases, you might even get paid to conduct studies or tests. While there is no more interesting scenario to analyze a solution than in production, some times we have to study solutions in a lab.

Davinci_iphone When a potential client mentions a robust lab study to base purchasing decisions or recommendations on I tend to get a little worried. Primarily, because testing WAN optimization in a lab doesn’t typically expose the complexities of production environments.  Furthermore, testing WAN optimization isn’t as straight-forward or as well understood as other standardized technologies. For starters, many of the popular automated test tools don’t support the application layer (OSI Layer 5-7) fluency needed to properly evaluate application acceleration for protocols like MAPI, CIFS, NFS, etc. Next, the boilerplate content in these tools doesn’t represent production datasets at the byte level so the ‘out-of-the-box’ deduplication test results aren’t representative. Many times the test tools are measuring per-packet or inter-packet metrics, while WAN optimizers are focused on per-transaction optimization by proxying TCP connections…which means we have three different sets of packets per transaction (client-side, inner channel & server-side).  So the test tool ‘timing’ reports aren’t always applicable or representative of the actual user experience.  Not to mention the application security infrastructure and requirements in a lab may not mirror the production environment (I.E. encrypted Outlook/Exchange, CIFS SMB Signing, Kerberos/NTLMv2 authentication, SSL requires smart-cards, etc).  These are just a few of the many hurdles you need to be aware of in test environments.

So now that we’ve discussed the challenges, let’s look at how you can structure a reasonable test lab for WAN Optimization…although in my opinion there is no substitute for production testing.  First, we need to emulate the production network performance characteristics in a dependable WAN Emulator, such as Apposite’s Linktropy. Next, we need to thoroughly understand the production applications on the real network so we know what to test and how to properly configure the test resources. If possible, setup a test client lab that connects to production servers so you don’t have to worry about building or configuring test servers. Don’t forget to check the applications’ security architecture such as settings for authentication, encryption and domain integration…many times these settings either make or break WAN optimization project success in production. Next, understanding the most common transactions for these applications will help us define individual test cases. Getting adequate sample content and data for these test transactions will provide the basis for measuring the effectiveness of deduplication.  The more samples/content the more representative your results will be. To test deduplication you’ll want to setup initial and subsequent test runs with the same or similar datasets. Typically, these are called cold, warm and partially warm tests.  For testing common applications like CIFS and HTTP you can use Riverbed’s Riverbench software to record time and throughput metrics, just ask your Riverbed Sales Engineer for a copy. A great place to find free performance test tools is Open Source Testing. If needed, setup test cases for the other features of the WAN optimization solution you’re considering.  Commonly these are Quality of Service, visibility, virtualization, etc. 

Prior to starting any testing thoroughly baseline the environment so you can work out any wrinkles…there is nothing more frustrating than starting formal testing only to find misconfigured speed/duplex settings or interface queues that are too short.

For post-test analysis you should use packet capture tools such as Pilot, Shark, and Wireshark to capture packets on the client-side, WAN and server-side of the WAN optimizers during each test. This will allow you to very quickly dive into the intricate details, and have hard data to backup the test results should they come into question.

In 2007 Network World setup their first WAN optimization test lab. There were gracious enough to put together a podcast on how they did it and the lessons learned along the way. I highly suggest you take the time to learn from it prior to setting up your own test lab for WAN optimization. 

When in doubt consult your Riverbed Sales Engineer.  They have extensive experience in testing WAN optimization and can be quite helpful.

Eleanor Roosevelt said it best – “Learn from the mistakes of others. You can’t live long enough to make them all yourself.”

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: