Cloud-based Load Testing with TFS Service

As you develop an application (and in some cases, right before you push it to production), you may ask yourself: “What will happen with 1000’s of people start using it?”.  After you panic for a while trying to figure out ways to make sure your application can handle the load, you find some load testing options.  If you are using Visual Studio Ultimate, you have built-in load testing capabilities.  Visual Studio load tests allow you to configure Load Test Scenarios consisting of a Test Mix (Web Performance Tests, Coded UI, and Unit Tests), Browser Mix, Network Mix, and Load Pattern.  You can get a lot of information on how to get your load test setup on the MSDN Site.

Once you create your test cases and setup your load tests, your next step is to find a suitable Test Rig.  A test rig is made up of a test controller and at least one test agent (but ideally more than one agent). 
image

The idea is that when you are ready to kick off your tests, your machine will communicate with the controller and the controller will find available agents to execute the tests.  Test results are gathered by the controller and once the load test completes, you can access and analyze test results.  By creating this test controller/test agent infrastructure, you are able to distribute load and run more realistic load tests.  A single machine usually cannot generate the type of load that is needed to prove that your application can handle load.  I usually test out my load tests from my machine, but when I’m ready to perform the “real” tests, those are executed from a proper test environment. You can read this Patterns & Practices guidance that covers Load testing.

Your test environment can make a huge difference between just running through your application’s functionality and actually generating meaningful data that will allow you to address any issues with your application.  There are a couple of problems with this:

  • Hardware is not cheap (especially if it’s going to sit around idle for long periods of time)
  • Hardware procurement process can take a long time
  • If your company has a dedicated load test environment, it may be in use by someone else and now you are waiting in line

To the Cloud!!

image

Wouldn’t it be great if if you could run meaningful Load Tests without having to worry about acquiring/installing/configuring your servers?  And wouldn’t it be great if your load test servers were available the moment you need them?  And wouldn’t it be great if you could just pay for the time that you use those servers?  Well, that sounds just like what Azure is really good at.  And that’s exactly what the Visual Studio ALM team delivered. 
This year at TechEd, Brian Harry announced Cloud-Based Load Testing with Team Foundation Service.  Just as its name implies, it’s a service that allows you to take advantage of Azure to run your load tests.  There is no need to configure controllers and agents, and it is available on demand.

Getting Started

It’s really easy to take advantage of this service.  If you already have a set of Load Tests, you are more almost there.  If you don’t have any load tests setup yet, follow this link to get started.  You don’t have to do anything different than you would if you were running your load tests on premises.  The following steps to get your load tests running on Azure require you to have Visual Studio 2013 Preview or later (Release Candidate was released last week).

For this example, I created a really simple Web Performance Test while browsing through this blog:
image

Then, I created a Load Test that uses the Web Performance test:
image

Running the load test with the test controller and test agent running on my laptop gives me these results:
image

image

This works, but I will eventually run into throughput issues with my machine and the results will no longer be valid with my machine not being able to keep up with the load.  To start using Azure-based Load Tests you open up the Test Settings file associated with your solution and change the Test Run Location:
 image

image

Before you can execute your load test on Azure, you need to make sure that you’ve connected to your TFS Service team project from Team Explorer.  Note that you don’t have to check in your files to TFS Service, you just need to be connected.  Once that’s done, you can run your load test.

After a couple of initializing steps where your load test is deployed to Azure, the controller and agents start doing their work

image

image

image

image

As the load test starts collecting data, you get feedback in the form of this chart:
image

Clicking on the details link, I can get some additional information about my test run:
image

As you may have noticed, my test case used 250 virtual user minutes.  While this service is in preview (and free for now), you can use up to 15,000 user minutes/month.  This will surely change once the service goes live and pricing is announced.

image

Clicking on the Download Report link gives us detailed information about the test run, the same information that you would get if you were running on premises:
image 

image 

To access previous runs, you can go to the Load Test menu in Visual Studio and select Load Test Manager.  This page comes up in Visual Studio:
 image

Here I can filter runs by State, Date, and who executed it, and selecting each one will allow you to view details for the test run.

And that’s all you need to have an entire Azure datacenter at your disposal for your load tests.  Enjoy!

Interested in learning more about Visual Studio ALM?  Checkout my Pluralsight courses on Microsoft Test Manager and Team Foundation Server Customization.


2 thoughts on “Cloud-based Load Testing with TFS Service

  1. Pingback: Friday Five - September 20, 2013 - The Microsoft MVP Award Program Blog - Site Home - MSDN Blogs

  2. Pingback: Tuesday – Speaking at MDUGJax: @VisualStudio Online — ALM Guide - Esteban Garcia

Leave a Reply

Your email address will not be published. Required fields are marked *


nine − 3 =

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>