Home > Visual Studio > Microsoft Visual Studio Team Suite Load Test Part 1 – VSTS vs Mercury LoadRunner

Microsoft Visual Studio Team Suite Load Test Part 1 – VSTS vs Mercury LoadRunner

I’ve been getting my hands dirty with Visual Studio Team Suite Load Test the past few months. In the coming few months, i will be posting several articles on my experiences with VSTS.

First i’ll start with a comparison between VSTS and Mercury LoadRunner. I’m including the version numbers as the newer versions of these 2 software promises much much more!

Point of Comparison Mercury Load Runner 6
Visual Studio Load Test 2008
Pricing Unlimited number of agent computers, pricing is per Virtual User (VU)

Useful if you have a whole lot of old computers or if you want to spread the load around in your network to find connection bottlenecks

Pricing is per physical CPU, Microsoft states that each logical CPU can take up to 1000VU

As this is on a per CPU basis, you may want to get a relatively powerful (read 4/6/8 Core CPU, >=8gb) machine to be the load test agent.

Also, DO NOT install this on a VM, be it ESX or Hypervisor, performance of the agent in a virtual environment is at least 50% worse as compared to having it installed on a physical machine.

Monitoring Capabilities By default, it does not monitor any server, client parameters, of course these are available if you pay

This means that during any test run, engineers need to be around to monitor the server loads

Able to monitor perfmon statistics (i.e windows machines), you can also specify the statistics to gather and save in your load test report.

It also comes with SQL Tracing for SQL Server, so you can monitor the long running queries with respect to your load test script.

With real time display of the server/client statistics, you can easily tell how the load test is running

On the downside, you will still have to monitor non-windows machines manually.

Virtual User Stepping You can specify User Stepping or Constant User Load You can specify User Stepping, Constant User Load or Goal based.

Goal based means VSTS will keep increasing the VU until a certain perfmon statistic is matched e.g web server 100% CPU

Testing Flexibility Able to test various systems, including web based, SAP etc

Databinding to text files is supported

Backend is something like C programming so you can easily change the parameters if required

Able to test most systems, any additional functionality can be done via a custom app and using VSTS to run the app

Databinding to various data sources are supported

There are 2 formats to testing, coded and scripted tests.

Coded tests are simple C#/VB codes with events and stuff for ultimate flexibility at the expense of ease of editing.

Scripted tests is similar to what Mercury has to offer

One thing to note: If you have installed the VSTS SDK, you can actually use a program to run thru the scripts and change the parameters for a scripted test, e.g changing the think time for all web requests to 5 seconds.

Categories: Visual Studio
  1. Galileo Performance
    October 22, 2009 at 2:02 pm

    Correction about Monitoring Capabilities By default, it does not monitor any server, client parameters, of course these are available if you pay

    All monitor are available free of charge

  2. kwanann
    October 22, 2009 at 2:55 pm

    hmm i remembered when i did it with LR 6, there was no options to monitor any server or client parameters, when you mentioned its available which version or loadrunner are you referrring to?

    • galileo performance
      October 22, 2009 at 4:13 pm

      Hy, since version 8.

  3. kwanann
    October 23, 2009 at 5:56 am

    Ah that’s good, at least LR kept up with MS🙂

    As mentioned in the post, i did my comparison using LR 6 where this feature was still a paid feature, thanks for bringing that up to my notice!

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: