Archive

Archive for October, 2010

nUnit Heart MSTest

October 1, 2010 Leave a comment

It’s nearly impossible to make everyone happy all of the time.  When it comes to nUnit and MSTest there are people who seem to be married to their test platform; I’ll admit that I’m rather attached to MSTest myself.  Some of the reasons given seem to have more with people being anti-Microsoft than pro-nUnit.  But the last thing I want to do is make people not want to test because they aren’t happy with the tools.  The way I see it any test, no matter what the platform, is much better than no test at all.

One thing I know about software development teams is you’ve got to pick your battles because you only get so many chips.  I knew that I needed MSTest so I could capture metrics, conduct load testing, use built-in Visual Studio tools to get code coverage, and more easily test my code.  I also knew that key players had their heart set on nUnit.  The nUnit test itself is not a bad test; I simply needed to leverage MSTest as a shell so I can do all these other great things.

Fortunately for our team we embrace outside of the box thinking and try to be flexible within reason.  Because of the desire to solve both problems we are now able to execute nUnit tests from both test harnesses, while only needing to write one test.  And it should work the opposite way (haven’t tried yet).  Depicted below is proof that it works.

Passing nUnit test The SAME test Passing in MSTest
image image

 

The solution was to create a base class that abstracts the complexity of it all.  I didn’t want developers to manage tons of attributes, and I didn’t want them to have to change the way they write tests. 

In the base class I added the nUnit attributes, using a custom class I created that inherits from the TearDownAttribute and SetupAttribute.  This workaround was required because the attributes weren’t being picked up in my inheriting class.  By adding the AttributeUsage attribute on top of the custom attributes and setting the Inherited property to ‘true’, I was able to solve that problem.   

Create the base class first

using System;
using nutest = NUnit.Framework;

[NUTestClass()] //For nUnit
public class MSTestAndNUnitCombo
{
[NUSetup()]
public virtual void Setup() { }

[NUTearDown()]
public virtual void TearDown() { }
}

[AttributeUsage(AttributeTargets.Method, Inherited = true, AllowMultiple = true)]
public class NUTearDown : nutest.TearDownAttribute { }

[AttributeUsage(AttributeTargets.Method, Inherited = true, AllowMultiple = true)]
public class NUSetup : nutest.SetUpAttribute { }

[AttributeUsage(AttributeTargets.Class, Inherited = true, AllowMultiple = true)]
public class NUTestClass : nutest.TestCaseAttribute { }

Unfortunately, I was not able to abstract the MSTest attributes on the Setup and TearDown virtual methods because Microsoft, in their infinite wisdom, made those attributes SEALED classes.  So, although the developer will still see some attributes we are still reducing complexity and forcing the developer not to forget two methods we always want implemented.

Next, create your unit test; Inherit from your Base Class; Alias your using statements; Add Attributes.

using mstest = Microsoft.VisualStudio.TestTools.UnitTesting;
using nutest = NUnit.Framework;

[mstest.TestClass()]
public class MSTestAndNUnitCombined : MSTestAndNUnitCombo
{
[mstest.TestInitialize()]
public override void Setup()
{
//TODO
}

[mstest.TestCleanup()]
public override void TearDown()
{
//TODO
}


/// This test still works in NUnit AND MSTest.
[mstest.TestMethod()]
[nutest.Test()]
public void FailingTest()
{
nutest.Assert.IsTrue(1 == 0);
}

/// This test still works in NUnit AND MSTest
[mstest.TestMethod()]
[nutest.Test()]
public void PassingTest()
{
nutest.Assert.IsTrue(1 == 1);
}
}

Finally, in the first PropertyGroup section of your nUnit test project you will need to add some XML.  If you added the project as an MSTest project first, this step isn’t required.  Also, if this step doesn’t work (perhaps the Guids used change in the future) simply add a new test project and look at the csproj file, then snag it’s setting (right-clicking and edit with notepad works well).

Finally add ProjectTypeGuids to the First PropertyGroup

<PropertyGroup>
. . . More XML will be found here . . .
<ProjectTypeGuids>
{3AC096D0-A1C2-E12C-1390-A8335801FDAB};
{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}
</ProjectTypeGuids>
</PropertyGroup>

There you have it.  Once this is all done, you can do some real cool thing with Visual Studio Test Tools.  One good tool is the Performance Wizard.  Simply right-click your unit test in the MSTest window and select the Performance Test option.  I have Ultimate, but I think this feature is available in Professional edition.

Performance Wizard Screen 1 Performance Wizard Screen 2 Performance Wizard Screen 3
image image image

After you run the profiler, you have some interesting metrics and a great baseline to measure true performance of your application.  Obviously the simple test above is not real-world, but you can run this against any of your unit tests.  I wasn’t able to gather metrics on my VM, however the Load Test does essentially the same thing (and more).  I have that working.

You will need to run Visual Studio as Admin and there are some limitations to what instrumentations you can run on a VM.  If you are going to run load tests on the VM, make sure you give it as much memory as possible. 

Metrics I just gathered using a Load Test

image

 

That’s all for this post.  If you have any questions, comments, or problems as you try to do this yourself I’ll gladly answer comments.  I am publishing this blog to the Kindle.  If you would like to subscribe visit this link

Categories: Uncategorized