Strangeness with NUnit runner instantiating attributes before running tests are run

So Scott Koon and I going through converting a project from using MBUnit / Gallio to NUnit instead, mainly for the speed benefits, but also for complying with the direction of our development moving forward.  One gotcha we just ran into is in the way NUnit handles custom attributes.

Say you have an attribute like this:

    [AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
    public class DogMeowsLikeRatAttribute : Attribute
    {
        public DogMeowsLikeRatAttribute()
        {
            if (!StateMaintainer.HasBeenTouched)
                throw new ImmaCryException("Oh snap! You broke it!");
        }
    }

and a class that uses the attribute:

    [DogMeowsLikeRat]
    public class WhatDoWeAttributeThisTooClass
    {
        public void FoShizzel()
        {
            Console.WriteLine("Nizzle izzle da bomb dapibizzle turpizzle go to hizzle shiz");
        }
    }

and a test for your class

    [TestFixture]
    public class TestThatClassyClass
    {
        [Test]
        public void WhoaNow()
        {
            StateMaintainer.HasBeenTouched = true;

            var subject = new WhatDoWeAttributeThisTooClass();
            subject.Should().NotBeNull("Cause I said so!");
        }
    }

and the state maintainer just to prove the point:

    public static class StateMaintainer
    {
        public static bool HasBeenTouched;
    }

When you run the test, you will receive an exception. Now the important thing is to look at the stack trace here:

<pre>------ Test started: Assembly: DoesNunitInstantiateAttributes.dll ------

Here's what you're a gonna do: Oh snap! You broke it!
Test 'M:DoesNunitInstantiateAttributes.TestThatClassyClass.WhoaNow' failed: Oh snap! You broke it!
	DoesNunitInstantiateAttributes.ImmaCryException: Oh snap! You broke it!
	Class1.cs(18,0): at DoesNunitInstantiateAttributes.DogMeowsLikeRatAttribute..ctor()
	at System.RuntimeTypeHandle.CreateCaInstance(RuntimeType type, IRuntimeMethodInfo ctor)
	at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
	at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit)
	at System.RuntimeType.GetCustomAttributes(Boolean inherit)
	at NUnit.Core.Reflect.GetAttributes(ICustomAttributeProvider member, Boolean inherit)
	at NUnit.Core.Reflect.HasAttribute(ICustomAttributeProvider member, String attrName, Boolean inherit)
	at NUnit.Core.Builders.NUnitTestFixtureBuilder.CanBuildFrom(Type type)
	at NUnit.Core.Extensibility.SuiteBuilderCollection.CanBuildFrom(Type type)
	at NUnit.Core.TestFixtureBuilder.CanBuildFrom(Type type)
	at NUnit.Core.Builders.TestAssemblyBuilder.GetFixtures(Assembly assembly, String ns)
	at NUnit.Core.Builders.TestAssemblyBuilder.Build(String assemblyName, Boolean autoSuites)
	at NUnit.Core.Builders.TestAssemblyBuilder.Build(String assemblyName, String testName, Boolean autoSuites)
	at NUnit.Core.TestSuiteBuilder.BuildSingleAssembly(TestPackage package)
	at NUnit.Core.TestSuiteBuilder.Build(TestPackage package)
	at NUnit.AddInRunner.NUnitTestRunner.run(ITestListener testListener, Assembly assembly, ITestFilter filter)
	at NUnit.AddInRunner.NUnitTestRunner.runMethod(ITestListener testListener, Assembly assembly, MethodInfo method)
	at NUnit.AddInRunner.NUnitTestRunner.MemberRun.Run(NUnitTestRunner testRunner, ITestListener testListener, Assembly assembly)
	at NUnit.AddInRunner.NUnitTestRunner.run(ITestListener testListener, Assembly assembly, IRun run)
	at NUnit.AddInRunner.NUnitTestRunner.RunMember(ITestListener testListener, Assembly assembly, MemberInfo member)
	at TestDriven.TestRunner.AdaptorTestRunner.Run(ITestListener testListener, ITraceListener traceListener, String assemblyPath, String testPath)
	at TestDriven.TestRunner.ThreadTestRunner.Runner.Run()

0 passed, 1 failed, 0 skipped, took 0.18 seconds (NUnit 2.5).</pre>

The test where the attributed class is instantiated has not been hit yet by NUnit.  Instead, it seems that NUnit is gleaning over the text fixtures in the assembly, finding all of the custom attributes, and instantiating those attributes before running any of the tests.  We ran into this issue because of an attribute that had an IoC container lookup (I know what you're going to say, but we didn't write it) in the constructor of the attribute. Due to the container not yet being initialized, an exception was thrown.

Hopefully Charlie Poole can provide some insight into this strange behavior on NUnit's part.

  1. #1 by Brad Wilson on September 14, 2011 - 3:09 pm

    This isn’t NUnit’s fault. They have to get all the attributes to find things like [TestFixture] and [Test], and the CLR is the one that’s responsible for instantiating attributes as a result. There’s no way to “get” the list of attributes without instantiating them.

    You’re going to have to fix your code. NUnit isn’t broken.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 887 other followers

%d bloggers like this: