Writing Data Driven Tests using xUnit

This is going to be a short tutorial (now updated for xUnit 2) on how to go about writing data driven tests using xUnit. The samples used will be easy to understand and relate to.

In case you are new to xUnit then all I will say is that xUnit is the next iteration for popular Nunit framework. It is better in many respects.

So what are data driven tests?

To keep it very simple, these are the tests which are run on different sets of data.

The input and the expected results are typically kept in same file. And the test reads the inputs and executes the test. Then it compares the result of the test with the expected result in the file and decides if test passed or failed. More details here.

As usual I will divide the tutorial into WHY and HOW.

WHY

There will be times when you will notice that you have tests which have same structure but the data on which they are working is different. No change otherwise in the code contained in the tests. These tests are good candidates for data driven tests. They reduce the number of tests in the file without compromising on the coverage.

HOW

Before you start off, remember to install via Nuget:

• xUnit.Net to get the basic xUnit functionality.
• xUnit.Net extensions to get support for data driven testing. [xUnit 2 update] This is no longer needed.

I will discuss three ways of going about Data Driven Tests using xUnit. There are more. I might cover them later.

Inline Data Driven Tests

In this the data is contained in the test file itself as the code below shows.

But first I will create a small class called CheckThisNumber. I will create a simple method called CheckIfEqual in this class. We will write the tests for this method.

namespace InlineDataDrivenTests
{
    public class CheckThisNumber
    {
        public int IntialValue { get; set; }

        public CheckThisNumber(int intialValue)
        {
            IntialValue = intialValue;
        }

        public bool CheckIfEqual(int input)
        {
            return IntialValue == input;
        }
    }
}

 

Time to write the inline data driven tests !!

using Xunit;

namespace InlineDataDrivenTests
{
    public class InlineDataDrivenTests
    {
        [Theory]
        [InlineData(1, true)]
        [InlineData(2, false)]
        [InlineData(-1, false)]
        [InlineData(0, false)]
        public void SampleDataDrivenTest(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

Let us go through important steps.

• Line 07: Notice the attribute Theory. In normal xUnit tests you use attribute called Fact.
• Line 08: Test is further decorated with InlineData attribute to tell xUnit about what kind of data driven testing will be done. Note the parameters in the parenthesis. These are the ones which will be used by the test case. You might see that Line 09-12 almost resemble a table.
• Line 12: The function takes two parameters. The type of the parameters is of the same type as the one in InlineData.
• Line 15: The number parameter getting used.
• Line 16: The expectedResult parameter getting used.

This is a single test. But actually it will run 4 times. With different data each time. As you can see below.

You just avoided writing 4 test cases which would have been the same code wise. And without sacrificing coverage.

Property Data Driven Tests

Now why we need this? Sometimes it might so happen that you want to share the data. Like there might be two test cases in two different files who will be working with same set of data. You might create the testcases in Inline data driven tests format but that is duplication. Not cool. Property Data Driven Test format allows for the reuse of the data.

Let us have some more code.

As usual we will have the CheckThisNumber class.

namespace PropertyDataDrivenTests
{
    public class CheckThisNumber
    {
        public int IntialValue { get; set; }

        public CheckThisNumber(int intialValue)
        {
            IntialValue = intialValue;
        }

        public bool CheckIfEqual(int input)
        {
            return IntialValue == input;
        }
    }
}

But this time we will add another class DemoPropertyDataSource.cs which will contain the test data. Here it goes.

using System.Collections.Generic;

namespace PropertyDataDrivenTests
{
    public static class DemoPropertyDataSource
    {
        private static readonly List<object[]> _data 
            = new List<object[]>
                {
                    new object[] {1, true},
                    new object[] {2, false},
                    new object[] {-1, false},
                    new object[] {0, false}
                };

        public static IEnumerable<object[]> TestData
        {
            get { return _data; }
        }
    }
}

• Line 07 : We create a List of Object Array. Note that Object is used to keep things generic.
• Line 10 – 13 : We create the test data.
• Line 16 : We define a public property called “TestData”.
• Line 18 : We return the list we created.

Now we will add two testfiles. To keep things simple I have kept the content inside them same. Just changed the names of the tests.

TestFile1.cs

using Xunit;


namespace PropertyDataDrivenTests
{
    public class TestFile1
    {
        [Theory]
        [MemberData("TestData", MemberType = typeof(DemoPropertyDataSource))]
        public void SampleTest1(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

TestFile2.cs

using Xunit;


namespace PropertyDataDrivenTests
{
    public class TestFile2
    {
        [Theory]
        [MemberData("TestData", MemberType = typeof(DemoPropertyDataSource))]
        public void SampleTest2(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

There is not much change in the test files except the highlighted line. MemberData attribute is used. And then we provide the details.
[xUnit 2 update] PropertyData has been retired. MemberData is now used as you can see above.

When we run the tests you will see that the tests are run against the same common set of test data. As you can see below.

We just shared the test data between testcases. Wo Hoo.

Excel Data Driven Tests

This is taking things one level up. Rather than storing the common test data in a class, why not put it in an excel file? Makes it easy for non technical person to come up with test data and populate the same.

However excel being excel, there are some hoops to jump through.

Here are the steps:

1. Create an excel sheet an put some data in it. Like this: 
   

   Value	Result
   1	TRUE
   2	FALSE
   -1	FALSE
   0	FALSE

2. Then you have to select the contents including the headers and create a named range. I will use the universal “TestData” name. How you do it? Select the contents -> Right click in it -> Choose Define Name option -> Put the name as “TestData”. Do not worry about the headers. xUnit will ignore the first row.
3. Also to make it work like a data source, you have to save the Excel in “Excel 97-2003 workbook format”. Use Save as to choose this option.
4. You usually will save the Excel inside the project folder. Then in Visual studio you add the excel file by right clicking project name and going for  “Add existing item” option.
5. In Visual Studio, go into the properties of the newly added excel file and set “Copy to Output Directory” to “Copy Always”

Now time for the test code. It is very similar to what we have before.

using Xunit;
using Xunit.Extensions;

namespace ExcelDataDrivenTests
{
    public class ExcelDataDrivenTests
    {
        [Theory]
        [ExcelData("ExcelDataSource.xls", "Select * from TestData")]
        public void SampleTest1(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

You might notice that not much has changed in the test file. Only in the highlighted line we specified that we are going to use ExcelData attribute and then supplied the details. We provided the name of the excel file and then a simple SQL statement to select everything from our named range “TestData”.

[xUnit 2 update]
They removed ExcelData attribute from xUnit. Instead they have provided sample code which we can change to our needs. So now you need to add this file to your project too.

ExcelDataAttribute.cs

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.OleDb;
using System.IO;
using System.Linq;
using System.Reflection;
using Xunit.Sdk;

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public sealed class ExcelDataAttribute : DataAttribute
{
    private static readonly string connectionTemplate =
        "Provider=Microsoft.ACE.OLEDB.12.0; Data Source={0}; Extended Properties='Excel 12.0;HDR=YES;IMEX=1;';";

    public ExcelDataAttribute(string fileName, string queryString)
    {
        FileName = fileName;
        QueryString = queryString;
    }

    public string FileName { get; private set; }

    public string QueryString { get; private set; }

    public override IEnumerable GetData(MethodInfo testMethod)
    {
        if (testMethod == null)
            throw new ArgumentNullException("testMethod");

        ParameterInfo[] pars = testMethod.GetParameters();
        return DataSource(FileName, QueryString, pars.Select(par => par.ParameterType).ToArray());
    }

    private IEnumerable DataSource(string fileName, string selectString, Type[] parameterTypes)
    {
        string connectionString = string.Format(connectionTemplate, GetFullFilename(fileName));
        IDataAdapter adapter = new OleDbDataAdapter(selectString, connectionString);
        DataSet dataSet = new DataSet();

        try
        {
            adapter.Fill(dataSet);

            foreach (DataRow row in dataSet.Tables[0].Rows)
                yield return ConvertParameters(row.ItemArray, parameterTypes);
        }
        finally
        {
            IDisposable disposable = adapter as IDisposable;
            disposable.Dispose();
        }
    }

    private static string GetFullFilename(string filename)
    {
        string executable = new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath;
        return Path.GetFullPath(Path.Combine(Path.GetDirectoryName(executable), filename));
    }

    private static object[] ConvertParameters(object[] values, Type[] parameterTypes)
    {
        object[] result = new object[values.Length];

        for (int idx = 0; idx < values.Length; idx++)
            result[idx] = ConvertParameter(values[idx], idx >= parameterTypes.Length ? null : parameterTypes[idx]);

        return result;
    }

    /// 

    /// Converts a parameter to its destination parameter type, if necessary.
    /// 
    /// The parameter value
    /// The destination parameter type (null if not known)
    /// The converted parameter value
    private static object ConvertParameter(object parameter, Type parameterType)
    {
        if ((parameter is double || parameter is float) &&
            (parameterType == typeof(int) || parameterType == typeof(int?)))
        {
            int intValue;
            string floatValueAsString = parameter.ToString();

            if (Int32.TryParse(floatValueAsString, out intValue))
                return intValue;
        }

        return parameter;
    }
}

And on running this code we see that the test runs four times using data provided from the excel sheet.

There are ways to run the tests using date from SQL server and OleDb too but at this point I am tired. . Maybe some day later.