Writing Data Driven Tests using xUnit

By | July 24, 2015

This is going to be a short tutorial (now updated for xUnit 2) on how to go about writing data driven tests using xUnit. The samples used will be easy to understand and relate to.

In case you are new to xUnit then all I will say is that xUnit is the next iteration for popular Nunit framework. It is better in many respects.

So what are data driven tests?

To keep it very simple, these are the tests which are run on different sets of data.

The input and the expected results are typically kept in same file. And the test reads the inputs and executes the test. Then it compares the result of the test with the expected result in the file and decides if test passed or failed. More details here.

As usual I will divide the tutorial into WHY and HOW.

WHY

There will be times when you will notice that you have tests which have same structure but the data on which they are working is different. No change otherwise in the code contained in the tests. These tests are good candidates for data driven tests. They reduce the number of tests in the file without compromising on the coverage.

HOW

Before you start off, remember to install via Nuget:

  • xUnit.Net to get the basic xUnit functionality.
  • xUnit.Net extensions to get support for data driven testing. [xUnit 2 update] This is no longer needed.

I will discuss three ways of going about Data Driven Tests using xUnit. There are more. I might cover them later.

Inline Data Driven Tests

In this the data is contained in the test file itself as the code below shows.

But first I will create a small class called CheckThisNumber. I will create a simple method called CheckIfEqual in this class. We will write the tests for this method.

namespace InlineDataDrivenTests
{
    public class CheckThisNumber
    {
        public int IntialValue { get; set; }

        public CheckThisNumber(int intialValue)
        {
            IntialValue = intialValue;
        }

        public bool CheckIfEqual(int input)
        {
            return IntialValue == input;
        }
    }
}

 

Time to write the inline data driven tests !!

using Xunit;

namespace InlineDataDrivenTests
{
    public class InlineDataDrivenTests
    {
        [Theory]
        [InlineData(1, true)]
        [InlineData(2, false)]
        [InlineData(-1, false)]
        [InlineData(0, false)]
        public void SampleDataDrivenTest(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

Let us go through important steps.

  • Line 07: Notice the attribute Theory. In normal xUnit tests you use attribute called Fact.
  • Line 08: Test is further decorated with InlineData attribute to tell xUnit about what kind of data driven testing will be done. Note the parameters in the parenthesis. These are the ones which will be used by the test case. You might see that Line 09-12 almost resemble a table.
  • Line 12: The function takes two parameters. The type of the parameters is of the same type as the one in InlineData.
  • Line 15: The number parameter getting used.
  • Line 16: The expectedResult parameter getting used.

This is a single test. But actually it will run 4 times. With different data each time. As you can see below.

Inline Data Driven Tests Sample

You just avoided writing 4 test cases which would have been the same code wise. And without sacrificing coverage.

Property Data Driven Tests

Now why we need this? Sometimes it might so happen that you want to share the data. Like there might be two test cases in two different files who will be working with same set of data. You might create the testcases in Inline data driven tests format but that is duplication. Not cool. Property Data Driven Test format allows for the reuse of the data.

Let us have some more code.

As usual we will have the CheckThisNumber class.

namespace PropertyDataDrivenTests
{
    public class CheckThisNumber
    {
        public int IntialValue { get; set; }

        public CheckThisNumber(int intialValue)
        {
            IntialValue = intialValue;
        }

        public bool CheckIfEqual(int input)
        {
            return IntialValue == input;
        }
    }
}

But this time we will add another class DemoPropertyDataSource.cs which will contain the test data. Here it goes.

using System.Collections.Generic;

namespace PropertyDataDrivenTests
{
    public static class DemoPropertyDataSource
    {
        private static readonly List<object[]> _data 
            = new List<object[]>
                {
                    new object[] {1, true},
                    new object[] {2, false},
                    new object[] {-1, false},
                    new object[] {0, false}
                };

        public static IEnumerable<object[]> TestData
        {
            get { return _data; }
        }
    }
}
  • Line 07 : We create a List of Object Array. Note that Object is used to keep things generic.
  • Line 10 – 13 : We create the test data.
  • Line 16 : We define a public property called “TestData”.
  • Line 18 : We return the list we created.

Now we will add two testfiles. To keep things simple I have kept the content inside them same. Just changed the names of the tests.

TestFile1.cs

using Xunit;


namespace PropertyDataDrivenTests
{
    public class TestFile1
    {
        [Theory]
        [MemberData("TestData", MemberType = typeof(DemoPropertyDataSource))]
        public void SampleTest1(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

TestFile2.cs

using Xunit;


namespace PropertyDataDrivenTests
{
    public class TestFile2
    {
        [Theory]
        [MemberData("TestData", MemberType = typeof(DemoPropertyDataSource))]
        public void SampleTest2(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

There is not much change in the test files except the highlighted line. MemberData attribute is used. And then we provide the details.
[xUnit 2 update] PropertyData has been retired. MemberData is now used as you can see above.

When we run the tests you will see that the tests are run against the same common set of test data. As you can see below.

Property Data Driven Tests Sample
We just shared the test data between testcases. Wo Hoo.

Excel Data Driven Tests

This is taking things one level up. Rather than storing the common test data in a class, why not put it in an excel file? Makes it easy for non technical person to come up with test data and populate the same.

However excel being excel, there are some hoops to jump through.

Here are the steps:

  1. Create an excel sheet an put some data in it. Like this: 
    Value Result
    1 TRUE
    2 FALSE
    -1 FALSE
    0 FALSE
  2. Then you have to select the contents including the headers and create a named range. I will use the universal “TestData” name. How you do it? Select the contents -> Right click in it -> Choose Define Name option -> Put the name as “TestData”. Do not worry about the headers. xUnit will ignore the first row.
  3. Also to make it work like a data source, you have to save the Excel in “Excel 97-2003 workbook format”. Use Save as to choose this option.
  4. You usually will save the Excel inside the project folder. Then in Visual studio you add the excel file by right clicking project name and going forΒ  “Add existing item” option.
  5. In Visual Studio, go into the properties of the newly added excel file and set “Copy to Output Directory” to “Copy Always”

Now time for the test code. It is very similar to what we have before.

using Xunit;
using Xunit.Extensions;

namespace ExcelDataDrivenTests
{
    public class ExcelDataDrivenTests
    {
        [Theory]
        [ExcelData("ExcelDataSource.xls", "Select * from TestData")]
        public void SampleTest1(int number, bool expectedResult)
        {
            var sut = new CheckThisNumber(1);
            var result = sut.CheckIfEqual(number);
            Assert.Equal(result, expectedResult);
        }
    }
}

You might notice that not much has changed in the test file. Only in the highlighted line we specified that we are going to use ExcelData attribute and then supplied the details. We provided the name of the excel file and then a simple SQL statement to select everything from our named range “TestData”.

[xUnit 2 update]
They removed ExcelData attribute from xUnit. Instead they have provided sample code which we can change to our needs. So now you need to add this file to your project too.

ExcelDataAttribute.cs

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.OleDb;
using System.IO;
using System.Linq;
using System.Reflection;
using Xunit.Sdk;

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public sealed class ExcelDataAttribute : DataAttribute
{
    private static readonly string connectionTemplate =
        "Provider=Microsoft.ACE.OLEDB.12.0; Data Source={0}; Extended Properties='Excel 12.0;HDR=YES;IMEX=1;';";

    public ExcelDataAttribute(string fileName, string queryString)
    {
        FileName = fileName;
        QueryString = queryString;
    }

    public string FileName { get; private set; }

    public string QueryString { get; private set; }

    public override IEnumerable GetData(MethodInfo testMethod)
    {
        if (testMethod == null)
            throw new ArgumentNullException("testMethod");

        ParameterInfo[] pars = testMethod.GetParameters();
        return DataSource(FileName, QueryString, pars.Select(par => par.ParameterType).ToArray());
    }

    private IEnumerable DataSource(string fileName, string selectString, Type[] parameterTypes)
    {
        string connectionString = string.Format(connectionTemplate, GetFullFilename(fileName));
        IDataAdapter adapter = new OleDbDataAdapter(selectString, connectionString);
        DataSet dataSet = new DataSet();

        try
        {
            adapter.Fill(dataSet);

            foreach (DataRow row in dataSet.Tables[0].Rows)
                yield return ConvertParameters(row.ItemArray, parameterTypes);
        }
        finally
        {
            IDisposable disposable = adapter as IDisposable;
            disposable.Dispose();
        }
    }

    private static string GetFullFilename(string filename)
    {
        string executable = new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath;
        return Path.GetFullPath(Path.Combine(Path.GetDirectoryName(executable), filename));
    }

    private static object[] ConvertParameters(object[] values, Type[] parameterTypes)
    {
        object[] result = new object[values.Length];

        for (int idx = 0; idx < values.Length; idx++)
            result[idx] = ConvertParameter(values[idx], idx >= parameterTypes.Length ? null : parameterTypes[idx]);

        return result;
    }

    /// 
    /// Converts a parameter to its destination parameter type, if necessary.
    /// 
    /// The parameter value
    /// The destination parameter type (null if not known)
    /// The converted parameter value
    private static object ConvertParameter(object parameter, Type parameterType)
    {
        if ((parameter is double || parameter is float) &&
            (parameterType == typeof(int) || parameterType == typeof(int?)))
        {
            int intValue;
            string floatValueAsString = parameter.ToString();

            if (Int32.TryParse(floatValueAsString, out intValue))
                return intValue;
        }

        return parameter;
    }
}

And on running this code we see that the test runs four times using data provided from the excel sheet.
Excel Data Driven Test Sample

There are ways to run the tests using date from SQL server and OleDb too but at this point I am tired. smiley. Maybe some day later.

24 thoughts on “Writing Data Driven Tests using xUnit

  1. Swathi Kudumula

    if we need to run tests reading data from CSV do we need to create CSVAttribute class for processing the data?

    Reply
    1. Pankaj K Post author

      That will be complicating things in my opinion. Simpler way will be to take the csv file itself as an input to the test via Theory attribute. In your test body you can read it one line at a time. Then split the lines to get the fields you want.

      Reply
  2. Antony Raj

    Do we have any option to push any data back to the excel sheet, say like the test results.

    Reply
  3. Pankaj K Post author

    I have not done this. For me ability to generate the html or xml report usually worked fine. If that is something useful for you then have a look at xunit console runner. It has options of generating xml and html report of the test run.

    Reply
  4. Mariana

    Hi, when I have a really long string in a cell on the .xls file, when the data is passed to the string variables on my tests, the strings come truncated. Missing the end, I noticed that it truncetes when exceeds over 250 chars in one string more or less and I can seem to find why this is happening. Please I’d appreciate some help!

    Reply
  5. Pingback: Data Driven Tests with xUnit.net – Travels.Through(code => new Tale(code));

  6. Rakesh

    could not able to use ExcelData attribute. saying type or namespace could not be found. I am having multiple rows of data in excel and want to implement using xunit.

    Reply
  7. Diana

    Hi, I’m trying to use your code, but I got an error in the line: IDataAdapter adapter = new OleDbDataAdapter(selectString, connectionString); it says namespace “OleDbDataAdapter” not found, do you have any idea, and in the line using System.Data.Oledb; is marked as not required, and I can’t continue. πŸ™

    Reply
    1. Pankaj K Post author

      Looks like you are missing reference to System.Data. Check this answer on stackoverflow. Also which version of .NET are you on?

      Reply
      1. Diana

        Thanks for your answer, I was trying the answer you mentioned, but I didn’t get the System.data listed πŸ™ I’m using visual studio 2017 with .Net core 2.0 with xunit

        Reply
        1. Diana

          I found the System.Data in a directory, but now when trying to add it as reference I got an error message: “The reference is not valid or not allowed.” πŸ™

          Reply
        2. Pankaj K Post author

          .NET core still does not have a 100% parity with the traditional .NET as we know and are familiar with. A quick google search throws up some interesting info like this and this and this.

          Reply
          1. subham

            just the OleDb data adapter is missing in .net core 2.0
            I think it can be done without that too?
            If you can help?

          2. Pankaj K Post author

            If I have time I will look into this. Meanwhile if you manage to find a fix do let me know. πŸ™‚

  8. subham

    Sure.!
    I have similarly tried something with Json data recently, by creating a Json data attribute, in .Net Core.
    It works for me.
    Will give it a shot for Excel too. πŸ™‚

    Reply
  9. Andy

    When I try to run the test case pointing to the .xlsx, I’m getting the following. Any ideas?

    The ‘Microsoft.ACE.OLEDB.12.0’ provider is not registered on the local machine.

    Reply
  10. Angy

    Hi, I’m trying to run this and works but if one Assert fails on my test, this doesn’t continues with the next excel row, this just stop execution. Any solution for this?

    Reply
    1. Pankaj K Post author

      Will like to see some code on this. My understanding is that the test runs for all the parameters.

      Reply
  11. Purna

    Thank you, i like your explanation on dragging the scenarios from excel sheets. This is what i am looking for.

    By any chance if you updated your code. Could you please let me know

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.