I have been writing unit tests for many years and have become a big fan of test driven development. When learning to write unit tests, almost all examples use the Arrange, Act, Assert (AAA) pattern. While there is nothing wrong with this pattern, over time I have come to dislike it. I find that it lends itself to code duplication and overly complicated tests.
I have often found myself in the position of having to debug complex tests written in the AAA pattern. When you are the author of such a test, debugging it is a lot easier as you understand the mindset behind how it was written and what it is trying to cover. Another developer looking at the test will need to determine these things before trying to fix any issues which can take time.
Having worked with other frameworks such as SpecFlow and Gherkin, I have come to the conclusion that I much prefer the pattern of Given When Then (GWT) over AAA. I feel that it makes you think about your testing at a deeper level and also promotes reuse of code within tests.
I have tried using both Gherkin and SpecFlows over the years within .NET projects and have never had a great experience. The following are reasons why I would not consider them in any project I was creating:
SpecFlow
In the .NET Framework this was a very heavyweight framework. I particularly disliked the way it auto generated ugly C# code behind the scenes. In the early days of .NET Core, there was no option to use SpecFlow. I believe this may no longer be the case but have not tried to use it recently.
Gherkin
I have tried a couple of implementations of Gherkin in .NET Core with varying levels of usability. The best one seems to be the Gherkin plugin for XUnit but it still lacks basic features I feel should be common place such as IntelliSense and code navigation between spec files and step files.
Custom Framework
A few years ago I was working on a contract for one of my clients along with some other contract developers. A couple of them mentioned how they prefer GWT and had written their own framework to use in .NET projects. This set me down the path of creating my own custom framework.
My requirements were simple. The framework should:
- Allow tests to be written in a GWT style, ensuring that tests are readable and keep the purpose of a test clear.
- Support for Async methods.
- Allow for Intellisense and code navigation
- Integrate with whichever test framework I choose for the underlying tests.
- Tests execution should be able to be performed by either MSTest or JetBrains ReSharper.
- Feedback step execution to console. This is useful for debugging failing tests.
Problem 1: How to write tests with a GWT style?
The following class and method is a simple piece of code that I used as a subject under test:
public static class MathOperations
{
public static int AddTwoNumbers(int number1, int number2)
{
return number1 + number2;
}
}I would like to write a test for this code to check that two positive numbers can be added together successfully. My aim is to have a test structured as per the following example:
[Test]
public void MathOperations_AddTwoNumbers_Adds_Positive_Numbers()
{
Given I have two numbers to add together;
When I use the AddTwoNumbers method to add my numbers;
Then The returned result should be the sum of the two numbers;
}In C# this syntax is not possible due to the spaces between words. I decided that using underscores would solve this issue. The next problem is telling the compiler that a particular statement maps to a piece of code. Well, surely this is simple and I just need to add a method named the same as the statement? Not quite! The compiler has no idea what “Given” is in this context.
I could solve this by removing the GWT statements from the beginning of each line but that would result in the test instantly becoming unreadable. So, Given, When and Then need to also be methods that take a parameter of the action to invoke.
To enable me to pass an action to another method, the Action data type can be used. I can then invoke that action within the function. For example:
private void Given(Action action)
{
action();
}This allows me to get to my goal of a much more human readable step in my test. As I am calling a function, parentheses must be added so that the final result looks as follows:
[Test]
public void MathOperations_AddTwoNumbers_Adds_Positive_Numbers()
{
Given(I_have_two_numbers_to_add_together);
...
}So, in conclusion, I think by having the test method name stating the scenario and the body having the GWT syntax, tests now look much more in line with SpecFlow and Gherkin.
Problem 2: Allow for async methods
The code up until now is not very friendly to asynchronous code. While it is possible to call async methods from within my GWT methods, it requires use of the .Result property. I do not like this approach as it is not as clear that the code is asynchronous and I would prefer to use the await operator.
To enable me to achieve this the Given, When and Then methods must be changed to return a Task and also be marked as async. This then requires the step method to return a Task, which then causes the parameter passed to one of the GWT methods to be of type Func<Task>. Finally, the call to one of the GWT methods will need to be marked with await and the test marked as async. This results in the following:
[Test]
public async Task MathOperations_AddTwoNumbers_Adds_Positive_Numbers()
{
await Given(I_have_two_numbers_to_add_together);
...
}
private async Task I_use_the_AddTwoNumbers_method_to_add_my_numbers()
{
//Do something
}
private async Task Given(Func<Task> action)
{
await action();
}The trade off here is that my GWT statements in my test need to be prefixed with await which makes them slightly less readable. After a lot of fiddling about, I have decided that this is a tradeoff I am willing to accept.
This now provides enough to get a fully working example. When put together, it looks like this:
public class MathOperations
{
public async Task<int> AddTwoNumbers(int number1, int number2)
{
return number1 + number2;
}
}[TestFixture]
public class Tests
{
private int _number1;
private int _number2;
private int _result;
[Test]
public async Task MathOperations_AddTwoNumbers_Adds_Positive_Numbers()
{
await Given(I_have_two_numbers_to_add_together);
await When(I_use_the_AddTwoNumbers_method_to_add_my_numbers);
await Then(The_returned_result_should_be_the_sum_of_the_two_numbers);
}
private Task I_have_two_numbers_to_add_together()
{
_number1 = 1;
_number2 = 4;
return Task.CompletedTask;
}
private async Task I_use_the_AddTwoNumbers_method_to_add_my_numbers()
{
var sut = new MathOperations();
_result = await sut.AddTwoNumbers(_number1, _number2);
}
private Task The_returned_result_should_be_the_sum_of_the_two_numbers()
{
Assert.That(_result, Is.EqualTo(_number1 + _number2));
return Task.CompletedTask;
}
private async Task Given(Func<Task> action)
{
await action();
}
private async Task When(Func<Task> action)
{
await action();
}
private async Task Then(Func<Task> action)
{
await action();
}
}You should notice that when a step method is calling an asynchronous method, it can be declared as async Task as it makes use of the await operator. Not all steps will be calling asynchronous code as seen in the example. In this case, the step method must be declared without the async operator. The method still has to return a Task so you will notice in this scenario I return Task.CompletedTask.
This is another tradeoff I was willing to accept. The alternative involved having a mix of both async and non async methods for the GWT methods. That, in turn, resulted in the test having a mixture of statements with and without the async keyword. I decided that it was more readable for all lines to be marked as async rather than a mixture.
Problem 3: Logging debug information to the test console.
When I run the test I do not see any information in the test output. It would be really useful, especially for debugging, if I could see the steps as they complete or fail. This is one feature of SpecFlow that I really like.
I am using NUnit for my testing so a quick google search provided me with the following link: https://github.com/nunit/docs/wiki/Text-Output-from-Tests-Spec.
It would appear that there is a simple TestContext.WriteLine() method that will achieve exactly what I am looking for. With this in mind, I updated my GWT methods as follows:
private async Task Given(Func<Task> action)
{
TestContext.WriteLine($"Given {action.Method.Name}");
try
{
await action();
TestContext.WriteLine("...done");
}
catch (Exception e)
{
TestContext.WriteLine($"Error - {e.Message}");
throw;
}
}
private async Task When(Func<Task> action)
{
TestContext.WriteLine($"When {action.Method.Name}");
try
{
await action();
TestContext.WriteLine("...done");
}
catch (Exception e)
{
TestContext.WriteLine($"Error - {e.Message}");
throw;
}
}
private async Task Then(Func<Task> action)
{
TestContext.WriteLine($"Then {action.Method.Name}");
try
{
await action();
TestContext.WriteLine("...done");
}
catch (Exception e)
{
TestContext.WriteLine($"Error - {e.Message}");
throw;
}
}Now when I run my test, the following is displayed in the test output:
This is much nicer and will allow me to pinpoint the failing step should a test fail.
Evaluation
Looking back on my original requirements, I believe that I have satisfied them all. Let’s recap.
Allow tests to be written in a GWT style, ensuring that tests are readable and keep the purpose of a test clear.
I feel that the finished product provides a mechanism for writing clear tests that are easy to read and understand.
Support for Async methods.
This has also been achieved with the small caveat in that each step must be prefixed with async. I am happy to accept this minor tradeoff for the benefits the framework provides.
Allow for IntelliSense and code navigation
All code is just C# code with no behind the scenes compiling or trickery going on. As a result, all IntelliSense and code navigation features that I would want are available.
Integrate with whichever test framework I choose for the underlying tests.
The use of TestContext.WriteLine() has unfortunately tied my implementation to NUnit. However, the code would be easily refactored to make it agnostic of the test framework.
Tests execution should be able to be performed by either MSTest or JetBrains ReSharper.
Because I am using an existing test framework and just creating a visual wrapper, all functionality will work with whatever test runner you choose to use.
Feedback step execution to console. This is useful for debugging failing tests.
You have seen how the GWT methods log to the console to provide this information.
Conclusion
Since developing this framework I have enhanced it in many ways and now use it in all of my projects where possible. I can honestly say it really works and other developers also find it easy to use over the standard AAA pattern. Furthermore, it provides a very simple implementation that is feature rich by virtue of it being standard C# code.
I am not going to share my entire library here but there is one key piece of functionality that it is worth mentioning as it makes the tests far easier to manage.
The example above was just a single file containing the test specifications, the step definitions and the GWT functions. The first thing I did was to move the GWT functions into their own class named Specifications.cs. I can then inherit this class.
When writing tests, I try to keep them in separate files for each subject under test. When you have a lot of tests for a particular class, this becomes a bit ugly so I decided to split the file and make use of partial classes.
Given the above example, I would now have two files; MathOperationsSpecs.cs and MathOperationsSteps.cs. I define them as follows:
[TestFixture]
public partial class MathOperationsSpecs : Specification
{
[Test]
public async Task MathOperations_AddTwoNumbers_Adds_Positive_Numbers()
{
await Given(I_have_two_numbers_to_add_together);
await When(I_use_the_AddTwoNumbers_method_to_add_my_numbers);
await Then(The_returned_result_should_be_the_sum_of_the_two_numbers);
}
}public partial class MathOperationsSpecs
{
private int _number1;
private int _number2;
private int _result;
private Task I_have_two_numbers_to_add_together()
{
_number1 = 1;
_number2 = 4;
return Task.CompletedTask;
}
private async Task I_use_the_AddTwoNumbers_method_to_add_my_numbers()
{
var sut = new MathOperations();
_result = await sut.AddTwoNumbers(_number1, _number2);
}
private Task The_returned_result_should_be_the_sum_of_the_two_numbers()
{
Assert.That(_result, Is.EqualTo(_number1 + _number2));
return Task.CompletedTask;
}
}You can see an example of this framework being used in another one of my posts about in-memory integration testing. Read about it here.
This solution works well for me personally and I hope you have found this article informative. My tests feel far simpler now and the framework allows me to utilise all of the benefits that GWT provides over AAA. I would love to hear your thoughts so please feel free to leave me a comment or questions below.