Microsoft Fakes

The fakes framework can be used to test any .NET method. Microsoft fakes mocks objects with no preexisting interfaces.

The fakes framework actually provides two ways to create mock objects.

1. Shims
2. Stubs

Getting started with Sims

Create a new class library in visual studio 2012. And add the class CartToShim

using System;

using System.Collections.Generic;

using System.Collections.ObjectModel;

namespace FakingExample

{

 public class CartToShim

{

publicint CartId { get; set; }

publicint UserId { get; set; }

public ReadOnlyCollection<CartItem> CartItems { get; set; }

private List<CartItem> cartItems = newList<CartItem>();

public DateTime CreateDateTime { get; set; }

public CartToShim(int cartId, int userId)

{

CartId = cartId;

UserId = userId;

CreateDateTime = DateTime.Now;

CartItems = new ReadOnlyCollection<CartItem>(cartItems);

}

public void AddCartItem(int productId)

{

var cartItemId = DataAccessLayer.SaveCartItem(CartId, productId);

cartItems.Add(new CartItem(cartItemId, productId));

}

}

}

Add CartItem class

namespace FakingExample

{

 public classCartItem

{

   public int CartItemId { get; set; }

   public int ProductId { get; set; }

   public CartItem(int cartItemId, int productId)

{

CartItemId = cartItemId;

ProductId = productId;

}

}

}

Add the DataAccessLayer class

using System.Data;

using System.Data.SqlClient;

namespaceFakingExample

{

publicstaticclassDataAccessLayer

{

public static int SaveCartItem(int cartId, int productId)

{

using (var conn = newSqlConnection(“”))

{

var cmd = newSqlCommand(“InsCartItem”, conn);

cmd.CommandType = CommandType.StoredProcedure;

cmd.Parameters.AddWithValue(“@CartId”, cartId);

cmd.Parameters.AddWithValue(“@ProductId”, productId);

conn.Open();

return (int)cmd.ExecuteScalar();

}

}

}

}

Add unti test project to the solution

In the untitest project, add the FakingExample project

Now right click on the FakingExample reference in the unit test project and select “Add Fakes Assembly”

This will create the following references

Now we are ready to do faking.

Add CartToShimTests test class to the test project

using Microsoft.QualityTools.Testing.Fakes;

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace FakingExample.Tests

{

[TestClass]

public class CartToShimTests

{

[TestMethod]

public void AddCartItem_GivenCartAndProduct_ThenProductShouldBeAddedToCart()

{

using (ShimsContext.Create())

{

int cartItemId = 42, cartId = 1, userId = 33, productId = 777;

Fakes.ShimDataAccessLayer.SaveCartItemInt32Int32 = (c, p) => cartItemId;

var cart = new CartToShim(cartId, userId);

cart.AddCartItem(productId);

Assert.AreEqual(cartId, cart.CartItems.Count);

var cartItem = cart.CartItems[0];

Assert.AreEqual(cartItemId, cartItem.CartItemId);

Assert.AreEqual(productId, cartItem.ProductId);

}

}

}

}

Now run the unti test. Notice that our test completely skipped over the SaveCartItem in DataAccessLayer, otherwise we would have received exception saying invalid connection string.

In my next post I will show the Stubs example.

Thanks

Vinay

When you start to solve a problem in TDD, one of the questions that most of the developers have is about the correct place to start testing the code.

Is it better to start with a high level functional test (Outside-In) at a user story and drill down into the low level tests as you  progress or to come with an initial low level design and then start writing tests from the lowest component (Inside-Out) in the design and progress to the higher level components?

A short answer would be that it depends on your / team’s coding preference or approach.

If you’re not sure how your code should look like or how it will interact or communicate with other parts of your system, it’s better to have a small design discussion with the team to identify the best possible design solution and start with the low level component and let it evolve as you write more tests.

But if your design already exists (Layered approach, where the layered components are identified like controllers, services, repositories etc.), you can start with the Outside-In approach.

As you see, Inside out testing implies a developer would probably consider the high-level design and break them up into low level components that interact with each other to provide the desired functionality. The developer thinks about how each component will be used by its client components and tests accordingly.

The advantage is that the team can deliver working software quickly as they have started with a proper design in his mind. However, since the code delivered was not written considering the high level user requirements, it may produce code that doesn’t go well with the YAGNI principle and also have a risk of not meeting the actual functionality.

In the Outside In approach, the part of system that has the closest correlation to the user requirements are tested first, guarantees that the critical parts of the application is tested first. Here the focus is more on how the user interacts with the system, rather than how the components interact with each other. The test cases generated hence supports the usability of the system.

In one line, in agile world, whole team is responsible for quality where as in traditional process, its QA’s job. Also, quality is never an after though in Agile, where as in waterfall, quality is assessed after building the product. QA is not the last line of defense but involved at the beginning of the project

Testing is a not a phase in agile development. Agile testing focuses on testing iteratively, as often as stable code base is available, until quality is achieved from end customer’s perspective. Development is considered as coding and testing. Rather, lot of agile teams does testing + coding. This is due to emphasis on quality even before start coding. Automated tests are designed for efficiency and maintainability. This leaves quality experts in the team for more exploratory testing.  This strategy is more in line with Agile Testing quadrants.

Enabling continuous quality is a key focus area of Update 1for VS2012. With the release for Update 1, code coverage results are available for automated test cases as well as manual tests. You can now record your coded ui tests and analyze the code coverage for these tests. Using the test explorer, you can now analyze the code coverage results for a group of tests or a single test.

Unit testing code that is spawned as a separate thread can be challenging.

We need to make sure our Asserts can run after the parallel thread completes, and not before.

Moreover, we need to make sure our unit test doesn’t complete before the parallel thread, leaving it orphan and causing errors for the test agent.

Consider the following code:

        private void LoadCustomerData()
        {
            this.IsLoading = true;

            // the following code will run a separate thread (use a BackgroundWorker if you like)
            Task.Factory.StartNew(() =>
                                      {
                                          // do some stuff here to load the data
                                          this.Customers = dao.GetCustomers();

                                          // finally indicate that loading is completed
                                          this.IsLoading = false;
                                      });
        }

Now consider a unit for this:

        [TestMethod]
        public void LoadData_should_bla_bla()
        {
            // Arrange
            var testObject = new SomeClass();

            // Act
            testObject.LoadData();

            // Assert
            Assert.IsNotNull(testObject.Customers);
        }

The test looks pretty straight forward. However, when you run this, it’ll not pass. You’ll most probably get the “agent terminated before execution completed” error or your assert will fail.

The reason is this: this.Customers is set by a separate thread, and as soon as you call LoadCustomerData(), it spawns that thread and returns immediately. Giving no time for the actual load to happen before you can do your Assert. In other words, your assert runs before the thread that loads the data.

OK, how to we solve this? Well, my first attempt would be to wait before I assert:

        [TestMethod]
        public void LoadData_should_bla_bla()
        {
            // Arrange
            var testObject = new SomeClass();

            // Act
            testObject.LoadData();

            while (testObject.IsLoading)
                ; // do nothing till loading completes

            // Assert
            Assert.IsNotNull(testObject.Customers);
        }

Again, at first look this looks pretty straight forwards. Your test would also work pass now. However, you’ll hit trouble when the test fails. What if something goes wrong when loading and IsLoading is never set to false. Well, your test will run indefinitely!

Ideally, your test should fail if for some reason IsLoading is not set to false. So, we could try a “wait with timeout” approach. Instead of waiting indefinitely, we can wait for some time (say 2 seconds) and throw a timeout exception after that (which will make the test fail). May be something like this:

        [TestMethod]
        public void LoadData_should_bla_bla()
        {
            // Arrange
            var testObject = new SomeClass();

            // Act
            testObject.LoadData();

            // Wait until IsLoading becomes false or for 2 seconds, whichever occurs first
            TestUtility.WaitWithTimeout(() => testObject.IsLoading == false, 2000);

            // Assert
            Assert.IsNotNull(testObject.Customers);
        }

There you go. This would work!

And here’s the code for this WaitWithTimeout function:

    class TestUtility
    {
        internal static void WaitUntil(Func<bool> whatToWaitFor, int howManyMilliseconds)
        {
            DateTime start = DateTime.Now;

            while (!whatToWaitFor())
            {
                Thread.Sleep(100); // just breathe a little instead of a continuous loop!

                if (DateTime.Now.Subtract(start).TotalMilliseconds > howManyMilliseconds)
                {
                    throw new TimeoutException("Wait timed out");
                }
            }
        }

Cheers!

-Anand

        private void LoadInitialData()
        {
            Loading = true;
            Task.Factory.StartNew(() =>
                                      {
                                          if (BranchTeams == null) // we only need to load first time
                                          {
                                              LoadBranchTeamData();
                                              if (App.StartupSurveyId > 0)
                                              {
                                                  SurveyFilter = App.StartupSurveyId;
                                              }

                                          }

                                          ApplyFilters();

                                          Loading = false;
                                      });
        }

        [TestMethod]
        public void LoadData_should_bla_bla()
        {
            // Arrange
            var testObject = new SomeClass();

            // Act
            testObject.LoadData();

            // Assert
            Assert.IsNotNull(testObject.Customers);
        }

I saw the following scenario somewhere on a live project. A fellow pair was trying to test drive a class and the situation is that he is trying to write a function to park a vehicle in a ParkingLot and wants to see if that has been successful. The class looks like the following.
public class ParkingLot {
private int capacity;
private List<String> parkingTickets;
public ParkingLot(int capacity) {
this.parkingTickets = new ArrayList<String>();
this.capacity = capacity;
}
public void park() {
if (parkingTickets.size() < capacity) {
String parkingTicket = new Random(100).toString();
parkingTickets.add(parkingTicket);
}
throw new RuntimeException("Parking is Full");
}
}
And the test class looks like the following for this:
public class ParkingLotTest extends TestCase {
private ParkingLot parkingLot;
@Override
public void setUp() throws Exception {
parkingLot = new ParkingLot(10);
super.setUp();
}
public void testShouldGiveParkingTicketWhenVehicleIsParked() throws Exception {
parkingLot.park();
// what do I assert here is the problem
}
}
Now the immediate problem is what to assert here? The park() methods behavior is that it adds a parking ticket to the list of parking tickets of the parking lot if there is space, otherwise it throws an exception saying parking is full. Soc, what my friend did here is the following:
public class ParkingLot {
private int capacity;
private List<String> parkingTickets;
public ParkingLot(int capacity) {
this.parkingTickets = new ArrayList<String>();
this.capacity = capacity;
}
public List<String> getParkingTickets() {
return parkingTickets;
}
public void park() {
if (parkingTickets.size() < capacity) {
String parkingTicket = new Random(100).toString();
parkingTickets.add(parkingTicket);
}
throw new RuntimeException("Parking is Full");
}
}
Note the added getParkingTickets() method. And then he changed the test code to assert that the there is one more ticket in the list, i.e the size of the list is increased by one.

The test code looks as follows now:
public class ParkingLotTest extends TestCase {
private ParkingLot parkingLot;
@Override
public void setUp() throws Exception {
parkingLot = new ParkingLot(10);
super.setUp();
}
public void testShouldGiveParkingTicketWhenVehicleIsParked() throws Exception {
int previousSize = parkingLot.getParkingTickets().size();
parkingLot.park();
int newSize = parkingLot.getParkingTickets().size();
assertEquals(1, newSize - previousSize);
}
}
Good enough!! We have meaningful assertion now. The test passes. All’s WELL!!

But wait….. What just we just do??  We added code for test!!

Was that the idea ? … Are we testing what we have .. or are we are adding code so that we could test what we had? Certainly, we just added something which was not suppose to be a part of the ParkingLots API. We added an extra public method, so that we could complete our test. We are writing CODE for TEST… not TEST for CODE

I am not claiming that we did not get this method in the ParkingLot’s API at all ever in future also, but what I am trying to say is that it wasn’t needed at that point in time. It may or may not be there in future. Only if a feature would drive it, it would would come, not otherwise.

TDD : Code Driven BY Test… not Code Driven FOR Test

So, what should have ideally happened in the above scenario?? There are two approaches to it.

• Mock parkingTickets object, so that you can set an expectation on the parkingTickets that when a park() is called with parkingLot having spaces, then add() method should have been invoked on the parkingTickets object.
• Look for something that logically proves that the park was successful. For e.g, what should logically be possible as a behavior of the ParkingLot on successful parking is that, it should be able to unpark(). So try an unpark and see if that goes successful.

In the above case though, the unpark() code is still not in, so I would rather go with the first approach.

@Test
public void shouldLoginForAValidUser() {
Assert.assertTrue(app.login(“blablauser”, “abcdef”));
}

@Test
public void shouldLoginForAValidUser() {
Assert.assertTrue(app.login(“sanjeev”, “secret”));
}

Have a look at the above two tests. They are doing exactly the same thing, i.e testing that the login is successful in case a valid user name and password is provided.

But, do you think they give you the same feeling in terms of seriousness. To me, honestly, they do not. The first test doesn’t give me a real serious feeling, however the second one does. As per my experience goes, an average programmer (most of us are) does not pay serious attention to the tests that do not look serious. So there are certain things about test data that I would like to point out in this post, which I have been talking about lately, with the teams that I have been on. I am not sure, if you would agree to them all, but I just wanted to share this feeling. And here goes the list:

• Try to keep the test data as close to the real world data as possible.
• Do not use, bla-bla like non-serious words in your tests
• Try to test only one thing in a test. Do not try to cover different scenarios in a single test. All the scenarios may belong to on method, but the very fact that you are able to identify them as scenarios, is good enough a reason for you to have separate tests for each of them.
• If possible, try to get the data from the production, use them in your tests (assuming you are working on a legacy application and the there is production data available). Note that, if you are using the production data, always sanitize the data. By sanitizing, I mean that you should take care that the sensitive fields like real emails, passwords, and, CC information etc if at all is there, should be removed and then the data should be used.

There are other advantages of using the real data (or close to real data) in your tests. For example, you get to see the various different formats in which a particular data could be used in the real world. For example, sometimes while testing the login functionality, we may just used a password like “secret”, or secret123 etc, but in the real world you may get scenarios like too short possword, too long password, or password with special characters etc. All these cases will help you in writing more scenarios for a particular functionality and thus making it bug free.

So, I think, that while writing test, it hardly takes any extra effort to have the data close to real world and the benefits are worth taking that extra time and effort, however less/more it may be.

I hope, you guys agree. Please let me know by leaving comments.