Fakes and Mocks and Stubs, Oh My!!!

August 21, 2008 at 3:50 am 3 comments

So we covered how to use EasyMock to write tests in one of the previous posts, but Mocking is not the only option when you want to test something that depends on a slow and expensive service. Mocking allows you to expect calls and return values when a call is made, and make sure the number of calls were correct and the order, but sometimes, you want a bit more, or not even that much. So in those cases, you have the options of using Stubs or Fakes. So without further delay, your options are :

  1. Mocks :
    Well, Mocks… What can I say. I have already ranted and raved about the awesomeness that is Mocks. Mocks and Mocking frameworks allow you to replace heavy and expensive services with Mocks, with which you can set expectations on what calls are made to the service, and return values. This gives you the advantage of knowing exactly what is called and returned, and makes your test deterministic and super fast. You can even set expectations on the number of calls and the order, though this is something that is not extremely recommended, since that makes the test dependent on the implementation, which is never a good thing.

    Advantages :

    • Fast, Reliable
    • Deterministic
    • Lightweight
    • Control over expectations and return values.

    Disadavantages :

    • Can become dependent on implementation of method being tested.
    • Can become a mockery if not careful, that is, can be testing interaction between mocks and nothing of the actual code. Especially when all that the method under test is doing is delegating to a bunch of service calls.
    • Can involve complicated setup for expectations, especially with unwieldy objects
  2. Stubs :
    Now, whereas with Mocks, you can specify what calls will be received and what will be returned making them somewhat intelligent, Stubs are at the other end of the spectrum. The dumbest of the dumb, the easiest of the easiest, these just stub out the methods of your expensive and heavy class. So a null Stub could return null for each method, or just return some constant value each time it is called. That means that regardless of what the method is called with, it returns the exact same thing, day in and day out. It doesn’t adjust, it doesn’t react, just returns. Of course, this means that you might not be able to test all the cases you want to inspect / test. But its simple, its easy, and of course, its fast.

    Advantages :

    • Fast, reliable
    • Simple
    • Consistent

    Disadvantages :

    • Dumb
    • Can’t exercise different cases without different stubs
    • Returns only one value consistently
  3. Fakes :
    And finally, we come to the smartest (figuratively) of the lot. The Fakes. For once, being a fake can be good. While Stubs return the same thing again and again, and mocks return what you tell it to, Fakes are smarter. The easiest way to explain a Fake is with an example. Say your code depends on a heavy service like a Database. A FakeDatabase would be an in-memory implementation of the DB, thus making it faster while at the same time, providing the same logic as the normal DB would. There are different types of Fakes, like a listening / recording Fake which records all the values passed to it.

    Advantages :

    • Can test while preserving behavior of dependencies
    • Faster than using actual services
    • Can test complicated logic
    • Most comprehensive testing approach

    Disadvantages :

    • Can be complicated to set up
    • Not as fast as mocks / stubs
    • Not as easy to define expectations
    • When / Where do you test the fake (especially the complicated ones) ?

Thats my 2 cents on the different approaches to testing with dependencies. Now that we know what we can do with those darn heavy dependencies, I’ll talk more on how to make sure you can use these different approaches to test your code soon.

Advertisements

Entry filed under: code, fake, java, mock, stub, testing, unit testing. Tags: , , , , , .

Your code smells!!! My code’s untestable !!

3 Comments Add your own

  • 1. closingbraces  |  August 21, 2008 at 4:38 pm

    Thanks for the good article – a very clear summary of the differences.

    My own take on “fakes” is that it can indeed be a lot of work to create really good fakes (including testing them, as you point out). But for widely used API or services I think it can make sense for someone to do this centrally, and produce a library of “ideal” fakes for that API. Everyone else can then just use the pre-written, pre-tested fakes.

    Ideally I’d like to see this as a responsibility of API design and implementation. If an API actually needs test-doubles or a fake service in order to be easily testable, the API ought to supply this. And this ought to be taken into account in the API design (e.g. avoid needing it if possible, or make it as straightforward as possible). Well, we can all dream…

    In terms of speed, if a fake is just doing normal in-memory stuff, it might involve slightly more than a mock but this shouldn’t normally be so much as to impact performance. You can churn through an awful lot of “plain” code before noticing any impact, and the code that you’re testing probably does as much or more than the fake. At least that’s been my experience.

    I’d say defining expectations is different for a fake, rather than harder, with a lot depending on the precise nature of the fake. If the fake lets you configure and inspect everything relevant to the object’s behaviour, and can record the API calls too, then you’ve got eveything you need. I don’t think there has to be anything making it intrinsically harder than strictly necessary.

    So on the whole I tend to regard “fakes” as the ideal – if you can get someone to develop them for you!

    At any rate, that’s my personal view on fakes, based on writing a set for the Servlet API.

    Mike

    Reply
  • 2. shyamseshadri  |  August 22, 2008 at 1:18 pm

    I agree with you on the fact that Fakes are the most ideal of the lot. It gives you the closest test to an integration test while still being an unit test. And yeah, Fakes are generally fast, but compared to Mocks and stubs, not so much. But I’ll play devil’s advocate here.

    Fakes can (can, and not necessarily must) be hard to setup to work similar to an existing service. Also, to test the interesting cases with a Fake, you might need to get the, say FakeDB, into a particular state before you can do your actual test. These might be like adding specific rows, and getting the DB to a particular state. Whereas with a mock, you can just tell it to expect a call and return whatever you need it to. (Stubs of course, can never do this…). Also might be tough to use a Fake Http service and stuff (not impossible, but harder than it needs to be).

    In the end, I feel it comes down to using what is most ideal for the situation. Use Fakes whenever you can, and whenever time and complexity permits, but use Mocks to get things going quickly and if you want to test edge cases easily and use stubs if you need to just get through the offending material.

    Reply
  • 3. closingbraces  |  August 22, 2008 at 4:01 pm

    Shyam,

    I see what you mean about the possible work involved in configuring a fake DB, as opposed to mocking it. Guess a lot depends on the nature of what’s being faked, as well as what facilities the fake provides and how well it’s been done.

    Absolutely agree about using whatever’s best for each situation. I see far too much clutching at names and silver bullets these days, rather than thinking about what precisely is needed in each specific situation. Probably guilty of that myself at times, but do try to avoid it.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


OmniscientOne

Blog Stats

  • 5,590 hits

Feeds

Pages

August 2008
M T W T F S S
« Jul   Sep »
 123
45678910
11121314151617
18192021222324
25262728293031

%d bloggers like this: