You are reading a translation of an old blog post published on my previous blog in French.
We decided during the main conference that we should use JUnit 4 and Mockito because we think they are the future of TDD and mocking in Java.
— Dan North, creator of BDD
The elegant syntax of Mockito probably explains its huge success in the Java ecosystem. How Mockito manages to keep our tests readable? What the code looks like behind this API? Let’s find it out.
Mockito is published under MIT license. The code presented in this article has been simplified for obvious reasons and must not be used outside this learning context. This article is based on the last version of Mockito (1.10.14) at the publication time.
A First Example
This basic interface follows the pattern Registry. To limit the number of calls to the registry, we decide to cache results using the pattern Decorator:
We configure this mock to associate an instance of DataSource with the name "dataSource".
3
We check that the registry has been called once with this name.
When running the class, the test succeeds. The goal is now to remove all Mockito import statements and to rewrite the minimal code to make the test successful again. We will try to match as much as possible the original code of Mockito, when naming classes, when writing algorithms, etc., even if some compromises will be necessary to keep this post relatively short.
Let’s Go!
Following the removal of imports, the code no longer compiles. We start by defining the signature of missing methods:
The code still doesn’t compile due to a reference to the type OngoingStubbing. The following code will fix this problem:
You can ignore this interface for now. It will be introduced later.
mock
Creating a mock means creating a proxy, an object that looks like the actual instance but whose behavior is preprogrammed. Since Java 1.3, we can use Dynamic Proxies to create dynamically a new class but only from an interface. This restriction is often problematic in practice, and most projects like Spring, Hibernate, or Mockito use a different library that manipulates the bytecode. The most famous library is cglib. This library works by extending the class, a more flexible approach (the class must just not be declared final).
Mockito abstracts cglib behind two interfaces: MockMaker and MockHandler.
An implementation of MockMaker is responsible for instantiating a proxy so that each call is delegated to the instance of the class MockHandler:
The class Invocation is a simple type grouping various properties related to a single method call:
cglib/ASM/Objenesis
We are approaching the low-level details of how Mockito is working. The code is relatively easy to understand due to the cglib API that relies internally on the API of another low-level library, ASM. Here is the implementation of MockMaker:
Don’t panic if you do not understand everything. The code is not as complex as it may seem. Let’s go through the code step by step.
We start by creating an instance of Enhancer, the main class in cglib, responsible for creating new classes dynamically.
We then describe what our mock must look like:
The most important line is the second one, where we specify the class of our mock. To understand the first line, you need to know that all classes generated by cglib implement by default the interface Factory. This interface allows, for example, to switch the callback. The method setUseFactory allows to disable this feature, but we are simply setting the default value and is therefore useless. The last line defines the kind of callback we are going to use. Several ones are available such as FixedValue to return the same value for every method call. The most flexible callback is MethodInterceptor that gives us full control of all call metadata to determine the result.
We create the Class object that will be used to instantiate our mock.
We call the method newInstance to create a new instance from the object Class we got just before:
This method requires a default constructor. This restriction may cause problems in some cases.
Imagine that the class to mock inherits another class:
Depending on what the parent constructor does, the result may be problematic.
Can we instantiate an object in Java without using a constructor?
Yes, by using the library Objenesis. Again, this library works by manipulating the bytecode that may differ based on the version of the JVM, the vendor, … (See the class StdInstantiatorStrategy if you are curious).
With this new knowledge, we can go back to the class MockHandler:
Objenesis creates a new instance of our dynamic proxy. Our mock is born. We associate it to an instance of MethodInterceptorFilter to connect cglib with our MockHandler.
Before closing this first section, you may have noticed that Mockito repackages cglib (and ASM):
Why repackage cglib?
The maintenance of Cglib is not perfect. A few unstable versions have been pushed to Central Maven. It forces projects like Mockito or Spring to repackage it in their namespace to guarantee a stable version.
It also raises another question.
Is cglib still the best solution?
The trend is clearly no. Major frameworks like Hibernate or Spring planned the migration to a different library like javassist.
To know more about CGlib, I recommend this excellent article to fix the missing official documentation.
when
Even if we are done with low-level bytecode manipulation, the following sections are not necessarily more straightforward. The Mockito API is simple to use but not necessarily to write.
A first glimpse…
When executing this line of code:
The method anyString is called first. We memorize the use of an ArgumentMatcher (in a kind of global variable).
The method registry#lookup(String) is then called (in fact, we call the interceptor of our mock). We memorize the invocation still using the global variable.
The method when is called. We know at this moment that we are configuring our mock.
The method thenReturn is called last. We exploit everything we memorized before, we save the expected result to return it when the mock will be really called during the test.
Let’s start with the global variable. This variable is an instance of the class MockingProgress:
This class will be modified in the last part. The code appears complex but the logic is simple: every time we know more about our position in the code, we communicate this position to this class, so we will be able to retrieve everything later.
Let’s talk about something more simple: the instances of ArgumentMatcher, based on the excellent library Hamcrest:
Mockito offers many matchers. For our example, only two matchers are necessary:
Their use requires that we call a factory method instead. This factory serves two purposes: communicates the matcher has been used and returns the right type for the code to compile (Note: instantiating a matcher directly instead of using anyString() would not compile):
Before moving to the last section, we need to introduce the class InvocationMatcher that we will encounter many times. This class associates an instance of Invocation (a method call) with the list of used matchers. Even if we receive the arguments in the object Invocation, the matchers are not present as attested by the previous code (anyString returns for example an empty string). Here is the code of this class:
1
When calling a method on a mock (when using when or verify), Mockito requires only literal values/variables, or only matchers. Under the hood, Mockito makes sure to only work with matchers. This is the role of the method argumentsToMatchers.
We still have a few classes to introduce like the class InvocationContainer. Unlike MockingProgress, shared between all mocks and all tests, every mock gets its own instance of InvocationContainer. This class keeps track of all stubbed invocations, which are all invocations using when to program the mock but also all invocations during the test execution that will be very useful to check assertions (verify).
1
stubbed contains all method calls recorded using when.
2
invocationForStubbing contains the current method call. We ignore for now the context (when, normal call, or verify).
3
registeredInvocations contains all the effective method calls (calls happening outside a call to when or verify).
4
This method is called every time we call a method on the mock. It can happen when exercising the mock or at the end of a test when calling verify. In the first case, the method addAnwser will be called just after (by the method thenReturn for example) to allow us to record this invocation in the list of stubbed invocations. For the second case, it is our only chance to take note of the call as there would not later method call like thenReturn to confirm the invocation context. Therefore, we add the invocation to the list of effective invocations, and we will remove it if a method like addAnswer is called just after.
The answers are represented by the interface Answer. They may describe a return value (thenReturn), an exception to propagate (thenThrow), etc.
The recording of all answers is done by one of the first abstractions we introduced in this article—the class OngoingStubbing that is returned by the method when. Here is the new definition of this class:
This second section is almost complete. We just need to assemble the different classes in MockHandler, the interceptor called on every method execution on our mock:
1
Each MockHandler is associated with an instance of a mock. It’s the best place to create the instance of InvocationContainer.
2
We dequeue all matchers to create an instance of InvocationMatcher.
3
We record a possible stubbing (will be confirmed later, or not).
4
If an answer has already been programmed, we simply return it.
We must not forget to update our initial implementation of the method when:
verify
Compared to the previous section, this last one will be far less challenging.
A first glimpse…
When executing this line of code:
The method times is called. This factory simply creates an instance of VerificationMode.
The method verify is called. We memorize the expected result passed in parameter (times(1)) in our global object MockingProgress.
The method anyString() is called. As usual, we memorize the matchers for later.
The method registry#lookup(String) is called again. We end up in the MockHandler implementation where the verification is really done. We check the matchers and look for a matching invocation.
Let’s start by introducing the interface VerificationMode:
As for matchers, several implementations are available. Only times is used by our example:
The verification is trivial by using the content of VerificationData containing all effective invocations, plus the invocations triggered by the verify functions. We just have to find the matching invocations and compare them with the expected number of invocations.
We need to revise our implementation of the method verify:
And MockingProgress too:
The last change occurs in MockHandler where the main logic resides:
Congratulations!
Congratulations, you have just written a minimal but operational version of Mockito in less than 500 lines. The complete source code is available here.
It’s Not Over!
Bonus: Multithreading
Most classes are used by a single mock instance. For example, each mock has its own instance of MockHandler. The only class to synchronize is the class MockingProgress, used as the global placeholder to support the flexibility of the Mockito API. Using the Java class ThreadLocal, making this class thread-safe is trivial:
Each method retrieves systematically the instance associated with the current thread using the method get defined on ThreadLocal. We just have to replace all references to this class:
By:
That’s done!
Bonus: Error Management
Reporting errors use without surprise Java exceptions, but these exceptions are not thrown directly when an error is detected. Mockito delegates this responsibility to the class Reporter, which centralizes all possible errors. A specific method exists for every kind of error. Here is an extract of this class:
The main advantage of this class is to centralize all error messages in the same place to make sure their formatting is consistent. Here is an example of use:
To remember
An API can be simple to use but not so simple to implement.
It is possible to instantiate a class in Java without calling the constructor using libraries like Objenesis.
To create a proxy for a concrete class in Java, we have to use bytecode manipulation using libraries like Cglib or Javassist.
Cglib is still ubiquitous in many popular frameworks, but most are migrating to Javassist.
Using ThreadLocal gives access to a global context for every thread of an application.
Try for yourself!
The rewrite of Mockito is missing many great features. Here are a few examples of some features omitted that you may find interesting to inspect:
Mockito supports the verification inOrder to make sure two mocks have been called in a precise order. We have seen that InvocationContainer is associated with a single mock. How Mockito manages to support this use case? +
Hint: The class InvocationImpl contains an attribute sequenceNumber.
Mockito supports the verification verifyZeroInteractions, which as its name suggests, makes sure no interaction besides the ones defined occurred. How does it work? +
Hint: The class InvocationImpl contains an attribute verified.
Mockito supports, for a single call to the method when, to chain several calls to the methods thenReturn, thenThrow, etc., that will match the result of the first execution, then the second, and so on. +
Hint: Compare OngoingStubbingImpl with ConsecutiveStubbing.
About the author
Julien Sobczak works as a software developer for Scaleway, a French cloud provider. He is a passionate reader who likes to see the world differently to measure the extent of his ignorance. His main areas of interest are productivity (doing less and better), human potential, and everything that contributes in being a better person (including a better dad and a better developer).