Let’s Smell Some Tests #3 — Testing Randomness in Java | by Krystian Szpiczakowski | May, 2022

Why you should avoid testing non-deterministic behavior

Photo by Lucas George Wendt on Unsplash

Hello, and welcome to the fourth article of the “Let’s Smell Some Tests” series. In the previous episode¹, we were looking at tests that exercised internal behavior, and we pointed out the drawbacks of this approach.

Today, we’re going to analyze another interesting topic: why testing untestable is a bad practice and why it leads to overcomplicated test cases.

Here’s what I’m using in today’s examples:

  • Java 17
  • JUnit 5.8.2
  • AssertJ 3.22.0
  • Mockito 4.5.1


Requirements for this use case are as follows:

  • When the game starts, spawn a new player on the map
  • The map is a matrix of tiles; the size of the map can be configurable
  • By default, place the player on a random tile

The first approach to implementation

A sample implementation of these requirements can look like the snippet below:

The test has passed, although it should be reworked

A short explanation:

  • The Board class consists of a matrix of objects, each of which is of the Tile type
  • The Tile class is represented by x and y properties, plus it holds a set of Localizable objects, like players; in the future the Localizable interface may also represent other objects like enemies, consumable items, and everything that can be bound to a tile on the map
  • Because we want to put a new player somewhere on the map, the Player class implements the Localizable interface, which comes with thegetCurrentLocation() function.
  • In order to instantiate a new Playerwe have to call a factory method create(Board board)
  • To meet the requirements, the Player class contains logic to select a random tile, where the player is supposed to start the game

Code can either be written so that it just works or will work and retain high quality. The current implementation represents the first option, unfortunately.

The code above compiles, the test passes, and theoretically, we could just move on and implement other functionalities, but let’s have a look at what could be done better:

  • The test case SpawnPlayerTest#testSpawnPlayerOnTheMap verifies too much by checking the board size (the name of the test suggests that it only focuses on spawning the player, which is not true)
  • The assertion logic is too complex (because the tile the player is assigned to is random, we are collecting all the tiles from the board, and we are checking whether the player has been assigned to one of them)
  • The logic for selecting a tile for the new player is hard-coded within the Player class (violation of Single Responsibility and Open-Closed principles)

Working with randomness can be tricky, and this is the case in our code. According to this example, we have a board whose dimensions are 5×4 tiles. This gives us twenty possible tiles that the tile for a spawning player can be selected from.

Now, there’s an important thing to say: Even if we know that there are twenty possible values ​​to assert, there is virtually no way to be sure which particular tile is going to be selected in the next test execution. Randomness is a thing that is a bit out of our control, and, in my opinion, it would be better if we accept this fact, instead of fighting against it in the test.

A solution that could better suit the current requirement can be described with a few points:

  • Extract logic selecting a tile for the player to a separate unit
  • Create an abstraction for this logic so that a different implementation could be used if requirements change
  • To spawn the player, pass the tile selector to the Player class through its constructor
  • Simplify the test SpawnPlayerTest#testSpawnPlayerOnTheMap

Step 1: Extract selecting a tile for the player

If requirements change in the future, we will need to write another class that will also implement the LocationSeed interface.

Step 2: Tell the player object where to get an initial tile

The constructor utilizes a provided LocationSeed object by calling the locationSeed.getTile() method. In the future, we can simply provide a different implementation of the LocationSeed interface when needed so that we are closed for modifications but open for extensions.

Step 3: Rework the test

I extracted the logic for board creation to a separate test method in a new test class. Now, this test verifies exactly what its name suggests.

From now on, the old test is simplified. It’s focused on spawning the player.

I used the mock() method from Mockito to create a fake implementation of the RandomLocationSeed class. This may seem a bit counterintuitive at the beginning because we wanted our test to verify if the spawning of the player works as expected.

Remember that we are supposed to test observable behavior, which is “spawn the player in a random place on the map.” However, we want to set up this test with dependencies we can control because randomness is an external factor we have no control over.

Then, I set up the mock by calling when() and then() functions also provided by Mockito. These functions are used to configure what should be done when the method is called. I simply queried the Board object to return one of its tiles, and whenever the getTile() function is called, then it returns a sample tile.

The last thing with regard to mocking LocationSeed is calling the verify() method, which checks whether the mock has been called at all.

At this point, I would suggest doing a last, small amendment to the test above. In fact, we’re dealing with the test with a stub, not with a mock. The main difference between them is that mocks don’t return a value (they act as commands), whereas stubs do return values ​​(they act as queries).

Calling the verify() method on a stub is a bad practice because it verifies internal behavior. Observable behavior in our case is to “spawn a player,” so querying for the tile is just an intermediate step of the business case. Hence, let’s delete invoking the verify() method, so that the final version of the test looks like this:

The final version of the test without checking whether the mock has been called
The final version of the test has passed

As you can see, programming games can be as much fun as playing games :).

Let’s summarize the lessons taken from this example:

  • Do not overcomplicate your tests
  • If your test does too much, ie, contains multiple assert statements and checks multiple objects, consider splitting this test into smaller ones
  • Not all functionalities are suitable for direct testing (like features depending on randomness)
  • Use mocks/stubs in places where things are out of control (testing randomness but also crossing application boundaries, like calls to a file system, web services/message buses, and so on)

[1]: Krystian Szpiczakowski, Let’s Smell Some Tests #2 https://betterprogramming.pub/lets-smell-some-tests-2-asserting-the-internal-behavior-in-java-1c0f34fe8bbc

Leave a Comment