Array
(
)

The underestimated benefits of unit testing a Mendix app

maandag november 8, 2021
Markus Travaille, Rolf Bolt

In the previous blog Direct Model Testing, a disruptive new paradigm for test automation we introduced a new paradigm for testing Mendix models. This blog focuses on the meaning of unit tests and the urge to start early on before technical debt will enhance costs and delay market introduction of your app functionality.

Benefits of unit testing

A zillion articles on the internet are available on the usefulness and necessity of executing unit tests. Most articles focus on benefits like:

  • you find and resolve bugs easier and earlier in the software development cycle, even before the total functionality is ready;
  • you can adapt your code with more confidence because unit tests alert side effects of your software change;
  • your solution will be better because it forces you to think about the logic on beforehand;

We will not evaluate those arguments in this blog again because they are well known. However, two benefits that are less frequently mentioned  are quite relevant in making a decision whether or not to implement unit tests as soon as possible:

  • implementing unit tests decreases the overall test effort;
  • unit tests enforce better application quality.

Let’s examine these relevant benefits.

Unit tests reduce overall test effort

This sounds counterintuitive, doesn’t it? Code contains a lot of units, so if you want to create tests for them, you need a lot of test cases and thus a lot of effort to create them. So it seems that unit testing enlarges your test efforts, right? Yes, but… that’s one half of the story. The missing part is that unit tests are related to test coverage. It appears that, for the same level of test coverage, implementing unit tests will decrease the number of tests dramatically compared  to (screen based) process testing. The latter being the most used method for testing Mendix applications.

Let’s explain with an example. Say we have a car assembly process that contains the following steps:

  • assemble engine, which is about assembling a diesel engine or a petrol engine;
  • assemble bodywork, which is about a bodywork type “middle class” and “premium class”;
  • assemble the engine in the car body.

Let’s assume that:

  • each assembly can be seen as a independent unit that is successful or unsuccessful;
  • a “middle class” car only can have a petrol engine and a “premium class” car can have all engine types.

If you are depending on process testing to test the whole assembly process you need 16 test cases to get full coverage:

engine types * bodywork types * engine-bodywork combinations = 2 * 2 * 4 = 16 testcases.

Assume that each assembly step is an independent unit which can be tested on unit level. In that case you only need to implement 8 test cases to get full coverage:

engine types + bodywork types + engine-bodywork combinations = 2 + 2 +4 = 8 test cases.

So for process testing without unit testing, the number of permutations to test is a multiplication, where unit testing of independent units the number of test cases turns out to be a sum. In real applications the number of test cases for process testing increases sharply because of the high number of parameters. This high number of permutations is one of the drivers for accepting low test coverage by business owners in order to keep costs at an acceptable level. It is for this reason that testers often use the test triangle to show optimal coverage of tests.

Controversy about “units” in Mendix

In the previous paragraph we used the term “unit” quite loosely. Before we discuss how unit testing benefits the application quality, we need to define the term “unit” more precisely. This is necessary since the term “unit” in low-code environments like Mendix does not have a well defined meaning.

One of the shortest definitions of the term “unit” we could find was; “Units are the smallest groups of code that can be maintained and executed independently. (Joost Visser 2016, O’Reilly). Added to that, the definition stated that a unit is always executed as a whole.

In ‘high code’ environments the Java methods and constructors are typically defined as units. This leads some people to think that ‘high code’ java extensions (microflow java actions) must be the smallest units in a Mendix model. This viewpoint can lead to the perspective that Mendix hardly needs any unit testing.

Other people will point to a ‘Microflow’ as the smallest executable part and thus as a unit. However, the problem with the ‘microflow is a unit’ approach is the fact that application logic is often dispersed in large microflow hierarchies where the underlying sub-microflows cannot be executed independently. So the smallest unit becomes equal to the whole microflow hierarchy or quite some large chunks of it. This viewpoint can also lead to the notion that unit testing is not a useful concept for Mendix apps.

Criteria to find and define units in Mendix

In order to find a unit in Mendix we need to look at the relevant characteristics of units:

  • they are independent;
  • they show predictable behavior;
  • they are relatively small;
  • they have a clear (single) purpose;
  • they can be tested directly by calling the unit;
  • they can be tested with a limited number of tests.

Summarized, these characteristics  lead  to the following statement: “a unit is a small piece of logic whose output can be fully predicted by a relatively limited number of manipulable input permutations”. Where input must be read as;

  • input parameters that are given to the unit;
  • retrieval of information by the unit during execution;

These input sources must be fully manipulable in order to induce predictable outputs.

The real units in Mendix

Elements in the Mendix model that comply with our statement are custom java actions, database rules and certain types of microflows.

As for the microflows, in our projects we detected three groups of microflows that comply to our statement:

Microflows that retrieve information. This group consists of:

  • ‘gets’: microflows that encapsulate logic to retrieve, filter, sort and join data from a database or by association;
  • ‘calls’: microflows that retrieve information by calling an API.

Microflows that manipulate data. These microflows are divided into two sub types:

  • ‘operations’: which create, change and delete objects in a controlled way;
  • ‘functions’: which manipulate non-object variables (string, integer, etc);

Microflows that evaluate correctness of data. These microflows are called ‘rules’.

When evaluating whether a microflow is a unit you have to assess if it meets the criterion  “limited input to predict the output fully”. If you create a large ‘operation’ which combines preconditions, logic, retrievals and functions in order to change an object, this microflow is not a unit anymore because the number of input combinations is quickly exceeding your imagination.

It appears that a unit can be a microflow, but not every microflow is a unit…

How unit tests enforce better application quality

So, how does  unit testing contribute to better application quality? It puts emphasis not only on the application logic, but also on the application structure (read software architecture). Therefore it is a driver to define a software architecture with microflow types that have specific characteristics.

Is that relevant? Yes it is! If your application is structured well, with independent components and units, it’s easy to adapt and maintain with lower cost and thus more agile. Postponing test efforts to the end of the project is often equal to postponing the (re)structuring of your application which results in a larger technical and test debt.

Is it difficult for your development team to identify units? Is the number of unit tests that can be written low? These signals indicate that your application maintainability and adaptability is low and that you face disproportionate investments to correct the situation later on. It serves as a call to action to scrutinize your app architecture rather sooner than later

Conclusion

Unit tests are important in your Mendix project because they contribute to your application quality in the short term and have a strong potential to shorten app delivery. Delaying unit testing to a later phase in your project could be an expensive decision and should not be taken lightly.

Markus Travaille, Chief Vision officer van Menditect en co-founder van EGALiT

Contact the author

Enter your contact details

Menu