Quality control is key when you want to maintain high speed Mendix app development. Low code projects have a tendency to slow down considerably after an initial blazing start. Especially when the size and complexity of the project increases, speed is going down. Proper testing of your Mendix app will help you maintain your speed.
Key factors for good testing are ‘start early’ and ‘test the smallest unit possible’. This approach is known as ‘shift left testing’ but we have transformed it into ‘single microflow testing’. Why? Because a microflow is the smallest unit in Mendix and should be tested early in the development cycle. In our previous blog “the benefits of unit testing” we explained that not every microflow is behaving as a simple predictable unit by nature. We also pointed out how to migrate to smaller independent sub-microflows that are easier to test.
So, if single microflow (unit) testing increases development speed and decreases overall cost, why do Mendix teams not widely adopt this? This article will try to shed some light.
Through countless customer interactions we have discovered three important blockages that prevent shift left testing in Mendix teams:
Let’s zoom in to these points in the next paragraphs.
First off, this feeling is colored by the fact that the costs are taken in advance and the benefits only become visible afterwards or not even at all. Therefore, investing in single microflow testing requires a leap of faith.
As a result, we wanted to quantify the investment, so we surveyed our customers and conducted a poll among Mendix developers. Based on the results, we estimate that building single microflow tests quickly takes up 20% of the development time. If a high test coverage is desired, this can be up to >30% of the total development time.
We zoomed in on the reasons why single microflow testing takes a lot of time. The main reason turns out to be that you need to manage a lot of test data and define proper test results. To understand this in detail let’s focus on four aspects:
If a microflow contains only primitive microflow parameters, the test data is usually quite manageable. However, if a microflow has one or more object parameters, the number of attribute values to be defined increases rapidly. Furthermore, an object parameter often needs to be associated with one or more related objects that also need to get the correct test data. One can imagine that filling in 100+ attribute values for a single microflow test is not uncommon. This is clearly a time consuming task.
A microflow can fetch existing data from the database or get it by calling an API (see image below). This data retrieval can also take place in one of the sub-microflows. All of these queries act as “implicit input parameters” for the behavior of the microflow under test.
It is generally understood that the most reliable test results are obtained, when the test is independent of existing data in the database. That requires creating all the “implicit input parameters” beforehand so that they can be retrieved in the microflow under test. Therefore, building reliable tests can be quite a bit of work.
Previous aspects address the amount of data involved in the execution of one test run for a single microflow. Although in real life, you want to execute the same test multiple times, with different sets of data, in order to improve test coverage. We use the term ‘data variation’ to describe this. For simple microflows this leads to a manageable number of permutations and 100% coverage is quite attainable. For more complex microflows, the number of permutations can increase dramatically (see our blog on “the benefits of unit testing”). The data variation of more complex single microflow needs optimization to get the right coverage, which takes time to design and to build.
When you have set up all the test data and defined the logic of your test script, then one final step has to be made: define asserts. An assert is a statement that tests an expected outcome of an executed test. If only the output parameter of a microflow has to be asserted, this is an easy task.
However, many microflows (and its submicroflows) are creating, changing or deleting objects while being executed (see image above). These “implicit output parameters” could need asserts as well. Figuring out which changes have occurred is time consuming and requires a thorough analysis.
In the past year we discovered that the background of a Mendix tester is considerably interfering with the perception of test automation. Roughly we see the two groups of testers struggling with test automation in Mendix; the citizen developers and the traditional testers.
Unit testing in high code projects has traditionally been the domain of developers with a formal software engineering background. For them, the concept of unit testing is often fully understood and the importance of this test type is appreciated.
Single microflow testing in Mendix is at the level of unit testing. It makes sense to consider test automation of some microflows belonging to the domain of the Mendix developer as well. But Mendix introduced the term ‘citizen developer’. People with no formal software engineering background who create business systems. Citizen developers often do not (yet) understand why single microflow testing is indispensable for guaranteeing app quality and do not focus on that by nature.
In addition, designing a test requires a different view on the application logic. From a ‘happy path’ way of thinking to an extensive focus on alternative paths. This can feel unnatural, and the fact that this is not perceived as the most creative work does not help either.
If development is done by ‘citizen developers’, is it logical to assume that single microflow testing can be done by ‘citizen testers’? We think it is. But it requires a paradigm shift for most testers.
Most testers are used to ‘black box’ testing, meaning they don’t need knowledge of ‘internal application logic’. However, for single microflow testing, this knowledge is indispensable and asks testers to adopt a more ‘white box’ approach. This requires testers to have a basic understanding of Mendix modeling and a good connection with the development team. We have seen examples of testers that have learned to build unit tests that support our view.
In order to determine if the investment in test automation is on track, it is necessary to have a way to measure its progress. Traditionally, test automation progress is measured in terms of test coverage. At this moment, no tools are available for the product owner to measure the coverage of single microflow tests.
This means that product owners that are willing to invest in long term quality have no short term means to measure test coverage. Single microflow testing can thus become an unmeasurable goal, which costs time, but progress cannot be measured. This requires quite a leap of faith of the product owner and does not make it easy to enforce the right test strategy to the Mendix team.
At Menditect we believe that these obstacles for single microflow testing should be solved. The importance of good quality at acceptable costs becomes more important as the adoption of the Mendix platform rises and Mendix apps become more business critical and complex.
Menditect sees the lack of single microflow testing as one of the sources for a continuous slowdown in release speed of Mendix apps. We want to solve this bottleneck because most Mendix apps become larger and more complex over time. To make this concrete Menditect strives to help its customers by:
We are going to publish a series of blogs on how to achieve these goals. Next week we will blog about “Single microflow test generation”. In it we share MTA’s new feature that reduces the configuration time of single microflow testing.
| info@menditect.com | |
| +3185 - 4010494 | |
| KvK: 82975795 | |
| Volg ons op LinkedIn |
Enter your information and choose a day and time when you would like to receive the demo (1,5 hrs).