In the previous blog “Mendix apps don’t need test automation? On the contrary” we evaluated a couple of app and project characteristics in order to assess if a low-code platform is diminishing the need for test automation. This blog focuses on what test automation properties for testing Mendix apps are necessary and need specific attention.
The promise of test automation is to lighten the test burden, especially for regression testing.
In an Agile/Devops context it is more and more realized that the purpose of test automation is not to detect errors, but to ensure that automated business processes continue to work.
In order to fulfill the promise of test automation some of the desired common properties are:
You can expand this list easily with more requirements or even features. In the following paragraphs we will zoom in on a couple of desired test automation properties for testing Mendix apps that need specific attention.
Mendix makes development faster, which means that the price per functionality is decreasing. In order to keep the balance between development and test investments, test automation costs must also become cheaper per functionality.
In practice this doesn’t appear to be the case. In fact, various parties indicate that they have stopped using test automation because it costs too much money in relation to the returns.
The ratio between the costs of development and the costs of test automation appears an important criterion for the selection of a test automation solution. This does not only concern the costs of the tooling, but also the costs of automating test cases.
On the web, a lot of definitions circle around what test coverage is. Like “the number of test levels that should be supported”, “the percentage of lines of code that are part of a test” or “the percentage of software requirements that are tested”. The evaluation of the best definition is subject for another blog.
For now, we adopt the working definition; “test coverage is the desired number of tests compared to the implemented number of tests”. As we have argued in our previous blog, lowering the desired number of tests’ in order to get a high test coverage is not a very good strategy to increase the test coverage for Mendix apps.
Since the application functionality increases very fast in typical Mendix apps, so must test automation development. It should not lag behind, creating an increasing backlog of test cases. If this criterion is not met, lower test coverage or delayed application deployment will be the result.
Mendix lowers the maintenance costs of apps because the low-code models are human readable without much training. This enhances speed in understanding logic and increases the ease of business functionality changes.
Furthermore Mendix models are consistent by themselves because consistency errors are detected at design-time. This intelligent feature enables error solving early in the process and lowers maintenance of Mendix apps even more.
The same characteristics should apply for test automation; easy to understand low- or no-code test cases that are mutually consistent and provide smart support for the test developer.
In that respect, test automation has to go a step further than Mendix. Test automation has no purpose by itself. It’s test logic is 100% dependent on the developed business functionality. If the Mendix app is changing, so must the test automation. For that, test tooling should provide smart indicators which helps the test automator to keep track of Mendix model changes.
If test automation tooling does not have this type of support, it forces the test automator to do intensive code or requirement analysis or, even worse, causes necessary test adjustments to become visible during test execution. Put a tool on your wishlist if it is capable of consistency checking and smart support because lower maintenance costs are to be expected.
Many test tools specifically support one test level (UI, Component/API, Unit). Using multiple test tools for one app increases the complexity and maintenance of test automation.
Common strategies to avoid the need for different test tools for each test level are:
As for the first strategy, screen-testing has a bad reputation for maintainability. This causes a clash between the desire of high test coverage for processes on the one hand and a bad maintainability on the other. The second and third strategy requires custom test code (unit tests or API definitions) within the business application which degrades maintainability as well.
Another aspect of test automation, test data management, also requires the use of specialist tools. The creation of synthetic data, anonymizing data and data variation management could be cumbersome tasks which can be facilitated by tooling.
The down part for having a test environment with a multitude of tools is that each tool asks for installation, knowledge and skills which will place a burden on the Mendix team. A better strategy would be to combine as much as possible in one one tool, which requires no additional skill sets from the Mendix team. The more requirements are covered, the higher you should rank the tool on your wishlist.
The Agile development process in Mendix projects emphasizes the importance of team cooperation and versatile skills of team members (T-shaped figures). This enables Mendix teams to be small and swift in coping with business demands.
Meanwhile, the tendency seems to be to append test developers as separate actors to Mendix teams. This trend is amplified by the fact that citizen developers and junior team members are not T-shaped which calls for adding test specialists to the teams. The specialistic nature of test tooling however can easily hamper cooperation, and worse, can lead to losing test- and tool knowledge when the test developer is leaving the team.
The following properties of test automation software will be of the essence in order to suit the citizen- and junior developer:
Choosing the test automation tool for Mendix asks for a high-speed, low maintenance solution that has strong support for small teams.
In the next blog ‘Direct Model Testing’, a disruptive new paradigm for test automation, we’ll throw some light on how to meet these criteria without writing new testautomation software.
| info@menditect.com | |
| +3185 - 4010494 | |
| KvK: 82975795 | |
| Volg ons op LinkedIn |
Enter your information and choose a day and time when you would like to receive the demo (1,5 hrs).