In the previous blog “Desired test automation properties for testing Mendix apps” we addressed criteria that should be met in order to make your test automation investments returning profits. This blog introduces Menditect’s totally different approach to test automation that incorporates these criteria.
When you think of it, test automation by itself is actually a strange phenomenon. First, you create business software that contains a lot of functionality, logic and rules in order to meet customer demands. Secondly, you create another application that is checking if the first application is working properly. To ridicule it, why not make a third application that is testing if the second application is doing its job right…and a fourth application…
The main problem of the test automation paradigm is that test software contains logic by itself which needs maintenance and scrutiny for correctness. And where the business application is dependent for its logic on business requirements, so is the test automation software solely dependent for its logic on the logic of the business application. In essence, you are coding the business logic twice, first in your business application and secondly in your test application.
Most test automation implementations are adopting two approaches that have negative side effects. Let’s evaluate some of them.
Black box testing
By definition, black box testing is banging on the outside of the application in order to find out if the inner workings of the application are correct. Test automation based on page-testing or API testing are examples of this approach. The disadvantages of this approach are:
Unit testing
Unit tests are usually implemented by adding extra code to your business application. It gives the opportunity to test the inner workings of the application logic by running the logic. Some disadvantages of this way of testing are:
The ideal test automation does not contain logic by itself but derives its logic completely from the business application logic. That is, the test automation is ‘knowing’ the inner workings without ‘programming’ this logic. On top of that, it would be great if the test can execute this logic directly. Just run the business application logic in the right order, provide the necessary test data and evaluate the correctness of the outcome.
Of course, when a change of business logic invokes a different data need, the tester must be alerted ‘design time’ to offer a different data source. This way of test automation and execution leads to seamless coupling between business logic on the one hand and the design and operation of tests on the other.
In addition to that, ideally it must be possible to create a ‘building’ of small executable tests into comprehensive all logic encompassing tests in a well structured and maintainable test framework such as the ISTQB framework. In that way, you start by defining unit tests for calling ‘low-level’ business logic and composing them into encompassing module- process- and API tests. Of course, this ‘building of tests’ must be checked ‘design time’ as well for consistency in order to keep maintenance issues to the minimum;
Last but not least, the ideal test solution must be capable of defining tests that support the testing of a business process that touches more than one business app during a test run. This prevents the necessity of a multitude of test automation environments.
This new paradigm will enhance test capabilities without writing test code and without invoking a maintenance burden. We call this way of testing ‘Direct Model Testing’.
First of all, it should be noted that Mendix models are key in the paradigm. These models contain all the business logic and business rules in the form of microflows and entities and are perfect touchpoints for the test application.
The Menditect Test Automation solution (MTA) utilizes the Mendix model by retrieving model components (microflows and entities) and exposing them to the test automator as testable components. The test automator creates teststeps from these components by selecting a model component and defining the desired data input. The picture below shows the selection page for finding a microflow to be turned into a teststep.
After configuring, the teststep can be executed directly on the backend server. When an assert is configured for the teststep, this will be evaluated on the fly. In other words, you can execute microflows by providing data to the microflow parameters, run them, collect the results and evaluate if the outcome is ok. In the same way you can select entities in order to inject, delete, retrieve and evaluate data objects directly. This way, all existing business logic can be re-used for test automation purposes for which no extra logic needs to be programmed.
Consistency checks
On top of easy configuration, MTA has unique consistency checking capabilities, like the Mendix Studio model consistency checker. You receive an alert in the test modeler immediately during ‘design time’ whenever your designed test is not conforming to the data types or logic in the business apps that are connected to your test definition.
Re-use of tests
Moreover, you can use the microflow result or the manipulated object in the next part of your test script. This enhances the possibility to re-use test components by chaining and organizing them into hierarchically structured module- and process tests.
Normally, re-use of tests by ‘chaining’ them into larger scripts is undesired because the implicit dependencies result in bad maintainability. But MTA supports consistency checking of the coherence of scripts as well. You get an alert when a previously designed test case is not delivering the correct information to a dependent test case (see picture below). This way the dependencies become explicit and are easy to maintain.
In order to speed up test automation even further, MTA supports data driven testing. For each model interaction, (microflow or object manipulation), a dataset can be configured that is used during test execution. By that, each test can be executed numerous times with different data without the need to copy teststeps.
The focus on utilizing the business logic directly in test configuration often leads to a couple of advantages.
Lower maintenance costs
First of all, you will experience a big drop in maintenance efforts and costs. The freed time and money can be spent on enhancing the application test coverage making the overall return of investment a better one.
Better collaboration between testers and developers
Secondly, developers and testers are in close contact with each other about testing the app. Not to say that it is now possible to enhance the T-shaped developer with easy to maintain test capabilities or the other way around, engaging citizen testers for designing and realizing test automation.
Third, experience learned that the re-use of models during ‘Direct Model Testing’ is a natural way of having code reviewing embedded in your development cycle. By this, the test actions will deliver a quality improvement of your app even before a test is executed.
Finally, executing tests directly on the model instead by using a page interface leads to fast test execution.
Changing the test automation paradigm to a ‘Direct Model Testing’ delivers exciting new ways of testing Mendix apps. We will continue to blog about utilizing ‘Direct Model Testing’ with subjects like ‘security handling of MTA’, ‘how to build process and component tests on top of unit tests’, ‘small steps to speed up the ROI of your test investment’ and so on. Please leave a comment on LinkedIn if you like us to write about some specific aspect of test automation with MTA.
| info@menditect.com | |
| +3185 - 4010494 | |
| KvK: 82975795 | |
| Volg ons op LinkedIn |
Enter your information and choose a day and time when you would like to receive the demo (1,5 hrs).