Array
(
)

Frequently asked questions about MTA

Read the frequently asked questions about MTA to know more about our product and services. For more information about Menditect you can also visit our documentation site

The impact of an application change on automated tests is often difficult to determine. How does MTA support that?

MTA helps the tester resolve inconsistencies in the test script by detecting where changes to the Mendix application affect the outcome. New test cases do need to be added by the test automaker. In MTA there is the possibility to create ‘manual’ test cases so that test cases can be added before the code is created (test driven development) or as a specification by a business tester for the development team that automates the tests.

Can I test multiple applications simultaneously within one test script so that interaction between systems can be tested?

Yes you can. Test scripts are called test suites in MTA. A test suite consists of multiple test cases that are called sequentially. For each test case, you specify which application will be accessed by MTA. Data can be passed from test cases to each other. This way you can pass data from one application to the next and test multiple applications in conjunction.

Does it cost a developer a lot of extra time to work with MTA?

In a healthy development environment, developers always test their work themselves. Initially building the test suites in MTA obviously takes some time. However, after that the maintenance of the configured tests is minimal. Each application revision can then be tested automatically with the push of a button.

What added value does MTA provide for small teams?

In small teams, there is often no room for a separate tester. This means that tests must be performed by the Mendix developer. Test automation and test execution are then quickly at the expense of new business functionality. MTA connects to the knowledge and thinking of the Mendix developer because the microflows and entities can be tested directly with MTA. For this no letter code needs to be written which keeps the test effort of the Mendix developer low.

Automatic testing does not work for our project as it is too maintenance sensitive, is it different with MTA?

MTA is adaptive and detects changes in the Mendix model by itself. In addition, MTA checks if interdependent test steps are still consistent. In most cases, after a model adjustment, the test suite is ready to be executed again within a few mouse clicks.

Can I have a Mendix application running on my local environment tested by MTA?

You can, but we don’t offer it by default. You’ll need a third party application (for example ngrok) to set up a tunnel, so that the local Mendix environment is accessible for MTA. See also our documentation

Can you reuse existing tests?

MTA supports reuse of tests in the following ways:

  • by building tests hierarchically allowing the smallest unit (unit test) to be used to build component and process tests with it;
  • by applying data variation which allows an existing test case to be reused for different situations without having to modify it;
  • by copying test cases and even entire test suites after which they can be easily adapted to an alternative but different scenario.

Reuse of test scripts is often discouraged in other environments due to maintenance issues. However, with MTA’s consistency checks, reuse of test cases is no longer an issue.

How can I check if the result of a test is correct with a predefined value?

You can do that by using asserts. An assert verifies that the result of a test step matches the configured expected result. Because a test step executes a microflow or reads or manipulates database data, an assert actually verifies that a microflow or database change was executed correctly.

Does MTA replace screen testing, or is screen testing still necessary?

With MTA, it is possible to directly test the microflow logic behind the screens without the need for calling these screens itself. In that way, MTA supports unit tests, component (API) and process tests and is able to test the user flow. Manual testing that also validates the user interface can be an essential step in monitoring the quality of an application.

Do all master data and other production data needed for testing need to be configured in MTA, or can existing data in the database be used?

Both are possible. In complex applications, there is often a hybrid situation. Some master data changes so little that it is not part of the test. These data can be selected from the database as ‘existing master data’. Master data that changes more frequently can be created on-the-fly in a test script and then reused in other test scripts. For repeatability and predictability of the test script, it is recommended to configure as much master data as possible in MTA.

Can I test an application with different roles?

Yes you can. MTA executes test cases with the users created in the testing application. In the test configuration, for each linked application you must specify which users (and therefore roles) are available to execute the test. Per test case you specify which user you want to execute the test case with. This allows the functionality to be tested with different roles.

Can I use Menditect Test Automation to create, modify or delete database entries?

Yes, that is possible. There are two types of test steps in MTA which you can use to manipulate database entries:

  1. A Microflow test step: With this test step you can call a microflow in the application under test. If a microflow in the application under test is available which manipulates a database entry, you can use this option.
  2. An Object test step: With an Object test step, it is possible to retrieve, create, change, delete objects directly without the need to create a microflow first.

How does Menditect Test Automation distinguish itself from existing screen tests and API tests?

MTA has the following unique features:

  • Direct Model Testing: MTA uses your existing Mendix model to define and execute test scripts directly without writing test code or the need to load pages. This saves execution time and speeds up your deployment process;
  • Consistency checking: The automatic consistency check verifies the consistency between test scripts themsleves and between the Mendix model. This speeds up test automation and reduces false negatives;
  • Test pyramid support: The environment supports unit-, component-, process and API tests which reduces the number of necessary testtools;
  • Multiversion microservice testing: Multiple Mendix apps can be tested in one test run! Multiple app versions can be added in different test configurations to suit all application version combinations in your production environment;
  • Data variation: Test scripts can be executed with different datasets in order to test the boundaries of your app with a data driven test approach;
  • Data injection: Labor-intensive setup and cleanup of master data can be automated so that the test conditions are optimal and correct every time the test is run.

What Mendix versions are supported?

See the MTA releasenotes for supported Mendix versions.

Can Menditect test non-Mendix systems as well?

MTA is specifically designed to test Mendix applications. However, MTA has two features that are useful in testing Mendix applications in combination with non-Mendix systems:

  1. Test design: In MTA it is possible to write and structure test designs for non-Mendix systems as well;
  2. API interaction: The API interaction of a Mendix application with non-Mendix applications can be modeled with MTA by calling microflows that handle these interactions.

When should I consider test automation?

Having doubts about whether test automation is a good idea?

Read our blog!

What technical level is required to work with MTA?

To work with MTA, some Mendix knowledge is required to understand the created business logic in the Mendix model.

In general, the Mendix Rapid Developer level is sufficient.

How will Menditect handle my personal data?

We value your privacy and observe due care in processing and protecting Personal Data. Please read our privacy statement carefully for more information.

 

What SLA (Service Level Agreement) does Menditect provide?

Can I partner with Menditect?

Yes, Menditect has several partners (Mendix partners, test partners, etc…). Please contact our sales rep on sales@menditect.com for more information on our partnership options.

Can I also test mobile Mendix Apps?

If a Mobile App is communicating with a Mendix Backend, MTA is suitable to be used for testing.

Because MTA is able to run tests directly on the Mendix model it is possible to test the interaction of the mobile app with the backend.

However, MTA is not testing the navigation- and page logic on the mobile device itself.

Can I test APIs?

MTA provides the following ways to test inbound and outbound APIs:

  • By directly executing the available microflow in the Mendix model that calls an API. Because MTA is capable of testing multiple applications with one test script, applications can trigger each other;
  • By creating a HTTP request or response object in MTA which contains a test JSON / XML message. This object can be used as an input parameter for a microflow that calls the outbound API or to mimic the arrival of a message;
  • By creating a test JSON / XML message in MTA and serve this to a microflow string input parameter after which the microflow calls the outbound API or processes the input as inbound message.

How secure is my data in MTA?

Menditect delivers a dedicated MTA environment to each customer which is hosted on a Mendix cloud node and will never be shared. This means that:

  • MTA data complies to the same security levels and measures as the Mendix cloud node;
  • Customer data will never be available to another customer or third party.

Furthermore, all data exchange between MTA and the application under test is HTTPS encrypted.

Menditect offers also a version that you can host yourself (in the Mendix cloud or on premise) provided that your Mendix license covers for this. In such a configuration Menditect cannot access any data stored in Menditect

Can I test conditional visibility?

No, conditional visibility can only be manipulated by setting attribute value or expression in the page definition. MTA does not test page definitions and has no options to test if conditional visibility is ok.

 

Can I test widgets?

No. Widgets and custom javascript are page elements and MTA is not testing page logic. However, widgets are often interacting with microflows which can be tested by MTA.

Can I test nanoflows?

No, nanoflows are executed in the browser. And because MTA is testing directly on the backend nanoflows cannot be tested.

Can Menditect also test whether certain products or deliverables are created (such as PDF files)?

Yes, provided they are stored in the database. If not, a custom check microflow can be built and called that checks if the file was created.

How long does it take to execute a test script?

MTA does not require screens to run tests. This means that loading screens does not unnecessarily slow down testing. Because MTA runs tests directly on the server, the speed of execution is higher than via screen tests. Ultimately, the speed of the cloud node that the client application is running on determines the speed of test execution.

What do I need to change in my Mendix model to be able to test with MTA?

Only the MTA plugin module needs to be imported and provided with a local user per environment, that’s it.

In some cases you might need to change the output parameter of a microflow.  See our documentation for more information

How does MTA differ from the Unit test module offered by Mendix?

  • In MTA, there is no need to make model modifications. Thus, the business logic remains separate from the test logic;
  • MTA is also readable for non-technical team members, such as information analysts, testers, product owners and end users;
  • MTA continuously checks the configured test scripts for consistency, both at the model level, and consistency between interdependent test scripts;
  • MTA provides data variation (data driven testing), which allows test suites to be executed with different input values each time;
  • MTA offers one central solution for insight in quality of the tested applications, because the result of recently executed test scripts are stored.

How can Menditect assist me with configuring my testscripts?

Menditect can help in several ways, from setting up MTA and writing new testscripts to automating and configuring existing manual testscripts. Many questions about the possibilities in MTA are answered in this FAQ section. More complicated configuration options are covered in the How To section.

But we also offer:

  • Deep dives at our office in Amersfoort, where testers or Mendix developers who want to use MTA are in control, and Menditect employees are available for support;
  • Hour based online expert services, using video conferencing;
  • Full day expert services on site.

Can I purchase Menditect Test Automation through my existing Mendix partner?

Yes, that is an option. You can contact your existing Mendix partner for this, or contact Menditect Sales via sales@menditect.com.

How can I get to know Menditect better?

It is possible to request a free demo of MTA. To do so, please fill out the contact form on the website.
Menditect also offers to go through an on-site pilot day with interested Mendix clients or partners. The client application will be connected to MTA and some unit tests and some process tests will be built, to show the potential of MTA.
Finally, Menditect hosts deep dives at its office in Amersfoort, where the client itself is in control in MTA, with Menditect being able to provide direct support.

Who has intellectual ownership of the test configurations and test data?

The intellectual property of test configurations and test data remains with Menditect’s client. The cloud node on which MTA runs is owned and maintained by Menditect. The ownership of the MTA code and models run on top of the cloud node rest entirely by Menditect. Click here for our end user license agreement.

Can I use Menditect outside of the Netherlands?

Yes, but Menditect Support currently operates only during Dutch working hours.

I am a large organization with multiple Mendix Apps. Is MTA suitable for me?

MTA’s ‘Direct Model Testing’ is designed for both small and large organizations that work with Mendix.

Menditect’s pricing and deployment model both fit small organizations with a single app and large organizations with multiple apps. The price per app decreases as the number of apps increases which provides a discount per app for larger organizations with multiple Mendix apps being tested with Menditect

Please contact our sales rep on sales@menditect.com for detailed information, questions or a custom offer.

I am a small organization with a complex Mendix App. Is Menditect suitable for me?

MTA’s ‘Direct Model Testing’ is designed for both small and large organizations that work with Mendix.

Menditect’s pricing and deployment model both fit small organizations with a single app and large organizations with multiple apps.

Please contact our sales rep on sales@menditect.com for detailed information, questions or a custom offer.

Can I export test designs and test configurations?

It is possible to export the test suite designs to Microsoft Word in order to support review processes.

Can I use existing or anonymized production data to test with?

Yes, you can. MTA has so-called ‘crud actions’ that allow data to be directly created, modified, deleted or retrieved from the database. In addition, you can create specialized microflows that allow for further anonymization of data, for example, randomizing zip codes or replacing a city name with a random name from a list of city names.

Can I get MTA to perform tests on a Mendix App that is running on-premise?

That depends on the situation. MTA can be used out of the box if the on premise app can be accessed from the Internet.

If the on premise app is not internet accessable, a custom solution is possible. In that case, contact sales@menditect.com.

How does MTA ensure that the person responsible is alerted when errors were detected during a test run?

This information is initially shown on the dashboard. Through the CI/CD integration it is possible to connect a test management system with which the test result can be retrieved and provided with an alert.

Can I use MTA to build or reset master data that my test scripts depend on?

Yes, you can, and is even recommended in order to be able to execute test scripts repeatably. MTA offers the possibility to perform on-the-fly so-called ‘crud actions’ on the database without the need to implement microflows for it. Of course, data can also be manipulated by having special microflows executed by MTA. Because of this flexibility, data can be created, modified, read and deleted directly at any point in the test script. This makes it possible to prepare data for the execution of a test script, but also to clean up or manipulate intermediate results in a script as well.

Can I perform process testing with MTA?

Yes, MTA is designed with process testing in mind.  MTA supports, unit- , component-,  process- and API tests. In addition, because MTA provides the ability to create objects on-the-fly, testing can be started at any point in a process.

What dashboard information does MTA provide?

The dashboard shows at a glance how production-ready the Mendix applications are for which test scripts have been created. The user can specify which application on which Mendix environment should be included in the dashboard.

The dashboard is divided into three detail levels:

  1. Overview of the latest test result of all selected applications per Mendix environment (Test, Acceptance or Production);
  2. Overview of successfully executed test cases of 1 application on a Mendix environment;
  3. Overview of the history of the aggregated test results over time.

 

Per visualized application is displayed:

  • The latest application revision that has been tested;
  • The number of test configurations in which the application occurs;
  • The number of test configurations for which the tests are executed;
  • The success rate of the last test runs.

 

From the dashboard you can navigate directly to a test configuration in order to analyse and solve errors immediately.

Can MTA be used for an application that uses SSO?

Yes. The users that are created with SSO in the Mendix application can be configured to be used by MTA. However, MTA is not logging on via SSO but uses the local user for testing.

Can I see the original test definition in case of a consistency error?

Yes, MTA automatically checks for model changes whether the test configuration is still consistent with the Mendix model. If this is not the case, a consistency error will be displayed, but the test script itself will not be modified. This makes it clear, for example, what the original data type of a microflow parameter is and what the originally specified value was. This way it is easier to determine how to reconfigure the test script to fix the consistency error.

A1078vierkant

Question not found?

Contact us and we will get back to you

Menu