Test all levels with MTA

Can you run all of this tests in CI/CD pipeline preferably Mendix Pipelines?
Yes. MTA provides a public API that can be used for test scheduling. Currently, we offer a separate Scheduler Module that can be used in a Mendix App to trigger the API endpoints. In the near future, Mendix will be offering POST and GET REST calls included in Pipelines, offering it without the need for a separate Scheduler Module. See also https://documentation.menditect.com/additional/howtos/schedule-tests/cicd-flow

Is menditect software used for testing the implementation of Mendix for customer application development based on Mendix?
Menditect Test Automation is built in Mendix, for Mendix. The MTA Application only connects to Mendix Apps. If you want to use the UI features in MTA, you can also test websites that are not built in Mendix, but you will still need to host a Mendix App to execute the tests on. See also https://documentation.menditect.com/additional/howtos/design-tests/test-ui-with-mta

Is populating the masterdata a microflow? Or does menditect use some oter method?
Optionally. If there is already a microflow in your Mendix App that generates Masterdata, this solution can be used. You can call this microflow from within MTA. You can also add Teststeps that Create data in your database without the need for any microflows and execute these in MTA (https://documentation.menditect.com/Teststep/create). You can even do this without the need for the Execution User to have access to the relevant Domain Model.

How to add menditect in mendix application?
You need a license to use MTA. Once you have obtained a license you can install MTA according to our documentation. See our Howto section on our documentation page https://documentation.menditect.com/additional/howtos

Are test cases developed inside Menditect App?
Test Cases are created and stored separately in MTA. The customer can choose to host MTA themselves or have Menditect host it in the Mendix cloud. Regardless however, the customer will remain owner of the Test Design and Test Results data.

Does coverage count the nr of activities executed?
Not at this time. Currently the coverage only counts the number of microflows that were executed (also submicroflows) and is therefore called microflow coverage. You can however specify which microflows count towards coverage with filters. In the future, we will also allow for counting which activities in microflows were executed. This will be called microflow activity coverage.

Is it possible to extract the testresults to a BI platform?
Yes, it is possible to export a JSON file in a predetermined format, with the results of a single Test Run.

How do you write the UI tests?
UI tests can be created by adding microflow Teststeps in MTA. In short, there are two types of microflows for UI testing: Locators and Actions. Locators define which Mendix Widget you want to do something with: for example Button, TextBox, or DropDown. Actions define what you want to do on them: Click, Fill with text, Select an option from a list.

Can you mock actions or submicroflow calls? e.g. Api call? Can you test Nanoflows and native mobile apps with the tool?
It is possible to mock data for API calls using a separate Stub module that we offer for MTA (https://marketplace.mendix.com/link/component/212962). Because MTA only tests the Backend of an App, triggering Nanoflows is not possible from MTA. However it is possible to trigger a Nanoflow by using the UI testing feature.

Is it possibly to execute only UI testing without having to test all the other levels (units, ..)?
Yes. In MTA you define the test level that you want to use. You can use MTA for all test levels or exclusively for UI testing or Unit/microflow testing.

Can we pause the UI test to debug and investigate the browser?
Yes, as long as the Test Case timeout is not hit, you can stop the test by using the Mendix debugger in Studio Pro; the test case timeout is 300 seconds by default, but adjustable for each Test Case.

Can you run Ui tests in headless mode?
Yes, by setting the headless option to True when executing the Test.

Can we used to test for non-Mendix or other low code platform based applications?
Menditect Test Automation is built in Mendix, for Mendix. The MTA Application only connects to Mendix Apps. If you want to use the UI features in MTA, you can also test websites that are not built in Mendix, but you will still need to host a Mendix App to execute the tests on.

Can Menditect test the On-Premise Mendix Apps?
Yes, MTA is a Mendix app that can be hosted as one of your Mendix apps. See also https://menditect.com/run-mta-on-your-own-mendix-environment/ The only requirement is that you use Mendix teamserver (GIT or SVN) to manage your commits. In the future MTA will also be available for connection with local GIT repositories, allowing a fully air-gapped deployement.

You mention it is PlayWright, why not use PlayWright then for these tests?
MTA makes it possible to store playwright scripts in MTA and keep all the Playwright code in the Mendix software. Therefore you don’t need to store and manage your Playwright test with any 3rd party tools. On top of that, Menditect has created a Playwright Starter kit that supports locators for all standard Mendix widgets.

Any API exposed by Menditect?
If for example I am using Jenkins to automatically deploy an application to another environment, can i run the tests from Menditect, get the scores etc… in an automatic way?
Yes, MTA has several public API’s that can be used by pipelines such as Jenkins. See also https://documentation.menditect.com/api and https://documentation.menditect.com/additional/howtos/schedule-tests/cicd-get-started

Do you need playwright knowledge to create UI tests?
If you already know Playwright you can use all the features of Playwright with MTA. However, for non Playwright experts we have added the Playwright Starter kit for Mendix. This allows you to use Playwright without the need of becoming a Playwright expert.

Do you also plan to integrate https://mxlint.com/?
MTA is a test tool, testing features on a Mendix runtime (dynamic testing). MxLint is a code analysis tool, checking rules against the Mendix models. Both tools have different purposes. We encourage you to use both to keep the quality of your code optimal. We have no integrations plans so far, but they can be run next to each other in Studio Pro or in a pipeline.

So you have to write playwright code (Typescript) for Ul tesitng, Does MTA help with generating Locators?
Menditect offers a Playwright Starter kit, a (still growing) library of microflows that are built with the Mendix framework in mind, using Playwwright Locators. You can use the microflows in this library without having to write any PlayWright code yourself.

What is the advantage of using MendiTect with PlayWright vs using pure PlayWright?
See above

What if I’m operating in Private Cloud or on Prem env? Assume that I can’t expose any data to the Mendix Cloud?
See question on deployment options.

How about custom marketplace widgets?
Menditect will provide best effort support to also include support for non platform supported Marketplace widgets. If you have a particular request you can reach out to support with this request.

Can non developer users work with the Ui testing module?
That is certainly our goal. We are working hard to make UI testing with MTA accessible for non technical testers. However, this is still work in progress.

Can you mock actions?
See above

Unfortunately, Menditect does not support load testing at the moment, and according to the roadmap, this feature is planned for delivery in two years. Can this be prioritized sooner?
Please reach out to support@menditect.com to provide some more context. If you could describe what problem you need solved and what the impact is of not having a solution, that would certainly help. Our roadmap is dynamic and based on feedback from the market we can adapt it to shifting priorities.

Can we combine teststep to create a reusable testblock?
You can use the DataVariation feature to run a single Test Case multiple times with optional variations in each run. Reusing particular pieces of a Test Case will be offered in the future. See also https://documentation.menditect.com/datavariation

As from a testmanagement perspective i would like to have my results from all testactivities in one place, aswell manual as automated. Is this possible?
Yes. Using the Archive function in MTA, it is possible to export a JSON file in a predetermined format, with the results of a single Test Run. The export can also be downloaded using the MTA Public API, see https://documentation.menditect.com/api#get-testrun-archive. You can also copy direct links of test cases, test suites en test configurations to a test management app, so that you can associate test cases from such a tool to the respective cases in MTA.

Can you share the the PRCING MODEL you propose for this solution?
You can reach out to Menditect sales to dicsuss pricing options.

Is MTA not free, right?
MTA is indeed a paid product. See also https://menditect.com/menditect-test-automation/pricing/