AI assisted test generation in MTA

Webinar summary

• Webinar Introduction and Objectives: Menditect hosted a webinar focused on simplifying and accelerating software testing for Mendix applications using AI. The presentation highlighted that as Mendix moves toward an agentic application platform, traditional manual and automated testing methods are insufficient for the speed required by AI-assisted development. The primary goal of the session was to demonstrate how to reduce the cost and complexity of test automation by utilizing AI to generate, analyze, and maintain tests.
• Menditect Test Automation (MTA) Overview: Menditect Test Automation (MTA) provides a platform for storing and running various types of tests, including unit testing (microflow testing) and end-to-end front-end testing. A key feature of MTA is its shared context with the Mendix model, which allows the AI to reason effectively about the application without requiring write access to the Mendix model itself. This approach prevents model pollution, and the AI only requires access to the model design, not the running application, to generate tests.
• Technical Integration via Model Context Protocol (MCP): MTA integrates with AI using the Model Context Protocol (MCP), an open standard for connecting AI clients with server tools. MTA functions as an MCP server, while MAIA, an LLM provide by Mendix, acts as an MCP client. This setup enables the AI to utilize diverse inputs—such as user stories, detailed specifications, and Mendix model information—to design tests. The actual execution of these tests remains deterministic, ensuring consistent results regardless of the AI’s involvement in the design phase.
• AI Capabilities in Test Generation and Data Management: The AI-assisted approach helps with both the creation and analysis of tests, including complex tasks like setting up and tearing down test data. By asking the AI what data is required, the system can automatically generate scripts to prepare the environment and clean up generated data. The upcoming MTA release (MTA 3.2) will include MCP tools and “MTA skills,” which are AI guidelines for building, analyzing, and maintaining tests, made available through the marketplace.
• Live Demo of Unit Test Generation: Menditect demonstrated the generation of a unit test for a car rental application, specifically testing a microflow attribute related to “free kilometers”. By providing a prompt to MAIA that included the MTA skills, the application context, and limits on data variations, the AI successfully built a test case with appropriate boundary conditions, documented the test, and executed it. The test passed, successfully verifying three different data variations.
• Efficiency Gains and Documentation: Data from internal testing indicates that generating tests with AI, particularly unit tests, is six to ten times faster than manual methods. The AI also automates the documentation process, ensuring that the intent behind each test is clearly defined. The token costs for these operations are minimal, estimated at under €0,10 for standard test generation tasks.
• Front-End Test Debugging and Maintenance: A live demonstration showed how to use Maya to analyze and repair a broken front-end test for a car rental booking flow. After the initial execution failed due to a date formatting error—where the system required European format rather than American—the AI identified the root cause through analysis, corrected the test steps, and successfully reran the test to completion. This capability positions MTA as an agent that can build, analyze, and repair tests autonomously.
• Strategic Business Value of AI-Assisted Testing: The adoption of AI-assisted testing reduces complexity, increases coverage, and allows for greater release confidence. By utilizing AI as a companion, developers can identify risks, design appropriate tests, and perform maintenance more efficiently. This approach addresses common bottlenecks in test automation, such as the initial setup and long-term maintenance of test scripts.
• Support for Legacy Mendix Applications: While the primary integration is designed for newer Mendix versions, MTA can support Mendix 9 and 10 applications by using the Mendix CLI (mx-cli). The MX CLI extracts model data into a format that LLMs can analyze, allowing the AI to generate regression tests for legacy apps to assist with migration confidence to Mendix 11.
• Roadmap and Free Trials: Menditect plans to launch assisted free trials this summer, allowing teams to test the platform’s capabilities on their own applications. The company intends to gather feedback to refine the product before a broader public release. The features will be available to all Menditect customers at no additional charge beyond the cost of the LLM they choose to use.