Reflect.run

Mobile app development has created a growing need for effective testing solutions that can handle the complexities of various devices and operating systems. While traditional desktop development testing methods fall short for mobile applications, new tools and approaches are emerging to fill this gap. This article explores one such solution, Reflect.run, which combines built-in mobile emulation capabilities with AI-powered test documentation to streamline mobile app testing.

Mobile Device Simulators

Chrome's built-in mobile emulation support allows testing on desktop using two device profiles: Mobile and Tablet. The mobile device profiles emulate specific iOS and iPad dimensions - iPhone 6, 7, and 8 with 667x375 pixels at 2x device scale factor for mobile, and iPad and iPad Mini with 1024x768 pixels at 2x device scale factor for tablet. Each test must be associated with a single device profile that cannot be changed after creation, addressing challenges with varying screen resolutions and JavaScript runtimes. This approach ensures consistent test behavior across different screen sizes while maintaining compatibility with existing test management tools.

Test Recording and Execution

After configuring the test properties and selecting a device profile, users begin recording by clicking "Begin Recording". This action initiates a cloud-based browser session loading the specified starting URL. During the recording process, all inputs are disabled to prevent false negatives, as indicated by the "Loading" vs. "Recording" status indicator at the top-right of the screen. For visual validation, users can add screenshots by selecting regions of interest, while textual validation is performed using specific options in the toolbar.

Once recording is complete, users verify their test by executing it through the Reflect.run interface. The platform captures detailed information about each test run, including console and network logs, allowing users to troubleshoot any issues through comprehensive documentation and AI-generated step descriptions. All recorded tests can be managed via the Reflect.run dashboard, where users can view both live and previously recorded test executions.

The recording and execution process supports multiple pages and sites, with features designed to maintain consistency across different devices and browsers. This includes support for hybrid mobile apps built with technologies like React Native, as well as comprehensive coverage for JavaScript applications across various runtime environments. By managing test execution within the cloud-based Reflect infrastructure, users can perform automated regression testing without the need for local installations or setup.

Test Playback and Validation

After recording and verifying their test through the Reflect.run interface, users trigger the automated playback process in several ways. The tool captures comprehensive documentation of each test run, including console and network logs, which it stores and retains according to the account's subscription tier.

Within the Reflect-run platform, tests can be executed either on-demand by clicking the "Run" button on the Test List page or within the specific test definition. For organizations managing multiple test cycles, Reflect allows combining these executions into logical units called Suites, which automate the process of environment configuration and result notification.

The playback process begins immediately after execution initiation, with Reflect connecting to its cloud browser infrastructure to perform the test run. All actions are recorded, including the initial page load and subsequent user interactions. The platform generates detailed documentation for each test step, including step descriptions automatically created using AI technology. These descriptions appear alongside brief animations in the test sidebar and are fully editable by the user, enhancing both documentation quality and test maintainability.

During playback, users have several options for additional documentation and troubleshooting. They can capture video recordings of the test execution, which are stored alongside the text-based documentation. Network logs are also captured and available for inspection, helping to diagnose issues related to page loading or API interactions. Reflect uses this information to generate detailed test results, including pass/fail outcomes displayed in a comprehensive results table accessible from the main application screen.

AI-Powered Test Documentation

Reflect.run's AI-powered test documentation generates comprehensive step descriptions and visual aids for automated tests. These AI-generated descriptions, displayed alongside brief animations in the test sidebar, replace traditional CSS selectors and XPath locators, allowing users to define test steps without technical coding knowledge. The system automatically translates plain-text instructions into actionable test steps, enabling rapid test creation - reportedly 10 times faster than code-based frameworks like Selenium and Playwright.

The platform supports three types of AI prompts: actions, true/false assertions, and questions. For complex testing scenarios, such as the Sauce Demo sample e-commerce site, Reflect's AI engine can break down instructions into multiple steps, ensuring thorough test coverage. While the AI handles many test actions effectively, it currently lacks support for file uploads, downloads, drag-and-drop operations, text highlighting, and native alert/prompt interactions.

To maintain test resilience over time, Reflect's AI Assistant uses selective element finding during runtime. If an expected element is not found, the AI generates a new selector based on the element's text content and the step's description. This approach ensures that tests remain valid even as applications evolve.

The platform's test documentation capabilities extend beyond basic step descriptions. For failed test steps, users can easily edit the AI-generated descriptions to improve test maintenance and clarity. These edited descriptions help maintain test suite accuracy while performing fewer maintenance tasks than traditional coding approaches.

Reflect's documentation system captures detailed information for each test step, including console and network logs. All test runs are stored and can be viewed through the platform's comprehensive results table, which displays run time, last execution date, and creation date for every test. The system stores test runs according to the account's subscription tier, with a default retention period of 30 days that can be extended through paid upgrades.

Real Device Testing

For organizations requiring real-device testing, Reflect.run offers specialized support that goes beyond its built-in mobile emulation capabilities. While the platform's native tools excel at simulating mobile environments through Chrome's device profiles, it acknowledges the limitations of this approach for comprehensive mobile testing.

To address these needs, Reflect provides an official support channel at support@reflect.run, where organizations can request assistance for specific use cases. The platform emphasizes the value of real-device testing for validating mobile applications across various hardware configurations and operating systems, though it notes that its native tools are not designed to replace these capabilities.

The support team works closely with customers to implement real-device testing solutions, though the exact nature of this support isn't detailed in the available documentation. For organizations that require this level of testing, Reflect encourages proactive communication with their support team to develop tailored solutions that meet their specific needs.