[End] Ask Katalon Anything - AI meets Web Testing

Announcements & Updates
1

Ask Katalon Anything 1200x630 (3)
Ask Katalon Anything 1200x630 (3)2400×1260 437 KB

Hey friends :waving_hand: @trust_level_0

It’s time - Ask Katalon Anything is officially LIVE! :rocket:

For the next few weeks, you can bring your questions directly to our Katalon Experts :bullseye:

:sparkles: They’re the minds behind the features you use every day.
:sparkles: They know the roadmap, the tips, the shortcuts (and the gotchas).
:sparkles: And they’re here to answer whatever’s on your mind.

So jump in with your toughest questions, your burning “what ifs,” and even that one tricky bug you can’t shake. These are real conversations with the people building Katalon with you. :light_bulb:

:spiral_calendar: Running now through 2025-10-21T16:59:00Z.

:round_pushpin: Happening right here in the Community → AKA Sept 2025

Thuy Ngo
Thuy Ngo261×261 62.4 KB
Vivian Nguyen
Vivian Nguyen261×261 57.8 KB
Duy Lam
Duy Lam261×261 69.7 KB
Tuan Tran
Tuan Tran261×262 45.7 KB
Duy Lam – Engineer (Katalon Studio) Thuy Ngo – Product Manager (Katalon Studio) Vivian Nguyen – Product Manager (Katalon TrueTest) Tuan Tran – Engineer (TrueTest Studio)
Duy builds impactful features for Katalon Studio. Passionate about creating solutions that empower testers, he enjoys seeing how the community applies Studio to solve real challenges. Thuy helps make Studio stronger and more adaptable. She works with the community and engineering teams to refine performance, Recorder, and AI-powered features. Always eager to hear your feedback! Vivian leads Katalon TrueTest, our AI-powered test automation platform. She focuses on turning real user interactions into smarter test coverage and is excited to exchange ideas with you. Tuan develops TrueTest Studio features that simplify test design and scaling. He’s focused on usability and performance, and always open to suggestions that help improve the experience.

:new: 1: Where to ask :question:

:new: 2. Guidelines :spiral_notepad:

  • If you have any questions about Web Tesing or TrueTest please raise them directly in the thread of that product (e.g. refer to Web Testing Q&A, ask @shin or @duy.lam). Ask one question at a time by replying to this topic, using the Reply button AT THE BOTTOM of each respective threads.

  • We will provide a weekly recap of all the questions asked in a separate thread. Hence, there should be no product-related discussion over here so that we can better focus on each product’s Q&A flow. Be sure to read through the existing questions in the summary by then to see if your question has already been asked by another member, before posting your own.

  • Please don’t reply to anyone else’s post. If you’d like to discuss a related topic in more detail, you can Create a new topic

  • After posting your question, please give our Product team members from 1-3 days to answer your questions as they might also be busy with other projects

  • And lastly, don’t forget to have fun! :wink:

Posts/replies that do not follow these guidelines may be removed to keep the AKA session flowing smoothly. Thanks! :+1:

:new: 3. Ask Katalon Anything Episode 8 Lucky Draw :sparkles:

We’re running a Lucky Draw to thank members who help grow the community during this round of Ask Katalon Anything (AKA)!

Lucky Draw Prize

:red_question_mark: How do I qualify for the Lucky Draw?
:backhand_index_pointing_right: Submit a question during the event. Our PMs and Engineers will review and select eligible questions.

:red_question_mark: How are winners chosen?
:backhand_index_pointing_right: From the pool of eligible questions, 5 lucky members will be randomly selected.

:red_question_mark: What’s the prize?
:backhand_index_pointing_right: Each winner will receive a $50 gift card.

:alarm_clock: When will the Lucky Draw take place?

  • The AKA event runs from September 23, 2025 to October 21, 2025.
  • The Lucky Draw will take place after October 21, 2025, once the event ends and we have verified all badge earners.
  • The winner(s) will be announced publicly in the forum on October 30, 2025.

If you have any questions, just reply here I’m happy to help.

Katalon Community Team

3 Likes
3

Hi team,

I am trying TrueTest and got a question that in what ways does TrueTest’s production-traffic capture improve coverage breadth and realism compared to conventional scripted test automation? Thank you

2 Likes
4

it will supposedly capture the real events from the clients and based on that would divide those journeys into High , medium and low scenarios that you can review , edit and connect

3 Likes
5

Hi DucLa, thank you for the question.
TrueTest improves coverage and realism compared to the conventional scripted test automation in three primary ways:

  • Automatic Discovery: Massively increases test breadth by automatically discovering and mapping hundreds to thousands of real user journeys you would never have time to script manually.

  • Eliminates “Happy Path Bias”: Improves realism by capturing the complex, real-world edge cases and non-linear workflows that scripted tests consistently miss.

  • Data-Driven Prioritization: Shows you with hard data which user paths are most critical, allowing you to stop guessing and focus testing resources on what actually impacts your customers.

2 Likes
9

Feedback for Proposed Katalon Studio Recording Agent
This concept sounds very promising and intriguing, especially for teams like ours that are still maturing in automation while balancing day-to-day manual testing. The idea of combining manual recording with natural language prompts could significantly reduce the initial scripting overhead and help newer automation testers become productive faster.

That said, from our team’s perspective, there are a few practical points that would make this feature truly valuable in an enterprise environment. Please note that this is purely from our team’s perspective but still could apply to other similar sized teams:

  1. Bridging Skill Gaps:
    Our test team currently has a mix of manual and beginner-level automation experience. A Recording Agent that allows a hybrid workflow recording some steps, then generating others via natural language would help bridge that gap nicely. It would make the transition to automation more approachable without overwhelming less technical testers.

  2. Reliability:
    While prompt-based generation sounds efficient, accuracy and element stability are more critical for us than speed. We’d prefer if the agent prioritized robust locator strategies, clear variable naming, and readable generated code that aligns with our existing test framework. Auto-generated steps are only useful if they can be easily maintained later.

  3. Domain Adaptability:
    The ability to plug in a domain-specific MCP server (e.g., for enterprise web apps or ERP systems) is intriguing. However, given our tech stack (.NET Blazor), we’d value compatibility and extensibility first — for example, if the Recording Agent can reliably detect and interact with Blazor components, that would be a game changer since Blazor is not currently on the officially supported framework list.

  4. Resource Efficiency:
    With a really small testing team, every new feature’s value comes down to time savings and reusability. If the Recording Agent helps us generate reusable test objects, data-driven templates, or shared keywords from natural language descriptions, it would definitely make a tangible difference to our productivity.

Overall, I believe this feature could be very useful for teams like ours that are still scaling up automation capability. If Katalon can balance convenience with maintainability — ensuring that the generated scripts are clean, stable, and framework-aligned — then it would absolutely make our testing life easier.

1 Like
10

Hi Duy and Thuy,
i’ve been exploring Katalon Studio recently and found the enhanced Recorder very powerful. I’m curious, are there any upcoming AI-powered enhancements planned to make test object recognition more resilient to dynamic locators or frequent UI framework changes (such as React or Angular)?

3 Likes
11

Hey @duy.lam and @Shin,

The idea of a Recording Agent sounds super interesting — especially the mix of manual recording and natural language prompts.

One thought I had: will the agent be able to auto-detect dynamic elements or handle test flakiness while generating steps through prompts? For example, if I record half a flow and then type something like “verify the user can update profile picture,” could it automatically identify the right locator strategy (XPath/CSS) for changing UI states?

I think this could really bridge the gap between manual and automation testers, especially for teams who are just starting to adopt AI-based testing.

Also, will the MCP integration allow teams to define context-aware logic (like specific API validations or DB checks) linked to the same test case?

If yes, that would make Katalon a complete hybrid solution — recording + reasoning + validation, all in one flow.

Really excited to see how this shapes up. Kudos to the team for thinking in this direction!

— Durgesh Sharma

1 Like
12

Dear Katalon Team,

The concept of a Recording Agent that combines manual recording with prompt-based test generation is both innovative and highly relevant to modern testing workflows. I believe this feature could significantly enhance productivity and accessibility in test automation.

Key Benefits:

  • Accelerated Test Creation: Natural language prompts allow faster generation of test cases compared to traditional step-by-step recording.

  • Ease of Use for Manual Testers: Reduces the learning curve by enabling non-technical users to contribute to automation.

  • Hybrid Flexibility: Offers the ability to record complex flows manually and supplement them with prompt-generated steps.

  • Domain-Specific Accuracy: Integration with custom MCP servers ensures that generated tests align with industry-specific requirements.

  • Improved Consistency: Standardized test generation minimizes human error and promotes uniformity across teams.

  • Scalable Automation: Enables rapid expansion of test coverage with minimal effort.

This approach aligns well with the evolving role of testers and the increasing adoption of AI in QA processes.

One question I have is:
Will the Recording Agent support seamless integration with existing test repositories and CI/CD pipelines to ensure smooth adoption in enterprise environments?

Thank you for inviting feedback and for the opportunity to contribute to the future of Katalon Studio.

1 Like
13

Hi team,

Are there any plans to integrate AI functions into Katalon?

I’m currently trying to compare PDF files across different environments. For now, I’m only comparing text and images using keywords. However, I’m facing several challenges — differences in resolution, image indexing, and special characters in the text make the comparison quite difficult. I believe AI could help simplify and improve this process.

1 Like
14

@Slash92 Not sure I understand the issue correctly. Your problem is how to extract the text and image content from a given PDF file, or how to assert the text/image (in your test case)?

1 Like
15

HI @Shin
I have a Web Testing Question
In complex test architectures, when multiple browser sessions are opened dynamically (for example, one for an application under test and another for Outlook or an admin portal), generally test automation platforms tend to lose driver context when switching back to the original browser.

So I want to know → How does Katalon internally manage web driver sessions across parallel or nested browser instances, and is there an advanced way to “attach” or “restore” the driver context of a previously opened browser without restarting the test execution?

1 Like
16

HI @taun
I have one question on True test, please answer

Many modern web UIs use animations, dynamic loaders, or personalized content that make pixel-perfect comparisons noisy.

How does TrueTest handle visual noise and dynamic components (e.g., rotating banners, timestamps, or animation transitions)?

Can users define “ignore regions” dynamically through code or metadata, rather than manually configuring them in the UI?

Regards,
Arvind Kumar C

1 Like
17

HI @taun
Question on Ture test

TrueTest uses AI to detect visual regressions between baseline and new versions. In real CI/CD pipelines, frequent UI tweaks (like color shade or pixel shift) can cause non-critical failures.

How does TrueTest’s AI model adapt over time to minimize false positives from non-functional UI changes — and is there a way to train or fine-tune the model based on project-specific visual baselines (e.g., using feedback loops or tagging reviewed results)?

Please answer
Regards,
Arvind Kumar C

1 Like
18

HI @duy.lam
I have a question on

When automating dynamic web apps with frequently changing DOM structures (like React or Angular), maintaining static test objects quickly becomes unmanageable.

Do you have any roadmap or a hidden technique to allow a fully dynamic repository, where object properties (XPath/CSS/attributes) are automatically learned or updated at runtime using AI-based DOM diffing — similar to how TestOps Smart Wait works?

Please reply

Regards,
Arvind Kumar C

1 Like
19

Surely it’s always one of our criteria for rolling out the whatever new feature to the enterprise customer. BTW, what type of testing your team is using Katalon so that it helps us prioritize the demand:

  • Web Desktop, Web Mobile or Mobile App?
  • The framework category for building app: SPA, web cross-platform (like Flutter, Corona) or mobile native (Android/iOS SDK)
  • Your CI/CD is cloud-hosting or self-hosting?

Duy

1 Like
20

@durgesh.sharma Thanks for your questions

One thought I had: will the agent be able to auto-detect dynamic elements or handle test flakiness while generating steps through prompts? For example, if I record half a flow and then type something like “verify the user can update profile picture,” could it automatically identify the right locator strategy (XPath/CSS) for changing UI states?

Just for brainstorming purposes: which use cases of Recording Agent, in a recording session, the prompt-based interaction is more beneficial than traditional browser-based interactions? This may sound simple, but I’d like to explore which domain/flow, during recording, when the natural language instruction (for a set of steps) is more productive than typing/clicking on elements

Could you elaborate more on this item? For instance, in the test execution phases (pre, in-progress, and post), what are the particular demands of context-aware logic?

Duy

1 Like