Katalon TestOps passing Project via command line

Feedback & Reviews Feature Requests
1

We currently have Katalon projects that are deployed and run through AzureDevOps. We run the same project in 3 different environments, Dev, QA, and Staging. Within the Katalon project you can define the integration for TestOps and select a project where the report data will be stored.

Since I run these projects in 3 different environments having all the data reported under one project doesn’t provide me with much value because I need to see the data represented separately for each environment. What I would like to see is the ability to set the TestOps project via command line during execution. This way when AzureDevOps executes the command line call I can specifically tell it what project to send the data to. Projects that were executed and run in the Dev environment would be shown in the Dev project in the TestOps site and projects that were executed in the QA environment would be stored in the QA project in testOps.

Currently all our test results are pulled back into AzureDevOps but I think I can get more useful data out of TestOps. However, if I can see the data split based on where the environment were the test was run it doesn’t provide me enough value. Along with this should be the ability to enable the TestOps integration from command line. I would not want the data from my Devs test run on their local machines to be pulled into TestOps. If I could enable and configure it at runtime I would be able to ensure the data is relevant to our environments and not part of some debugging process.

Without this the data becomes messy pretty quick so I will continue to just use the AzureDevOps reporting.

2

Hi @kgurtsak,
The testops CI can solve your problem. You can try it out. When run your katalon project with local agent, its results will be submitted to the configured testops project.

3

Thanks Thong,

Moving to TestOps CI would be too big of a change right now for what we have in place. We are using AzureDevOps as our repo for our projects and it is handling all of the deployments and executions of our test code. I wouldn’t want to move to something that is still in beta and I’m not sure of the pricing model. It looks like it would be a good option but we would have to have the on-prem version of TestOps also because of our local tesing environments.

My alternative option is to create multiple com.kms.katalon.integration.analytics files, one for each environment, and have these deployed prior to each execution. Each file would point to the TestOps project that I want the execution to be recorded to. I would also configure our CICD to remove this file after execution so that only results from the CICD run would be recorded. It’s a little more configuration that I wanted to do but without a cmd argument to pass in project information I think it’s the cleanest way. I don’t want to tear down the whole architecture for some reporting since I am getting results pulled into Azure but I wanted to see if TestOps would provide more detailed information.

2 Likes