Hi Community members, 
Our team noticed that there had been several threads on the forum regarding Desktop Testing with Katalon as of late. As a result, we will be dedicating June to producing Desktop Testing-related articles for our KShare series!
Let’s get started!
So, you have a list of elements from an object’s locator → you want to click on one of the elements in the list. → Do you wish to use images to determine the desired element?
If you see yourself in this kind of situation, then stick with us till the end of this article for a tutorial on how to do so.
Scenario
There are 2 combo boxes , which can be found with the locator below:
//Window/ComboBox
Now, we expect to click on the combo box including the default value named Telex. What should we do? Read more below to find out 
Instructions
Step 1: Capture an image containing the text Telex
We recommend you to use the key-chord: Win + Shift + S instead of using any tools like Snipping Tool since it supports you capturing precisely the text.
Step 2: Add the custom keywords below into your Custom Keyword file
@Keyword
def clickElementContainsImage(WindowsTestObject testObject, String image, float minScore = 0.5) {
def element = findElementContainsImage(testObject, image, minScore);
if (element == null) {
KeywordUtil.markFailedAndStop("Cannot find any elements that contain images");
return;
}
element.click();
}
@Keyword
def RemoteWebElement findElementContainsImage(WindowsTestObject testObject, String image, float minScore = 0.5) {
List<RemoteWebElement> elements = Windows.findElements(testObject);
def driver = Windows.getDriver();
def window = driver.manage().window();
def winWidth = window.getSize().getWidth();
def winHeight = window.getSize().getHeight();
def winX = window.getPosition().getX();
def winY = window.getPosition().getY();
// println "> Window(${winX}, ${winY}, ${winWidth}, ${winHeight})";
def desktop = WindowsDriverFactory.getWindowsSession().getDesktopDriver();
def device = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice();
def screenWidth = device.getDisplayMode().getWidth();
def screenHeight = device.getDisplayMode().getHeight();
// println "> Screen(${screenWidth}, ${screenHeight})";
def bounds = new DesktopScreenRegion().getBounds();
def scale = bounds.getWidth() / screenWidth;
def sikuliImage = resizeImage(image, scale);
Target target = new ImageTarget(sikuliImage);
target.setMinScore(minScore);
for (def elementI : elements) {
def info = elementI.getText();
def x = winX + elementI.getLocation().getX();
def y = winY + elementI.getLocation().getY();
def width = elementI.getSize().getWidth();
def height = elementI.getSize().getHeight();
def region = new DesktopScreenRegion((int)(x * scale), (int)(y * scale), (int)(width * scale), (int)(height * scale));
def result = region.find(target);
if (result) {
// println ">>> (${x}, ${y}, ${width}, ${height}) ${info}";
return elementI;
} else {
// println "> Region(${x}, ${y}, ${width}, ${height}) ${info}";
}
}
return null;
}
Usage
In the custom keyword named findElementContainsImage and clickElementContainsImageyou will notice a parameter named float minScore = 0.5
0.5 is the default value of the exact index when you use those custom keywords. However, you can update that number to make sure the keyword can recognize your image exactly.
For example, in this scenario, we will need to update the minScore from 0.5 to 0.9
CustomKeywords. 'Windows ImprovedKeywords.clickElementContainsImage' (DropownList, Path, 0.9)
Result
If you find this article helpful, then don’t forget to show your support by leaving a like 
or a heart 
and share it with your colleagues or teammates!
