UI Automation Activities
Unified UI Automation activities work across desktop apps and web pages using a single set of activities. They use indication-based selectors that adapt to UIA, MSAA, Java, SAP, and Playwright automatically — no need to switch between desktop and browser activity sets.
ClickClick
Click any element — desktop or web. Automatically routes to UI Automation or Playwright based on the selector.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | No | — | Element selector (desktop or web) |
ClickType | String | No | Single | Type of mouse click Allowed: Single, Double, Right |
InputMethod | String | No | Simulate | How to send the click (desktop only) Allowed: Simulate, WindowMessages, HardwareEvents |
Timeout | Int32 | No | 30 | Maximum seconds to wait for the element |
CvFallback | Boolean | No | False | Use Computer Vision as primary strategy |
ImageTemplate | String | No | — | Path to image template for CV matching |
ImageMatchThreshold | Double | No | 0.8 | Confidence threshold for image matching (0.0 - 1.0) |
UseImageFallback | Boolean | No | True | Automatically fall back to image-based matching if element not found by selector |
ClickClick
Clicks on a UI element identified by its selector
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | No | — | UI element selector (e.g., ' |
ClickType | String | No | Single | Type of click to perform Allowed: Single, Double, Right |
InputMethod | String | No | Simulate | How to send the click to the application. Simulate: Uses UI patterns, no mouse movement, works in background. WindowMessages: Uses Win32 messages. HardwareEvents: Moves actual mouse cursor. Allowed: Simulate, WindowMessages, HardwareEvents |
ImageTemplate | String | No | — | Path to image template file for computer vision matching |
UseImageFallback | Boolean | No | True | Automatically fall back to image matching if UI Automation fails |
ImageMatchThreshold | Double | No | 0.8 | Confidence threshold for image matching (0.0 - 1.0) |
Close Application/BrowserClose
Close the current application or browser session.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
CloseMode | String | No | Auto | What to close: Auto (detect from context), Browser, Desktop Allowed: Auto, Browser, Desktop |
ForceKill | Boolean | No | False | Force-terminate the process if graceful close fails within the grace period |
GracePeriod | Int32 | No | 5 | Seconds to wait for graceful close before force-killing |
CloseAll | Boolean | No | False | Close all running instances of the process (Desktop mode only) |
Execute JavaScriptExecute JavaScript
Execute JavaScript code in the context of the current browser page.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Script | string | Yes | — | JavaScript code to execute in the page context |
Result | string | No | — | Variable to store the script return value |
BrowserSessionId | string | No | — | Browser session ID (auto-detected from parent scope if omitted) |
Extract TableExtract Table
Extract tabular data from any table or grid — desktop or web.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | Table or grid element selector |
OutputVariable | DataTable | No | extractedTable | Variable name to store the extracted DataTable |
IncludeHeaders | bool | No | true | Whether to treat the first row as column headers |
MaxRows | Int32 | No | 0 | Maximum rows to extract (0 = unlimited) |
ColumnFilter | String | No | — | Comma-separated list of column names to keep |
ScrollAndAccumulate | bool | No | false | For virtualized grids: scroll the table and accumulate all rows (deduplicates automatically) |
RowSelector | String | No | tr | CSS selector for rows within the table |
CellSelector | String | No | td,th | CSS selector for cells within a row |
HeaderSelector | String | No | th | CSS selector for header cells |
Timeout | Int32 | No | 30 | Maximum seconds to wait for table extraction |
Extract UI TableExtractUITable
Extract data from a table or grid into a DataTable variable. Works with both desktop (DataGrid, ListView) and web page tables.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | string | Yes | — | Selector for the table, grid, or list element |
OutputVariable | string | No | tableData | Variable name to store the extracted DataTable |
IncludeHeaders | Boolean | No | True | Extract column headers from the table |
RowIndex | Int32 | No | -1 | Extract only this row (0-based). -1 for all rows. |
ColumnIndex | Int32 | No | -1 | Extract only this column (0-based). -1 for all columns. |
Get TextGet Text
Read text from any element — desktop or web. Automatically routes to UI Automation or Playwright based on the selector.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | Element selector (desktop or web) |
OutputVariable | String | No | getTextValue | Variable name to store the extracted text |
Timeout | Int32 | No | 30 | Maximum seconds to wait for the element |
ImageTemplate | String | No | — | Path to image template for OCR fallback |
ImageMatchThreshold | Double | No | 0.8 | Confidence threshold for image matching (0.0 - 1.0) |
Get TextGetText
Extracts text from a UI element and stores it in a variable
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | UI element selector to extract text from |
OutputVariable | String | No | — | Variable name to store the extracted text |
UseImageFallback | Boolean | No | True | Automatically fall back to image matching if UI Automation fails |
ImageMatchThreshold | Double | No | 0.8 | Confidence threshold for image matching (0.0 - 1.0) |
HoverHover
Hover over any element — desktop or web.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | Element to hover over (desktop or web) |
Timeout | Int32 | No | 10 | Maximum seconds to wait for element location |
Launch ApplicationLaunch Application
Launch or attach to any desktop application. Establishes the correct automation scope for child activities.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
ApplicationType | String | No | Auto | Auto-detect or force Desktop application Allowed: Auto, DesktopApp |
ApplicationPath | String | No | — | Desktop application path or UWP AppId |
Arguments | String | No | — | Command-line arguments for the application |
Selector | String | No | — | Application-level selector for window identification |
WindowTitle | String | No | — | Window title (or partial) to match |
ProcessName | String | No | — | Process name to attach to (e.g., 'notepad', 'excel') |
AttachMode | String | No | LaunchNew | LaunchNew opens a fresh instance. AttachExisting finds a running one. AttachOrLaunch tries to attach first, then launches if not found. Allowed: LaunchNew, AttachExisting, AttachOrLaunch |
Close | Boolean | No | True | Close the application after child activities complete |
MaximizeWindow | Boolean | No | False | Maximize the window after opening |
WaitForReady | Boolean | No | True | Wait for the application to be ready for input |
WaitTimeout | Int32 | No | 30000 | Maximum time to wait for the application window |
Launch ApplicationLaunch Application
Launches an application by path or indicated target
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
ApplicationPath | String | Yes | — | Path to the executable (e.g., 'notepad.exe', 'calc.exe', 'C:\Apps\MyApp.exe'). Use 'Indicate application' to select. |
Arguments | String | No | — | Command-line arguments to pass to the application |
WorkingDirectory | String | No | — | Working directory for the application |
WaitForReady | Boolean | No | True | Wait for the application window to be ready for input |
WaitTimeout | Integer | No | 5000 | Timeout in milliseconds to wait for the application to be ready |
Open | Boolean | No | False | If true, open new instance. If false, use existing window if available. |
Close | Boolean | No | False | If true, close application after completing child activities. If false, leave application open. |
MaximizeWindow | Boolean | No | False | If true, maximize the application window after launch. |
Launch BrowserLaunch Browser
Launch a Playwright-enabled browser. Child activities inside this container use the Playwright engine for web automation.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
BrowserType | String | No | Chrome | Browser to launch (must have Genzbots extension installed) Allowed: Chrome, Edge |
Url | String | No | — | URL to navigate to after launching the browser |
BrowserSessionId | String | No | — | Existing browser session ID to reuse (skips launch) |
Close | Boolean | No | True | Close browser after completing child activities |
MaximizeWindow | Boolean | No | False | Maximize the browser window after launch |
BrowserSessionIdOutput | String | No | — | Variable to store browser session ID for use in subsequent activities |
Open ApplicationOpenApp
Launches an application using its executable path
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
AppPath | String | Yes | — | Path to the executable or application name (e.g., 'notepad.exe', 'C:\Apps\MyApp.exe') |
Arguments | String | No | — | Command-line arguments to pass to the application |
WaitForReady | Boolean | No | True | Wait for the application to be ready for input |
Select ItemSelect Item
Select an item from a dropdown — desktop or web.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | Dropdown/combobox element selector |
Value | String | Yes | — | Item to select |
SelectBy | String | No | value | How to match: value, label, or index Allowed: value, label, index |
Timeout | Int32 | No | 10 | Maximum seconds to wait for element location |
Send HotkeySend Hotkey
Send keyboard shortcuts or key presses — desktop or web.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Key | String | Yes | — | Key or combination to send, e.g. {Enter}, {Ctrl+C}, a |
Selector | String | No | — | Element to focus before sending key. If empty, sends to active window/page. |
DelayBetweenKeys | Int32 | No | 10 | Milliseconds to wait between each keystroke |
DelayAfter | Int32 | No | 100 | Milliseconds to wait after sending all keys |
Timeout | Int32 | No | 5 | Maximum seconds to wait for element focus |
Send KeysSendKeys
Sends keyboard keys to the focused application or a specific element
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
TargetMode | String | No | ForegroundWindow | Target for the keys: Foreground window or a specific element Allowed: ForegroundWindow, SpecificElement |
Selector | String | No | — | Selector for the target element (optional - only used when TargetMode is SpecificElement) |
Keys | String | Yes | — | Keys/text to send. Use {Key} for special keys: {Enter}, {Tab}, {Ctrl+C}, {Alt+Tab}, {F1}-{F12}, {Up}/{Down}/{Left}/{Right}, etc. |
DelayBetweenKeys | Int32 | No | 10 | Delay in milliseconds between each keystroke |
SendAsHardwareKeys | Boolean | No | False | Send keys using hardware scan codes (more compatible with some applications) |
DelayAfter | Int32 | No | 100 | Delay in milliseconds after sending all keys |
Take ScreenshotTake Screenshot
Take a screenshot of an element, window, or full page — desktop or web.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | No | — | Element to screenshot. If empty, captures full window/page. |
OutputPath | String | No | — | File path to save the screenshot (.png, .jpg, .bmp). Auto-generated if empty. |
FullPage | bool | No | False | Capture the entire scrollable page (web only) |
OutputVariable | String | No | — | Variable to store the screenshot file path |
Timeout | Int32 | No | 10 | Maximum seconds to wait for element location |
Type IntoType
Types text into a UI element (supports variables and credentials)
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | No | — | UI element selector (e.g., ' |
Text | String | Yes | — | Text to type. Supports ${Variables} and ${Credential:Name.Password} |
ClearFirst | Boolean | No | False | Clear existing text before typing |
InputMethod | String | No | Simulate | How to input text: Simulate (patterns, no mouse), WindowMessages (Win32 messages), HardwareEvents (actual keyboard) Allowed: Simulate, WindowMessages, HardwareEvents |
ImageTemplate | String | No | — | Path to image template file for computer vision matching |
UseImageFallback | Boolean | No | True | Automatically fall back to image matching if UI Automation fails |
ImageMatchThreshold | Double | No | 0.8 | Confidence threshold for image matching (0.0 - 1.0) |
Type IntoType Into
Type text into any element — desktop or web. Automatically routes to UI Automation or Playwright based on the selector.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | No | — | Element selector (desktop or web). Either Selector or ImageTemplate is required. |
Text | String | Yes | — | Text to type into the element. Supports variables ({variableName}) and credentials. |
ClearFirst | Boolean | No | False | Clear the field before typing new text |
InputMethod | String | No | Simulate | How to send keystrokes (desktop only) Allowed: Simulate, WindowMessages, HardwareEvents |
Timeout | Int32 | No | 30 | Maximum seconds to wait for the element |
CvFallback | Boolean | No | False | Use Computer Vision as primary strategy |
ImageTemplate | String | No | — | Path to image template for CV matching |
ImageMatchThreshold | Double | No | 0.8 | Confidence threshold for image matching (0.0 - 1.0) |
Use ApplicationUseApplication
Attaches to an already running application window to perform actions inside it
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
WindowTitle | String | No | — | The title (or partial title) of the window to attach to (e.g., 'Untitled - Notepad') |
ProcessName | String | No | — | The process name of the application (e.g., 'notepad', 'chrome') |
BringToFront | Boolean | No | True | Whether to bring the application window to the foreground |
WaitTimeout | Integer | No | 10000 | Maximum time to wait for the window to be available |
Close | Boolean | No | False | If true, close application after completing child activities. If false, leave application open. |
ApplicationPath | String | No | — | The application path or AppUserModelID for UWP apps (e.g., 'Microsoft.WindowsCalculator_8wekyb3d8bbwe!App') |
Open | Boolean | No | False | If true, always launch a new application instance. If false (default), attach to an existing window. |
Arguments | String | No | — | Command-line arguments to pass when launching the application (used when Open is true) |
WorkingDirectory | String | No | — | Working directory for the application (used when Open is true) |
MaximizeWindow | Boolean | No | False | If true, maximize the application window after attaching or launching |
WaitForReady | Boolean | No | True | Wait for the application to be ready for input |
Wait For ElementWait For Element
Wait for any element to appear, disappear, or reach a state — desktop or web.
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | Element selector (desktop or web) |
State | String | No | Visible | State to wait for: Visible, Enabled, Exists, Appear, Disappear, Clickable Allowed: Visible, Enabled, Exists, Appear, Disappear, Clickable |
Timeout | Int32 | No | 30 | Maximum seconds to wait for the element state |
ElementFound | bool | No | — | Variable to store whether element was found |
Wait For ElementWaitForElement
Waits for a UI element to become visible, enabled, or exist
| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
Selector | String | Yes | — | UI element selector to wait for |
State | String | No | Appear | State to wait for: Appear (element becomes visible), Disappear (element goes away), Visible, Enabled, Exists, Clickable Allowed: Appear, Disappear, Visible, Enabled, Exists, Clickable |
ElementFound | bool | No | — | Variable to store whether element was found |