QAID Docs

Automated Tests

The Automated Tests step is the primary test generation method in QAID. It creates AI-powered tests based on user journeys and page purpose.

Overview

Automated Tests (scenarios) verify complete user journeys:

  • "User should be able to submit the contact form"
  • "User should navigate from homepage to pricing page"
  • "User should complete the signup flow"

QAID's AI analyzes each page to understand its purpose and generates meaningful tests.

How Automated Tests Work

AI Analysis

QAID's AI examines each page to understand what users would want to accomplish, then generates appropriate tests for those goals.

Reliable Selectors

All selectors in scenario tests come from actual elements discovered on your site. QAID never invents or guesses selectors, ensuring tests are reliable and accurate.

Self-Healing Selectors

QAID's self-healing engine automatically re-locates elements when selectors change — using live page analysis and content matching. Tests continue passing through routine UI updates without manual maintenance. When a selector is healed, the new one is recorded for future runs.

Visual Regression per Scenario

Each scenario can optionally capture visual regression baselines:

  • Enable "Visual Regression" toggle when creating/editing a scenario
  • First execution captures the after-screenshot as the baseline
  • Subsequent executions compare against the baseline using pixel comparison
  • Results appear in the scenario's Visual tab

Viewing Scenarios

Pages with Scenarios

The main view shows pages with scenario counts:

  • Page URL
  • Number of scenarios
  • Scenario status breakdown

Filtering

Filter scenarios by:

  • Tags: Filter by assigned tags
  • Status: Passing, Failing, Pending
  • Page: Specific page

Scenario List

Click a page to see its scenarios:

  • Scenario name and description
  • Status badge (pass/fail/pending)
  • Tags
  • Last execution time

Creating Scenarios

Creating a New Scenario

  1. Click on a page to open the scenario view
  2. Click "New Scenario"
  3. Fill in the scenario details:
    • Description: What should happen (e.g., "User submits contact form")
    • Expected Result: What success looks like
    • Tags: Categorization labels

Writing Good Descriptions

Good Examples:

  • "User should be able to request a product demo"
  • "User navigates from homepage to pricing and sees plan options"
  • "User fills out and submits the newsletter signup form"

Poor Examples:

  • "Test the button" (too vague)
  • "Click things" (no clear goal)
  • "Form test" (unclear purpose)

Using the Browser Preview

The scenario creation modal includes a browser preview:

  1. See your page live
  2. Click elements to understand the flow
  3. Validate your scenario makes sense
  4. Capture screenshots for reference

Generating Test Code

After defining the scenario:

  1. Click "Generate Test"
  2. QAID generates the Playwright test code
  3. Review the generated test
  4. Save the scenario

Scenario Details

Viewing Details

Click a scenario to see:

  • Name and Description
  • Status: Current pass/fail state
  • Mapped Elements: Elements used in the test
  • Missing Elements: Elements the AI wanted but couldn't find
  • Test Code: The generated Playwright script
  • Execution History: Past runs

Mapped Elements

Shows which elements from your site are used:

  • Element name
  • Selector that will be used
  • Confidence level (how sure QAID is about the selector)

Missing Elements

If the AI couldn't find elements it needed:

  • Reason why element is missing
  • Suggestions for alternatives
  • Impact on test reliability

Executing Scenarios

Single Scenario Execution

  1. Open a scenario
  2. Click "Run"
  3. Watch execution progress
  4. View results when complete

Batch Execution

  1. Go to Test Plan step
  2. Select multiple scenarios
  3. Click "Run Selected"
  4. View consolidated results

Execution Progress

During execution you see:

  • Current step being executed
  • Screenshots of each state
  • Pass/fail indicator per step
  • Final result

Execution Results

After completion:

  • Overall pass/fail status
  • Detailed step-by-step results
  • Screenshots at each step
  • Timing information
  • Error details (if failed)

Managing Scenarios

Editing Scenarios

  1. Open the scenario
  2. Click "Edit"
  3. Modify description, expected result, or tags
  4. Regenerate test if needed
  5. Save changes

Deleting Scenarios

  1. Open the scenario
  2. Click "Delete"
  3. Confirm deletion

Tagging Scenarios

Tags help organize scenarios:

  • Add tags when creating/editing
  • Use suggested tags from existing scenarios
  • Examples: "smoke", "critical", "regression", "signup"

Automation Readiness

QAID evaluates each scenario for automation:

StatusMeaning
Automate NowAll elements found, high confidence
Needs AttentionSome elements missing or low confidence
Manual OnlyCan't automate reliably

AI-Generated Scenarios

QAID can automatically generate scenarios:

  1. Click "Generate Scenarios" on a page
  2. QAID analyzes the page and creates relevant scenarios
  3. Review and adjust generated scenarios as needed

Best Practices

Writing Scenarios

  • Focus on user goals, not technical details
  • Be specific about expected outcomes
  • One goal per scenario
  • Use realistic user language

Managing Scenarios

  • Tag scenarios consistently
  • Group related scenarios
  • Remove outdated scenarios
  • Keep scenario count manageable per page

Improving Success Rate

  • Add missing data-testid attributes
  • Simplify complex multi-step scenarios
  • Verify authentication is configured

Troubleshooting

Scenario Generation Fails

Possible causes:

  • Required elements not yet extracted
  • Page structure changed
  • Authentication required

Solutions:

  • Re-run element extraction
  • Check page is accessible
  • Verify authentication settings

Missing Elements in Scenario

The AI wants elements that don't exist:

  • Element may require interaction to appear
  • Element may be in an iframe
  • Element may be dynamically loaded

Solutions:

  • QAID's runtime element discovery automatically handles hidden elements
  • Re-run element extraction if elements have changed
  • Use recording to capture complex interactive sequences
  • Check iframe handling for cross-origin iframes
  • Wait for page fully loaded

Tests Failing Intermittently

Possible causes:

  • Timing issues (element loads slowly)
  • Dynamic content (dates, counters)
  • A/B testing on the site

Solutions:

  • Add appropriate waits in test
  • Account for dynamic content in assertions
  • Ensure consistent test environment

Low Selector Confidence

If confidence is below 80%:

  • Add data-testid to the element
  • Use more specific HTML structure
  • Consider alternative selectors

Recording Tests

You can also create scenarios by recording user interactions:

  1. Install the QAID Chrome Extension
  2. Start recording from the extension panel
  3. Interact with your website normally
  4. Stop recording when done
  5. The recorded actions are converted to a scenario with self-healing selectors

See Chrome Extension for detailed recording instructions.

Diff Analysis

Previous Page

Visual Regression

Next Page

On this page

OverviewHow Automated Tests WorkAI AnalysisReliable SelectorsSelf-Healing SelectorsVisual Regression per ScenarioViewing ScenariosPages with ScenariosFilteringScenario ListCreating ScenariosCreating a New ScenarioWriting Good DescriptionsUsing the Browser PreviewGenerating Test CodeScenario DetailsViewing DetailsMapped ElementsMissing ElementsExecuting ScenariosSingle Scenario ExecutionBatch ExecutionExecution ProgressExecution ResultsManaging ScenariosEditing ScenariosDeleting ScenariosTagging ScenariosAutomation ReadinessAI-Generated ScenariosBest PracticesWriting ScenariosManaging ScenariosImproving Success RateTroubleshootingScenario Generation FailsMissing Elements in ScenarioTests Failing IntermittentlyLow Selector ConfidenceRecording TestsRelated Topics