Documentation

Landing Page Test

Compare two landing page variants with identical AI personas before launch to pick the higher-converting design with statistical confidence.

Scenario

Your design team has two versions of a B2B SaaS landing page ready to ship. Rather than waiting weeks for real traffic to produce statistically significant A/B test results, you can use SimuTest to run 100 AI-simulated evaluators against each variant simultaneously. Both variants receive identical persona packs so the comparison is controlled.

Why this approach works

Identical personas ensure the comparison is apples-to-apples — the only variable is the page design
Results available in minutes rather than weeks, unblocking shipping decisions
AI thinking traces explain why users prefer one variant, not just that they do
Run before any real user sees either variant — zero risk to conversion rates

Configuration

Define both variants in the same simutest.yaml. Use the same personas.pack and identical task string for a controlled comparison.

version: 1

defaults:
  model: claude-sonnet-4-20250514
  sessions: 100
  viewport:
    - desktop
    - mobile

tests:
  - name: "Landing Page — Variant A (Control)"
    url: http://localhost:3000/landing-a
    task: "Understand what this product does and sign up for the free trial"
    criteria:
      - cta_findability
      - content_clarity
      - visual_hierarchy
      - persuasion_conversion
    personas:
      pack: "b2b-saas-evaluators"

  - name: "Landing Page — Variant B (Treatment)"
    url: http://localhost:3000/landing-b
    task: "Understand what this product does and sign up for the free trial"
    criteria:
      - cta_findability
      - content_clarity
      - visual_hierarchy
      - persuasion_conversion
    personas:
      pack: "b2b-saas-evaluators"

Running the Comparison

Use the simutest compare CLI command for a quick one-liner, or use the SDK's compare() method for programmatic access with custom reporting.

CLI

# Compare two variants using the CLI
simutest compare \
  --baseline "Landing Page — Variant A (Control)" \
  --treatment "Landing Page — Variant B (Treatment)" \
  --config simutest.yaml

Node.js SDK

import { SimuTest } from '@simutest/sdk';

const simutest = new SimuTest({ apiKey: process.env.SIMUTEST_API_KEY! });

async function runLandingPageTest() {
  // Run both variants in parallel
  const [variantA, variantB] = await Promise.all([
    simutest.test({
      url: 'http://localhost:3000/landing-a',
      task: 'Understand what this product does and sign up for the free trial',
      sessions: 100,
      criteria: ['cta_findability', 'content_clarity', 'visual_hierarchy', 'persuasion_conversion'],
      personas: { pack: 'b2b-saas-evaluators' },
    }),
    simutest.test({
      url: 'http://localhost:3000/landing-b',
      task: 'Understand what this product does and sign up for the free trial',
      sessions: 100,
      criteria: ['cta_findability', 'content_clarity', 'visual_hierarchy', 'persuasion_conversion'],
      personas: { pack: 'b2b-saas-evaluators' },
    }),
  ]);

  // Compare the results
  const comparison = simutest.compare(variantA, variantB);

  console.log('Winner:', comparison.winner);
  console.log('Confidence:', comparison.confidence + '%');
  console.log('Score delta:', comparison.scoreDelta);

  // Post as PR comment (in CI context)
  if (process.env.GITHUB_TOKEN) {
    await simutest.postPRComment(comparison, {
      token: process.env.GITHUB_TOKEN,
      repo: process.env.GITHUB_REPOSITORY!,
      pr: Number(process.env.PR_NUMBER),
    });
  }
}

runLandingPageTest().catch(console.error);

Evaluation Criteria

These four criteria are specifically tuned for landing page performance evaluation.

Criterion	What it measures	Signal
cta_findability	How quickly and confidently users locate the primary call-to-action	Time-to-CTA, click confidence
content_clarity	Whether users can accurately describe the product's value after reading the page	Comprehension accuracy, reread rate
visual_hierarchy	How well the layout guides users through the intended reading path	Scroll depth, section dwell time
persuasion_conversion	Effectiveness of social proof, objection handling, and urgency elements	Conversion intent, hesitation points

Expected Results

SimuTest outputs a side-by-side comparison table with per-criterion scores, a winner recommendation, and a confidence percentage based on the score distribution across sessions.

Metric	Variant A	Variant B	Delta
Overall score	6.8	7.9	+1.1
CTA findability	6.2	8.4	+2.2
Content clarity	7.1	7.6	+0.5
Visual hierarchy	6.9	8.1	+1.2
Persuasion/conversion	6.5	7.8	+1.3

Winner: Variant B — 94% confidence. Variant B outperforms the control on all four criteria, with the largest improvement in CTA findability (+2.2). Ship Variant B.

PR Comment Example

When run in CI, SimuTest can post A/B comparison results directly to the pull request as a comment. This gives reviewers full test context without leaving GitHub.

## SimuTest A/B Results

| Metric               | Variant A (Control) | Variant B (Treatment) | Delta   |
|----------------------|--------------------|-----------------------|---------|
| Overall score        | 6.8                | 7.9                   | +1.1 ✅ |
| CTA findability      | 6.2                | 8.4                   | +2.2 ✅ |
| Content clarity      | 7.1                | 7.6                   | +0.5    |
| Visual hierarchy     | 6.9                | 8.1                   | +1.2 ✅ |
| Persuasion/conversion| 6.5                | 7.8                   | +1.3 ✅ |

**Winner: Variant B** (confidence: 94%)

### Top findings

**Variant A issues**
- CTA button blends into the hero section — 43% of sessions missed it on first scroll
- Value proposition unclear: 61% of sessions couldn't articulate what the product does after reading the hero

**Variant B improvements**
- High-contrast CTA button found in median 4.2 seconds vs 11.8 seconds for Variant A
- Benefit-led headline improved comprehension scores by 22%

_Generated by [SimuTest](https://simutest.dev) · 100 sessions per variant_

Configure PR comments by setting GITHUB_TOKEN and PR_NUMBER in your CI environment. See the CI/CD Integration guide for GitHub Actions workflow examples.

On this page