Documentation

Python SDK

Run AI-powered UX tests from Python with sync and async support.

Installation

Install the SDK from PyPI using pip:

pip install simutest

Requires Python 3.9 or later. The package includes type stubs for IDE support.

Quick Start

Import SimuTest, create a client with your API key, and call test(). The call blocks until all sessions complete.

import os
from simutest import SimuTest

client = SimuTest(api_key=os.environ["SIMUTEST_API_KEY"])

results = client.test(
    url="http://localhost:8000",
    task="Complete the checkout flow",
    sessions=100,
    model="claude-sonnet",
    personas="auto-diverse",
    criteria=["task_completion", "checkout_friction", "dark_patterns"],
    viewport="desktop",
)

print(results.summary)
# {"overall_score": 7.4, "sessions_completed": 97, "top_issue": "..."}

for session in results.sessions:
    print(session.persona["name"], session.scores["task_completion"])

Configuration

Pass keyword arguments to the SimuTest constructor:

from simutest import SimuTest

client = SimuTest(
    api_key=os.environ["SIMUTEST_API_KEY"],   # required
    base_url="https://api.simutest.dev",       # optional, default shown
    timeout=300,                               # optional, seconds (default: 300)
)

Parameter	Type	Required	Description
api_key	str	Yes	Your SimuTest API key
base_url	str	No	API base URL (default: `https://api.simutest.dev`)
timeout	int	No	Request timeout in seconds (default: 300)

Running Tests

The client.test() method accepts the following parameters:

results = client.test(
    # Required
    url="http://localhost:8000",
    task="Complete the onboarding flow",

    # Session configuration
    sessions=100,                  # number of simulated sessions
    model="claude-sonnet",         # AI model to use
    personas="auto-diverse",       # 'auto-diverse' or list of persona dicts

    # Evaluation criteria
    criteria=[
        "task_completion",
        "checkout_friction",
        "trust_credibility",
        "navigation_clarity",
        "dark_patterns",
    ],

    # Viewport
    viewport="desktop",  # 'mobile' | 'desktop' | 'tablet'

    # Authentication (optional)
    auth={
        "method": "credentials",
        "login_url": "http://localhost:8000/login",
        "username_env": "TEST_USERNAME",
        "password_env": "TEST_PASSWORD",
        "success_url": "http://localhost:8000/dashboard",
    },
)

Parameter	Type	Description
url	str	URL of the page to test
task	str	Natural language task for agents to complete
sessions	int	Number of simulated user sessions (default: 100)
model	str	AI model: `'claude-sonnet'` \| `'claude-opus'`
personas	str \| list	`'auto-diverse'` or a list of persona dicts
criteria	list[str]	Evaluation metrics to score
viewport	str	`'mobile'` \| `'desktop'` \| `'tablet'`
auth	dict	Authentication configuration (optional)

Results Object

The test() call returns a TestResult object with the following shape:

# Results object structure (pseudo-typed for reference)

results.test_id          # str — unique test identifier
results.status           # str — 'completed' | 'failed'

results.summary          # dict
  .overall_score         # float — 0–10
  .sessions_completed    # int
  .sessions_failed       # int
  .top_issue             # str
  .scores                # dict[str, float] — per-criteria scores

results.sessions         # list[SessionResult]
  [i].session_id         # str
  [i].persona            # dict — name, age, tech_savviness, description
  [i].scores             # dict[str, float]
  [i].completed          # bool
  [i].thinking_trace     # list[dict] — step, thought, action, url, timestamp
  [i].duration_ms        # int

Pytest Integration

Use session-scoped fixtures to run the test once and share results across multiple assertions. This avoids running expensive simulations multiple times per test suite.

import os
import pytest
from simutest import SimuTest

@pytest.fixture(scope="session")
def simutest_client():
    return SimuTest(api_key=os.environ["SIMUTEST_API_KEY"])


@pytest.fixture(scope="session")
def checkout_results(simutest_client):
    return simutest_client.test(
        url=os.environ["PREVIEW_URL"],
        task="Complete a purchase with the default cart items",
        sessions=100,
        criteria=["task_completion", "checkout_friction", "dark_patterns"],
    )


def test_overall_ux_score(checkout_results):
    assert checkout_results.summary["overall_score"] >= 7.0


def test_no_dark_patterns(checkout_results):
    assert checkout_results.summary["scores"].get("dark_patterns", 10) >= 8.0


def test_task_completion_rate(checkout_results):
    completion_rate = (
        checkout_results.summary["sessions_completed"] / 100
    )
    assert completion_rate >= 0.85, f"Completion rate too low: {completion_rate:.0%}"


def test_checkout_friction(checkout_results):
    assert checkout_results.summary["scores"]["checkout_friction"] >= 6.5

Async Support

Use AsyncSimuTest to run tests concurrently with asyncio.gather(). This is useful when comparing multiple variants or viewports in a single test run:

import asyncio
import os
from simutest import AsyncSimuTest

async def run_ux_tests():
    client = AsyncSimuTest(api_key=os.environ["SIMUTEST_API_KEY"])

    # Run multiple tests concurrently
    mobile_task, desktop_task = await asyncio.gather(
        client.test(
            url="http://localhost:3000",
            task="Sign up for a free trial",
            sessions=50,
            viewport="mobile",
        ),
        client.test(
            url="http://localhost:3000",
            task="Sign up for a free trial",
            sessions=50,
            viewport="desktop",
        ),
    )

    print("Mobile score:", mobile_task.summary["overall_score"])
    print("Desktop score:", desktop_task.summary["overall_score"])

asyncio.run(run_ux_tests())

On this page