/ docsConceptsllms.txt
On this page
Apps & targetsScenarios & compiled plansRuns & evidenceDiscoveryScopeCredits, budgets & plansModelsSelf-healing & pinningSchedules & cyclesVisual baselines

Concepts

Ten ideas carry the whole product. Each section below states what the thing is and the one rule about it that surprises people, and what it costs where it has a price of its own.

Apps & targets

A target is an app registered in your workspace: an iOS / Android build or a web URL, plus an ownership attestation — you can only test what is yours, and every run, discovery and scope generation is blocked on an un-attested target before any billing happens. Plans hold 2 / 5 / 10 apps, enforced when you add one; a downgrade keeps every existing app usable and only refuses additions.

Scenarios & compiled plans

A scenario is ordered natural-language steps, in whatever language you write in. Compiling produces a deterministic plan. With a map from Discovery, the compile takes the app's real selectors from it; without one, it works from the steps alone. Runs replay the plan, which is why a compiled scenario's runs are cheap (often zero AI tokens) and reproducible. Plans are versioned — you can pin which version replays; the compile pipeline's per-step decisions are readable through the CLI and the MCP server (the app does not show them yet). A step whose element no person has verified — by recording it on a device, or by pinning a selector — stays untrusted, and a schedule refuses a scenario with an untrusted step when the schedule is created or changed.

A scenario Scope proposed is approved and lands as a draft, compiles with the selectors the compiler chose, and runs in Strict to Passed with no AI. 0:30

Runs & evidence

A run executes a scenario on a device and ends in one of five ways. Passed: every step passed, evidence captured, at least one check made. Failed: a step failed, or a passed step has no evidence. No checks: nothing was checked, or no evidence could be captured. Error: the run could not reach a verdict — the engine or infrastructure broke, or the run could not start, e.g. a Strict run of a scenario with steps only AI could find (shown as Precondition not met, the steps listed). Cancelled: someone stopped the run. Only a Passed run is green, and Error is never dressed up as a test verdict (and never billed like one). Every run carries its evidence: per-step results with screenshots, timing, the model used, and logs that arrive already redacted. A run on one of our Android devices whose scenario turns capture on keeps its network capture too, with sensitive headers and saved secrets masked, and every run's Network tab says why when it has none. A run on your own Android phone keeps one too, once the machine it is plugged into has turned capture on and set the phone up: masked there before it is sent, and masked again here. A deterministic replay that used no AI honestly shows exactly that, rather than inventing a model name.

Discovery

Discovery crawls a target screen by screen with an AI agent and produces the screen map future compiles are grounded in, plus proposed scenarios for review. It is priced by the ceiling you choose (Quick · Standard · Thorough), quoted before the crawl starts, charged exactly the quote on completion (or only what it used, up to the quote, when our AI service stopped it early or when it failed on your own device after spending tokens or lost the connection to it) and nothing when the platform fails it. It can be cancelled mid-crawl, and a cancelled discovery is still charged the price it was quoted. One started before discoveries were quoted keeps the contract it started under — the tokens it used, at least 1 credit when it used any — except when the platform fails it: a discovery that fails on our side costs nothing on either branch, and your credit ledger shows it as Not charged, with the reason.

Discovery walks the app with the phone shown live, maps 12 screens, and draws the app's screen map. 0:26

Scope

Scope needs a scope document and a completed Discovery. Generation refuses unless the app has a scope document — a .pdf or .docx, attached as a file or given by its address on the app's page — and a completed Discovery, whose screen map it works from. Pointing NthCase at an app and letting it explore is Discovery, not Scope.

Scope generation reads that document — your requirements, your spec — and derives what is worth testing from it. Discovery learns from the app as built; Scope learns from the app as intended. They meet in the middle: both feed scenarios you review before anything runs.

A test-scope PDF is attached to the app, Generate scenarios from Scope shows its quote, and the scenarios it proposes arrive beside what they cover. 0:21

Credits, budgets & plans

The plan's weekly budget replaces the allowance every seven days — nothing rolls over. Top-up credits are a separate balance that never expires and survives plan changes; cancelling forfeits the remaining allowance and keeps the top-up. A plan change mid-week lifts the allowance only upward. Plans and credits can't be bought yet: a redemption code gives a workspace a month of a plan, whose weekly budget comes with it, and a code that carries credits of its own adds them to the top-up. When that month is over the plan ends, and its allowance is forfeited as a cancel forfeits it; the top-up stays. Plans also carry entitlements checked where they are used: the model list, runs-at-a-time capacity (1 / 3 / 10 in flight, judged on every trigger — past it a new trigger is refused, not queued), and API access for keys (judged on every request).

Models

Seventeen models, Claude and OpenAI, each with a public cost multiplier derived from provider prices, with a floor under the cheapest — the same job priced across the whole ladder, and chosen for each compile, run and discovery. Starter includes two (Claude Haiku 4.5 and GPT-4o mini); Pro and Team can use every model on the list. When you don't choose in the app, the declared default drives — a deliberate pick, not whatever happens to sort cheapest. A command-line, MCP or REST call that names no model gets the AI service's own default instead.

IdModelMultiplier
claude-haiku-4-5Claude Haiku 4.5 (the default)0.35×
claude-sonnet-5Claude Sonnet 50.65×
claude-sonnet-4-6Claude Sonnet 4.61×
claude-opus-4-8Claude Opus 4.81.7×
gpt-4o-miniGPT-4o mini0.15×
gpt-4.1-nanoGPT-4.1 nano0.15×
gpt-4.1-miniGPT-4.1 mini0.15×
gpt-5.6-lunaGPT-5.6 Luna0.15×
o4-minio4-mini0.3×
gpt-4.1GPT-4.10.55×
o3o30.55×
gpt-5.1GPT-5.10.65×
gpt-4oGPT-4o0.7×
gpt-5.6-terraGPT-5.6 Terra0.8×
gpt-5.4GPT-5.41×
gpt-5.5GPT-5.52×
gpt-5.6-solGPT-5.6 Sol2×

Self-healing & pinning

What heals is a step whose selector no longer finds its element: the engine asks for a replacement, at most twice for that step, on the web, Android and iOS alike. What does not: a check of the page's address or title or of whether something exists, a wait with nothing to find, and a step that failed for any other reason — a wrong screen, a closed tab, a device that refused to be driven. On Android and iOS a check whose element is missing fails as a check and is not healed.

What a heal does depends on the run's mode. A Strict run, the default and the mode of every scheduled run, never substitutes: the step fails, and the replacement rides out as a proposal for a person to confirm. An Assisted run accepts the AI's pick and the step passes, marked self-healed, so its green means the pick was accepted, not verified. Either way the heal is for that run only; nothing a run heals changes what later runs replay.

Strict does not start a run that has a step only AI could find. When the compiled plan holds a step's element only as a description, a model would have to find it while the run goes. A Strict run of such a plan ends Error before any device is leased, shown as Precondition not met with those steps listed, and nothing is charged; the run's page offers Run in Assisted beside them. An Assisted run runs it: on a phone the element is found by vision and the step reads Located via vision, and on the web it is resolved live from its description. Where the address found answers to several elements, the step's place in a list or its words pick one, and the step reads Chosen by its place in the list or Chosen by its words; neither is a heal. An AI-judged check — a visual judgement the step's author asked for, decided by a model looking at a screenshot — is not such a step: Strict runs it, and the run marks it AI-judged.

Pinning makes it stick. On the run's page a self-healed step offers Pin to scenario, and a proposal offers Confirm & pin. A step that passed on the address the compiler chose, which a schedule waits on until a person confirms it, offers Confirm this element, which pins the address that run used. Each writes the selector onto the scenario's step, where later compiles keep it and it counts as verified, so a schedule can rely on it. Pinning spends no AI tokens and is not charged; the compile and the runs after it are. It puts the scenario back to Draft, and only a compile puts the pinned selector into the plan: the scenario's page asks for that compile before it runs, and a run started from Re-run on a run's page, the CLI or the MCP replays the old plan until then.

In an Assisted run a step is found by vision, Pin to scenario writes it onto the step, and after the compile a Strict run replays it with no AI. 0:33

A check that read the wrong element. On the web, Android and iOS, when a check finds its element but that element does not hold the value it expects, the failed step offers This isn't the right element, which opens a field to type the selector you meant, and Pick it on the screenshot, which opens the step's own screenshot: click the element there and its selector fills that field. Pin my selector asks before it pins, because a pick pins the element the check reads, not its result. The pin replaces that check's earlier pin rather than adding a second one, and the check keeps its value as your step wrote it. It then waits for a compile, as the pins above do.

A check pinned on purpose to the product's name fails in Strict, the price is picked and pinned, and after the compile a Strict run passes with no AI. 1:02

Every pin says where it came from: Pinned via Web, MCP, API key, CLI or Recorder — a selector recorded on a device is a pin too — with the person who pinned it named on hover. A pin made before the product kept this reads plain Pinned. On a run's page a pin made after that run reads Pinned after this run, because that run did not use it. Unpin, on the scenario's page or the run's, takes a pin off its step after a confirm; the scenario goes back to Draft and needs a recompile.

Only the app pins. Whether a step healed, and to what, reads the same from the command line (nth runs detail) and the MCP server (get_run_detail), and both can start a run in either mode. Neither can pin a run's selector: the MCP tools grouped as Plans & self-heal pins choose which compiled plan version replays, and take a pin off a step (unpin_selector, and nth scenarios unpin on the command line), but they put none on.

Schedules & cycles

A schedule runs one or more scenarios on a cadence, in a timezone. The New schedule drawer offers Hourly, on the hour; Daily or Weekly, at the time you set; and Custom (cron), for a cron expression of your own. The timezone is your browser's unless you choose another, and before you save, the drawer lists the next three occurrences in it. A schedule takes only a Ready scenario whose every element a person recorded, pinned or confirmed, as Self-healing & pinning describes. Creating or changing a schedule with any other scenario is refused, and the drawer lists each action no person has settled, with its reason.

At each time the cadence names, the schedule fires a cycle on its own: one run per scenario, with nobody pressing anything. Those runs are Strict, and each is marked Scheduled on its page and in Runs. The schedule's page lists its cycles and the runs of each.

A schedule is created from two settled scenarios after an unsettled one is refused, fires on its own at 23:27, and both runs pass with no AI. 1:04

The drawer's estimate, per cycle and per month, is advisory: it leaves out device time beyond a run's included minutes, which depends on how long the run takes. Each run is charged as any run is, when it ends (How a run is priced).

Pause stops the cycles the cadence would fire. Resume counts the next one from now: a cycle missed while the schedule was paused is never fired. Run now fires one cycle at once, on a paused schedule too. A schedule pauses itself when the workspace runs out of credits or has no plan, and, when you turn it on, after the number of failed cycles in a row you chose. The same schedules can be read and changed from the CLI, the MCP server and the REST API.

Visual baselines

A step about how a screen looks — a layout, colours, an image, a chart — compiles to a baseline check: it compares a picture of the whole page, or of one element, with a picture a person approved, pixel by pixel. A picture belongs to a screen — its size, pixel ratio, orientation and light or dark theme — and a run is compared only against the picture approved for its own screen, so one step can have one picture per screen. Until a picture is approved for a run's screen, the step reads unverified on that run, and it is never green.

You approve on the run's page, where the picture is. A baseline step there says whether its screen has an approved picture, who approved it and when, and links to See baselines. When no picture is approved for its screen, the run's picture is shown with Approve; when another run's picture is approved for that screen and this step failed, Replace with this picture is offered after a confirm; a step that passed, or whose own picture is the approved one, offers neither. Approving runs no model and is not charged.

A step's picture is approved as its baseline, and the next run compares its screen with it pixel by pixel and finds no pixel that differs. 0:22

Each approved picture records whether it covers the page or one element, which element, the screen it was taken on, the run and step it came from, when, and who approved it through which door. The scenario's page lists them under Baselines, step by step and one per screen, with the picture shown through a short-lived signed address, as a step's screenshot is. A picture approved before the product recorded its screen, or who approved it, says so instead of guessing.

Deleting a baseline cannot be undone. Delete one on the scenario's page, with the MCP server's delete_baseline, or with nth scenarios delete-baseline: the next run of that step on that screen asks for its picture to be approved again. The MCP server and the command line read the list too (list_baselines, nth scenarios baselines). Approving stays a signed-in person's, from the run's page: a workspace key, the command line and the MCP server have no way to approve a picture.