End-to-end testing for Web, Android and iOS

You can afford to test every release

The careful click-through you’d do by hand runs on real devices while you keep building. Changes you sign off stay signed off, so each release is easier to trust.

Write the test in plain words, or ask your AI tool to. You see the estimated cost before it runs.

NTHCASECHECKOUT-DECLINESPLAN V1 · RUN 1
Open the storefront and dismiss the cookie banner · 1.2sAdd the first product to the basket · 0.8sGo to checkout and fill the delivery form · 2.4sPay with the declined test card · 1.9sAssert the decline message names the reason · 0.6s
NTHCASECHECKOUT-DECLINESPLAN V1 · RUN 2
not run
not run
Open the storefront and dismiss the cookie banner · 1.2sAdd the first product to the basket · 0.8sGo to checkout and fill the delivery form · failedPay with the declined test card · not runAssert the decline message names the reason · not run

It worked yesterday.

same plan, v1yesterday: passed today: step 3 failed

See NthCase in two minutes

Chapters 13

Test from the AI tool you build with.

Connect NthCase to any AI tool that supports MCP, then ask for a test in the chat. Your AI tool writes the test, runs it on a real device and tells you the result, without leaving the chat.

MCP server

46 tools · one API key
https://mcp.nthcase.com/mcpAuthorization: Bearer gp_…
  • The server runs on our side, so there is nothing to install. all 46 tools
  • Your AI tool connects in one of two ways: it signs in, so you copy no key, or it sends your workspace's API key in the header above. Claude signs in, for example; Cursor sends the key. How to connect
  • In an AI tool that supports MCP Apps, trigger_run opens a panel in the chat that follows the run: the device live while it runs, each step as it passes, fails or waits, the credits so far and the verdict. When a device cannot be watched, the panel says why.
  • The estimate_schedule tool tells an agent what a schedule will cost before it spends your credits.
NthCase · Live run
Checkout survives a declined card
Acme Storefront · Pixel 8 · Android 14
Running
Credits spent2 cr
Steps
  1. 1Open the storefront and dismiss the cookie bannerPassed
  2. 2Add the first product to the basketPassed
  3. 3Go to checkout and fill the delivery formPassed
  4. 4Pay with the declined test cardRunning
  5. 5Assert the decline message names the reasonWaiting
Open in NthCase
The live run panel, as it opens in an AI tool that supports MCP Apps.

Your sentences become a plan every run replays.

Write each step the way you would ask a friend to check your app, in the language you work in, or record them on a real device. No code, no selectors, no ids. The AI model you choose turns your steps into a fixed test plan once, and every run replays that plan.

Scenario

Open the Web tab, wait for it to load, scroll to Acme's pricing section and verify the "Growth" plan shows "$29 / mo".

your model
Compiled plan · v1your model
  1. 1tap "Web" tab
  2. 2waitFor page loaded
  3. 3scroll to Acme's pricing
  4. 4assert "Growth" · "$29 / mo"
Replays v1 on every runyou choose the version

Already wrote a spec? Start from it.

The spec or long prompt you gave your AI tool already says what your app should do. Upload it as a .pdf or .docx on your app's page, and NthCase proposes scenarios from it. Each one waits until you approve it; then you compile and run it like any other.

  • Proposing scenarios works from the screen map a Discovery makes first, so your app needs one. A discovery is quoted before it starts.
  • Your app's page then lists each requirement in the document as covered, with the scenarios that cover it, or as a gap.
Acme checkout, the specPDF
  1. 1A declined card says why the payment failed.
  2. 2A guest can check out without an account.
  3. 3The basket keeps its items after sign-in.
  4. 4A promo code lowers the total before payment.
/apps/storefront/suggestions
Proposed scenarios
AI-drafted scenarios for Acme Storefront. Approve one to add it to Scenarios, then compile and run it.
  • Guest checks out without an account
    4 steps
    ApproveDismiss
    1. 1Open the storefront and add the first product to the basket
    2. 2Go to checkout and continue as a guest
    3. 3Fill the delivery form and pay with the test card
    4. 4Assert the confirmation shows an order number
  • Basket keeps its items after sign-in
    5 steps
    ApproveDismiss
  • Promo code lowers the total
    4 steps
    ApproveDismiss

Press record and use your app.

Open your app on a real device from NthCase: a web app in a browser, a phone app on one of our devices or on your own phone. Then press Record and use it. Each tap, typed value, scroll or Back becomes a step, named for you in plain words. No Discovery is needed first.

  • Each recorded step keeps the element you touched as a pin a person made, shown as Pinned via Recorder. A run finds it without asking a model, and a schedule accepts it. A scenario you only recorded compiles without asking a model too.
  • Type into a field with a saved login, and the step records its name, never its value.
  • Recording captures what you do, not what you check. A check, such as “the total shows $24.00”, is a step you write in words: the compile asks a model to place it, and a schedule takes it once a person has confirmed, on a run's page, the element it reads.
  • Recording itself costs no credits; compiling and running what you recorded are charged as any compile and run.
Guest adds a product to the basketscenario
  1. 1Tap 'Accept' on the cookie bannerPinned via Recorderresource-id=com.acme.storefront:id/accept_cookies
  2. 2Tap the featured productPinned via Recorderresource-id=com.acme.storefront:id/featured_product
  3. 3Tap 'Add to basket'Pinned via Recorderresource-id=com.acme.storefront:id/add_to_basket
  4. 4Tap the basketPinned via Recorderaccessibility-id=Basket
  5. 5Check the basket's total shows '$24.00'

5 is written in words

NTHCASEGUEST-ADDS-TO-BASKET● REC
Tap 'Accept' on the cookie bannerTap the featured productTap 'Add to basket'Tap the basket

Run it on our devices or on yours.

Pick the device for each run: one of ours, the phone on your desk, or a device you already rent from BrowserStack. Wherever it runs, it replays the same plan.

  • Our devices

    Emulators, simulators and real phones, with nothing to install. A run's floor is 2 credits on an emulator, a simulator or the web, and 4 on one of our real phones.

    What a run costs
  • The phone on your desk

    Plug it into your computer and start the NthCase agent, nth-agent. Runs reach the phone through a connection the agent opens, so nothing connects in to your machine. A run's floor here is 2 credits.

    Connect your own phone
  • Your BrowserStack account

    New, and not finished yet. Name the devices you already rent, and each one becomes a device in your workspace. Your access key never leaves your machine: NthCase never receives it. A run here is priced as one on your own phone.

    Connect your cloud-lab account
/runs
  • 4e70d5d5
    Checkout survives a declined card
    Pixel 8Android 14 · Emulator
    Passed2 cr
  • 9b31c0a2
    Checkout survives a declined card
    Pixel 7Android 14 · PhysicalYour own device
    Passed2 cr
  • c58e1f47
    Checkout survives a declined card
    Google Pixel 9Android 15.0 · Physical · Cloud lab
    Passed2 cr

When your app changes, the plan stays fixed.

Every run replays the plan you compiled. If a release moves or renames something a step looks for, the engine asks the AI model for a replacement, at most twice for that step, and puts what it found on the run's page. The plan changes only when you say so.

Go to checkout and fill the delivery formfailed
renamed by the release
3Go to checkout and fill the delivery form

This step could not be verified. The AI proposes a selector — review it before pinning.

proposed: text=Continue to paymentwas looking for: the Continue button under the delivery form
Strict
The default, and every scheduled run. The step fails, and the replacement waits for you to confirm it.
Assisted
The run uses the replacement and marks the step self-healed, so you can see it was not checked by a person.
Pin it
On the run's page in the app, pin the selector to the scenario and compile again. Every run after that replays the new plan. Pinning spends no AI tokens; the compile does.

It sees what changed on screen.

Write a step about how a screen should look, and it compiles to a check against a picture a person approved, of the whole screen or of one element. Every run after that compares its screen with that picture, pixel by pixel.

  • One picture per screen: its size, pixel ratio, orientation, and light or dark theme. A run is compared only with the picture approved for its own screen.
  • Until a picture is approved for that screen, the step fails, so the run is never green. The picture the run took waits on the run's page.
  • Approving happens on the run's page, by a signed-in person. It runs no model and costs nothing.
  • The comparison is pixel by pixel, with a small tolerance for anti-aliased edges and colours that barely differ, and it asks no model. It is not the check that asks a vision model what a screen shows.
Approved picture
This runfailed

one screen: 1280×720, 1x, landscape, light

/runs/b7a1e3c9
2Check the product page looks as approvedThis screen has an approved baseline1280×720 px at 1x, landscape, light theme
Step 2 failed

1,906 of 921,600 pixels (0.206814%) differ from the approved baseline, more than the allowed 0.0002% (1 of 921,600 pixels; a pixel counts when its colour differs by more than 0.2 and it is not an anti-aliased edge).

Replace with this picture

Write it once. Reuse it in every test.

A step can run another scenario of the same app, at its place. So any flow is written once, as a scenario of its own, and reused wherever it is needed: sign in, fill the basket, accept the cookie banner, whatever several tests share.

  • The run reads as one list of steps: the reused scenario's own steps stand where that step stood.
  • Change it in one place. When it compiles again, the scenarios that run it are rebuilt with no model call. A rebuild stops, and says why beside that step, when the new steps leave the app on another screen than the steps after them expect.
  • A loop is refused: a scenario can't run itself, directly or through the scenarios it runs.
  • In the builder it is Add a step that runs another scenario. Your AI tool can write one too.
Basket keeps its items after sign-inscenario
  1. 1Add two products to the basket
  2. 2Run scenario Sign inplan v1 · Ready
  3. 3Open the basket and check both products are in it
NTHCASEBASKET-KEEPS-ITEMSRUN 1
Add two products to the basketOpen the sign-in page and type the saved test loginTap Sign in and wait for the account pageOpen the basket and check both products are in it

When you grow, put it in CI and on a schedule.

Check every release on a real device, or check your app every night, with the tests you already wrote.

From your pipeline

The nth CLI starts a run, waits for the verdict and sets an exit code your pipeline can check. It uses your workspace's API key.

Install it with npm install -g @nthcase/cli, or with dotnet tool install -g nthcase.

$ nth trigger 7c2f9a14-5b3e-4d8a-a1f0-3e9b6c2d8f51 --wait
Run 4e70d5d5-8f1c-4a3b-9d62-0b7e5c1a9f34 Passed. 2 credits. https://nthcase.com/runs/4e70d5d5-8f1c-4a3b-9d62-0b7e5c1a9f34
$ echo $?
0

On a timetable

Schedules run your scenarios on a timetable you set in the app, and every scheduled run is Strict. A schedule only takes scenarios whose steps a person has confirmed, by recording them on a device or pinning their selectors.

Checkout survives a declined card, nightly
Every day about 02:04 · UTC · 1 scenario
Passed
est. ~2 cr / cycle
0the run passed
1the run failed
2the run ended in Error: it could not reach a verdict, because the engine or infrastructure broke or the run could not start — not a test verdict
3the timeout ran out before the run finished
4someone cancelled the run
10the command or its arguments are wrong
11the key is missing or the API did not accept it, or its address or version needs changing
12the API could not be reached

Each run ends with a verdict and the evidence behind it.

Every run records each step's result and time, per-step screenshots and the model used. It also keeps the logs, and the network capture when its scenario turns capture on, with sensitive headers and saved secrets masked.

/runs/4e70d5d5Checkout survives a declined card
PIXEL 8 · ANDROID 142 crPassed
StepsLogsNetwork
Assert the decline message names the reason0.6s
  1. 1Open the storefront and dismiss the cookie banner1.2s
  2. 2Add the first product to the basket0.8s
  3. 3Go to checkout and fill the delivery form2.4s
  4. 4Pay with the declined test card1.9s
  5. 5Assert the decline message names the reason0.6s
ModelNo AI — deterministic replay
  • Passed: every step passed, evidence captured, at least one check made.
  • Failed: a step failed, or a passed step has no evidence.
  • No checks: nothing was checked, or no evidence could be captured.
  • Error: the run could not reach a verdict — the engine or infrastructure broke, or the run could not start, e.g. a Strict run of a scenario with steps only AI could find (shown as Precondition not met, the steps listed).
  • Cancelled: someone stopped the run.

No step of this example run asked a model, so its page says “No AI — deterministic replay”. A step can still ask one when its element is not found, as When your app changes shows.

Everything you run is priced in credits. Here are the rules.

A run costs its AI tokens or its floor, whichever is greater.

2 cr floor
per run on our emulators, simulators or web, and on a device you connected yourself.
4 cr floor
per run on our physical devices.
1 credit
includes 2 device-minutes on our emulators or 1 on our physical devices.
1 cr floor
for Compile, Scope and scenario generation, when they spend tokens.
Error
waives the flat fee and the device time. A run that never reaches a verdict pays only its tokens. A run we refuse to start costs nothing. A run that loses its connection to your own device pays as if it had finished.

On your own device, you pay no device time. A run on a device you connected yourself has the 2-credit floor, however long it runs. We charge device time only on our own devices.

A discovery can save tokens on every run. A test compiled after a discovery uses the elements on its map, so its runs do not have to ask the AI model to find them. A test compiled without one can ask the model on every run, and every ask spends tokens.

A discovery is quoted before it starts. It costs exactly that quote when it completes or when you cancel it. One that our AI service stopped early, or one that fails on your own device, costs only what it used, and one the platform fails costs nothing. A run is charged the tokens it used, at 1 credit = 8,000 AI tokens on the reference model; a discovery's ceiling is quoted at 12,500 reserved tokens to the credit, because you pay for the ceiling whether or not the crawl uses it.

You choose the model, and the estimate follows. Each time you compile a test, run it or start a discovery, you pick the AI model, from Claude and OpenAI, and the estimate or quote you see before you confirm updates with your choice. See the models

Run · Checkout survives a declined cardemulator
Modelyour model
Estimate for this run2 cr
Balance1,194 cr
After this run1,192 cr

Here is what each plan will cost.

Plans and credits can't be bought yet. A redemption code gives a workspace a month of a plan, and the plan's weekly credits come with it.

Each plan has a price in dollars a month and a weekly credit budget that resets every seven days. You can spend the budget on any operation, with any model your plan includes.

Starter

$29/mo once on sale

For a solo developer starting with automated tests.

  • Weekly budget60 credits / week
  • Apps2 verified apps
  • Runs1 run at a time
  • Models2 models
  • APINo API access

Pro

$99/mo once on sale

For a team that runs tests on every push.

  • Weekly budget250 credits / week
  • Apps5 verified apps
  • Runs3 runs at a time
  • ModelsEvery model
  • APIMCP server + CLI · API keys

Team

$349/mo once on sale

For growing product teams and agencies.

  • Weekly budget800 credits / week
  • Apps10 verified apps
  • Runs10 runs at a time
  • ModelsEvery model
  • APIMCP server + CLI · API keys
Get started

Credits from a code: a redemption code can also add credits of its own. They sit in a separate balance from the weekly budget. They never expire, never reset, and are kept when your plan changes.

Want to try Pro first? Tell us what you'd test.

Every plan includes members and roles. Before an app can run, you confirm that you own it or that its owner allows you to test it.

Common questions

Each answer links to the page that says more.

What is NthCase?

NthCase runs end-to-end tests of web, Android and iOS apps on real devices. You write each test as steps in plain sentences and choose the AI model that compiles it. You see the cost in credits before anything runs: an estimate for a run, and an exact quote for a discovery.

Do I need to write code to create a test?

No. A test is a scenario: a list of steps written as sentences, in the language you work in, with no selectors and no ids. Compiling turns the steps into a fixed plan once. Every run replays that plan.

Can I test from inside my AI tool?

Yes, from any AI tool that supports the Model Context Protocol. Add the NthCase server to it: a tool that supports sign-in connects by signing in, as Claude does, and any other sends your workspace's API key, as Cursor does. Your AI tool can then write a test, run it on a real device and read the result. In a tool that shows Model Context Protocol apps, a run opens a panel in the chat that follows it.

Which platforms and devices can I test on?

You can test web, Android and iOS apps. A run can use one of our emulators, simulators or physical devices. It can also use a phone plugged into a machine of yours, or a device you already rent from a cloud lab. The cloud lab option is new and not finished yet.

Is my cloud-lab access key shared with NthCase?

No. When your tests run on devices you rent from a cloud lab, the NthCase agent on your machine holds your access key. It adds the key only to what it sends the lab itself, never writes it to disk and never prints it. NthCase never receives it.

How does pricing work, and when am I charged?

Your plan gives you a weekly budget of credits, and it resets every seven days. Everything you run is paid from it in credits. A compile costs its AI tokens, with a floor of 1 credit. A run costs its tokens or its floor, whichever is greater: 2 credits on an emulator, 4 on one of our physical devices. A discovery is quoted before it starts, and a completed discovery costs exactly that quote, or only what it used if our AI service stopped it early.

Which AI models can I use?

Models from Claude and OpenAI. You choose the model each time you compile a test, run it or start a discovery, and the price you see before you confirm follows your choice.

What happens when my app's screens change?

If a step's selector no longer finds its element, the engine asks the AI model for a replacement, at most twice for that step. Strict is the default mode, and every scheduled run uses it. A Strict run fails the step and offers the replacement for you to confirm. An Assisted run accepts the replacement and marks the step self-healed. A heal lasts for that run only. To keep it, pin it on the run's page and compile again, and later runs replay it.

Do I have to run a Discovery first?

No. You can write, compile and run tests without one. A Discovery explores your app and keeps a map of the real elements on its screens. A test compiled after a discovery uses the elements on its map, so its runs do not have to ask the AI model to find them. A test compiled without one can ask the model on every run, and every ask spends tokens. You need one to turn a scope document into scenarios, because that works from the map. A discovery is quoted before it starts.

Can I turn a spec I already wrote into tests?

Yes. Upload it on your app's page as a .pdf or .docx, once a Discovery has mapped the app. NthCase proposes scenarios from it, and each one waits until you approve it. Your app's page then lists each requirement in the document as covered or as a gap. Each run is quoted before it starts, and costs its AI tokens, never more than that quote. A run that fails costs nothing.

Where is my app build stored, and is it safe?

We scan an uploaded build for viruses and store it with Cloudflare, in the European Union. A build with malware never runs anywhere. A build that has not been scanned clean cannot run on our phones. Before a run on our devices installs your build, it removes any earlier copy of your app and its data. A live session removes the build it installed when it ends. We never remove an app that was on a device before we arrived.

Can I use my own devices?

Yes. Run the NthCase agent on the machine your phone is plugged into. It joins your workspace with an agent token. Runs reach the phone through an outbound tunnel that the agent opens, so nothing connects in to your machine. A run there costs its tokens or the 2-credit base floor, whichever is greater, and no device time.

Can I run tests from CI, the CLI or an AI assistant?

Yes. One workspace API key works with the CLI, the REST API and the Model Context Protocol server. nth trigger --wait waits until the run has a verdict, then sets an exit code your pipeline can check. Install the CLI from npm (npx @nthcase/cli) or NuGet (dotnet tool install -g nthcase). An AI assistant can write, run and read tests with that server's 46 tools. An assistant that supports sign-in connects by signing in, as Claude does.

How are my test credentials and secrets handled?

A saved login's secret is write-only. You type it once, and no screen or API call returns it again. It is encrypted with AES-GCM. It is decrypted in one place only: the worker that drives the browser or device, at the moment a step needs it. A step names the login by its label, never by its value. Every secret a run used is masked in its logs. When a scenario turns network capture on, the capture masks sensitive headers, fields and address parameters and the run's saved secrets; the privacy page says what it keeps as sent. Screenshots are not masked, because they are pictures of the screen.

If you can type it, you can test it.