# We gave cloud agents a phone to QA
*October 14, 2026*
Our cloud agents now test mobile pull requests on cloud simulators and leave screenshots and a recording behind. The agent collects the evidence, and a person decides whether the change is right.
More and more of the code in that app is written by coding agents. Of the 144 pull requests opened against it in the last two weeks, 99 were opened with Claude Code or Cursor. The agents are good at writing the change. What they couldn't do was show it to you.
A mobile diff doesn't tell a reviewer much. To see the change, they have to set up a simulator, build the app (sometimes onto a real device), sign in, and find an incident, alert, or schedule that reaches the new code. That easily takes 30 minutes or more. So mobile PRs tend to sit open, waiting for someone with half an hour to spare.
We set ourselves a goal: a mobile PR should reach its reviewer with proof that the change works. Not a green tick, but screenshots and a recording of the change running in the app, with each claim in the PR linked to the screenshot that shows it. And not against a shared environment, but against a copy of our backend on the agent's own machine, with its own database, our seed data, and any backend changes the PR depends on.
To get there, we gave cloud agents a phone (an iOS simulator or Android emulator in the cloud, which is close enough for an agent) and taught them to do what a QA tester does: read the change, work out what should be different on screen, then tap through the app to check. There's one part of the job we deliberately didn't give them. The agent never decides whether the change is right. It collects the evidence, and a person makes the call.
Two weeks in, the agent has QA'd 42 PRs, and 26 of them have merged. This is the story of how we built it, what it costs, and what's next.
## You can't review a mobile change from a diff
For our web dashboard, this has been solved for a long time. Cursor cloud agents run on a Linux machine with a browser, so when one opens a dashboard PR, it attaches a screenshot or a recording of the change to the description. The reviewer sees the change before they read the code.
Mobile had nothing like it. We post an Expo build link on our PRs, with a QR code that opens the change on your phone, but that only gets you as far as the sign-in screen. You still need an account in an organization with the right data, and then an incident, alert or schedule that takes you down the new code path.
The agents writing those PRs were worse off than the reviewers. The same Linux machine that can screenshot a web page has no phone to run the app on. A mobile agent can type-check, lint and run the unit tests, but it has never seen the screen it just changed. It writes UI blind, then hands it to a person to look at.
So writing a change had got cheap, and checking it hadn't. That was the bottleneck we wanted to solve.
## Doing the same for mobile
We wanted mobile to work like the web: the agent that writes a change should also run it, look at it, and attach what it saw to the PR. A QA run has three parts.
**A Cursor cloud agent with our whole product.** Cursor runs each cloud agent on its own Linux VM, and an agent's environment can check out more than one repo. Ours checks out two: core and mobile-app. Core is our product: a Go monolith with a React frontend. When a cloud agent starts, it creates its own Postgres database, seeds it with incidents, alerts, schedules and users, and runs core against it on port 4001. Metro, React Native's JavaScript bundler, serves the app on port 8081.
That means every PR can test the product end to end, on data nobody else is using. We've even built fake versions of third-party integrations, so an agent can check a change that, say, posts a Slack message into an incident channel without a real Slack workspace. It's the same environment our cloud agents write code in, so an agent can change the backend and the app in one go, then test them together.
**A simulator in the cloud.** On iOS, that's EAS Simulator, Expo's hosted iOS simulator, which is still experimental. On Android, it's an Android emulator. Each agent starts its own and installs the latest development build of our app on it.
**A tunnel between them.** The simulator runs outside the agent's VM, so the two need a way to reach each other. We use local egress: the simulator's outbound traffic to exactly two ports, Metro and core, goes through a tunnel back to the VM. The app loads its JavaScript from Metro and talks to the backend as if both were on its own `localhost`.
The agent drives the simulator with `agent-device`, Callstack's command-line tool for driving iOS and Android devices. Every tap, scroll and screenshot is one command, and the whole session is recorded as a video, which becomes the recording in the report.
We could have pointed the simulator at staging, but a local backend wins twice over:
* **A mobile PR can be tested with its backend change.** When a mobile PR depends on a core PR, the VM checks out that branch of core too, and the agent tests both together.
* **Nothing the agent does touches shared state.** It can resolve alerts and post updates, and nobody else sees them.
There are two ways to start a run, and both follow the same instructions, a `pr-qa` skill checked into the repo:
* **A Cursor agent QAs its own PR before review.** It opens the PR as a draft and runs QA. If QA finds a bug, the agent fixes it, pushes and QAs again, then publishes the evidence into the description.
* **A person adds the `run-qa-agent` label to a PR.** A GitHub workflow starts a Cursor agent through Cursor's API and posts a "QA running" comment, which the report replaces when it's done.
In the first two weeks, agents started 31 of the 43 runs on their own PRs, and people started 12 with the label. The split is moving one way: 17 to 9 in the first week, then 14 to 3 in the second. Of the 26 QA'd PRs that have merged, 20 were QA'd by the agent that wrote them. For five of those, Cursor's own bot did everything: it opened the PR, built the change, QA'd it and attached the evidence. A person looked at the screenshots and merged it.
That's where we think this goes. The label stays useful for PRs a person wrote, but the default should be what our web agents already do: the agent that writes a mobile change QAs it before anyone is asked to review. A person's job is to look at the evidence.
## Teaching the app to be driven
None of this works if the agent can't find the buttons. agent-device reads the app's accessibility tree, the same one a screen reader uses, and when we first pointed it at our app it couldn't target a fifth of the controls. Rows and header buttons had no role, cards merged everything inside them into one element, and text inputs had no identifier.
One PR across 163 files fixed that. Every pressable control now has a role and a testID, icon-only buttons have a label, and containers no longer swallow the controls inside them. A lint rule keeps it that way, and a Claude Code hook runs it whenever an agent finishes a turn, so agents fix their own misses.
Then we made every screen reachable without tapping through Home. Expo Router already turns each screen into a deep link, so we wrote down the query parameters each of the app's 100-plus routes reads, with a test that fails when a screen is missing, and a script that fills in real IDs from our seed data. The agent opens `/alerts/` directly. And it never needs a password: a development build pointed at the local backend signs in as a seeded user, behind a flag that's off in every production build.
## What the reviewer gets
Every run ends with a report in the PR: in the description when an agent QA'd its own PR, or as a comment when someone added the label. A line at the top says which commit was QA'd, whether the simulator's build had the PR's native code, and whether the evidence is complete. Below it are three things.
**Claims, each with screenshots and a link into the video.** The agent turns the PR into claims a reviewer can check on screen, such as "Notes show their text in place on the timeline, on both screens." Each claim gets the screenshots that show it, a note on how the agent got there and what the screenshot shows, and a ▶ timestamp that jumps to that moment in the recording. Anything it couldn't check, like another language or dark mode, goes in a "Not checked" list instead of being skipped quietly.
**The full recording.** The whole simulator session is recorded, so every report links the complete video, along with the device it ran on. When a still isn't enough, the reviewer can watch the agent do it.
**What it cost, and which models ran it.** The end of the report shows the cost of the agent run and the models it used, next to a timeline of how long each step took. Every reviewer can see what each run cost.
The agent collects the evidence, but it doesn't decide whether the change is right. A check in CI marks the PR ready for review only when every claim has a screenshot and there's a recording. From there, it's the reviewer's call.
## Making it fast
Our first runs took far too long, and most of that time wasn't QA. The agent waited for the simulator, fought the pop-ups a development build throws up, and tapped through the app like a new user to find the screen it needed. A few simple ideas fixed most of it:
* **Do the slow part in the background.** The simulator boots while the agent reads the PR and decides what to check.
* **Go straight to the screen.** Every screen in the app has a deep link, so the agent jumps to the screen a PR changed instead of tapping its way there.
* **Tap with agent-device, not computer use.** Cursor agents can also drive a screen with computer use which is slower. We steer the agent to use agent-device only, which navigates screen with fast CLI.
* **Let the platform record.** The cloud simulator already records every session, so the agent doesn't have to.
A typical run now takes about 11 minutes from start to evidence, and 90% finish within 33 minutes. On the alert timelines PR, the longest stretch wasn't waiting at all: the agent spent 11 minutes fixing a label its own first pass had caught, then checked the fix.
## What it costs
The cost in each report comes from Cursor's Admin API, with one catch: Cursor records a single usage event per agent run. When the same agent built the feature and then QA'd it, the cost covers both.
| Run | Runs | Median cost | Range | Median QA time |
| --- | --- | --- | --- | --- |
| QA only (someone added the label) | 8 | $6.92 | $3.31–$31.94 | 14 min |
| Built and QA'd by the same agent | 25 | $17.11 | $2.61–$96.88 | 10 min |
| All runs | 33 | $14.39 | $2.61–$96.88 | 11 min |
The QA-only row is the closest we have to the price of QA on its own: about $7 a PR. The most expensive QA-only run, $31.94, was the upgrade to Expo SDK 57, which had a lot of app to check. At the other end, a Cursor agent built and QA'd a small layout fix, "Sit the incident lead avatar next to the name", from start to finish for $2.61.
Across all 33 runs, the agents spent $800, mostly on Claude Opus 5.5.
Set that against checking a PR by hand: 30 minutes or more of an engineer's time to set up a simulator, build the app and find the right data. A QA-only run takes about 14 minutes, and nobody has to sit and watch it.
## What's next
* **QA by default.** Today a run starts when someone adds the label or a Cursor agent QAs its own PR. We'd like every PR that changes the app to arrive with evidence without anyone asking.
* **Stale evidence.** Evidence is tied to a commit, but a new push doesn't yet mark the old report as out of date.
* **More seed data.** Cover requests, shift handovers and alert groups need seed changes in core before the agent can reach them.
* **QA cost on its own.** Cursor bills a run as a whole, so we can't yet say what QA alone costs when the same agent also wrote the code.
* **Native changes.** A PR that changes native code gets a "Stale native build" note today. Building those PRs on demand would close the gap.
Coding agents moved our bottleneck from writing changes to checking them. Giving them a phone moved most of the checking back to the agents, and left the judgement with people.
If this kind of work sounds interesting, [we're hiring](https://incident.io/careers)!