ablohDocumentation
Documentation
Set up.
Two ways in. Both end at the same setup pull request, and merging it turns the check on.
From the app
- 01Sign in at app.abloh.dev and install the GitHub App on your repository.
- 02abloh reads your repository, drafts the setup, and shows you what it could not settle. Answer what is open and the setup pull request opens. Your CI runs the setup trial on it.
- 03Merge it when the trial is green.
From your terminal
- 01Run npx -y @abloh/cli init from your repository’s root.
- 02It opens the setup pull request. Your CI runs the setup trial on it.
- 03Install the GitHub App from the link in the pull request, and merge it when the trial is green.
The check appears on every pull request after the merge, and nothing gates anyone before it. What the setup pull request contains is below.
The CLI.
Or run every command through npx -y @abloh/cli <command> with nothing installed. Node 20.6 or newer.
Opens the setup pull request from your repository's root. --road github|local|both skips the question; --answers-file answers the rest from a file.
Measures a diff on your machine and writes the report to disk. Uploads nothing.
Your stored local runs, newest first. The last 20 per repository are kept.
Sign this machine in, so local AI steps use abloh's model and your plan's usage.
Usage spent this period, and when it resets.
Exit codes are boring on purpose: 0 for success, 2 for a usage error, and 1 for a draft nobody confirmed or a run that is enforcing with a bad gate. The verdict lives in the artifacts and the check, never in the exit status.
What the setup pull request contains
Whichever door you came through, abloh reads your CI, picks the job your tests already run in (and the matrix leg, when that job fans out), and drafts two things: abloh.yml, the config file, and a one-step diff that puts abloh into that job right after your suite — the action pinned to a full commit SHA, GitHub OIDC in place of any long-lived token. From the terminal, init asks only what your repository could not settle by itself, and where the check should live: GitHub pull requests, locally, or both. It pushes the abloh/setup branch with your own git.
Your own CI runs the five-stage setup trial on that pull request: borrow your build, wire into your runner, your suite on untouched code, one planted change, and a publication dry run that sends nothing. When your CI has no job it can ride, the diff is a job of its own with the setup script .abloh/setup.sh.
The config file.
abloh.yml is read from the pull request’s merge base, so changing it is reviewed like code. Setup writes it with a comment under each line saying where the value came from. Every key:
Share of mechanical breaks in changed lines your tests must catch. Default 70.
true fails the check below threshold. Blocks merges only where you mark the check required. Default false.
The package to measure in a monorepo. --subdir overrides. Default: picked from the diff.
The job in your CI that abloh's step rides, as .github/workflows/<file>.yml::<job>. Absent when there is one obvious test job.
The one matrix leg the step runs on, one value per dimension. Every dimension or none.
How the suite runs, from scripts.test. Default: detected at run time.
The runner invoked directly, skipping the non-test parts of scripts.test. Default: abloh invokes the runner.
Untracked paths your tooling writes during a run. Undeclared ones abort the run. Tracked paths cannot go here.
Committed files your own install rewrites. Abloh measures them as your install left them and says which changed. Not a test file, and not a file the change edits.
Variable NAMES your run needs, never values, and never from this file. Setup reads them from your workflow. A local run takes the value from your shell at build time: it enters no image layer and no docker history. abloh keeps no value of yours: a hosted run that needs one refuses with a sentence naming the variable.
The recorded API responses your tests replay: a file you produce with abloh run --record-network and commit like a fixture.
Files whose change forces a reinstall instead of container reuse. Default: the digested set.
chromium, for a suite that drives a browser. Default: none.
Services your tests reach on localhost during a measurement, started beside the suite. Default: none.
install lets abloh stage @vitest/coverage-v8, pinned to your vitest version, into its own cache in the job that measures - nothing is written into your repository. skip turns it off and the changed-line check reports itself as declined. Only vitest repositories are asked. Default: install.
Digest-pinned image proposed tests are proved in, name@sha256:digest. Written by setup. Default: none, no proposed tests.
Runtimes added to that image beyond node, bun or deno. Default: none.
The shell script that builds your project when abloh runs its own job or a local run: .abloh/setup.sh, yours to edit. default: none, findings only, no proposed tests.
Where that script runs. Default: repo root.
The older way to spell one install step, before environment.setup. Still read, and the script wins where both exist. A pull-request check borrows what your own workflow built and needs no install command. A local run needs one to prove a proposed test, or a setup script.
Rules your code must always keep, one line each. AI-planted bugs target the ones a reader could break by accident.
Abloh measures inside the environment your own CI already builds. Your workflow installs and builds exactly as it does for your own test job, and abloh copies that prepared tree into a sealed image. Every mutant and every generated test runs inside it, with no network and none of your job's environment.
That means reproducibility is as stable as your own CI is, and no more. Two runs at the same commit on the same runner image, with the same lockfiles and the same tool versions, execute against the same bytes and give the same answer. Abloh records the fingerprint of what it inherited so you can tell when that stopped being true.
When your CI environment changes underneath you, abloh does not hide it. The run names the field that moved, and any verdict carried forward from before the change is re-earned rather than reused.
Abloh does not claim to reconstruct your environment from a recipe, and it does not claim your dependency tree was frozen while your suite ran. It claims something narrower and more useful. What it measured is what your own CI would have run.
Private registries
If your .npmrc points a scope at a private registry, your install needs a token. abloh init reads that token’s name out of your own workflow and writes it into environment.requiredVariables, with the workflow line beside it as the receipt. The value is never written into the file.
A pull-request check needs nothing further. It borrows the tree your own workflow already installed with your own token, so no credential of yours reaches abloh at all.
A local run installs for itself, so it needs a value. Set the variable in the shell you run abloh in and it is handed to the image build as a build secret: it reaches the install, and it enters no image layer and no docker history. The token stays on your machine.
abloh stores no registry credential of yours, anywhere. That is why a run on abloh’s own machines has no value for one, and when such a run needs a token it refuses, names the variable, and points you at connecting the check. npm, pnpm, yarn and bun are covered, and any registry your .npmrc or .yarnrc.yml names, including GitHub Packages, Artifactory, Nexus and Verdaccio.
Run it locally.
The same measurement, on your machine: it diffs base to head, plants its bugs and writes the full report to disk. A dirty working tree is refused; add --uncommitted to measure the working tree. No check is opened and nothing is uploaded, on any plan.
The range measured. base is required; head defaults to HEAD.
Measure the working tree instead of the last commit. Untracked files are still refused.
One package in a monorepo. Without it, auto-selected when every changed file sits in one package.
Which abloh.yml (your config file) to read. Defaults to your repository's root.
Raw score only: no AI review of survivors, no model called.
Measure and review, but propose no test fixes.
Capture what your tests' outbound HTTP calls answered, into <file>, for later runs to replay with the network off. This run REACHES THE NETWORK and is not a sealed measurement.
abloh run --help lists the rest: seeded replays, triage concurrency, time budgets.
Coverage and classic mutation need nothing. The AI steps need a model: signed in with abloh login they use abloh’s model and draw on your plan’s usage; with MODEL_API_KEY set they use your own model and draw nothing, signed in or not. abloh speaks the OpenAI chat-completions protocol (OpenAI, Azure OpenAI, compatible gateways); an API that speaks only its own native protocol is not reached. The three variables are read from the process environment only, never from abloh.yml, which is committed, and never from a .env. With none set the run still finishes, and each skipped step names the variable that turns it on.
Your provider’s key. Alone, calls go to OpenAI.
Optional — provider URL (needed off plain OpenAI: Azure, gateway, local).
Optional — bearer / api-key (only if your gateway rejects the default).
Tests that call an API
abloh measures with the network off, so a test that calls a real API cannot run inside a measurement. Record what the API answered where the network is allowed, then replay it from the file.
That run reaches the network and is not a sealed measurement; it says so in its output. It writes every outbound request and response into the file with the credentials it can name stripped: authorization, cookie and api-key headers, published token shapes (JWTs, GitHub, Slack, Stripe, AWS, Google) and any field whose name says it holds a secret. Read the file before you commit it; the scrubber removes only what it can name.
Commit it and point environment.recordings at it. Every run after that replays those calls from the file with no network, identically every time. A call the file does not hold fails, named recording-missing, with the request to record.
HTTP only, every client from fetch to jsdom’s XMLHttpRequest; a WebSocket is not recorded. Matching is on the method, the URL and a hash of the body; header noise, query order, the default port and the fragment are ignored. A recording holds what the API said on the day you captured it: re-record when the API changes. The file is byte-stable, so an unchanged API produces no diff and a changed one shows exactly what moved.
What leaves your CI.
Only the source lines mutation testing modified. Nothing else.
Your code and tests run only on your GitHub Actions runner. abloh never clones your repository.
contents: read and id-token: write. No write access.
One, ABLOH_MODEL_TOKEN, in your repository. GitHub hands it to the step whose own line names it and to no other step. It buys model calls for that repository, reads nothing, and you rotate or revoke it from the API keys page.
Mutations stay in the runner’s working tree. abloh cannot push commits, branches or tags.
Runs in an unprivileged container, network off, repository mounted read-only.
What abloh sends, and when
abloh run uploads nothing: no check, no result, no diff, no file name. Its AI steps send the changed code they reason about to abloh’s model when you are signed in, to your own provider when MODEL_API_KEY is set, and nowhere when neither: signed out with no key, nothing leaves the machine.
abloh init sends one thing: one anonymous event when it finishes, saying which step ended the run, the refusal category if any, your package-manager and test-runner family, and the abloh version. It carries no repository name, no path, no account, no command, no environment value and no free text; every field is one of a fixed list of words.
The first init on a machine that sends one says so on its own output, the inits after it do not repeat the line, and abloh init --help still names the switch. Turn it off for one run with --no-telemetry, or for good with ABLOH_NO_TELEMETRY=1; DO_NOT_TRACK=1 is honoured too. Offline, failed or switched off, init behaves in exactly the same way: the send is best effort and never changes or delays what init writes.
The MCP server.
Serves the tests abloh has proved — each executed against the planted fault and failing on it, passing on your code — to any Model Context Protocol client. Three read-only tools; none touches your working tree, and applying a test stays your agent’s act.
Lists the tests abloh has already proved will catch a gap — each was executed against the planted bug and failed on it, passed on your code. One entry per proposal, with the key to fetch it by.
Reads one proposal in full: the body, the file it belongs in, the name it runs under and the gap it closes. Meant to be applied as written — what was proved is that exact test.
The narrow list: only the proven tests somebody on your team read in Slack and deliberately staged for a coding agent, bodies included. An empty answer means nobody has staged anything for that run, not that the run proposed nothing.
Generate a key in workspace settings. Both values below come from there; both files carry your key, so keep them out of commits. Claude Code reads .mcp.json at your repository’s root; Cursor reads the same JSON from .cursor/mcp.json beside it or ~/.cursor/mcp.json for every project:
{
"mcpServers": {
"abloh": {
"command": "npx",
"args": ["-y", "@abloh/mcp"],
"env": { "ABLOH_API_KEY": "…", "ABLOH_ORG_ID": "…" }
}
}
}Codex, in ~/.codex/config.toml:
[mcp_servers.abloh]
command = "npx"
args = ["-y", "@abloh/mcp"]
env = { ABLOH_API_KEY = "…", ABLOH_ORG_ID = "…" }