Docs
Get started
DeployAngel compares each release with the last good one and tells you when you can stop watching. Setup takes a few minutes: add the agent, give it a token, and deploy as usual. It works with Ruby, Python, and .NET apps, wherever they run; supported frameworks lists the versions.
1. Install the agent
Create an account and an app, then create its app token. The setup checklist has a button for it, or use Settings, API tokens, "Deployed application". Then add the agent for your framework.
Supported frameworks
| Language | Frameworks | Jobs and schedules | Agent |
|---|---|---|---|
| Ruby 3.1+ | Rails 7.1+, Hanami 2.2+, Sinatra 3.0+ | Every ActiveJob adapter (Solid Queue, Sidekiq, GoodJob, ...) and native Sidekiq jobs; Solid Queue, GoodJob, sidekiq-cron, sidekiq-scheduler, and whenever schedules | the deployangel gem |
| Python 3.10+ | Django 4.2+, FastAPI 0.100+, Starlette 0.27+, Flask 2.3+ | Celery 5.3+ and RQ 1.16+; Celery Beat, django-celery-beat, and django-crontab schedules | the deployangel package |
| .NET 8+ | ASP.NET Core, and worker services | Hangfire 1.8+ jobs and recurring jobs | the DeployAngel NuGet package |
On any of them, work that cron, Heroku Scheduler, or a Kubernetes CronJob starts can be declared too (scheduled work).
Each has a one-page setup guide too: Rails, Django, FastAPI, Flask, and ASP.NET Core.
Rails
Rails 7.1 or later on Ruby 3.1 or later. Add the gem and run bundle install:
# Gemfile gem "deployangel"
There's nothing else to add. It records every ActiveJob adapter (Solid Queue, Sidekiq, GoodJob, ...) and native Sidekiq jobs, and reads Solid Queue, GoodJob (gem 0.1.23 or later), sidekiq-cron, sidekiq-scheduler, and whenever schedules.
Hanami
Hanami 2.2 or later on Ruby 3.1 or later, with gem 0.1.30 or later. Add the gem and run bundle install, then install it where the app is configured:
# Gemfile
gem "deployangel"
# config/app.rb
require "hanami"
require "deployangel/hanami"
module Bookshelf
class App < Hanami::App
DeployAngel::Hanami.install(self)
end
end
It records each request by the route it matched, including slices' routes, and the exceptions Hanami renders as error pages, and starts reporting when the app boots, as config.ru does. Native Sidekiq jobs are recorded with nothing more to add. Hanami answers requests it can't parse with a 500, so a scan during verification can fail a release that didn't cause it: answer them with a 400.
Sinatra
Sinatra 3.0 or later on Ruby 3.1 or later, with gem 0.1.30 or later. Add the gem and run bundle install, then register the extension before the app's routes, so it knows the file each one is in:
require "sinatra/base" require "deployangel/sinatra" class Bookshelf < Sinatra::Base register DeployAngel::Sinatra # routes end
A classic app registers it at the top level, after require "sinatra". It records each request by the route Sinatra ran, with the app's mount point from config.ru, and the exceptions Sinatra renders as error pages. Sinatra answers some requests it can't parse with a 500, so a scan during verification can fail a release that didn't cause it: answer them with a 400.
Django, FastAPI, and Flask
Python 3.10 or later, with Django 4.2, FastAPI 0.100, Starlette 0.27, or Flask 2.3 or later. Add the package to your requirements (it has no dependencies of its own):
pip install deployangel
and one line for your framework:
# Django: settings.py
INSTALLED_APPS = [
# ...
"deployangel.django",
]
# FastAPI, where the app is created (deployangel.starlette.init for Starlette)
import deployangel.fastapi
app = FastAPI()
deployangel.fastapi.init(app)
# Flask, where the app is created
import deployangel.flask
app = Flask(__name__)
deployangel.flask.init_app(app)
Celery is recorded automatically in Django, including Celery Beat schedules in settings or in django-celery-beat. Elsewhere, add deployangel.celery.init(celery_app) where the Celery app is created, so workers report too. For RQ, run workers with rq worker -w deployangel.rq.Worker, or --worker-class deployangel.rq.Worker with django-rq.
The Python agent reports when the framework's debug mode is off, or set DEPLOYANGEL_ENVIRONMENT to name the environment yourself. It reports file paths relative to the working directory, which on Heroku and in most containers is the repository's root; set DEPLOYANGEL_ROOT if your processes start somewhere else. Under uWSGI, turn on enable-threads.
ASP.NET Core
.NET 8 or later, for ASP.NET Core apps and worker services. Add the package (it has no dependencies beyond the .NET and ASP.NET Core shared frameworks):
dotnet add package DeployAngel
and register it where the host is built:
// Program.cs builder.Services.AddDeployAngel();
For Hangfire 1.8 or later, add DeployAngel.Hangfire and call UseDeployAngel() in Hangfire's configuration, in every process that runs Hangfire, so job runs and recurring job schedules report too:
dotnet add package DeployAngel.Hangfire
builder.Services.AddHangfire(config => config
.UseDeployAngel()
.UsePostgreSqlStorage(...));
Keep the debug symbols (.pdb files) that dotnet publish writes next to your assemblies, which it does by default. The agent reads them to tell which source file each exception, route, and job is in, without your source code being deployed or sent. The .NET agent reports when the host's environment is Production (see staging), and reads its settings from environment variables, an optional DeployAngel section in appsettings.json, and AddDeployAngel(options => ...), in that order. The deployangel command is a .NET tool: dotnet tool install --global DeployAngel.Tool.
The token
Then set the token as DEPLOYANGEL_TOKEN in the app's environment, on every process: web servers and job workers. The next section shows how for your host.
What to expect on day one. For an app's first 24 hours of telemetry, DeployAngel learns what's normal for it: which routes and jobs run, and when. Deploys during that time are still checked, and you'll hear right away if one fails, but they can't be cleared, and they end quietly with no "not cleared" alert. The first deploy after that can be cleared. The clock starts when the agent first reports, so installing it a day before a deploy you care about means that deploy can be cleared.
Check the setup. Once it's deployed, run bundle exec deployangel doctor (deployangel doctor for Python and .NET; gem 0.1.24 or Python 0.1.11). It checks your checkout and what DeployAngel has seen so far: which processes report and the release each runs, the routes, jobs, and scheduled jobs the agent found, and any setup problem, such as a worker that isn't running the agent, with what to do about it. The app page shows the same problems, and lists your scheduled jobs as soon as the agent reports. A day after the agent first reports, members who get release alerts receive one email saying what DeployAngel learned in that first day.
Each process sends one small summary a minute: request and job counts, response times, and exceptions with numbers, IDs, emails, and quoted values in their messages replaced. Other words in a message are kept, so if your app's messages might hold personal or health data, turn messages off. It never sends request bodies, cookies, parameters, or records from your database. See the privacy policy for exactly what's collected. If DeployAngel is ever unreachable, your app keeps running normally.
2. Set it up for how you deploy
DeployAngel needs to know which release each process is running. On most hosts, the agent works it out with no setup. When nothing names the release, the agent (gem 0.1.16, Python package 0.1.5, and every .NET package) uses a fingerprint of the app's code, so every app is verified. The commit adds each release's commits and pull requests, and the sections below say how to pass it where the host doesn't.
The fingerprint is the hash of the file digests the agent already sends, reported as a version like code:3f9a2c1d4e5b. The same code gives the same fingerprint, so redeploying unchanged code isn't a new release. Without a commit, a release page shows which files changed from the digests, but not commits or pull requests, and a deploy registered by commit from CI can't be matched to it. The agent logs a warning suggesting DEPLOYANGEL_REVISION. If one release's processes hash different files, such as per-server settings written at deploy time, the dashboard warns, since each process then looks like a separate release.
The sections follow the dashboard's setup tabs: containers, Kamal, Heroku, then Render, Fly.io, and the rest: Render, Fly.io, Railway, Coolify, Dokku, and App Engine, Elastic Beanstalk, Capistrano, and your own servers.
Containers: compose, Swarm, ECS, Kubernetes, Cloud Run
The agent runs inside your app's process, so containers work like any other host. Set DEPLOYANGEL_TOKEN in the container's environment: under environment: in compose, in an
ECS task definition, from a Kubernetes Secret, or with --set-env-vars on Cloud Run. Web and worker containers built from the same image report the same
release, and each container counts as its own instance, merged every minute.
That's enough to be verified, by the fingerprint. Plain Docker doesn't tell the app which commit it's running, and .dockerignore usually leaves .git out of the image, so to see each release's commits and pull requests, bake the commit in when you build. This command adds the two lines to the end of your Dockerfile's last stage, where a value that changes with every commit doesn't invalidate the cache for earlier layers:
bundle exec deployangel install docker # Rails deployangel install docker # Python, .NET # It adds: ARG GIT_SHA ENV DEPLOYANGEL_REVISION=$GIT_SHA # then, when building docker build --build-arg GIT_SHA=$(git rev-parse HEAD) .
On GitHub Actions with docker/build-push-action, pass build-args: GIT_SHA=${{ github.sha }}. A .NET app built in a git checkout carries its commit in its assembly's version, which the agent reads, but not when the Docker build's context leaves out .git.
On ECS, including Fargate, the agent reads the release from the container's image without the build arg: the
image tag when it's a commit or a version like v1.4.2, else the image digest, for a moving tag like
latest. Baking the commit in still works, and shows each release by its commit.
Kamal
Pass the token to the app through Kamal's secrets:
# .kamal/secrets
DEPLOYANGEL_TOKEN=$DEPLOYANGEL_TOKEN
# config/deploy.yml
env:
secret:
- DEPLOYANGEL_TOKEN
Kamal reads $DEPLOYANGEL_TOKEN from the shell that runs kamal deploy, so set it there, and in CI if you deploy from it, or fetch it from your password manager like your other Kamal secrets:
export DEPLOYANGEL_TOKEN=da_live_…
The agent reads the release from KAMAL_VERSION, which Kamal sets in every container. Optionally, to register each deploy and link it to your CI run, add a post-deploy hook:
bundle exec deployangel install kamal # Rails deployangel install kamal # Python, .NET
It writes .kamal/hooks/post-deploy, or, if you already have one, tells you the line to add. The hook needs a "CI deploys" token in DEPLOYANGEL_API_TOKEN wherever you run kamal deploy, and it never fails a deploy. Without it, DeployAngel still notices each new release.
Heroku
The easiest way will be the DeployAngel add-on, which sets the token and signs you in from Heroku. It subscribes to your app's releases too, so you only turn on dyno metadata (the labs:enable lines below). It's in alpha, so installing it needs an invitation: join the Heroku alpha. Without the add-on:
heroku config:set DEPLOYANGEL_TOKEN=da_live_… heroku labs:enable runtime-dyno-metadata heroku labs:enable runtime-dyno-build-metadata
Dyno metadata tells the agent which release it's running, and build metadata which commit (HEROKU_BUILD_COMMIT). Both take effect on your next deploy. Then connect Heroku in the app's Settings, under Integrations, so every release is registered, including config changes and rollbacks.
Render
In the Render dashboard, add DEPLOYANGEL_TOKEN to the web service's environment variables, and to each background worker's, since Render runs workers as separate services. An environment group shared by both sets it once. That's all: the agent reads the release from RENDER_GIT_COMMIT, which Render sets, and DeployAngel notices each new release when the agent reports it.
Fly.io
fly secrets set DEPLOYANGEL_TOKEN=da_live_…
The agent uses the image tag Fly.io gives each deploy as the release. Fly.io doesn't tell the app its commit, so to see each release's commits and pull requests, add it to the image with deployangel install docker and pass it when you deploy:
fly deploy --build-arg GIT_SHA=$(git rev-parse HEAD)
Other platforms: Railway, Coolify, Dokku, App Engine
Add DEPLOYANGEL_TOKEN to the app's environment variables in the platform's settings, for web and worker processes alike. On Railway, Coolify, and Dokku, that's all: the agent reads the commit from
RAILWAY_GIT_COMMIT_SHA on Railway, SOURCE_COMMIT on Coolify, or GIT_REV on Dokku, and
DeployAngel notices each new release. On Railway, a deploy that didn't come from GitHub, such as railway up, is
identified by its deployment ID instead. On DigitalOcean App Platform, also set DEPLOYANGEL_REVISION: ${_self.COMMIT_HASH} in the app spec.
On a platform that doesn't tell the app its commit, such as Google App Engine, write it to a REVISION file in the app's root before each deploy, so it's uploaded with the code:
git rev-parse HEAD > REVISION gcloud app deploy
Or set DEPLOYANGEL_REVISION to the deployed commit, if the platform lets you set it per deploy. A platform you give a container image to, such as Cloud Run, is set up like containers.
Elastic Beanstalk
Add DEPLOYANGEL_TOKEN to the environment properties of each environment that runs the app, web and worker tiers alike:
eb setenv DEPLOYANGEL_TOKEN=da_live_…
Elastic Beanstalk doesn't tell the app which commit it's running. Set DEPLOYANGEL_REVISION in the same
update-environment call that deploys each version, so the two always change together. This works on every
platform, Docker included, with no build args. Then wait for the environment and register the release:
aws elasticbeanstalk update-environment \ --environment-name "$EB_ENV" \ --version-label "$VERSION_LABEL" \ --option-settings "Namespace=aws:elasticbeanstalk:application:environment,OptionName=DEPLOYANGEL_REVISION,Value=$GITHUB_SHA" aws elasticbeanstalk wait environment-updated --environment-names "$EB_ENV" bundle exec deployangel release --commit="$GITHUB_SHA" --version="$VERSION_LABEL" bundle exec deployangel verify --wait --until=initial
In a Python or .NET app, run deployangel without bundle exec. update-environment returns as soon as the deploy starts, so don't skip the wait: registering early starts
the 15-minute clock before the new version is running. If the deploy fails and Elastic
Beanstalk rolls back, the environment still returns to Ready, but the new commit never reports, so verify
exits 2. Leave DEPLOYANGEL_RELEASE_VERSION unset: a release the agent names by version is matched by it,
so it would have to equal --version exactly.
Capistrano
For Rails apps deployed with Capistrano; without it, see the next section. Set DEPLOYANGEL_TOKEN in the app's environment on each server, for example in its .env file or systemd unit. The agent reads the release from the REVISION file Capistrano writes. To register each deploy, add one line to the Capfile:
# Capfile require "deployangel/capistrano"
and set DEPLOYANGEL_API_TOKEN (a "CI deploys" token) wherever you run cap. To make cap wait for the release's first check, and exit with an error if it fails, add set :deployangel_wait, "initial" to config/deploy.rb.
Your own servers: deploy scripts, Fabric, Ansible
For Rails, Python, or .NET apps on servers you manage, deployed by a script, Fabric, or Ansible. Set DEPLOYANGEL_TOKEN in the app's environment on each server, for web and worker processes alike:
DEPLOYANGEL_TOKEN=da_live_…
If the app runs from a git checkout, as with deploys by git pull, Fabric, or Ansible, that's all: the agent reads the commit from .git in the app's directory or up to 3 above it, without running git. Otherwise, write the commit to a REVISION file on each deploy, once the new code is in place and before the app restarts:
git rev-parse HEAD > REVISION
The agent reads REVISION from the directory the app starts in, or from DEPLOYANGEL_ROOT if you set it. Or set DEPLOYANGEL_REVISION to the deployed commit. Without either, the fingerprint still identifies each release.
To register each deploy, so a release that never boots ends as "Not cleared: this release never reported", run this at the end of your deploy script, Fabric task, or Ansible playbook, with a "CI deploys" token in DEPLOYANGEL_API_TOKEN where it runs:
bundle exec deployangel release --commit="$GIT_SHA" # Rails deployangel release --commit="$GIT_SHA" # Python, .NET
3. How deploys are registered
You don't have to do anything: when the agent first reports a new release, DeployAngel registers the deploy and starts verifying it. The releases running when you install the agent are the baseline, so the next deploy gets the first verdict.
Each release is compared with recent traffic from the healthy releases before it: ones that didn't fail and weren't replaced before they could clear. Usually that's the release just before it, over at least the last hour. For a quiet app, the comparison reaches back across several releases, up to 7 days, until there's enough traffic. A release that ended not cleared, such as one that reached the time limit, can still be part of it, and the verdict marks the comparison provisional. The verdict's baseline_quality says which of these happened.
Registering deploys from CI is optional. It labels each release with the CI run, links back to it, and starts verification as soon as the deploy finishes. Create a "CI deploys" token and save it as a secret. On GitHub Actions, add the verify-release action after the deploy step, for Rails, Python, and .NET apps alike. It registers the deploy, waits for the first check (see step 5), puts the verdict on the job's summary, and fails the step when production rejects the release, so a bad release fails the job that shipped it:
# .github/workflows/deploy.yml, after the deploy step
- uses: DeployAngel/verify-release@v1
with:
api-token: ${{ secrets.DEPLOYANGEL_API_TOKEN }}
Waiting holds the runner for about 15 minutes, which private repositories pay for. With GitHub connected, the verdict also shows on the commit, so wait: none registers the deploy without the wait. Elsewhere, or to only register, run deployangel release after deploying, and deployangel verify --wait --until=initial to fail the job on a bad release:
- run: bundle exec deployangel release
env:
DEPLOYANGEL_API_TOKEN: ${{ secrets.DEPLOYANGEL_API_TOKEN }}
The CLI comes with the Ruby gem and the Python package, and as a .NET tool for .NET apps (dotnet tool install --global DeployAngel.Tool). In a Python or .NET project, drop bundle exec and run deployangel directly. In GitHub Actions, GitLab CI, CircleCI, and Buildkite, it fills in the commit, the run, and the link by itself. Elsewhere, pass --commit=$GIT_SHA. Redeploying a commit that's already been deployed isn't a new release, so register it if you want it verified again.
A registered release has to start reporting within 15 minutes of being registered. If no process reports it by then, it ends as "Not cleared: this release never reported", naming the earlier release if that one kept running instead, and deployangel verify exits 2. That catches a release that never booted, such as one whose migration failed. If the release does start reporting within 24 hours, as after a slow rolling deploy, DeployAngel withdraws that verdict and verifies it from then, unless a newer release rolled out first. DeployAngel only notices an unregistered release once the agent reports it, so register deploys from CI if a release that dies on boot is a risk for you, and register when the deploy finishes, not when it starts.
4. Staging
Add staging as an environment of the same app in the dashboard. It gets its own token and its own verdicts, and its notifications are quiet by default. Staging needs nothing extra in Rails or Python: the Rails agent (gem 0.1.17 and later) reports from every Rails environment but development and test, so RAILS_ENV=staging works, and the Python agent reports whenever debug mode is off. The .NET agent reports only from the environments it lists, production by default, named after ASP.NET Core's environment in lowercase (Staging is staging), so list staging:
builder.Services.AddDeployAngel(options =>
{
options.Environments = new[] { "production", "staging" };
});
5. Wait for a verdict in CI or a coding agent
With a "CI deploys" or "CLI & coding agents" token in DEPLOYANGEL_API_TOKEN, the CLI waits for a release's verdict and exits with a code a pipeline can act on:
bundle exec deployangel verify --wait --until=initial
- 0 cleared, 1 failed, 2 not cleared
- 6 no problems in the first 15 minutes, but not cleared yet
- 3 still verifying when the wait timed out
The CLI reference lists every command and exit code.
A busy app usually clears within 15 to 20 minutes, before a slow query meets the evening's data or a route sees its daily peak. So for the rest of the release's first day, DeployAngel keeps comparing latency, app-wide and per route, every few minutes. It compares the last hour with the same hours over the past week, so ordinary peak traffic doesn't read as a slowdown, and it judges the app as a whole route by route, holding each request to its own route's usual latency, so an hour of dashboard views or verdict polls, each as fast as usual, doesn't either, and a slowdown of fast routes beneath a slow page still shows. If the app's p95 rises 1.5x and at least 150 ms, or a route's rises 2x and at least 300 ms, and it holds for 15 minutes, you get a late regression alert (in an hour of fewer than 200 requests, only a large slowdown counts), like a failure. The clearance isn't revoked: a slowdown hours later is as often load or a dependency as the release. The watch ends early when a newer release rolls out. The release page shows this watch and the others still running after a verdict under "Since it cleared": latency, the volume of routes and jobs too quiet to judge at clearance, the scheduled jobs that weren't due yet, and the rarely used paths waiting for first use, each with what it has confirmed so far and when it ends. The release's timeline runs on to when watching ends. If a late regression was a false alarm, anyone on your team can dismiss it on the release page with a reason: it stays listed, but the release counts in later baselines again and later releases stop tracking it.
How a verdict can change
A verdict changes only on new evidence, or on a decision someone on your team records. Each change is in the release's history, with the evidence or the person and their reason. These are all the ways:
- Cleared, then failed: a scheduled job the clearance was still watching misses its run. The clearance is revoked, and you're alerted.
- Not cleared, then verified: a release that never reported starts reporting later, as in a slow rolling deploy, and is verified like any other.
- Replaced before it cleared: the release takes its successor's verdict, since that verdict covers its changes too.
- Failed, then verified again: someone accepts every failing finding as an intended change (a drop made on purpose), and no newer release has rolled out. Undoing the acceptance judges the finding again.
Nothing else changes a verdict: not AI investigation, not your feedback on whether it was right, and not time passing. Late regressions, like a slowdown later in the day or a rarely used path failing on first use, alert you without changing the verdict.
Set up your coding agent
To have a coding agent do the whole setup, not just this step, use "Set up with your coding agent" on a new app's page in DeployAngel. It gives you a prompt with a one-time setup code, not tokens, and points the agent to a step-by-step guide written for agents. The agent redeems the code with deployangel setup --code=…, which saves the app's tokens in the project's .deployangel/credentials.json, kept out of git, without printing them, so they never appear in the agent's conversation. Then it installs the agent, gives production the token, registers deploys, sets up coding agents, and checks it all with deployangel doctor.
Coding agents can wait for the verdict themselves, read why a release failed, confirm a fix worked, pick up what an earlier session left open, and learn what production does with code before changing it, through DeployAngel's MCP server. Its token is read-only, so the agent gets production evidence without database or infrastructure credentials. To set up Claude Code, Cursor, and Codex for a project, run this in it:
bundle exec deployangel install agents # Rails deployangel install agents # Python, .NET
It adds the MCP server to .mcp.json (Claude Code), .cursor/mcp.json (Cursor), and .codex/config.toml (Codex), and the instructions below to AGENTS.md, with a CLAUDE.md that imports it. In a uv or Poetry project it runs deployangel through uv run or poetry run. It never replaces an entry that's already there, and running it again only refreshes its own instructions. Commit the files: they hold no secrets.
The server and CLI need DEPLOYANGEL_API_TOKEN, a "CLI & coding agents" token from the app's Settings, in the environment your agent runs in, for example in an .envrc with direnv. Claude Code asks once before it starts a project's MCP server, and Codex reads .codex/config.toml only in projects you've marked as trusted.
Or install the plugin, which brings the MCP server and the same instructions, as a skill, to every project:
# Claude Code claude plugin marketplace add DeployAngel/agent-plugins claude plugin install deployangel@deployangel # Codex: add the marketplace, then install deployangel from the plugin list codex plugin marketplace add DeployAngel/agent-plugins
The plugin is on GitHub, and in review for the Cursor Marketplace; until it's listed there, use install agents in Cursor. Codex starts MCP servers with a limited environment, so if the plugin's server reports a missing token, add env_vars = ["DEPLOYANGEL_API_TOKEN"] to [mcp_servers.deployangel] in your Codex config.
To add the server by hand instead:
# Claude Code claude mcp add deployangel -- bundle exec deployangel mcp # Rails claude mcp add deployangel -- deployangel mcp # Python (or: -- uvx deployangel mcp), .NET # Codex: in ~/.codex/config.toml (Python, .NET: command = "deployangel", args = ["mcp"]) [mcp_servers.deployangel] command = "bundle" args = ["exec", "deployangel", "mcp"] env_vars = ["DEPLOYANGEL_API_TOKEN", "DEPLOYANGEL_URL"]
Any other MCP client, such as Cursor, VS Code, or Zed, runs the same command. To add the instructions by hand, put this in your agent's instructions, such as AGENTS.md or CLAUDE.md (in a Python or .NET project, drop bundle exec):
## Production verification (DeployAngel)
After deploying, or pushing a commit that deploys, wait for DeployAngel's
verdict: call the `wait_for_verification` MCP tool with the commit and
`until: "initial"`, calling again while it's in progress, or run
`bundle exec deployangel verify --commit=<sha> --wait --until=initial`.
- Exit 0 / verified: the release is cleared. Report the clearance line, what
it says it doesn't cover ("Not assessed", "Not run since the deploy"), and
anything DeployAngel is still watching, then move on.
- Exit 6: no problems so far, but NOT cleared. Report "no problems so far, not
yet cleared" and the expected clearance time. DeployAngel keeps verifying and
alerts on failure.
- Exit 7: warnings at the initial check. Report them and review the findings.
The release is NOT cleared.
- Exit 2 / inconclusive: the release is NOT verified. Do not claim success.
- Exit 1 / failed: read the findings and exceptions (`get_exception`),
investigate the likely cause, and propose a fix. `next_steps` names the last
cleared release a rollback would go to. Do not roll back or change
production without explicit approval.
- A failed release whose failing finding is a volume or checkpoint drop or a
slowdown you made on purpose (a route or job you removed, a page that now
does more work): you can't accept it. Tell the user why you think it was
intended, with your evidence, and give them the finding's `accept_url` so a
person decides. It stays failed until then.
- Exit 3: still verifying; run the command again.
- Not cleared yet: call `get_exercise_plan` (or `bundle exec deployangel plan`).
If its status is "exercisable" or "waiting_for_activity", run
`bundle exec deployangel exercise --url=<production URL>`: it sends the
plan's read-only requests and records them on the release. Offer to
exercise what it skips: routes that change data only with a test account
or after asking. If the status is "warm_up", nothing run can clear it.
- Confidence `medium` clears a release just as `high` does (high also needs 80%
coverage and a full baseline), so don't call a medium clearance weak.
Warnings (`suspicious: true`) aren't failures: report them as "watching
closely:" and the `watching_closely` items, never as "suspicious".
- After deploying a fix for a failed release, read `fix_check` in the verdict
(`--format=json` with the CLI): `fixed` means every earlier problem is
healthy again. Otherwise report what's still happening or not confirmed yet.
- At the start of a session, before changing code: call `list_deployments`
and `list_late_regressions` (or run `bundle exec deployangel status`). If a
recent release failed, isn't cleared yet, or has a late regression, mention
it before other work.
- Before changing code that serves requests or runs as a job: call
`get_production_context` with the files you plan to change (or run
`bundle exec deployangel context <files>`). If an item's clearance outlook
is `rarely_used` or `later_today`, a clearance won't cover your change soon
after deploying: say so, and plan to exercise it or report a check that
covers it. Files in `unmapped_files` aren't known to be unused.
Before changing code
deployangel context, or the get_production_context MCP tool, says what production does with code before you or your agent change it (gem 0.1.44, Python 0.1.30, .NET 0.1.12). For each route and job the files serve, it gives how often production ran it in the last week and how many runs failed, whether a release's clearance would cover a change to it at this time of day, its schedule and critical flows, and failed releases and late regressions on it in the last 90 days. A rarely used route won't be covered by a clearance, so that's when to plan a smoke test or deployangel exercise.
bundle exec deployangel context app/jobs/refund_job.rb app/controllers/refunds_controller.rb # Rails deployangel context shop/views.py --route="POST /refunds" # Python, .NET
Files map to routes and jobs through what the agent reported (Rails conventions, or the files Python and .NET agents send for each route) and through the stack traces of recent exceptions. Models, services, and libraries usually map only through exceptions raised in them, so a file DeployAngel can't map is listed as unmapped, never as unused.
6. Low traffic? Run a smoke test after each deploy
DeployAngel clears a release only once enough of the app has run on it. A quiet app can take hours to get there, or reach the 72-hour limit first and end up "not cleared". DeployAngel's servers never send traffic to your app, but traffic you send from your side, like a smoke test against production, is real traffic, and it counts like any other.
For an app that's normally quiet, a release can clear on as few as 30 requests, if none fail and most of the routes the app normally serves (80% of them) have run at least three times each. So a script that visits your main pages three times each, once the new release is serving, is usually enough. These clearances have Medium confidence, since a few requests can't rule out a small rise in errors. Latency is still checked on those requests: from 30, a large slowdown, with a fifth or more of requests over their own route's usual limit, fails the release. Under 200 requests a clearance never claims the release is as fast as before, and its report says what was checked. Every release is still watched for at least 15 minutes.
Not sure what to run? deployangel plan says what stands between a release and clearance, in the rule's numbers, and lists what to exercise: the routes the app normally serves that are short of their runs, routes the release changed that haven't run, and critical flows. Routes that change data are marked, so you can use a test account for them. The release page shows the same list under "To clear sooner", and coding agents get it from the get_exercise_plan MCP tool. Running a plan's items doesn't clear a release by declaration: only the requests themselves count.
In GitHub Actions, give the verify-release action your production URL as exercise-url: when a release isn't cleared at the first check, the step sends the release's plan of read-only requests from the runner and waits again, so a quiet app clears on its own deploy. Routes that change data are never sent.
Exercise a release from your side
deployangel exercise does the read-only part of the plan for you, from wherever it runs: your machine, CI, or a coding agent's shell. It requests the plan's GET routes that have no path parameters, spreads any request shortfall across them, and then records what it sent on the release, which lists it under "Exercised from your side", so scripted requests are never mistaken for users. It skips and names routes that change data or need a path parameter, for a smoke test with a test account, and stops requesting a page after its first 404, sending its share to the pages that answered. It sends at most 200 requests, about 5 a second, with the user agent DeployAngel-Exercise, and doesn't follow redirects. A "CLI & coding agents" token can run it.
bundle exec deployangel exercise --url=https://example.com # Rails deployangel exercise --url=https://example.com --dry-run # Python, .NET: show what it would send
# .github/workflows/deploy.yml, after the deploy step
- name: Smoke test production
id: smoke
run: ./bin/smoke-test https://www.example.com # your own script
continue-on-error: true
- name: Report it to DeployAngel
run: bundle exec deployangel check --name="smoke test" --status=${{ steps.smoke.outcome == 'success' && 'pass' || 'fail' }}
env:
DEPLOYANGEL_API_TOKEN: ${{ secrets.DEPLOYANGEL_API_TOKEN }}
Reporting the result is optional, since the requests count either way, but it helps. A failing check fails the release at once, because your own test caught it. A passing one lists the test in the clearance report, and --covers=sign-in,checkout marks the critical flows with those names, which you set up in the app's Settings, as checked for the release. Reporting needs a "CI deploys" token. In CI, the CLI finds the release from the commit.
- Visit what your users normally use, not just the home page: DeployAngel counts the routes that are normally active.
- Start the test once the new release is serving. Requests to the old release don't count toward the new one.
- Side effects are yours to manage: use a test account, and avoid steps that send real email or take payments.
- A new app's first 24 hours are a warm-up while DeployAngel learns what's normal. Releases deployed then can't clear, smoke test or not.
7. Webhooks, PagerDuty, Honeybadger, and AppSignal
Besides email and Slack, DeployAngel can post each verdict to your own URL, for tools like incident.io or Teams, or a service of your own. PagerDuty has its own channel. Add one in the app's Settings, under Notifications, and choose its events like any other channel. DeployAngel shows the webhook's signing secret once, when you add it. If the secret leaks or you lose it, use the webhook's rotate button (↻, "Rotate signing secret") there: the old secret stops working at once, and the new one is shown once.
Each event is a POST with a JSON body:
{
"schema_version": 1,
"id": "delivery_4821",
"event": "failed",
"drill": false,
"sent_at": "2026-10-02T14:03:00Z",
"app": { "id": 12, "name": "shop", "environment": "production" },
"text": "Release v185 failed verification (shop)\nHTTP 5xx rate: 0.2% -> 6.8% …",
"verdict": { "schema_version": 1, "deployment": { … }, "verification": { … }, "findings": [ … ] }
}
eventisfailed,cleared,inconclusive,revoked(a cleared release that later failed),late_regression,drill(a fire drill),job_missed,job_failed,job_stuck, orjob_recovered(a scheduled job between releases),error_new(errors the app never raised before),volume_droppedorvolume_recovered(a route or job class that dropped between releases, with avolumeobject in place of a verdict),telemetry_stoppedortelemetry_resumed(the agent stopped reporting from every process, with atelemetryobject in place of a verdict),error_spikedorerror_spike_ended(an error the app already knows that spiked between releases, with anerror_spikeobject in place of a verdict),acceptedoracceptance_withdrawn(a failure someone accepted as an intended change, or took that back),drop_reminder(a drop that has held the releases after it for a day, waiting on someone to accept or fix it; sent once more a week later, then the app page keeps showing it), ortestfrom the "Send test" button. Revoked releases, late regressions, fire drills, job alerts, new errors, error spikes, silences, acceptances, and reminders go wherever failures go.drillistrueonly for a fire drill. Its event isdrill, neverfailed, so anything that acts onfailed, like paging someone, ignores it.textis the same summary Slack gets, ready to forward to a chat tool.verdictis the same documentdeployangel verify --format=jsonand the API return, described in the verdict document. It's empty for test events, job alerts, new errors, and error spikes.jobcomes only with job alerts: the job'skey,name,schedule,problem(missedorfailed),expected_by,last_run_at,opened_at, andresolved_at.errorscomes only with new errors: a list with each error'sfingerprint,exception_class,message,top_frame,first_seen_at, andfirst_seen_release. The fingerprint works withdeployangel exceptionfor the stack trace.error_spikecomes only with error spikes: the error'sfingerprint,exception_class,message, andtop_frame, itsusualcount for that hour and theobservedcount,hour_started_at,opened_at, and, once it's back to usual,settled_hour_started_atandresolved_at.id, also sent as theDeployAngel-Deliveryheader, is the same on every retry of one delivery, so you can ignore repeats.
Every request carries a DeployAngel-Signature header: t= a Unix timestamp, and v1= the HMAC-SHA256 of the timestamp, a dot, and the raw body, keyed with the secret. Refuse requests whose signature doesn't match, or whose timestamp is more than a few minutes old. In Rails:
timestamp, signature = request.headers["DeployAngel-Signature"].scan(/(?:t|v1)=([^,]+)/).flatten
expected = OpenSSL::HMAC.hexdigest("SHA256", ENV["DEPLOYANGEL_WEBHOOK_SECRET"], "#{timestamp}.#{request.raw_post}")
valid = ActiveSupport::SecurityUtils.secure_compare(expected, signature.to_s) &&
Time.at(timestamp.to_i) > 5.minutes.ago
Answer with any 2xx status within 10 seconds. Anything else, including a redirect, counts as a failure, and DeployAngel tries again up to four more times over the next few minutes. Webhook URLs must use HTTPS and a public address: DeployAngel won't send to private or internal networks. Settings lists each channel's recent failed and retrying deliveries, with the error, until a later delivery to that channel goes through.
Honeybadger and AppSignal
If your team already watches Honeybadger or AppSignal, DeployAngel can put each verdict there too. Add it in the app's Settings, under Notifications, and choose its events like any other channel. Each one is set up for a single app environment, so add one per environment.
- Honeybadger: paste the project API key, from the Honeybadger project's settings, and pick its region (US or EU). Each verdict becomes a
deployangel.verdictevent in Honeybadger Insights, withevent,message,app,environment,release,commit,verdict,confidence, andurl(the release's page), so you can chart it, add it to a dashboard, or alert on it. "Send test" sends atestevent, and tells you if Honeybadger refused the key. - AppSignal: paste the front-end API key for the app and environment, from its App settings in AppSignal. It can only send data. Each verdict becomes a marker on the app's graphs: ✅ cleared, ❌ failed, ⚠️ clearance revoked or a late regression, ⏳ not cleared. AppSignal accepts markers without checking the key, so after "Send test", look for the 🧪 marker on the app's graphs.
Both get only the verdict's one-line summary and the release's identifiers, never the findings or AI output. Keys are stored encrypted and shown only by their last four characters. Staging apps send every verdict here by default, since markers are quiet.
PagerDuty
DeployAngel can page your on-call when a release fails. In PagerDuty, add an Events API v2 integration to the service that should be paged, and copy its integration key. In DeployAngel, add a PagerDuty channel in the app's Settings, under Notifications, paste the key, and pick your account's region (US or EU). Use a service's integration key: ruleset and event orchestration keys, which start with R, aren't accepted. Each channel is set up for a single app environment.
- Problems page. A failed release, a revoked clearance, or a late regression triggers an alert at
errorseverity, with the same summary and details Slack gets and links to the release. - One alert per app. A failed release stays failed, so its alert resolves when the next release clears: the fix you deploy. Further problems before then add to the same alert instead of opening new ones. Untick "Cleared" to resolve alerts yourself.
- Clearances and "not cleared" become change events, which show on the service's timeline and next to its incidents without paging anyone. PagerDuty shows change events on plans with Recent Changes.
- New errors become change events too, never a page: one new error rarely needs someone woken up, and a release that fails on it pages as a failure.
- Error spikes page, in an alert of their own for each error, resolved when it's back to usual: a spike is under way, and has an end.
- Fire drills page too, so you can check the alert reaches on-call: at
infoseverity, which is low urgency where the service's urgency follows severity, marked as a fire drill, and in an alert of their own. Resolve it when you're done. - "Send test" triggers an
infoalert. PagerDuty accepts events without checking the key, so look for the test alert on the service, then resolve it.
Staging apps page only for failures by default, and clearances resolve them. The key is stored encrypted and shown only by its last four characters.
Run a fire drill
A new app can go weeks before a release fails. To see what happens when one does, and to check your alerts reach the right places, run a fire drill: on the app's page, or in its Settings, under Fire drill. It needs about an hour of the app's traffic first.
DeployAngel stages a failed release where your busiest route fails for 30% of its requests with a new exception, FireDrill::SimulatedError, and judges it against your app's real traffic with the same rules as every release. Nothing is deployed, no traffic is sent to your app, and none of your real releases, baselines, or statistics change. You get the release page a real failure gets, with how soon after rollout it would have failed and an example of its AI investigation (a drill has no commits for a real one to read), and its alert goes where failures go, marked as a fire drill: email only to you, Slack, PagerDuty (at info severity), and webhooks with "event": "drill". Honeybadger and AppSignal get nothing. A new drill replaces the last one, and drills are removed after 7 days.
Monthly reports for your clients
Each app gets a monthly report as a PDF, to send to the people who own it. It covers the month's releases and how each fared, every problem in plain words with the release that resolved it, upgrades and database changes, scheduled job alerts, and request, error, and job counts compared with last month. A failed release you marked as a false alarm in its verdict feedback is listed as dismissed, with your note, never as a problem.
Download any month from the app page's Monthly reports card, this month so far included. On the 1st, members get last month's reports by email, a PDF for each production app; turn that off in your email settings. Account owners set the name and logo the reports carry under Account, Monthly report branding: a PNG or JPEG up to 250 KB. Until a name is set, reports show none. Monthly reports come with the Starter, Team, and Business plans, and during the beta.
Share a release page
To show someone a release without inviting them, such as a client or a reviewer, open the release in the dashboard and choose "Share". Anyone with the link can see the verdict, findings, coverage, and history, without signing in. It never shows your team, feedback, or anything written by AI, and it doesn't link back into your dashboard. What changed (commits, pull requests, and changed files) is left out unless you include it, since it can come from a private repository. "Stop sharing" turns the link off at once; sharing again makes a new link.
Errors, and alerts for new ones and spikes
Each app's Errors tab lists the errors it raised in the last 21 days, most recently seen first: the exception and its message, the route or job it was raised in most, how often it happened, a bar for each day, and the release it first appeared in. Open one to see it release by release. Each release shows how many times it happened there and its rate per 1,000 requests and job runs, so a busy release and a quiet one compare fairly, along with the release's verdict and whether that verdict flagged the error. The daily chart marks each release, in red when it failed, so you can see which deploy started it. Below are the routes and jobs it came from and a sample stack trace. On a release page, a new or spiking exception links to its error.
The agent counts exceptions each minute rather than sending each one, so an error has counts, where it came from, and one sample stack trace, never the request that raised it.
When your app raises an error it never has before, DeployAngel alerts wherever release problems go: email, Slack, webhooks, and PagerDuty, where it's a change event on the service's timeline that pages nobody. Each error is announced once, and errors that appear together come in one message, with each one's message, when and in which release it first appeared, where it was raised, and how many times so far. It's never a verdict, so no release fails over it. Some new errors stay quiet:
- During an app's first day, while DeployAngel learns what's normal, since every error is new then.
- For a day after an agent update that changes how errors are identified.
- When a failed release or a late regression already told you about it.
An error in a release that's still being verified waits up to 15 minutes for the verdict, so if the release fails on it, you hear about it once, as the failure.
An error your app already raises can suddenly run far more often with no deploy: a dependency that's down, bad data, an expired credential. DeployAngel compares each hour with the error's usual count for that hour of the day, and alerts once when it's at least 10 times usual and at least 30 times, and once more when it's back to usual. A busier hour raises what's usual to match, and an error that has had bursts like it at that hour before isn't news. Errors in their first day are left to the new-error alert, and nothing is judged while a release is being verified, or in an hour with a deploy in it, since a restart raises errors of its own. It's never a verdict, so no release fails over it.
Both alerts are on for production and off for staging. Switch them on the Errors tab, which also marks each error that was announced, and each one spiking now.
8. Connect GitHub
Optional. Verdicts come from production telemetry alone, and the agent already reports which files changed between releases. Connecting GitHub adds what changed in each release: pull requests, dependency and framework upgrades, and migrations. When a release fails, it also ranks the changed files most likely behind it. And each release's verdict shows on its commit, as a check named "DeployAngel (production)", where your whole team already looks.
In the app's Settings, under Integrations, choose "Connect GitHub" and install DeployAngel's GitHub App on the repositories you want. Back in DeployAngel, pick the app's repository. Recent releases are analyzed again, so they get GitHub's view of what changed too. The app can read code and metadata in the repositories you choose. It writes only verdicts: the check, and a comment on pull requests when a release fails. It never changes code, issues, or a pull request's contents.
The check follows the release: queued while DeployAngel waits for the release to report, in progress while it's verified, then green when it clears, red when it fails, and neutral when it ends not cleared, which is never a pass. A clearance revoked later turns it red. Its details have the same summary Slack gets and a link to the release. It needs a commit, so releases registered without one don't get it. Turn it off under Integrations. If GitHub refuses a check because your installation hasn't accepted the Checks permission, Integrations says how to accept it.
The check lands on the deployed commit, so with squash or merge commits it doesn't show on the pull request. So when a release fails, or its clearance is revoked, DeployAngel also comments on each merged pull request the release shipped (the newest 10). The comment names the release, the range it covers, and the other pull requests in it, with the same summary Slack gets and a link to the release. A coding agent that takes instructions from pull request comments can pick it up from there, and the comment gives it the deployangel verify command that returns the findings. Clearances don't comment, since a comment notifies everyone on the pull request and the check already shows them. Turn comments off under Integrations. If GitHub refuses one because your installation hasn't accepted the Pull requests permission (read and write), Integrations says how to accept it.
Releases need their commit: the agent reports it on most hosts (see step 2). If production runs a commit that isn't in the repository, such as one built from a fork, the release page says so instead of guessing.
9. AI investigation
Optional, and off until you turn it on. When a release fails, Claude reads the evidence DeployAngel already gathered and explains the likely cause, the files it suspects, and what to check next. It's labeled as inference and never changes a verdict.
In the app's Settings, under AI investigation, check "Investigate failed releases with AI". Each failed release gets one investigation, up to 5 a day across your account during the beta. If the beta's monthly AI budget runs out, investigations pause until the 1st, and failed releases say so. It sees only the failing findings, new exceptions, and what changed; if GitHub is connected, that includes short excerpts of the few most relevant changed files. See the privacy policy for what's sent to Anthropic.
10. Troubleshooting
Start with deployangel doctor: it names most setup problems and what to do about each.
"The agent doesn't say which release it's running"
The agent is reporting, but couldn't find its release, so nothing it sends can be tied to a deploy. Agents from gem 0.1.16 and Python 0.1.5, and every .NET agent, fall back to a fingerprint, so this means an older agent, file digests turned off (DEPLOYANGEL_FILE_DIGESTS=false), a .NET app deployed without its .pdb files, or a Ruby app outside Rails. Set DEPLOYANGEL_REVISION to the deployed commit, or follow the steps for your host above. On Heroku, enable dyno metadata. If only some processes are affected, it's usually a job worker missing the setting.
The app page says "Agent hasn't reported yet"
Check that DEPLOYANGEL_TOKEN is set on every process, that the app can reach https://api.deployangel.com, and that it's running in an environment the agent reports from (production by default; DEPLOYANGEL_ENABLED=true forces it on). Then check the setup step for its framework (step 1): Rails needs only the gem, while Hanami, Sinatra, Python, and .NET apps each add one line, which is easy to miss.
"Not cleared: this release never reported"
No process reported the registered release within 15 minutes. If the verdict says another release kept running, the rollout failed, was rolled back, or was still going. Otherwise the new version may never have started (check the deploy and boot logs, for example a failed migration). Or the release DeployAngel was told about doesn't match the one the agent reports. Make sure your CI registers the same commit the app runs, for example --commit=$GIT_SHA from the build that was deployed, and registers after the deploy finishes.
A release failed on 500s from requests no route matched
Scanners send requests a framework can't parse: a multipart body with two boundaries, form fields nested hundreds deep, JSON that doesn't parse. Some frameworks answer them with a 500 rather than a 400, sometimes before routing, so the exception shows up on a route like POST unmatched. DeployAngel judges a release by what it answers, so a scan during verification reads as a new exception and a rise in 5xx, and can fail a release that didn't cause it. Muting these exceptions in your error tracker doesn't help, since the response is still a 500. Answer them with a 400, which is also the right response: after that they show as 400s and don't count against a release.
We sent the same malformed requests to each framework, at the versions named below. Django, Flask, and FastAPI already answer them with a 400 or a 422, or parse them leniently, so there's nothing to add. The exception is a Django view that parses JSON itself with json.loads(request.body): catch json.JSONDecodeError and return a 400, or use Django REST framework, which does.
Rails (tested on Rails 8.1 with Rack 3.2) answers almost all of them with a 400. The gap is Rack::MethodOverride, which reads a POST body above Rails' exception handling: a multipart body it can't parse becomes a 500 on any path. Answer those with a 400 just above it. Rack 3.1 or later marks every parse error with Rack::BadRequest; on an older Rack, upgrade it first.
# lib/middleware/reject_malformed_requests.rb
class RejectMalformedRequests
def initialize(app)
@app = app
end
def call(env)
@app.call(env)
rescue Rack::BadRequest
[ 400, { "content-type" => "text/plain" }, [ "Bad Request\n" ] ]
end
end
# config/application.rb, after Bundler.require
require_relative "../lib/middleware/reject_malformed_requests"
# and in the Application class
config.autoload_lib(ignore: %w[assets middleware tasks])
config.middleware.insert_before Rack::MethodOverride, RejectMalformedRequests
Middleware can't be autoloaded, hence the require and lib/middleware in autoload_lib's ignore list. The DeployAngel agent sits above it, so it still counts these requests, as 400s.
Hanami (tested on 2.3) answers all of them with a 500, and multipart and JSON bodies fail before routing. Hanami picks the status for an error by its exact class name, so list the parse errors as bad requests:
# config/app.rb
module MyApp
class App < Hanami::App
config.render_error_responses.merge!(
%w[
Rack::Multipart::BoundaryTooLongError Rack::Multipart::EmptyContentError
Rack::Multipart::MissingInputError Rack::Multipart::MultipartPartLimitError
Rack::Multipart::MultipartTotalPartLimitError Rack::QueryParser::InvalidParameterError
Rack::QueryParser::ParameterTypeError Rack::QueryParser::ParamsTooDeepError
Rack::QueryParser::QueryLimitError Hanami::Middleware::BodyParser::BodyParsingError
].to_h { |name| [ name, :bad_request ] }
)
end
end
Hanami renders its error pages in production, so these get public/400.html if you have one.
Sinatra (tested on 4.2) answers some with a 500, and in a classic app, where method override is on, a bad multipart body escapes to the server. Sinatra reads every request's parameters anyway, and Rack keeps what it parsed, so parse them first in config.ru and answer 400 if Rack can't. It costs nothing extra (Rack 3.1 or later):
# config.ru
require_relative "app"
class RejectMalformedRequests
def initialize(app)
@app = app
end
def call(env)
Rack::Request.new(env).params
rescue Rack::BadRequest
[ 400, { "content-type" => "text/plain" }, [ "Bad Request\n" ] ]
else
@app.call(env)
end
end
use RejectMalformedRequests
run App
ASP.NET Core (tested on .NET 10) answers a bad form with a 400 when it binds it: [FromForm] parameters, an IFormCollection parameter, or MVC model binding. Reading it yourself with Request.ReadFormAsync() or Request.Form throws, and the request becomes a 500. Bind the form instead, or answer those errors with a 400 in Program.cs, after UseExceptionHandler if you use it:
app.Use(async (context, next) =>
{
try
{
await next(context);
}
catch (Exception e) when (e is InvalidDataException or IOException
&& context.Request.HasFormContentType && !context.Response.HasStarted)
{
context.Response.Clear();
context.Response.StatusCode = StatusCodes.Status400BadRequest;
}
});
Nobody gets notification emails
Team emails go only to members who have verified their email address, each to their own inbox, and each member chooses which they get in their profile's Email section: every release alert the app sends, failures only, or none, plus the weekly report and the day-one email. Every email has a link to unsubscribe from it, which works without signing in. If the dashboard shows a reminder to verify, use the link we emailed, or "Send it again". Which verdicts send email is set per app, in its notification settings, where "Send test" checks delivery.
Still stuck? Contact us.
Reference
CLI
The gem and the Python package include the deployangel command, and for .NET it's the .NET tool DeployAngel.Tool, with the same commands, output, and exit codes. In a Python project it's installed with the package (or run it with nothing installed, as uvx deployangel), without bundle exec. In a .NET project, install it once with dotnet tool install --global DeployAngel.Tool, or run it with nothing installed as dotnet tool exec DeployAngel.Tool -- <command>. It doesn't boot Rails, so it starts quickly in CI or a coding agent's shell. It reads its token from DEPLOYANGEL_API_TOKEN. A "CI deploys" token can run every command. A "CLI & coding agents" token can run all of them except release and check. bundle exec deployangel help prints a summary.
| Command | What it does |
|---|---|
release | Registers a deployment so its verification starts right away (step 3). Options: --commit=SHA, --version=V, --kind=code|config|rollback|promotion, --provider=NAME, --url=RUN_URL. In CI, the commit, a build label, and a link to the run are filled in for you. Options you pass win. |
verify | Prints a release's verdict, or waits for one with --wait. Pick the release with --commit=SHA, --version=V, or --deployment=ID. Other options: --until=initial|verdict|closed, --timeout=30m (in seconds, or with m or h), --format=text|json, and --all-findings to include passing findings. |
status | The latest deployment and its verification. Takes --format. |
exception FINGERPRINT | An exception's sanitized message and stack trace (the app's frames, or library frames when it has none). Fingerprints are in the verdict document's exceptions. Takes --format. |
context FILE… | What production does with code before you change it (step 5): the routes and jobs the files serve, how often they ran, whether a clearance would cover a change to them, and their recent failures. Also --route="POST /x" and --job=Job, and --format. Gem 0.1.44, Python 0.1.30, .NET 0.1.12. |
check | Reports a smoke test's result (step 6): --name=NAME --status=pass|fail, plus optional --covers=flow,flow, --details-url=URL, and --commit=SHA or --deployment=ID. |
exercise | Sends a release's exercise plan's read-only requests to production and records them on the release (step 6). --url=PRODUCTION_URL is required; also --commit, --version, or --deployment, --max-requests=N (up to 200), and --dry-run. |
install kamal | Adds a Kamal post-deploy hook that runs release (Kamal). |
install docker | Bakes the commit into the image as DEPLOYANGEL_REVISION, from a GIT_SHA build argument (containers). |
install agents | Sets up Claude Code, Cursor, and Codex in the project: the MCP server in each one's config, and instructions to wait for a verdict after deploying (step 5). |
doctor | Checks the setup (gem 0.1.24 or Python 0.1.11): the agent in the project's dependencies, how production will know the release, and, with a token, what DeployAngel has seen of the app, with any setup problem and what to do about it. Only errors exit 5, so it's safe in CI. It also takes the app's own DEPLOYANGEL_TOKEN, so heroku run bundle exec deployangel doctor shows the release that process reports. Takes --format. |
mcp | Runs the MCP server over stdio, for coding agents (step 5). |
version | Prints the gem's version. |
Without a release named, verify and check use the commit checked out in the current directory, or the CI build's commit. Output is text in a terminal and JSON otherwise, so piping it to jq works; --format overrides that. The JSON is the verdict document.
How long verify --wait waits depends on --until:
initial: until the first check, 15 minutes after the release's first telemetry, or a verdict if one comes sooner. "No problems so far" isn't a clearance.verdict, the default: until the release is cleared, fails, or ends not cleared.closed: after a verdict, also until DeployAngel stops watching for scheduled jobs that weren't due yet, since a missed one turns a clearance into a failure. That can take up to 8 days, so give it a long--timeout.
A failure always returns at once. Without --wait, verify reports where things stand and exits with 3 if the release is still being verified.
| Exit code | Meaning |
|---|---|
| 0 | Cleared |
| 1 | Failed |
| 2 | Not cleared (inconclusive): it ended without enough evidence to clear, for example at the time limit. Never treat it as a pass. |
| 3 | Still being verified: the wait timed out, or --wait wasn't given |
| 4 | No deployment found for that commit, version, or ID |
| 5 | Bad options, a missing or invalid token, or a network error |
| 6 | With --until=initial: no problems in the first 15 minutes, but not cleared yet |
| 7 | With --until=initial: warnings in the first 15 minutes, not cleared yet |
Configuration and checkpoints
The agent is set up with environment variables, and most apps need only the first one.
| Variable | What it does |
|---|---|
DEPLOYANGEL_TOKEN | The app's "Deployed application" token, on every web and worker process. Without it, the agent stays off. |
DEPLOYANGEL_REVISION | The deployed commit. It wins over whatever the agent finds on the host (step 2). |
DEPLOYANGEL_RELEASE_VERSION | Optional. A label for the release, like v185. |
DEPLOYANGEL_ENABLED | true or false turns reporting on or off in any environment. When it isn't set, the Rails agent reports everywhere but development and test, the Python agent whenever debug mode is off, and the .NET agent in the environments it lists (production by default). |
DEPLOYANGEL_ENVIRONMENT | Python and .NET. The environment's name, which Python frameworks don't have. Without it, it's development in debug mode and production otherwise in Python, and ASP.NET Core's environment name in lowercase in .NET. |
DEPLOYANGEL_ROOT | Python and .NET. In Python, the app's root, which file paths are relative to; defaults to the working directory. In .NET, where the app runs, searched for a REVISION file and a git checkout; defaults to the host's content root. |
DEPLOYANGEL_SOURCE_ROOT | .NET only. The repository's root as the build saw it, which file paths are relative to. The package's build step records it in the app's assembly, so you shouldn't need it. |
DEPLOYANGEL_EXCEPTION_MESSAGES | false sends exceptions with their class and where in your code they happened, never a message. Grouping and verdicts work the same. For apps whose messages might hold personal or health data. |
DEPLOYANGEL_FILE_DIGESTS | false stops the agent sending file digests: relative paths and hashes, never file contents. DeployAngel uses them to tell which routes changed in a release. Turning them off also turns off the code fingerprint. |
DEPLOYANGEL_API_TOKEN | For the CLI and MCP server, in CI or on your machine: a "CI deploys" or "CLI & coding agents" token. The app itself doesn't need it. |
DEPLOYANGEL_URL | The API's address, for the agent and the CLI. Defaults to https://api.deployangel.com; you shouldn't need to change it. |
Two settings live in code. In Rails, in an initializer:
# config/initializers/deployangel.rb
DeployAngel.configure do |config|
# Rails environments to report from. Default: production only.
config.environments = %w[production staging]
# Critical flows, by route, job class, or checkpoint.
config.critical_flows = {
"checkout" => [ "POST /orders", "job:OrderConfirmationJob", "checkpoint:order.created" ]
}
end
In Django, in a DEPLOYANGEL dict in settings; elsewhere, with deployangel.configure before init:
# settings.py
DEPLOYANGEL = {
"environments": ["production", "staging"],
"critical_flows": {
"checkout": ["POST /orders/", "job:shop.tasks.send_confirmation", "checkpoint:order.created"],
},
}
# FastAPI, Flask, or a Celery worker
deployangel.configure(environments=["production", "staging"])
In .NET, in AddDeployAngel, or a DeployAngel section in appsettings.json:
// Program.cs
builder.Services.AddDeployAngel(options =>
{
options.Environments = new[] { "production", "staging" };
options.CriticalFlows["checkout"] = new[] { "POST /orders", "job:Shop.Jobs.ConfirmationJob.Send", "checkpoint:order.created" };
});
Critical flows declared here appear next to the ones set up in the app's Settings, and every clearance report says whether each one ran. They're updated each time a process starts. Whether every flow must have run before a release can clear is set in Settings.
Cron, Heroku Scheduler, whenever, and django-crontab
DeployAngel reads the schedules your app's own scheduler keeps (Solid Queue, GoodJob, sidekiq-cron, sidekiq-scheduler, Celery Beat, Hangfire), so it notices when a job misses a run. Every job it finds is listed on the app page, with its last run and next run. GoodJob's cron is read from the processes that run it; one you turn off in GoodJob's dashboard still looks missed after the next deploy, since the agent can't see that switch. Work started from outside the app, by cron, Heroku Scheduler, or a Kubernetes CronJob, needs to be declared, by the name it runs under and its schedule, read in the server's time zone unless you name one:
# Ruby (gem 0.1.22 or later)
config.recurring_jobs = {
"NightlyInvoiceJob" => "0 3 * * *", # a job class, e.g. from rails runner
"rake invoices:send" => "every day at 4am", # a rake task: recorded with no code change
"nightly import" => "30 2 * * *" # DeployAngel.task("nightly import") { ... }
}
# Python, in DEPLOYANGEL or deployangel.configure (package 0.1.10 or later)
"recurring_jobs": {
"shop.tasks.send_receipts": "0 3 * * *", # a Celery or RQ task
"manage.py send_invoices": "every day at 4am", # a management command: recorded with no code change
"nightly import": "30 2 * * *", # with deployangel.task("nightly import"): ...
}
// .NET, in AddDeployAngel(options => ...)
options.RecurringJobs["nightly import"] = "30 2 * * *"; // await DeployAngelAgent.RunTaskAsync("nightly import", async () => { ... });
Code that isn't a job, a rake task, or a management command is recorded when you wrap it in DeployAngel.task (Ruby), deployangel.task (Python, as a with block or a decorator), or DeployAngelAgent.RunTaskAsync (.NET), which records its duration and whether it raised. The whenever gem's config/schedule.rb and django-crontab's CRONJOBS are read automatically: their rake tasks, job runners, and functions are expected on schedule with nothing to declare.
Alerts when a scheduled job misses a run
While DeployAngel verifies a release and watches it afterwards (up to 8 days), a job that misses its run can fail the release or revoke its clearance. Between releases, it keeps watching the jobs your scheduler declares and alerts when one misses a run or its run fails, with no setup. The alert goes wherever release problems go: email, Slack, webhooks, and PagerDuty. You get one message when it happens and one when the job runs again, never repeats in between. It's never a verdict, so no release fails over it, and a job that was already missing before a deploy doesn't fail that deploy either.
A job counts as late once a tenth of its interval has passed since it was due, at least 15 minutes and at most an hour, so a nightly job gets an hour. A failed run waits out the time a retry gets first. A tick due while a deploy restarted your scheduler isn't expected, since schedulers don't make it up. Alerts are on for production and off for staging. Switch them, or mute a single job, on the app page's scheduled jobs card. Jobs DeployAngel only infers from history, without a declared schedule, never alert.
Any background job, scheduled or not, that's still running long after it normally finishes alerts too: after ten times its usual (p95) time over the past week, and at least 5 minutes. It hasn't failed, so nothing else would say so; look for a lock, a call with no timeout, or a loop. You get one more message when no run of it is going that long.
Alerts when a route or job drops between releases
A release's verdict catches a route or job class that collapses after a deploy. Between deploys, DeployAngel compares each hour's normally active routes and regular job classes with their usual volume for that hour, and alerts once when one drops to a fifth of normal, like a partner's webhook that stops arriving on a day nobody deployed, and once more when it's back. It's on for production apps and listed on the app page under "Traffic between releases", where it can be turned off. No release is failed for it.
Alerts when the agent stops reporting
Every agent process sends telemetry each minute, even when the app is idle. If none of a production app's processes has reported for 15 minutes, DeployAngel alerts once, wherever failures go: the app may be down, or the agent can't reach DeployAngel. It says so again when telemetry is back, with how long it was silent. Apps that scale to zero, with hours of silence in the past week, aren't alerted for it.
Checkpoints
Errors and response times don't show work that quietly stops happening. Count the business events that matter with one line each:
# Ruby
DeployAngel.checkpoint("order.created")
DeployAngel.checkpoint("webhook.stripe.processed", count: events.size)
# Python
deployangel.checkpoint("order.created")
deployangel.checkpoint("webhook.stripe.processed", count=len(events))
// .NET
DeployAngelAgent.Checkpoint("order.created");
DeployAngelAgent.Checkpoint("webhook.stripe.processed", events.Count);
DeployAngel learns each checkpoint's normal rate relative to your traffic, and fails a release after which one drops sharply or stops, even if every request and job still succeeds. Only drops count, and a checkpoint without enough traffic never keeps a release from clearing. The call never raises, never touches the network, and does nothing where the agent isn't reporting. Names use letters, numbers, and . _ : -, up to 100 characters. Keep them to a fixed set, with no IDs in them, since only 100 distinct names are counted each minute.
Rails apps get two checkpoints with nothing to add (gem 0.1.40 and later): rails.mail.delivered, counted from Action Mailer, and rails.storage.stored, from Active Storage uploads. Counts only, never a recipient, filename, or key. Turn them off with config.derived_checkpoints = false or DEPLOYANGEL_DERIVED_CHECKPOINTS=false.
No clearance can check results. A release that returns wrong answers without changing error rates, latency, volumes, or exceptions passes every check, and every clearance report says so. A checkpoint on what matters is how to catch that. The app's Settings list the checkpoints it reports and suggest a few for its busiest routes that change data, with a name to start from and the line to add.
One kind of wrong answer needs no checkpoint: a normally busy route that answers far more of its requests with 401, 403, 404, or 422 after a release, as when sign-in or a form's CSRF check breaks, shows as a warning on the release page. It never fails a release or makes it "cleared with warnings", since a client without a token or a crawler guessing ids raises those too.
Routes and jobs need no checkpoint. A normally busy route or job class that gets far less traffic after a release, with no errors, fails it the same way, by the same rule: a webhook that stops arriving, or a job that's no longer enqueued. A drop already under way before the release rolled out, like a partner's outage that began just before the deploy, didn't come from the release, so it's a warning instead. Background jobs are also judged all together, so workers that stop, every job class at once, fail the release too. That count starts 10 minutes after the rollout, since a deploy restarts the workers, and an app with no web traffic, like a worker fed by another system's queue, is judged against its quietest recent days at those hours. Checkpoints are for events inside a route or job, like an order created by one of several requests.
When a drop was intended, a route or job the release removed isn't counted. For any other, like a webhook you switched off, someone on your team can accept the finding as intended on the release page, with a reason. If nothing else failed the release, it's verified again, and the releases after it aren't failed for that change. Coding agents can't accept a finding: the verdict document gives them an accept_url to hand to you.
The notice that a failure was accepted waits five minutes, so undoing a mis-click sends nothing. Anyone on the team can undo an acceptance, until a release that carries the change clears; the drop is then judged again. Until someone decides, every release after the failed one waits on the drop: its page and "not cleared" reason name the release to accept it on, and after a day, the people who got the failure get one reminder.
The verdict document
Everything that reports a verdict gives you the same JSON document: deployangel verify --format=json, the MCP server, the verdict field of each webhook, and the API at GET https://api.deployangel.com/api/v1/deployments/ID/verification (send the token as Authorization: Bearer …). One parser covers all of them.
This is version 1 (schema_version). New fields can appear at any time, so ignore any you don't recognize. New values can also appear in fields like state, verdict_reason, and signal, so decide what to do from verdict, which is only ever one of the four values below. Removing a field or changing what one means will come with a new version. A failed release, shortened:
{
"schema_version": 1,
"deployment": {
"id": 512,
"version": "v185",
"commit": "3f2a9c1e8b7d40f6a5c2e91d0b4f7a3c6e8d1b25",
"kind": "code",
"released_at": "2026-10-02T13:41:00Z",
"first_seen_at": "2026-10-02T13:42:00Z",
"environment": "production",
"provider": "heroku",
"source_url": null,
"promoted_from": null
},
"verification": {
"state": "closed",
"verdict": "failed",
"verdict_reason": "regression_detected",
"confidence": "high",
"coverage": 0.62,
"suspicious": false,
"watching_closely": [],
"baseline_quality": "normal",
"observation_started_at": "2026-10-02T13:42:00Z",
"initial_check": null,
"expected_clearance_at": "2026-10-02T14:20:00Z",
"time_limit_at": "2026-10-05T13:42:00Z",
"watching": false,
"watch_until": null,
"latency_watch_until": "2026-10-03T13:42:00Z",
"verdict_revised_at": null,
"superseded_by": null
},
"summary": "v185 failed verification. HTTP 5xx rate failing on application; Route 5xx rate failing on route:POST /orders. Most affected: POST /orders (31.2% 5xx).",
"findings": [
{
"signal": "http_5xx_rate",
"scope": "application",
"status": "failing",
"test": "binomial_one_sided",
"threshold": "failing: p < 0.001 and >= 2x and >= +0.5 pp; warning: p < 0.01 and >= 1.5x",
"precision": "exact",
"baseline_n": 48210,
"observed_n": 3104,
"p_value": 0.0,
"baseline_value": 0.002,
"observed_value": 0.068
},
…
],
"exceptions": [
{
"fingerprint": "9c41e07b2a6d4f18b3e5c07a91d2f468",
"exception_class": "NoMethodError",
"message": "undefined method 'amount' for nil",
"count": 68,
"new": true,
…
}
],
"clearance": {
"cleared": false,
"cleared_at": null,
"time_to_clearance_seconds": null,
"statement": "v185 failed verification",
"suspected_routes": [
{
"route": "POST /orders",
"observed_5xx_rate": 0.312,
"baseline_5xx_rate": 0.004,
"requests": 218
}
],
…
},
"investigation": null,
"poll_after_seconds": null,
"dashboard_url": "https://app.deployangel.com/…",
…
}
verification
| Field | Meaning |
|---|---|
state | pending: waiting for the release's first telemetry. observing: being verified. waiting_for_activity: outside the app's usual active hours, so the clock is paused. watching: has a verdict, but scheduled jobs that weren't due yet are still being checked. superseded: a newer release rolled out first; this one gets its verdict when the newer one does. closed: done. |
verdict | null until there is one, then verified (cleared), failed, or inconclusive (not cleared). |
verdict_reason | null for an ordinary clearance. Otherwise regression_detected; missing_recurring_job (a clearance revoked while watching); time_limit, warm_up, critical_flows_unverified, no_release_telemetry, or other_release_running (not cleared); or, for a superseded release, cleared_by_successor, superseded, or superseded_by_failed_release. |
confidence | high, medium, or low. A release clears only at high or medium, and medium is a full clearance: high also needs 80% coverage, a full baseline, and exact measurements. |
coverage | The share, from 0 to 1, of the app's normally active routes, job classes, and due scheduled jobs that have run on the release. null during warm-up. |
suspicious | true if any finding is a warning. Warnings don't fail a release, and a release can clear with them; the dashboard says "watching closely". |
watching_closely | Those warnings in words, as the dashboard shows them, such as "p95 latency on GET /orders went from 300 ms to 420 ms". Empty when there are none. |
baseline_quality | What the release is compared against. normal: one healthy earlier release, usually the one just before it. extended: several earlier releases, to get enough traffic. provisional: includes a release that wasn't cleared. fallback: the previous release failed, so an earlier one. sparse: little traffic to compare with. unavailable: nothing to compare with, so the release can't clear. |
initial_check | null until 15 minutes in, then result (no_problems_so_far, warnings, or no_activity_yet), coverage, and recorded_at. It's a milestone, not a verdict. |
expected_clearance_at | An estimate from how long the app's recent releases took to clear, or null without that history. |
time_limit_at | 72 hours after the first telemetry, when a release that hasn't cleared ends not cleared. It can end sooner, after 8 hours of the app's active time. |
watching, watch_until | Whether scheduled jobs are still being checked after the verdict, and until when. |
latency_watch_until | Until when latency is still compared with the same hours of the past week, after the verdict. A slowdown found by then is a late regression. |
verdict_revised_at | When a clearance was revoked, or null. |
superseded_by | The deployment ID of the release that replaced this one. |
deployment
id is what --deployment takes. version and commit identify the release, and either can be null. kind is code, config, rollback, or promotion. Verification is timed from first_seen_at, when the release's first telemetry arrived, not from released_at. source_url links to the CI run or deploy, if known. For a promotion from staging, promoted_from names the staging release and its verdict. That's context only: it never changes this release's verdict.
findings
Each finding is one signal tested on one scope. Only findings that aren't passing are listed, unless you ask for all of them with --all-findings.
| Field | Meaning |
|---|---|
signal | http_5xx_rate, route_5xx_rate, p95_latency, job_failure_rate, job_duration, queue_latency, new_fingerprint, fingerprint_amplification, checkpoint_rate, volume_drop (a route or job class that runs far less than the release before, with no errors), missing_recurring_job, first_use_failure, or external_check (a failed smoke test). |
scope | application, or a kind and a name: route:GET /users/:id, job_class:InvoiceJob, fingerprint:9c41e07b2a6d4f18b3e5c07a91d2f468, checkpoint:order.created, recurring_job:…, check:smoke test. |
status | failing, warning, accepted (a person accepted it as an intended change, so it neither fails nor holds the release), insufficient_data, not_observable, or pass. |
baseline_value, observed_value | Before and after. Rates are from 0 to 1, durations are in milliseconds, and the rest are counts. For checkpoint_rate and volume_drop, before is the count expected for this much traffic. |
baseline_n, observed_n | The sample sizes behind each value. |
test, threshold, p_value, precision | The statistical test used, the rule it was held to (in words), its result, and whether the values are exact or approximate. |
accept_url, accepted_as_intended | On a failed release, a failing volume_drop, checkpoint_rate, or p95_latency finding may be a change made on purpose, such as a retired webhook or a page that now does more work. accept_url is where a person can accept it as intended, with a reason, on the release page; once they have, accepted_as_intended has their name (by), note, and at, and for a slowdown the p95 it was accepted at (value, in milliseconds): slowing much further than that still fails. On a later release that's waiting on the same drop, held_by names the failed release it started in (deployment_id, version, commit), and accept_url points there. The API can't accept a finding, so a coding agent never accepts its own failure. |
clearance and exceptions
clearance is the report you see on the release page: cleared, cleared_at, time_to_clearance_seconds, a one-line statement, and lists of missing_evidence (what it still needs before it can clear), not_observable, and still_watching. For a failed or suspicious release, suspected_routes lists up to three routes with the worst error rates.
For a cleared release, scope says what the clearance doesn't cover, so "cleared" is never read more broadly than its evidence: not_assessed (app-wide signals that never had enough data, with their rule and how much there was), critical_flows_not_run, changed_not_run (rarely used items changed in the release that haven't run since), and within_limits (app-wide latency or job duration that passed because it rose past its warning ratio but not its rise in ms, such as 218 ms to 1,002 ms against a warning at 2× and +1,000 ms). lines has each as a sentence, which the release page and the CLI show under the verdict. It's null unless the release is cleared.
exceptions lists up to 10 exceptions that are new in the release, each with its fingerprint, exception_class, sanitized message, application frame, and count. deployangel exception FINGERPRINT gets the stack trace.
Everything else
summary: one line of text built from the findings. It never comes from AI.drill:trueonly for a fire drill, which webhooks deliver as eventdrill. The API and CLI never return one.source_correlation:"unavailable", or what changed in the release (commits, pull requests, and the changes most relevant to the findings) once release analysis has run.investigation:nullunless AI investigation ran. It's labeled "AI inference, not a verdict", and includeslikely_cause,suspected_files, andrecommended_next_action.critical_flows,late_regressions,rare_items_pending, andcoverage_detail: the detail behind the clearance report.fix_check: whether this release fixed what went wrong before it, ornullwhen nothing did. A failed release stays failed, so its fix is a later release. This lists each problem since the last cleared release, from the releases that failed and the late regressions found in between, looked up in this release's own evidence. Itsstatusisfixed(cleared, and every problem is healthy again),healthy_so_far(still being verified),still_failing,partly_confirmed, ornot_confirmed, with astatementsuch as "v185 fixes all 2 problems from v184: they're healthy again". Each of itsitemshas the earlierrelease, alabel, astatus(healthy,not_seen,not_run,too_little_data,still_elevated,still_failing,seen_again, ornot_judged), and atextline. The release page and the clearance message show it too.poll_after_seconds: 60 while the release is being verified,nullonce it's done. Don't poll more often.dashboard_url: the release's page in DeployAngel.