Rails autoscaling on CPU and response time, not guesswork

Most autoscaling advice assumes you’ve already got a metrics pipeline, a dashboard, and an opinion about what “high load” means for your specific app. Most small Rails apps have none of that, which is usually why autoscaling gets skipped entirely in favor of just running a bigger server than needed, all the time, to be safe.

The simpler version: set a minimum and a maximum, and let the two numbers that actually correlate with a struggling Rails process — CPU and p95 response time — decide what happens between them.

How it actually decides

App → Settings → General → Autoscaling takes a minimum and maximum replica count. Railyard adds or removes web replicas between those two bounds based on:

• Target CPU per replica (%) — the average CPU a replica is allowed to sit at before another one gets added.
• p95 response time — the 95th-percentile response time target; a Rails process queuing requests behind a slow query or a blocked thread shows up here before it shows up as an outage.

Leave Max blank and nothing reactive happens — the app just runs a fixed replica count, which is the right default for anything that doesn’t actually see variable load.

One real constraint worth knowing going in: replicas share the app’s own server, so this helps with CPU-bound or slow-request load specifically, not a server that’s genuinely out of capacity — and it never scales up while the server’s memory is already high, since adding a replica to a memory-starved box makes things worse, not better.

PATCH /api/applications/:id/autoscale
{ "min": 1, "max": 4, "cpu": 70, "p95_ms": 400 }

Pre-warming for a spike you can actually predict

Reactive scaling is good at responding to load; it’s not good at responding to load that’s about to happen in the next sixty seconds, because by the time CPU is elevated enough to trigger a scale-up, the spike has already started hurting response times. Scheduled windows solve the specific case where you already know when the spike is coming — a product launch at a fixed time, a daily traffic pattern, a weekly batch job that always runs at the same hour.

mon,tue,wed,thu,fri 08:00-18:00 3

Days, a start-end range, and a minimum replica count, in the app’s own time zone. The window only raises the floor — the usual CPU/p95 rules still decide everything above that minimum, so a bigger-than-expected spike inside the window still scales further. It pre-warms; it doesn’t replace reactive scaling.

What this replaces

The alternative most teams actually run today is a fixed, oversized server count chosen once and rarely revisited, because building a real autoscaling signal from scratch is a project in itself. Two numbers that already correlate with Rails-specific strain — CPU and p95 — turn out to be enough to automate the common case without needing a metrics stack bolted on first.

Full reference, including how this interacts with automatic failover: Autoscaling and failover.