# We stopped a Railway migration 3 weeks out. How to know to stop or go?

> We took a production Rails app to three weeks before a Railway cutover, then called it off. Here's what Metal, backups, logs, and rollback look like in production, and the questions to answer before you migrate.

- Date: 2026-11-18
- Authors: Ilya Cherepanov, Kirill Kuznetsov, Kirill Yakovenko, Travis Turner
- Categories: Infrastructure
- URL: https://evilmartians.com/chronicles/we-stopped-a-railway-migration-3-weeks-out-how-to-know-to-stop-or-go

---

Railway is cheap, fast to set up, and adding thousands of new users a day, so plenty of teams are sizing it up for production right now. Evil Martians got most of the way through one of those migrations before backing out, and along the way we learned what the platform asks of you once real traffic and a real database are involved. This post shares what we found, so engineers weighing Railway, or any cheap and fast-growing PaaS, can decide with the full cost in view.

Earlier this year, Evil Martians started moving a high-load Rails production off [Engine Yard](https://www.engineyard.com) and onto [Railway](https://railway.com). The runbook was written, the date was on the calendar, and we'd already lowered our DNS TTLs (time-to-live values, so the switch would propagate fast). 

>We called it off three weeks before the switchover. 

That said, **we like Railway, we've put other teams on it, and we'd recommend it again for the right workload**. But we stopped because Railway was halfway through a hardware transition, and we couldn't predict which CPU our database would end up on during the maintenance window: the same MySQL dump had imported in 71 minutes on one Railway service and 382 minutes on another.

Railway sells building blocks rather than a managed stack, and its pricing page prices these blocks, not the work of turning them into something a business can depend on. Below are the five parts of that work we wish someone had priced for us before we started, followed by the questions we now answer before putting anything on a platform like it.

## 1. For us, "Metal" was a hardware lottery

Railway runs services on physical the hosts it owns and operates. They call this [Railway Metal](https://docs.railway.com/platform/railway-metal). The pitch is no noisy neighbors and predictable performance. Our first benchmark said otherwise. The same MySQL dump took 71 minutes to import on one service and 382 minutes on another (in the same week, on the same plan). 

Here are the four import runs we did during the evaluation:

| Service | Region | CPU seen by the container | Dump | Import time |
|---|---|---|---|---|
| mysql-tmp | metal | Intel Xeon Ice Lake, 48-core (2021) | actual | **382 min** |
| mysql-tmp-2 | gcp (virtualized) | — | actual | **71 min** |
| mysql | metal | AMD EPYC 9655P, 96-core (2024) | scrubbed | 97 min |
| mysql-replica | metal | Intel Xeon Gold 5520+, 48-core (2023) | scrubbed | 133 min |

The two actual-dump runs were on different infrastructure, and the two metal-vs-metal runs used a smaller scrubbed dump, so this isn't a clean within-tier benchmark. The spread is still the point.

> The dump, week, and plan: 71 minutes on one service, 382 on another within one plan.

Why? Railway was mid-migration between two hardware generations. Gen 1 Metal ran on Intel Xeons; this is hardware Railway itself now says [wasn't built for the demand mix it sees today](https://blog.railway.com/p/railway-metal-gen-2). Gen 2 is AMD Zen 5c EPYC: 96 cores, 192 threads, DDR5, live in US West, US East, and Amsterdam as of June 2026, with Singapore still to come. The hardware arrives in batches and each site fills over a few weeks, so during the rollout the fleet is a mix of both. In our table, 382 minutes is Gen 1 and 97 is Gen 2.

The 71-minute row is neither. Railway's fleet is growing into demand, and when its own capacity runs short, it bursts onto rented VMs in GCP and AWS. By Railway's own account in June 2026, [40 percent of its workloads were running on cloud burst capacity](https://blog.railway.com/p/railway-metal-gen-2), up from none at the end of 2025. In our runs, that rented GCP VM was the fastest machine we touched, beating both generations of Metal. The share of burst capacity isn't constant either, and it's presumably highest when the platform is busiest. 

So, in any given week, a "metal" service can be on 2021 Xeons, 2024 EPYCs, or a hyperscaler's virtualized hardware that behaves like neither. You can't pick a hardware generation, and [there's no way to opt out of Metal](https://docs.railway.com/platform/railway-metal).

For most web workloads, none of this is visible. For large SQL imports, batch processing, ML inference, or video encoding (anything where the worst case matters more than the average price) it decides whether the platform works for you. It did for us, and we'll come back to that towards the end.

## 2. Managed services exist, but the ceilings are hard

Railway's set of managed services is growing but still small. Today it's [point-in-time recovery (PITR) for PostgreSQL](https://docs.railway.com/volumes/point-in-time-recovery), built on [pgBackRest](https://pgbackrest.org) with continuous WAL (write-ahead log) archiving with a restore window of roughly four weeks, and [one-click HA (high-availability) PostgreSQL on Patroni](https://docs.railway.com/databases/postgresql-ha) with sub-30-second failover. HA shipped in March 2026, and PITR [reached beta in May](https://railway.com/changelog/2026-05-15-railway-for-ios). That's two database features in one quarter, and Railway's [summer update](https://blog.railway.com/p/railway-summer-update-2026) frames Q3 2026 as when the product catches up.

*Read also Evil Martians' article*: https://evilmartians.com/chronicles/martian-kubernetes-kit-a-smooth-sailing-toolkit-from-our-sre-team

We expect that pace to continue. But you're migrating onto today's platform, not next year's, and today, the ceilings are hard:

- **HA PostgreSQL is a template, not a service.** You provision [seven containers](https://railway.com/deploy/postgres-ha-patroni): three etcd nodes, three PostgreSQL nodes, and an HAProxy. Then you pay for seven, patch seven, and monitor seven. Railway handles the failover, and your team handles everything else that can go wrong with seven containers.
- **PITR is PostgreSQL-only.** MySQL, Redis, ClickHouse: you're back to [volume backups](https://docs.railway.com/volumes/backups), and scheduled backups top out at once per day. If your RPO (recovery point objective) is six hours on a non-PostgreSQL database, that's a system you build yourself.
- **Manual backups cap at 50% of volume size.** This part is [documented](https://docs.railway.com/volumes/backups). But what isn't documented: the API started returning errors after the tenth manual backup on a volume, and support was never able to tell us why. We ran into this more than once.
- **3,000 read and 3,000 write IOPS, every volume, every plan, every size.** [Pro self-serves capacity up to 1TB](https://docs.railway.com/volumes/reference), and the IOPS (input/output operations per second) number doesn't move with it. There's no faster tier to upgrade into. If your database wants more, you shard, or you leave.

The IOPS ceiling is the most structural limit on the platform, and almost nobody writes about it.

> **Our advice**: price the backup and recovery path for every non-PostgreSQL datastore before you price the compute. On Railway it's a system you run, not a box you tick.

## 3. Logs are for debugging, not for keeping

Start with what exists (because it's more than Railway's reputation suggests). There's a [filter query language](https://docs.railway.com/observability/logs) (`@key:value`, numeric ranges, Boolean operators), structured JSON logs, and HTTP logs you can filter by path, method, status, and latency. There's a customizable [Observability dashboard](https://docs.railway.com/observability) with widgets for logs, CPU, memory, network, disk, and spend, and since July 2026, [HTTP panels](https://railway.com/changelog/2026-07-31-railway-plugin-for-chatgpt) covering requests by status class, p50/p90/p95/p99 latency, and 5xx rate. Pro plans get threshold Monitors that alert by email and in-app, and webhooks carry deploy and crash events.

Here's what's missing for production:

- **No native log drains.** Railway [says so itself](https://docs.railway.com/observability/logs): forwarding to [Datadog](https://www.datadoghq.com), [Better Stack](https://betterstack.com), or [Axiom](https://axiom.co) means running [Vector](https://vector.dev), [Fluent Bit](https://fluentbit.io), or an [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) yourself, and budgeting for the collector _and_ the destination. [Datadog's log pricing](https://www.datadoghq.com/pricing/?product=log-management) alone can match your entire Railway bill on a mid-sized service.
- **A hard limit of 500 log lines per second, per replica**, with [everything over it dropped](https://docs.railway.com/observability/logs). The load spike you most want to reconstruct afterward is the one the platform throws away.
- **Logs and metrics are stored for days, not months.** [Metrics retain 30 days](https://docs.railway.com/observability/metrics). Logs retain [7 days on Hobby, 30 on Pro, and up to 90 on Enterprise](https://docs.railway.com/observability/logs).
- **Sometimes the logs vanish.** This happened three times (that we know of) on our services, for several hours each. Support's explanation was that a log forwarder broke on the underlying node, stdout stopped being collected, and the data was unrecoverable, in the console and through the API. Railway's own status page records a [build log delivery incident in May 2026](https://status.railway.com/incident/M0XS8E8V) where logs from the affected window "may be permanently missing." We only caught ours because someone on the team had an alert on the _absence_ of logs.

> **Our advice**: treat on-platform logs as a debugging convenience. Forward everything off-platform from day one, and never let Railway hold the only copy of anything you'd need for compliance, an audit, or an incident review.

## 4. There's deliberately no automatic rollback

Railway won't roll back a deployment that fails. The feature was [requested, and Railway declined it](https://station.railway.com/feedback/automatic-rollback-on-crash-fc64766f), on the grounds that you should keep bad deploys from being promoted, rather than recover from them afterward.

That's more reasonable than it seems. **With a [healthcheck](https://docs.railway.com/deployments/healthchecks) configured, a build that fails to boot is never promoted.** Railway waits for a 2xx before switching traffic, with a 300-second default timeout, and the previous deployment keeps serving.

The real issues are narrower than "no rollback", and they are sharper:

**Railway stops watching the healthcheck once the deployment is live.** The docs [state this plainly](https://docs.railway.com/deployments/healthchecks): "Railway does not monitor the healthcheck endpoint after the deployment has gone live." A service that passes its check and starts crash-looping ten minutes later has no automatic revert. Manual rollback exists (you redeploy a previous deployment), but it's bounded by your plan's [image retention](https://docs.railway.com/pricing/plans#image-retention-policy), 72 hours on Hobby and 120 on Pro, and it needs a human to notice.

**Volume-backed services can't have overlapping deployments.** To prevent corruption, Railway [won't run two deployments mounted to the same volume](https://docs.railway.com/volumes/reference), so every stateful redeploy takes real downtime, healthcheck or not. 

**Horizontal scaling is manual.** [Vertical autoscaling](https://docs.railway.com/deployments/scaling) works: services scale up to your plan's CPU and memory limits on their own. 

**Internal traffic is DNS-based.** Public traffic is fine: nearest-region routing and random distribution across replicas, though [no sticky sessions](https://docs.railway.com/deployments/scaling), and connection draining on redeploy that [defaults to zero seconds](https://docs.railway.com/deployments/reference) until you set `RAILWAY_DEPLOYMENT_DRAINING_SECONDS`. Service-to-service traffic over `*.railway.internal` [resolves through internal DNS](https://docs.railway.com/networking/private-networking/how-it-works), and during scale-up we watched new replicas lag behind in it.

The pattern is the same in each case: Railway ships the cheap, useful version of a primitive and stops there. The savings can be spent on the necessary engineering.

## 5. Growing pains cut both ways

Railway is onboarding [12,000+ new users a day and running 50M+ builds a month](https://blog.railway.com/p/railway-summer-update-2026). That growth is also a source of instability. On our own services, we saw internal DNS lag, replicas that were slow to register, and some unplanned downtime that looked capacity-related. Latency between US daytime and nighttime traffic differed by around 50ms, which we suspect is again the "hardware lottery" at play: at peak, the fleet is busier and leans harder on rented cloud capacity. 

In May 2026, [Google suspended Railway's production GCP account](https://blog.railway.com/p/incident-report-may-19-2026-gcp-account-outage) and took the platform down for roughly eight hours. In July, [a US East availability zone went down](https://blog.railway.com/p/incident-report-july-2-2026-us-east-services-outage) it was down for about four.

To Railway's credit, their response was serious: a distributed router across Metal, AWS, and GCP, and the Gen 2 rollout as [the largest hardware order in the company's history](https://blog.railway.com/p/railway-metal-gen-2). Every fast-growing PaaS goes through this phase, and until Railway is out the other side, its reliability tracks with growth rate, and your plan should assume as much.

## So, should you migrate to Railway? Seven questions to answer first!

We didn't, but this was about our workload, not about Railway itself. The platform is cheap, and you'll spend money and engineering time around it to make it production-ready. Whether the combined bill still beats [Render](https://render.com), [Heroku](https://www.heroku.com), or even [AWS](https://aws.amazon.com) depends on your workload. We've seen it go both ways.

*Read also Evil Martians' article*: https://evilmartians.com/chronicles/migrating-whop-from-postgresql-to-planetscale-zero-downtime

Here's the checklist we now walk through before putting anything on Railway, or any cheap-and-growing PaaS:

1. **What's the worst acceptable runtime for any CPU-bound operation in your stack, and does a 5x spread break it?** Cheap platforms are cheap on average and unpredictable at the tails. Size for the tail.
2. **Where does your state live, and what RPO and RTO (recovery time objective) can you hit there?** If it's PostgreSQL, the platform now meets you halfway: PITR and HA exist. If it's anything else, backups, recovery, and the IOPS ceiling are your problem. Price that first.
3. **Which managed services are you giving up, and what do the substitutes cost?** Write the list, price it, and add it to the platform bill before you compare anything.
4. **What does your observability look like if the platform's own logs vanish for a few hours?** If the answer is "we'd be blind," build the forwarding first. If compliance needs those logs, the off-platform copy is the only one that counts.
5. **When a deploy crashes twenty minutes after passing its healthcheck, what's the recovery path, and how long does it take at 3 AM?** If it involves waking someone up, measure it in minutes.
6. **What's your tolerance for being part of a platform's scaling curve?** Early-stage at high growth is less stable than late-stage at slow growth. Both are valid choices; they're different ones.
7. **What's your exit?** Work out what leaving would cost before you commit to arriving.

For us, it came down to the first question above. In the middle of a CPU-generation migration, with a scheduler free to put us on any of three very different machines, we couldn't sufficiently predict performance to guarantee a reliable maintenance window. Ours was three to four hours. The outlier run was six and a half, and there was no way to know which machine we'd draw on the night. For a database a business runs on, that's a bet we weren't willing to take.

We stopped three weeks before cutover, with no regrets. We also don't regret the time spent on the evaluation. We now know exactly what Railway is, which is way more than the pricing page was going to tell us. 

---

If you're weighing a platform move right now and would rather not run the evaluation yourself, this is the kind of work Evil Martians lives for, including [Whop's 750GB PostgreSQL-to-PlanetScale migration with zero downtime](https://evilmartians.com/chronicles/migrating-whop-from-postgresql-to-planetscale-zero-downtime.md). Drop us a line and we'll benchmark on the hardware you'll actually get, price the recovery path before the cutover, and write a runbook you can promise a maintenance window on.

---

**Moving a production workload between platforms?** Evil Martians runs Railway platform evaluation and migration for production apps: benchmarks on the hardware you actually get, backup and recovery paths priced before the switchover date, and a runbook you can promise a maintenance window on. [Contact Evil Martians](https://evilmartians.com/contact-us.md)

## Related

- [Ilya Cherepanov](https://evilmartians.com/martians/ilya-cherepanov.md)
- [Kirill Kuznetsov](https://evilmartians.com/martians/kirill-kuznetsov.md)
- [Kirill Yakovenko](https://evilmartians.com/martians/kirill-yakovenko.md)
- [Travis Turner](https://evilmartians.com/martians/travis-turner.md)
- [Blog](https://evilmartians.com/chronicles.md)

Site index for LLMs: https://evilmartians.com/llms.txt
