Railway Security Incident: Timeline, Credential Exposure and Platform Changes

Railway’s October security incident exposed a difficult trade-off for a cloud platform: keeping customer workloads available while stopping new deployments to contain an intrusion. Existing applications stayed online, but builds and several platform tools were interrupted as Railway investigated unauthorized access to internal systems. The company says encrypted environment variables from at most 1.85% of workspaces may have been affected, and recommends that customers rotate sensitive credentials stored before October 21.

Railway’s forensic work was still underway when it published this preliminary account, a follow-up to its October 21 security disclosure. The company says it brought in external incident response specialists and planned to publish another report when the investigation concludes. Its account describes an attacker moving through several internal systems, reaching credentials that enabled some encrypted customer variables to be decrypted.

What Railway says happened

Railway’s monitoring first flagged abnormal activity at 13:58 UTC on October 20, after elevated latency triggered an alert for the engineering on-call. The company initially identified a possible privilege escalation in its build infrastructure and halted build pipelines while engineers investigated. It deployed mitigations and restricted access to affected systems, then reported that the exploit had been addressed and could no longer be reproduced.

At 19:53 UTC, however, Railway found further evidence that the attacker could expand access across multiple parts of the platform in parallel. That changed the scope of the response: engineers added monitoring, traced additional access paths, restricted credentials, and paused builds again. Railway says the intrusion began at 13:58 UTC on October 20 and that it confirmed the attacker had lost access by 03:32 UTC on October 22.

The account says the attacker was likely seeking AI provider API keys to conduct model distillation. Railway’s preliminary evidence suggests the intruder had access to modified models and substantial computing resources, which the company says enabled sophisticated attempts against its infrastructure. Railway has not disclosed the specific techniques or full scope of the exploits while the investigation continues.

The sequence of events illustrates why the first mitigation did not end the incident. Railway initially believed it had blocked the identified exploit, but later evidence showed the attacker could reach other parts of the system. Engineers then focused on finding access paths and limiting how far the intrusion could spread.

How the response affected customers

Builds and deployments were paused during parts of the response. Customers also experienced intermittent outages affecting the dashboard, API, command-line interface, and support tools. Railway says deployed applications remained online throughout, and that the first system it restored was the ability to disable existing deployments. By October 22, it reported that API access and deployments across Pro, Enterprise, Hobby, Trial, and Free tiers had returned. Some supporting features were reactivated gradually.

During the investigation, Railway found that the attacker had accessed encrypted environment variables and, using compromised credentials, decrypted a subset. Railway says the encryption design limited the potential reach: it could establish that at most 1.85% of workspaces had variables that may have been compromised, or about 100,000 workspaces. That figure is a ceiling for potentially affected workspaces, not confirmation that every one of them had a secret decrypted.

The company says it could not determine exactly which variables were decrypted because full keys used envelope encryption. It notified customers with confirmed impact and also contacted a larger group it assessed as higher risk based on preliminary evidence about the attacker’s targeting. Railway identified two patterns that could raise risk: workspaces containing many environment variables, and environments holding substantial numbers of AI provider keys.

Railway says individual user accounts were not accessed. It has not ruled out that source code may have been reached and said it would address that question in its final forensic report. The company also invalidated Railway access tokens as a precaution.

Provider notifications and credential rotation

At 09:25 UTC on October 21, Railway says it notified vendors about services whose keys appeared in the attacker’s activity and asked them to revoke potentially affected credentials. The company names GitHub, GitLab, Anthropic, OpenAI, OpenRouter, SpaceXSI, and Stripe among the providers it contacted, along with others. For some services, Railway supplied all keys stored on the platform, rather than only keys it knew had been accessed.

That broad approach means a vendor’s revocation notice does not prove that a customer’s key was exposed. Railway says some providers requested revocation of all keys it held for their services, extending the number of customers affected by key resets beyond the estimated 100,000 potentially exposed workspaces.

Railway recommends that customers revoke and reapply sensitive variables stored before October 21. It also published guides for checking activity and rotating credentials. Its additional recommendations are to remove public network access to databases, review GitHub organization audit logs and installed app access, and check registry activity logs, including those for Docker Hub.

For credential handling, Railway points customers to its documentation on sensitive variables and keys. It also links to GitHub’s guidance for reviewing organization audit logs and checking installed app access, plus Docker’s documentation on organization activity logs. Customers with questions can contact [email protected].

Infrastructure changes Railway reports

Railway says it shut down potentially compromised internal tools, rotated keys twice, and replaced credentials for its storage buckets. It rebuilt infrastructure tooling from validated source and checked that the same privilege escalation patterns could not be repeated. The company also added a way to revoke Railway application permissions in bulk when it detects suspicious activity.

One significant change was accelerating a planned move to a fourth generation of builder infrastructure. Railway describes the previous generation as isolated, multi-tenant environments and the new one as single-tenant environments. The rollout had been planned for the following week but was completed sooner in response to the incident. That architectural change may reduce exposure between build environments, though the company has not yet published technical details that would let customers assess the change independently.

Railway says it reinitialized internal systems with only required dependencies, new credentials, and additional hardening. It also says it performed other mitigations that it is not detailing while the investigation continues. These changes address containment and recovery; the open question is what the completed forensic account will establish about the original access path and any exposure beyond encrypted variables.

Recovery status and customer questions

In an update dated October 27, Railway said the incident was resolved for existing users and that services had been restored for its five million existing users. It said it had paused new signups while prioritizing reliable service during recovery, and expected to reopen them that week. The company’s statement that systems were safe to use reflects its own validation of the redeployed systems; customers should still follow its credential rotation guidance.

Railway says it cannot refund customers for costs incurred through compromised AI keys. It directs affected customers to contact their AI token provider, then submit a support request through Railway’s support portal with timestamps, prompts, and other evidence of unauthorized usage so the company can raise the request on their behalf.

Some customers who signed in again may not have seen their expected GitHub connection because Railway rolled its GitHub App installations. The company says users whose GitHub account email differs from their Railway account email may need to choose “Log in using email.”

The practical takeaway is to treat stored credentials as needing rotation, then inspect activity in connected services and registries. Railway has restored core operations and describes substantial infrastructure changes, but its final report remains necessary to clarify the intrusion’s reach, the status of any source code exposure, and the technical details behind its containment measures.