Deployments slow to start

Timeline

1. 23:59 UTC Railway status page · Reported Deployments slow to start
2. 23:59 UTC Railway status page · Investigating We are aware of an issue causing deployments to take longer than normal to initialize. Users may experience delays when starting new deployments. We have identified a potential cause and are actively working to restore normal behavior. We will provide updates as we learn more.
3. 00:20 UTC Railway status page · Monitoring We have identified the cause of the delay: a networking issue on a single host in our infrastructure caused a backlog of deployments to queue up. We have removed the affected host from rotation and are scaling additional capacity to work through the backlog. Deployments are beginning to progress again. We will continue to monitor and provide an update once things have fully returned to normal.
4. 00:37 UTC Railway status page · Investigating We are continuing to investigate an issue with deployment initialization. While we identified and removed an affected host from rotation, some of our deployment workers are experiencing instability as they work through the backlog of queued deployments. We are actively working to stabilize the deployment system and clear the backlog. We will provide further updates as we make progress.
5. 03:05 UTC Railway status page · Investigating We continue to work through a backlog of queued deployments. Our deployment workers have stabilized following configuration changes, and throughput is gradually returning to normal levels. Some deployments may still experience delays as the backlog clears. We are actively working to reduce the remaining queue. We will provide further updates as we make progress.
6. 03:33 UTC Railway status page · Monitoring Deployments are continuing to recover. We have stabilized our deployment pipeline and cleared most of the backlog of queued deployments. Throughput has returned to nominal levels and new deployments are progressing through the queue. Some delays may still be experienced as the remaining backlog fully clears. We are actively monitoring and will provide a further update shortly.
7. 03:38 UTC Railway status page · Resolved Deployment delays have been resolved. Our deployment pipeline has stabilized. The backlog of queued deployments has been cleared, and throughput has fully returned to normal. We apologize for any inconvenience. If you continue to experience issues, please contact support at https://station.railway.com.