Resolved -
We've deployed mitigations and all services are operating normally. Traffic may continue to reroute to nearby regions for the next few hours, but we will restore traffic gradually after verifying system availability in the IAD1 region.
May 8, 06:39 UTC
Monitoring -
We implemented a fix and are monitoring the results. Backlogs for Workflows and Queues will be processed within the next few hours.
May 8, 04:54 UTC
Update -
New workflow runs and queue messages are being processed now. Processing the backlog of existing workflow runs and queued messages is still being investigated.
May 8, 02:33 UTC
Update -
We are continuing to work on a fix for this issue.
May 8, 02:18 UTC
Update -
New messages to Vercel Workflows and Vercel Queues are being queued, but message processing is paused. Queued messages will be processed when service is restored.
May 8, 02:13 UTC
Update -
Traffic to the IAD1 region has been re-routed to nearby regions. Functions configured in IAD1 will be invoked in a different region if failover regions are configured, but if no other regions are configured, the function will still be invoked in IAD1.
May 8, 01:47 UTC
Identified -
Some Vercel functions that run in the IAD1 region are experiencing elevated invocation failures. We are investigating the issue and will share more information as it becomes available.
May 8, 01:18 UTC