Elevated API and Dashboard Error Rates

Incident Report for Vercel

Resolved

All systems normal. We will continue to monitor, sharing a full postmortem after completing a thorough investigation with our upstream provider.
Posted Feb 12, 2021 - 06:05 UTC

Update

Further mitigations have been put in place by our upstream provider and we are now seeing reduced error rates on the Vercel Dashboard and APIs. We are continuing to monitor.
Posted Feb 12, 2021 - 05:04 UTC

Update

We are again experiencing elevated error rates on the Vercel Dashboard and API. Existing deployments and live traffic are not affected by this issue. We continue to work with our upstream provider to resolve this.
Posted Feb 12, 2021 - 02:55 UTC

Monitoring

A mitigation has been put in place by our upstream provider and we are now seeing reduced error rates on the Vercel Dashboard and API. We are continuing to monitor.
Posted Feb 12, 2021 - 02:30 UTC

Update

Full availability has been restored to all deployments. We are continuing to work with our upstream provider to restore full usability to the Vercel Dashboard and API.
Posted Feb 12, 2021 - 02:20 UTC

Update

A mitigation has been put in place and will shortly restore full availability to all deployments. We are still working with our upstream provider to resolve the issues impacting the Vercel Dashboard and API.
Posted Feb 12, 2021 - 01:23 UTC

Identified

After working with our upstream provider, a mitigation is being put in place to restore full availability to deployments. Further updates will be shared when available.
Posted Feb 12, 2021 - 01:20 UTC

Update

We are currently investigating reports of elevated error rates on the Vercel Dashboard and API. Some deployments may be experiencing degraded availability. We will share updates as they become available.
Posted Feb 12, 2021 - 00:46 UTC

Investigating

We are currently investigating reports of elevated error rates on the Vercel Dashboard and API. Existing deployments and live traffic are not affected by this issue. We will share updates as they become available.
Posted Feb 12, 2021 - 00:35 UTC
This incident affected: Dashboard (Dashboard), Build & Deploy (Builds), CDN (ARN1 - Stockholm, Sweden, BOM1 - Mumbai, India, CDG1 - Paris, France, Europe, CLE1 - Cleveland, East US, DUB1 - Dublin, Ireland, GRU1 - São Paulo, Brazil, HKG1 - Hong Kong, HND1 - Tokyo, Japan, IAD1 - Washington DC, USA, ICN1 - Seoul, South Korea, LHR1 - London, UK, PDX1 - Portland, West US, SFO1 - San Francisco, CA, USA, SIN1 - Singapore, SYD1 - Sydney, Australia), Observability (Drains, Logs), API, and CI/CD (Deploy Hooks, Git Integrations, Integration Webhooks).