Previous incidents

TLDR: A temporary slowdown caused by saturation in our API gateway layer increased response times until they exceeded the edge-network timeout, causing a 524 HTTP response for some API requests.

Timeline in PST
01:00 AM First elevated 524 error responses detected
06:35 AM Rolled back recent backend release (no improvement)
07:19 AM Rolled back related network changes (no improvement).
08:22 AM Scaled up API gateway
09:36 AM Scaled up API gateway further
10:00 AM Reverted the previous night's...

The issue has been resolved: https://neonstatus.com/aws-us-west-oregon/incidents/01JYM23FB7HR82VPZR9DBKVPP8#01JYM4F8Y8MZARWJWRFKV01AP5.

Our database provider has reported this issue as resolved from their end

TL;DR

In response to hallucinations reports in the Success Evaluation feature, we updated our integration with Gemini LLM to use Structured Output. This inadvertently changed the type of the call.analysis.successEvaluation field from string | null to string | number | boolean | null, introducing a breaking change for customers with strict type validation and those using Vapi Server SDKs.

Timeline (all in PT)

• June 12, 11:32pm: Enterprise and Startup users report hallucinations in Su...

It is resolved.

Summary:
We experienced an issue related to API key validation within our WebSockets implementation when sending the API key more than once.

Details:
The issue arose during API key validation within our WebSockets implementation. Our system validates that the API key provided during the initial message is the same as proceeding messages.

A recent change introduced during a release caused a mismatch in how API keys were compared. Specifically, the system was comparing hashed API keys agains...

API was down due to user error in routine maintenance. Service has since been restored