EU Region login API errors

On September 10 at 8:42 AM PDT, our monitoring detected elevated errors with the Auth API in the EU region. The issue was identified promptly and fully resolved by 8:49 AM PDT. Total impact duration was approximately 7 minutes.

Impact

• Region affected: EU
• Service affected: Authentication API
• Customer impact: Customers in the EU experienced failed authentication attempts during the incident window.

Timeline (PDT)

• 08:42 – Incident detected. Elevated Auth API errors in the EU region reported.
• 08:43 – Engineering and DevOps teams engaged.
• 08:46 – Root cause isolated to a Terraform-related configuration change involving a VPC endpoint.
• 08:49 – Configuration change rolled back; Auth API errors resolved.

Root Cause

The outage was caused by a configuration change in our production tenant during an ongoing Terraform migration. As part of the migration, a new VPC endpoint was being added to production environments. During Terraform apply, a subnet association was also being created. A network disruption occurred between the creation of the VPC endpoint and the subnet association (behavior not seen in prior testing) leading to temporary Auth API connectivity failures in the EU region. A new Terraform migration plan has been created that eliminates the possibility of disruption.