Incident Report

Summary

On Thursday, 18 September 2025, between 07:22 and 08:35 US/Pacific, Google Workspace products experienced elevated login failures globally for a duration of 1 hour 13 minutes. Some users may have been unable to login to various Google Workspace services during that time.

This is not the level of quality and reliability we strive to offer you, and we are taking immediate steps to improve the platform’s performance and availability.

Root Cause

A shift in traffic led to a resource contention issue within Google's authentication system. The system's capacity for handling concurrent requests was exhausted in several east coast regions, creating a bottleneck that blocked new login requests. This resulted in elevated login failures and errors for users across various Google services.

Remediation and Prevention

Google engineers were alerted at 07:31 US/Pacific via internal monitoring. After identifying the bottleneck, engineers made configuration changes to increase the replica count of the affected frontend service component by 50%. This additional capacity relieved the resource contention, and normal service was fully restored by 08:35 US/Pacific.

Google is committed preventing a repeat of this issue in the future and is completing the following actions:

• We will tune the current authentication frontend's deployment parameters to ensure ongoing system stability
• Longer term, we plan to eliminate this class of resource contention by migrating to a new architecture.

Mini Incident Report

We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Workspace Support using the help article https://support.google.com/a/answer/1047213.

(All Times US/Pacific)

Incident Start: 18 September 2025, 07:20

Incident End: 18 September 2025, 08:30

Duration: 1 hour, 10 minutes

Affected Services and Features:

All Google Workspace products

Description:

Multiple customers experienced elevated login failures within various Workspace products. From preliminary analysis, the incident was suspected to be caused by a resource contention issue in Google’s internal authentication system. Google will complete a full incident report in the following days that will provide a full root cause.

Customer Impact:

Multiple users may have been unable to login to various Google services during the impacted time.

Summary: We have mitigated an issue where some customers experienced login issues to Google services.

Description: The issue with Google services has been resolved for all affected customers as of Thursday, 2025-09-18 08:30 US/Pacific.

From preliminary analysis, we have identified that the issue is caused by a resource contention issue. Our engineers have mitigated the issue by adding in additional resources.

We thank you for your patience while we worked on resolving the issue.

Symptoms: Some customers experienced login issues to Google services.

Workaround: None at this time.

Summary: We have mitigated an issue where some customers experienced login issues to Google services.

Description: Between 07:22 and 08:30 PDT, customers experienced login issues to Google services.

Our engineering team has identified the cause and applied appropriate mitigations by 08:30 PDT. Customers should now be able to login to the Google services.

We are closely monitoring the environment for stability.

We will provide an update by Thursday, 2025-09-18 09:20 PDT with current details.

We apologize to all who are affected by the disruption.

Symptoms: Some customers experienced login issues to Google services.

Workaround: None at this time.