Skip to content

Cloudflare Experiences Disruptive Service Due to API Malfunction Self-Caused by the Company

Cloudflare experienced an outage caused by a glitch in its API that resulted in overload, leading to disruptions in their services. Here's an explanation of the issue and the subsequent resolution.

Cloudflare Experiences Downtime due to API Malfunction Caused by Internal Programming
Cloudflare Experiences Downtime due to API Malfunction Caused by Internal Programming

Cloudflare Experiences Disruptive Service Due to API Malfunction Self-Caused by the Company

Cloudflare Faces Second Incident in 2023, Less Severe Than Summer Outage

Cloudflare, a prominent network services provider, recently encountered another incident in September, leading to a widespread outage of numerous APIs and the Cloudflare Dashboard. In a Sept. 13 blog post, Tom Lianza, Cloudflare's vice president of engineering, and Joaquin Madruga, vice president of engineering for the developer platform, detailed the incident.

The September incident was triggered by a bug in Cloudflare's Tenant Service API. This issue resulted in repeated, unnecessary calls to the Tenant Service API, causing it to overload. As the Tenant Service is part of Cloudflare's API request authorization logic, this overload affected other APIs and the dashboard, causing API requests to return 5xx status codes when authorization evaluations failed.

In June 2023, Cloudflare was linked to a significant outage that impacted sites like Spotify, Google, Snapchat, Discord, Character.ai, and more. However, the September incident was less disruptive than the summer outage.

Cloudflare accidentally included a 'problematic object in its dependency array,' which was recreated, treated as new, and caused it to re-run. This eventually led to the API call executing many times during a single dashboard render instead of just once.

The blog post expresses regret for the disruption caused by the September incident at Cloudflare. The company is currently investigating the issue and working on improvements to its systems and processes to prevent such incidents in the future.

No further details were provided about the specific impact or duration of the September incident. It's worth noting that earlier this month, Cloudflare experienced another incident, but it was less disruptive than the summer outage.

In a separate incident, Cloudflare faced a DDoS attack last week, but the group 'Lizard Squad' claimed responsibility for the self-triggered DDoS attack on Cloudflare last weekend.

Cloudflare continues to be a vital player in the network services industry, and while incidents like these can cause disruptions, the company is dedicated to learning from them and enhancing its systems to provide a more reliable service to its users.

Read also:

Latest