Coinbase says a routine infrastructure update disabled a critical network gateway, disrupting transfers, debit card purchases, and onchain services for about 50 minutes. Its new postmortem explains the error and why the company’s normal recovery tools also failed.
Key Takeaways
- Coinbase traced the 50-minute outage to an undetected Kubernetes resource name collision during a routine infrastructure update.
- The incident disrupted transfers, Coinbase Card debit purchases, institutional settlements, and onchain services across multiple platforms.
- The outage also disabled Coinbase’s normal rollback process, prompting new safeguards and infrastructure changes.
Coinbase Explains What Went Wrong
Crypto exchange Coinbase (Nasdaq: COIN) has identified the software error behind last week’s outage that left customers unable to complete transfers, make purchases with Coinbase Card, and access several onchain services for about 50 minutes.
According to a postmortem published on July 21, engineers were deploying what appeared to be a routine, low-risk update when an undetected Kubernetes resource name collision caused the update to modify the wrong network component.
Coinbase said:
“This was caused by an unintended misconfiguration of an important network component as a result of a routine configuration update. At no point were customer funds at risk.”
Kubernetes is software Coinbase uses to organize and run applications across its cloud infrastructure. The resource name collision caused the update to alter part of the company’s Istio ingress gateway instead of only the intended component. The gateway acts like a traffic controller, directing requests between Coinbase’s internal services.
Once the gateway went offline, systems responsible for processing trades, transfers, settlements and payment authorizations could no longer communicate.
Cards Declined and Transfers Stalled
The disruption affected Coinbase’s retail, institutional and developer platforms from approximately 12:37 p.m. ET to 1:25 p.m. ET.
Some retail customers could not complete deposits, withdrawals or off-platform transactions. Transfers already underway appeared stuck but were processed after service returned. Coinbase Card debit purchases were declined, while credit card purchases continued.
The outage also interrupted onchain swaps through Coinbase DEX on Base and Solana. Coinbase Exchange and Prime customers experienced delayed or failed transfers and settlements, while developer platform customers could not onboard users, move funds, or use onramp services.
The impact highlights the growing importance of reliability as Coinbase expands beyond spot crypto trading. Its planned “everything exchange” products, including tokenized stocks and perpetual futures, are widening the range of financial activity dependent on the company’s infrastructure.
Coinbase has separately described plans to make the platform a broader financial hub, with new trading, payments and money-management services spanning stocks, derivatives, stablecoin payments and developer tools. That expansion raises the stakes when shared infrastructure fails.
Why Recovery Took Longer Than Expected
The gateway was restored at 1:20 p.m. ET, and Coinbase declared the incident mitigated three minutes later. Remaining transaction backlogs cleared over the following hours.
Recovery took longer because Coinbase’s deployment tools relied on the failed gateway. Engineers could not use the standard rollback process and instead manually launched an emergency deployment through the company’s cloud provider using temporary privileged access.
Coinbase is now adding checks to block naming conflicts, separating recovery tools from the infrastructure they manage and testing emergency access procedures more regularly.
Coinbase emphasized:
“Our goal is always zero downtime. We are actively redesigning our infrastructure to ensure that, in the highly unlikely event of another failure like this one, we can swiftly revert and recover.”
Author: Kevin Helms
Source: Bitcoin
Reviewed By: Editorial Team
