Avalanche Crippled by Bug Triggered by Unusually High Volume, Engineer Says

The blockchain, which had been touted for its ability to handle a high volume of transactions, was slowed to a crawl by an unusually high number of transactions.

AccessTimeIconFeb 14, 2021 at 9:59 p.m. UTC
Updated Sep 14, 2021 at 12:11 p.m. UTC
10 Years of Decentralizing the Future
May 29-31, 2024 - Austin, TexasThe biggest and most established global hub for everything crypto, blockchain and Web3.Register Now

An Ava Labs engineer gave a rundown of the small code bug that severely crippled the Avalanche blockchain last week

In a Sunday Medium post, blockchain engineer Patrick O'Grady wrote that increased congestion on the network triggered a “non-deterministic bug” related to how the high-throughput, proof-of-stake blockchain keeps track of transactions. 

Funds were never at risk, O'Grady notes, though the high-profile misstep has a valuable lesson for the blockchain industry.

Avalanche launched in September 2020 with the claim it could process 4,500 transactions per second. It’s backed by prominent cryptocurrency firms including Mike Novogratz’s Galaxy Digital, Bitmain and Initialized Capital. It also has an academic stamp of approval, having been designed by Emin Gün Sirer, a computer science professor at Cornell University.

The blockchain is usually grouped with other so-called Ethereum killers, or blockchains designed to solve the scalability problems that have plagued the second-largest blockchain since inception. While positioned to steal market share from Ethereum, Avalanche also has been billed as a way to complement and connect – rather than strictly compete  – with its forbear. 

Avalanche has three “default chains,” including the so-called “contract chain” that supports the Ethereum Virtual Machine and its Solidity coding language. It’s this chain that was part of this week’s issue. 

You can read a full accounting of the problem that arose. But in short, in order to boost transaction throughput, Avalanche’s three chains remain separate and distinct from each other, each performing within a set range of transaction-types, up until the moment an asset has to hop over to another chain. That process was placed under an incredible strain, following the launch of a new decentralized money market called Pangolin. 

An atypical amount of users and volume created an atypical amount of blocks to be processed. This, O’Grady notes, triggered a bug that was creating false cross-chain “mints.” In O’Grady’s words: “This caused some validators to accept some invalid mint transactions, while the rest of the network refused to honor these transactions and stalled the [contract]-chain.”

Importantly, no double-spends occurred. “The bug did not affect regular transactions, coin transfers, asset transfers, coin destruction, or smart contract invocations. Avalanche never allowed any user to successfully send the same funds to two recipients,” O’Grady wrote.

A read of the issue was ready just hours after the initial issue, though a fix was harder to come by. Given Avalanche’s decentralized nature, it would be impossible to get all the nodes to collude and rollback problematic transactions. 

Instead, as O’Grady writes, a solution was found through incremental deployment of a patch – basically the way any software is updated. 

Blockchains are complex things, built by human beings, but run by machines. An issue that was small enough to bypass during an initial inspection can snowball as a network grows. In Avalanche’s case, the bug didn’t bring down the network but it did pour ice water over some of the boasts made about the network’s ability to handle high-throughput prior to launch.

AVAX, the blockchain’s token, is trading hands at around $41.20, down from $53 on Feb. 11 when the problem occurred. 

Disclosure

Please note that our privacy policy, terms of use, cookies, and do not sell my personal information has been updated.

CoinDesk is an award-winning media outlet that covers the cryptocurrency industry. Its journalists abide by a strict set of editorial policies. In November 2023, CoinDesk was acquired by the Bullish group, owner of Bullish, a regulated, digital assets exchange. The Bullish group is majority-owned by Block.one; both companies have interests in a variety of blockchain and digital asset businesses and significant holdings of digital assets, including bitcoin. CoinDesk operates as an independent subsidiary with an editorial committee to protect journalistic independence. CoinDesk employees, including journalists, may receive options in the Bullish group as part of their compensation.


Learn more about Consensus 2024, CoinDesk's longest-running and most influential event that brings together all sides of crypto, blockchain and Web3. Head to consensus.coindesk.com to register and buy your pass now.


Read more about