Here’s Why Solana Ceased Block Production for 7 Hours on Saturday

Millions of transactions per second and heavy traffic contributed to the network’s disruption, developers explained on Tuesday.

AccessTimeIconMay 3, 2022 at 8:28 a.m. UTC
Updated May 11, 2023 at 5:25 p.m. UTC

Bots tied to a new non-fungible token (NFT) project built on Solana caused a seven-hour network outrage on Saturday, project developers confirmed in a post on Tuesday. No new blocks were produced by the network during that time.

Solana processes an average of 2,700 transactions per second (tps), blockchain explorers show, with an upper peak of over 710,000 tps on a standard network, as per developer documents.

However, on Saturday night millions of transactions flooded the network each second. This led to network validators – or entities that process transactions on and support the network – to run out of memory and crash, developers said.

“An enormous amount of inbound transactions (6 million per second) flooded the network, surpassing 100 Gbps of traffic at individual nodes,” developers wrote. “There is no evidence of a denial of service attack, but instead evidence indicates bots tried to programmatically win a new NFT being minted using the popular Candy Machine program.”

Validators ran out of memory while trying to clear up transactions in other, abandoned blocks. The number of total forks that validators had to evaluate exceeded their capacity to do so automatically, which led to a manual intervention to fix the issue.

Bots had swarmed the popular NFT minting tool known as Candy Machine on Saturday, as reported. This was mainly to take part in a new NFT project mint.

However, that mint used a fixed price for users instead of a Dutch auction – the usual process followed by crypto projects – which incentivized bots to send a huge number of transactions in hopes of winning the mint. This caused spam on the network.

Meanwhile, developers noted Solana has suffered intermittent congestion issues several times in the past months – with most of these resulting from bot activity targeted at NFT mints.

Changes are coming to combat those problems. Developers said the upcoming v1.10 release, currently stabilizing on testnet, includes memory use improvements to prolong the time nodes can endure slow or stalled consensus, which would help mitigate issues like those seen on Saturday.

Fees for prioritizing transactions are coming to Solana as well, developers added. This could help mitigate the entirety of Solana getting affected because of a single project.

Solana's SOL tokens trade at over $88 at writing time, as per CoinGecko data. Prices are down a nominal 1.8% in the past 24 hours.


DISCLOSURE

Please note that our privacy policy, terms of use, cookies, and do not sell my personal information has been updated.

The leader in news and information on cryptocurrency, digital assets and the future of money, CoinDesk is a media outlet that strives for the highest journalistic standards and abides by a strict set of editorial policies. CoinDesk is an independent operating subsidiary of Digital Currency Group, which invests in cryptocurrencies and blockchain startups. As part of their compensation, certain CoinDesk employees, including editorial employees, may receive exposure to DCG equity in the form of stock appreciation rights, which vest over a multi-year period. CoinDesk journalists are not allowed to purchase stock outright in DCG.

CoinDesk - Unknown

Shaurya is the Deputy Managing Editor for the Data & Tokens team, focusing on decentralized finance, markets, on-chain data, and governance across all major and minor blockchains.


Learn more about Consensus 2024, CoinDesk’s longest-running and most influential event that brings together all sides of crypto, blockchain and Web3. Head to consensus.coindesk.com to register and buy your pass now.


Read more about