Solana Still Doesn’t Know What Caused Weekend Network Outage

Woman holding broken network cable and Solana logo

Two days after a major outage, the Solana Foundation says it’s still investigating why the Solana network went down for almost 20 hours following a network upgrade.

“The cause of this is still unknown and under active investigation,” the Foundation wrote in a blog post published on Sunday night. A Solana Foundation spokesperson told Decrypt in an email on Monday that there are still no updates to the information that’s already been shared in its blog.

That’s a big blow for a network that’s become a major player in the industry in just two years.

During the outage, which started just before 6 a.m. UTC on Saturday, the network was unable to process user transactions. This means all on-chain activity, including NFT and DeFi trading, came to a screeching halt. After engineers recommended restarting the network, validators had to downgrade to an earlier version of software used to run nodes. They restarted the network by 2 a.m. UTC on Sunday.

On Saturday, the price of Solana’s token, SOL, took a hit on the news that the network was experiencing an outage. It started the day trading at $23.03, but fell 6% to $21.71 before the outage ended on Sunday morning, according to CoinGecko. By Monday afternoon, SOL had almost made up the outage-induced losses and was trading at $22.46.

But backlash on Twitter was fiery and persistent through the weekend.

“DeFi doesn’t work on a chain that goes down, no matter how low the fees are,” wrote a user. Who goes by 0xShitTrader on Twitter and says they’re an office manager for Ellipsis Labs.

The company has been building Phoenix, a decentralized limit order book, on Solana. And has previously said it was attracted to building on the network because of its high throughput and low transaction fees.

The outage also brought out some heavy criticism from Paul Brody. Who heads up blockchain initiatives at global accounting firm EY.

“Solana is like a lifestyle-friendly blockchain because you can have nights and weekends off,” Brody wrote on Twitter Saturday. “When can we all just admit that Solana is a joke. We’re so far beyond farce here. How can you [build] mission critical infrastructure on this nonsense,” he continued in another tweet.

Solana co-founder Anatoly Yakovenko said on an episode of Decrypt’s gm podcast last year that outages are “the biggest challenge for us, and the number one priority.” At the time, he said he saw a new validator client, Firedancer, as a “long-term fix.”

The client has a built-in fail-safe that switches validators to “vote-only” mode in the event of an outage. That means the network prioritizes voting transactions, which are needed to restart the network, over regular user transactions.

Saturday’s Solana outage is the latest in a string of issues the Solana network has faced. From September 30 until October 1, the network saw degraded performance that turned into a 7-hour outage.