How Walrus Protocol is Reinventing Decentralized Storage

Decentralized storage protocol Walrus is set to launch its testnet in just a few weeks—and developers are already creating novel applications that leverage its unique properties, Mysten Labs Chief Scientist George Danezis told Decrypt.

The Walrus devnet hackathon Breaking the Ice saw over 60 submitted projects and 288 registered participants, with a $50,000 prize pool awarded to 10 winning teams. Among the hackathon projects that have caught Danezis’ attention are Walrus Pass, a ticketing and subscriptions app, and Link Forge, a social networking app.

“What’s really interesting about Walrus Pass is that it combines so many things,” he said, including ZK-login for one-click Gmail logins, Sui for ticket and subscription purchases, and Walrus in order to “store tickets and QR codes and all of the stuff you actually need to show at the door.” The app, he added, “provides this kind of end to end experience that you would need to address this as a whole product.”

Link Forge, meanwhile, is “a bit of a clone of LinkTree,” enabling users to upload a profile picture and links, giving them a presence on the decentralized web that’s akin to their Web2 profile. “It highlights that Web3, broadly speaking, is ready to start playing the role that Web2 technologies played,” said Danezis. “And the second thing is, it highlights the fact that Web3 is naturally social.”

Both apps—indeed, all the apps in the hackathon—make use of Walrus’ “secure, decentralized blob store,” in which blobs, or binary large objects, are stored in a non-hierarchical architecture. “It takes very large chunks of data, megabytes up to gigabytes, and it doesn’t really care what they are,” said Danezis. It stores them by “taking each file and then splitting it and encoding it into many, many small chunks that it distributes across the storage nodes,” giving it resilience and the ability to reconstruct the file from available chunks by reencoding it. “It can store anything you want,” said Danezis.

What is Red Stuff?

Walrus’ secret sauce is its new data encoding algorithm, Red Stuff, an erasure coding algorithm that leverages fountain codes to chop big files into small chunks and encode them.

“Red Stuff is special in that it has some very key properties for what we're trying to do,” said Danezis. “First of all, it's extremely fast—you can basically encode gigabytes per second of data,” he explained. “The second property that it has is that it allows you to reconstruct missing chunks very efficiently,” he added, explaining that it uses a dual encoding scheme that “allows you to just request very small chunks from everybody else to reconstruct your own little chunk,” instead of the resource-intensive approach of downloading the entire file, reencoding it and obtaining any missing elements. “That is extremely efficient, which allows storage nodes to heal effectively on the cheap, rather than having to spend a lot of time to reconstruct what they should have,” he said.

Walrus also leverages Sui’s “very fast” blockchain to perform metadata management, storage node management and payments. “Because all of that stuff can happen at a whole different scale and at a very low cost,” Danezis said, Walrus can employ novel erasure coding techniques like Red Stuff, which “require a little bit more complex metadata management than traditional techniques that just replicate files.”

That, in turn, means that Walrus “ends up storing much less data in the network, by about a factor of five—so in order to store one megabyte, you store about five megabytes,” Danezis said. “So that's the key advantage of Walrus, while at the same time not sacrificing on other dimensions such as security or decentralization.”

Testnet and beyond

With the Walrus testnet on the horizon, Danezis is already looking ahead to future applications of the decentralized storage protocol.

Walrus Sites is one such application, enabling users to upload their entire website to Walrus. “Gateways allow you to just browse this website from your web browser without needing any special wallets or software,” he explained. Decrypt is already using Walrus to provide decentralized, censorship-resistant storage of its news articles. “There are natural synergies between news organizations and secure, decentralized storage,” said Danezis, adding that users can verify that articles are genuine and trace their edit history on-chain.

Walrus is also eying applications such as AI, he added. “In AI today, there are great discussions around, ‘How do I know what my models were trained on? How do I know if something is generated from a model?’” Danezis said. That data has to be stored somewhere in order for it to be audited, he explained, and the need for “everybody to be able to actually check what it is and make sure that they're using the same data,” makes Walrus a natural fit.

Another possible application is to store the terabytes of data that make up the history of blockchains, he said. “Walrus is a perfect store for storing some of that data, as well as for storing data that you need to know is available in the future—such as those that are used as part of L2 roll-ups on Ethereum.”

The testnet phase will serve as an opportunity to improve the performance and robustness of Walrus’ infrastructure, Danezis said, adding that, “On the other side, you know, there is a whole adventure in order to really make sure that the community takes full ownership of the project.”