Following on from our Modular Series introduction last week, we are excited to unveil our deep dive into all things surrounding Data Availability.

⚔️ A Call to Arms

Think you have what it takes to enter the Castle and contribute to research, community initiatives, due diligence analysis, and advising/servicing projects in the space? Or maybe you want to upskill and shadow community members who have already walked a successful path as an intern?

Fill out an application form!

The Data Availability Problem

The Data Availability problem refers to the question of how peers in a blockchain network can be sure that all the data of a newly proposed block is actually available. If the data is not available, the block might contain malicious transactions that are being hidden by the block producer. Even if the block contains non-malicious transactions, hiding them might compromise the security of the system, as nobody can verify the state of the chain anymore. Therefore, one can’t be sure all transactions are actually correct. In other words, data availability refers to the ability of all participants (nodes) to access and validate the data on a network. This is a prerequisite for functioning blockchains or L2 scaling solutions and ensures that the networks are transparent, secure, and decentralized.

This data availability problem is especially prominent in the context of rollup systems, which inherit security from the base layer they are built upon, where they post the transaction data. It is hence very important that sequencers make transaction data available because DA is needed to keep the chain progressing (ensure liveness) and to catch invalid transactions (safety in the optimistic case). Additionally, data needs to be available to ensure that the sequencer (basically the rollup block producer) doesn’t misbehave and exploit its privileged position and the power it has over transaction ordering.

Data availability poses limitations to contemporary rollup systems: even if the sequencer of a given L2 were an actual supercomputer, the number of transactions per second it can actually process is limited by the data throughput of the underlying data availability solution/layer it relies on. Put simply, if the data availability solution/layer used by a rollup is unable to keep up with the amount of data the rollup’s sequencer wants to dump on it, then the sequencer (and the rollup) can’t process more transactions even if it wants to.

Hence, strong data availability guarantees are critical to ensure rollup sequencers behave. Moreover, maximizing the data throughput of a data availability solution/layer is crucial to enable rollups to process the number of transactions necessary to become execution layers supporting applications ready for mass adoption.

How can we ensure that data on a given data availability layer is actually available, or in other words, that a sequencer has actually published complete transaction data?

The obvious solution would be to simply force the full nodes in the network to download all the data dumped onto the DA layer by the sequencer. However, this would require full nodes to keep up with the sequencer’s rate of transaction computation, thereby raising the hardware requirements and consequently worsening the network’s decentralization (as barriers of entry to participate in block production/verification increase).

Not everyone has the resources to spin up a high-spec block-producing full node. But blockchains also can’t scale with low validator requirements. Consequently, there is a conflict here between scalability and decentralization. Additionally, centralizing forces in PoS networks (as stake naturally concentrates to a small number of validators), augmented by centralization tendencies caused by MEV, lead researchers to the conclusion, that blockchains and decentralized validator sets don’t guarantee all the nice security properties, we hoped for in the first place. Hence, many researchers argue that the key to scaling a blockchain network while preserving decentralization is to scale block verification and not production. This idea follows the notion that if the actions of a smaller group of consensus nodes can be audited by a very large number of participants, blockchains will continue to operate as trustless networks, even if not everyone can participate in block production (consensus). This has been the core thesis of Vitalik’s endgame article (December 2021), where he states:

Luckily, there are approaches that address the data availability issue by leveraging error-correcting techniques like erasure coding and probabilistic verification methods such as data availability sampling (DAS). These core concepts play a key role in building scalable networks that can support the data availability needs of rollup-based execution layers and will be discussed in depth over the course of this report.

Data Availability & Calldata on Ethereum

Before diving into these concepts, let’s have a look at the status quo of data availability on Ethereum to understand why these solutions are actually needed.

As already mentioned in the previous section, the capacity of the DA layer to handle the data that a rollup sequencer posts onto the DA layer is crucial. Currently, rollups utilize calldata to post data to Ethereum L1, ensuring both data availability and storage. Transaction calldata is the hexadecimal data passed along with a transaction that allows us to send messages to other entities or interact with smart contracts.

However, calldata is limited to ~10KB per block, and with a fixed price of 16 gas/byte is very expensive. The limited data space on Ethereum and the high cost of L1 block space limit rollup scalability and currently are the main bottlenecks L2 scaling solutions face. To quantify this in terms of cost, call data cost accounts for 90% of rollup transaction cost.

To overcome these limitations and reduce transaction costs on their execution layer, some L2 teams have come up with the idea of using off-chain DA solutions, instead of posting the data to Ethereum L1. This can be a fully centralized solution (such as a centralized storage system) or what is referred to as a data availability committee (DAC).

In the case of a DAC, the protocol relies on a set of trusted entities that guarantee to make data available to network participants and users. When we remove data availability requirements from Ethereum, the transaction cost of the L2 becomes much cheaper. However, having a DA off-chain comes with a trade-off between cost and security. While Ethereum L1 provides very high liveness and censorship resistance guarantees, off-chain solutions require trust in parties that are not part of the crypto-economic security system that secures Ethereum mainnet.

Not surprisingly, there are initiatives aiming to reduce the cost rollups incur when posting data to L1. One of them is EIP-4488, which reduces the calldata cost from 16 to 3 gas per byte, with a cap on calldata per block to mitigate the security risks arising from state bloat. However, it does not directly increase L1 data capacity limit but rather balances the cost of L1 execution with the cost of posting data from L2 execution layers in favor of rollups, while retaining the same maximum capacity. This rebalancing means that the average data usage of available capacity will go up, max capacity doesn't. The need to increase max capacity still remains, however, at least in the short term, EIP-4488 allows us to enjoy lower rollup fees (as it could reduce costs by up to 80%).

The long-term solution to this is Danksharding. The mid-term, intermediate solution on the other hand is EIP-4844, which aims to increase the blockspace available to the rollups. EIP-4844 is expected to massively reduce the rollup cost base. But more on this in the next section.

Use our referral system to spread the word about the Chronicle!

EIP-4844 a.k.a. Proto-Danksharding

With transaction fees on L1 spiking with increased on-chain activity, there is great urgency to facilitate an ecosystem-wide move to rollup-based L2 solutions. However, as already outlined earlier in this report, rollups need data. Right now the cheapest data option for rollup is calldata. Unfortunately, this type of data is very expensive (costing 16 gas per byte).

To illustrate this, let’s have a closer look at Optimism and Arbitrum’s cost. The analysis is based on data provided by Avail.

The amount of data posted to Ethereum L1 by Optimism (just OP mainnet) has increased steeply over the course of 2022 as the network gained traction among users. The peak was reached in December 2022 when Optimism posted roughly 5,000 MB of data to Ethereum L1. This has since slightly decreased again and average data posted to Ethereum per month amounts to around 3,000 MB.

So what did Optimism pay to post that data to Ethereum? Interestingly, the chart looks very different when we look at the total cost (in ETH) rather than the total amount of data posted. This makes sense though, as there's no multidimensional EIP-1559, and gas prices for rollups are driven up by general execution on L1 when activity on Ethereum increases.

To quickly recap for readers who are not familiar with EIP-1559:

EIP-1559 is an Ethereum Improvement Proposal introduced to modify how transaction fees work in the Ethereum network. It was implemented in the "London" hard fork in August 2021. It aimed to make transaction fees more predictable, improve the user experience, and address various other economic and security considerations. The main components are:

• Base Fee: Instead of users setting gas prices directly in a bidding system, the network sets a "base fee" for each block based on network congestion. When the network is more than 50% utilized, the base fee increases, and when it's less than 50% utilized, the base fee decreases. This provides a predictable fee mechanism.

• Fee Burn: The base fee is not given to miners (validators in the context of Ethereum 2.0). Instead, it's burned (destroyed), which has a deflationary effect on the total supply of Ether.

• Inclusion Fee (Tip): In addition to the base fee, users can provide a tip to miners to prioritize their transactions. This allows for faster transaction processing when needed.

The total cost and the average cost per MB that Optimism incurred for posting data to Ethereum in the period from January 2022 to August 2023 look as follows:

The data clearly shows that rollup cost is heavily influenced by activity both on the rollup (amount of data that is posted), as well as on Ethereum (gas price on l1). Because while the cost of calldata might be fixed at 16 gas per byte, the total gas price for a transaction is not just about the calldata. The gas price (which is denominated in gwei and is variable, determined by supply and demand dynamics) determines how much ETH the rollup will pay for each unit of gas consumed. When the Ethereum network is congested, there's higher competition for block space, which drives up this gas price. This means that, even if you're only paying for calldata and this cost is "fixed" in gas terms, the actual ETH cost can vary significantly based on network activity. It is important to note though, that activity (in the form of transaction volume) on the rollup is not the only determinant of the amount of data being posted to L1. Data compression plays an important role here as well. The Optimism Bedrock upgrade for example introduced significantly improved data compression, which had a clear impact on the cost.

Arbitrum One on the other hand started gaining traction slightly later, which is mirrored in the amount of data posted, which started to increase significantly in January 2023, peaking at almost 5,000 MB in April 2023 (which marked the top of what we could refer to as a mini bull market). Similar to Optimism, transaction volume and consequently data posted to Ethereum L1 started to decrease after that.

If we look at the actual cost that was incurred (in terms of total ETH paid), the chart is very similar to Optimism, with the peak having been reached in May 2023.

The granular data & cost data for Arbitrum One are summarized in the table below.

What is interesting to see, is that with the introduction of Arbitrum Nitro on the 31st of August 2022, Arbitrum One had a constantly lower DA cost than Optimism mainnet. This changed when Optimism caught up through the Bedrock upgrade in June 2023. Other than this, the picture is the same as for Optimism. Increased activity on Ethereum L1 often results in higher gas prices due to the competition for block space. When gas prices rise, rollup providers must pay more to post their proofs/commitments to Ethereum, and this increased cost is often passed on to the users of the rollup.

So how can this issue be addressed? Simply lowering calldata costs might not be a good idea: as it would increase the burden on nodes. While archive nodes need to store all data, full nodes store the most recent 128 blocks. An increased amount of calldata to store would therefore increase storage requirements even more, worsening decentralization and introducing bloat risks if the amount of data dumped onto L1 continues to increase. At least not before Ethereum has reached the ETH 2.0 stage that is referred to as “The Purge”.

The Purge involves a series of processes that remove old and excess network history and simplify the network over time. Aside from reducing the historical data storage, this stage also significantly lowers the hard disk requirements for node operators and the technical debt of the Ethereum protocol.

Nevertheless, rollups are already significantly reducing fees for many Ethereum users. Optimism and Arbitrum for example frequently provide fees that are ~3-10x lower than the Ethereum base layer itself. Application-specific validity rollups, which have better data compression, have achieved even lower fees in the past.

Similarly, Metis, an unconventional optimistic rollup solution, has achieved $0.01 transaction fees by implementing an off-chain solution for DA & data storage. Nonetheless, current rollup fees are still too expensive for many use cases (or in the case of Metis they come with certain security tradeoffs).

The long-term solution to this is danksharding, a specific implementation of data sharding, which will add ~16 MB per block of dedicated data space that rollups could use. However, data sharding will still take some time until it’s fully implemented and deployed.

Until that stage, EIP-4844 provides an interim solution by implementing the transaction format that will be used in danksharding (data blobs), without actually implementing any sharding on the validator set. Instead, data blobs (Binary Large Objects) are simply part of the beacon chain and carefully downloaded by all consensus nodes.

To prevent state bloat issues, these data blobs can be deleted after one month. The new transaction type, which is referred to as a blob-carrying transaction, is similar to a regular transaction, except it also carries these extra pieces of data called blobs. Blobs are extremely large (~125 kB), and are much cheaper than similar amounts of call data. This EIP-4844 primarily addresses L2 transaction fee issues and is expected to have a large impact on those (see figure 6).

So, EIP-4844 (also referred to as proto-danksharding) is a proposal to implement most of the groundwork (eg. transaction formats, verification rules) of full danksharding and is a pre-stage to the actual danksharding implementation. Rollups will have to adapt to switch to EIP-4844, but they won't have to worry about adapting more stuff when full sharding is rolled out, as it will use the same transaction format.

In a proto-danksharding paradigm, all validators and users still have to directly validate the availability of the full data. Because validators and clients still have to download full blob contents, data bandwidth in proto-danksharding is targeted to 0.375 MB/block instead of the full 16 MB that is targeted with danksharding.

However, this already allows for significant scalability gains as rollups won’t be competing for this data space with the gas usage of existing Ethereum transactions. Basically, this new data type is coming with its own independent EIP-1559-style pricing mechanism. Hence, layer 1 execution can keep being congested and expensive, but blobs themselves will be super cheap for the short to medium term.

While nodes can safely remove the data after a month, it's extremely unlikely for it to truly be lost. Even a single commitment can confirm that "this blob was committed inside block X” to prove to everyone else that it’s true (this is the 1-of-N trust assumption). Block explorers will likely keep this data, and rollups themselves will have an incentive to keep it too since they might need it. In the long term, rollups are likely to incentivize their own nodes to keep relevant blobs accessible.

Rollup-Centric Ethereum 2.0 & Data Sharding

Alongside the Proof of Stake (PoS) consensus mechanism, the other central feature in the ETH 2.0 design is Danksharding (a form of data sharding). While in the old ETH 2.0 roadmap the key idea of how to scale the network was execution sharding, where L1 execution was split among a set of 64 shards, all capable general-purpose EVM execution, Ethereum has pivoted to a rollup-centric roadmap, introducing a limited form of sharding, called data sharding: these shards would store data, and attest to the availability of ~250 kB-sized blobs of data (double the size of EIP-4844 blobs), transforming the Ethereum base layer into a secure, high-throughput data availability layer for rollup-based L2s.

As we’ve learned, making data available is a key responsibility of any rollup’s sequencer and ensures security & liveness of the network. However, keeping up with the amount of data that sequencers post onto the data availability layer can be challenging, especially if we want to preserve the network’s decentralization.

Under the Danksharding paradigm we use two intertwined techniques to verify the availability of high volumes of data without requiring any single node to download all of it:

• Attestations by randomly sampled committees (shards)

• Data availability sampling (DAS) on erasure-coded blobs

Suppose that:

• You have a big amount of data (e.g. 16 MB, which is the average amount that the ETH 2.0 chain will actually process per BLOCK initially)

• You represent this data as 64 “blobs” of 256 kB each.

• You have a proof of stake system, with ~6400 validators.

How do you check all of the data without…

• …requiring anyone to download the whole thing?

• …opening the door for an attacker who controls only a few validators to sneak an invalid block through?

We can solve the first problem by splitting up the work: validators 1…100 have to download and check the first blob, validators 101…200 have to download and check the second blob, and so on.

But this still doesn’t address the issue that the attacker might control some contiguous subset of validators. Random sampling solves the second issue by using a random shuffling algorithm to select these committees.

DAS is in some ways a mirror image of randomly sampled committees. There is still sampling going on, in that each node only ends up downloading a small portion of the total data, but the sampling is done client-side, and within each blob rather than between blobs. Each node (including client nodes that are not participating in staking) checks every blob, but instead of downloading the whole blob, they privately select N random indices in the blob (e.g. N = 20) and attempt to download the data at just those positions.

To cover the extreme case where an attacker only makes 50-99% of the data available, Danksharding relies on a technology called erasure coding, an error correction technique. The key property is that, if the redundant data is available, the original data can be reconstructed in the event some part of it gets lost. Even more importantly though, it doesn’t matter which part of the data is lost: as long as X% (tolerance threshold) of the data is available, the full original data can be reconstructed.

Rollups under the Danksharding paradigm offer similar scalability to the computational shards in the old roadmap. This is also thanks to the final piece of the puzzle: data shards. Data sharding essentially means that not every validator will continue to download the same transaction data as nodes do currently (validators also have to run nodes). Instead, Ethereum will essentially split its network of validators into different partitions (or committees) called “shards.”

Let’s say Ethereum has 1000 validators that currently store the same transaction data. If you split them into 4 groups of 250 validators each that will now store different data, you have suddenly quadrupled the amount of space available for rollups to dump data to.

This introduces a new problem. If validators within a shard will only download and store the transaction data that is dumped to their shard, validators within one shard won’t have guarantees that the entirety of the data dumped by a sequencer was indeed made available. They would only have a guarantee for their shards, but not that the rest of the data was made available to other shards.

For that reason, we run into a situation where validators in one shard cannot be sure that the sequencer was not misbehaving because they do not know what is happening in other shards. Luckily, this is where the concept of DAS comes in. If you are a validator in one shard, you can sample for data availability using data availability proofs in every other shard! This will essentially give you the same guarantees as if you were a validator for every shard, thereby allowing Ethereum to safely pursue data sharding.

With executable shards (the old vision of 64 EVM shards) on the other hand, secure cross-shard interoperability was a key issue. While there are schemes for asynchronous communication, there’s never been a convincing proposal for true composability. This is different in a scenario where Ethereum L1 acts as a data availability layer for rollups. A single rollup can now remain composable while leveraging data from multiple shards. Data shards can continue to expand, enabling faster and more rollups along with it. With innovative solutions like data availability sampling, extremely robust security is possible with data across up to a thousand shards.

A Closer Look at Danksharding

Now that we understand the concept of data sharding, let’s have a look at what the specific danksharding implementation looks like. To make the data sharding approach outlined in the previous section feasible on Ethereum, there are some important concepts we should be aware of.

In simplified terms, danksharding is the combination of PBS (proposer-builder separation) and inclusion lists. The inclusion list component is a censorship resistance mechanism to prevent builders from abusing their magical powers and force non-censorship of user transactions. PBS splits the tasks of bundling of transactions (building blocks) and gossiping the bundle to the network (broadcasting). There can be many builders, of course, but if all builders choose to censor certain transactions there might still be censorship risks within the system. With crList, block proposers can force builders to include transactions. So, PBS allows builders to compete for providing the best block of transactions to the next proposer and stakers to capture as much of the economic value of their block space as possible, trustlessly and in the protocol (currently this happens out of the protocol).

On a high level, proposers collect transactions from the mempool and create an inclusion list, which is essentially a list that contains the transaction information to be included in the block. The proposer convoys the inclusion list to a builder who reorders the transactions in the inclusion list as they wish to maximize MEV. In this way, although block proposers have no say when it comes to how transactions are ordered, proposers can still make sure all transactions coming from mempool enter the block in a censorship-resistant manner, by forcing builders to include them. In conclusion, the proposer-builder separation essentially builds up a firewall and a market between proposers and builders.

Additionally, in Danksharding’s latest design, the beacon blockchain will contain all the data from shards. This is achieved by having both Beacon blockchain and sharding data validated by a “committee” composed of validators (the shards). In this way, transactions from the same beacon blockchain can freely access shard data and can be synchronized between rollups and the Ethereum base layer, greatly improving data availability. This greatly simplifies rollup structure as problems such as confirmation delay will no longer exist and opens up new possibilities of cross-rollup composability (e.g. synchronous calls of various L2s with an L2-powered but L1-based AMM contract).

The Data Availability Landscape

So Danksharding is a combination of mechanisms that aim to securely scale the data handling capacity of the Ethereum base layer, significantly improving on the data availability guarantees that rollups get from the L1. But as mentioned before already, this is still very much under development. Development times are longer due to constraints and the necessity to align with consensus, but there are a lot of DA solutions already in the market tried by other protocols. So in this section, we will recap the different data availability solutions that we currently see on the market.

Even if ETH 2.0 is not fully rolled out yet, the spectrum of data availability solutions that Ethereum’s various L2s can leverage to tackle the DA problem has grown rapidly over the course of the past few years, as shown in the figure below.

In the bottom left of the chart, we find pure validiums, offering comparably low-security guarantees. A validium uses zero-knowledge (ZK) proofs for transaction validity and stores transaction data off-chain with a centralized data provider. Validiums offer the lowest cost per transaction in the above landscape. However, the security guarantees are comparably weak as accessing the latest state in a validium requires off-chain data to be available. Hence, a risk that the data provider misbehaves or goes offline does exist.

To address these security concerns most current validium designs utilize a DAC rather than a single data provider (as in the pure validium). Basically, DAC-based solutions can be thought of as validiums with multiple nodes, where nodes or members of the committee are trusted parties that keep copies of data off-chain and make data available. Examples of DAC-based validiums are Deversifi or ImmutableX.

Celestiums on the other hand are a novel type of L2 network based on Celestia’s DA layer using Ethereum for proof verification or dispute resolution. Depending on whether it’s a validity rollup or an optimistic rollup. Celestiums are a form of a permissionless DAC scaling solution with additional economic guarantees on data availability because the decentralized committee can be slashed in case of malicious behavior. We will dive deeper into Celestia’s DA solution later on in this report.

Traditional (or “true”) Ethereum rollups are the final group in the top right corner of the above graph. Rollups directly use the underlying Ethereum L1 as the DA layer, which comes with strong security guarantees, but also high costs since every transaction has to be posted on-chain through calldata.

Alternative Data Availability Solutions

Now that we understand how data availability is currently addressed in the Ethereum ecosystem, let’s dive in and explore these different DA solutions in detail.

Off-Chain DA Solutions

As we’ve already touched upon earlier, there are two main types of DA solutions:

• On-chain DA

• Off-chain DA

This categorization is visualized in Fig. 13 below.

While a "pure validium" means that DA can go off-chain with no guarantees, as off-chain data service providers can go offline at any time, Starkware’s StarkEx, zkSync’s zkPorter, and Arbitrum’s Nova are examples of validium scenarios that rely on a DAC. In these cases, data availability is basically provided by the rollup itself.

But as hinted before, there are also specialized, more decentralized data availability solutions that aim to strike a balance between scalability, security & decentralization. In the following subsections, we will discuss these solutions in depth.

Celestia

While most blockchain ecosystems agree that the future is multi-chain, they differ in their vision. Some approaches that have started to gain traction over the past years are Ethereum (rollups), Polkadot (parachains), and Cosmos (zones).

According to its rollup-centric roadmap, Ethereum envisions the future to evolve around rollup-based L2 solutions. Unfortunately, rollups tend to be rather expensive to operate/use and are less flexible than L1 chains (at least pre-EIP-4844 / $ETH 2.0). One of the big advantages rollups have though, is shared security.

Cosmos, on the other hand, is an ecosystem of interoperable but sovereign application-specific L1 blockchains known as “zones”. While zones can be cheaper to use and more flexible than rollups, they don’t share full security with each other, and spinning up a validator set to secure the network can be hard & expensive in itself.

What Celestia does is combine elements from both Ethereum’s and Cosmos’ approaches, as it merges Cosmos' vision of sovereign & interoperable zones with Ethereum's rollup-centric shared security approach (see Fig. 15).

Originally, Celestia was created to answer the question of what is the least a blockchain can do in order to offer shared security for other blockchains. They came up with two main things:

• It must order transactions

• Ensure data availability for chains plugged into the Celestia DA/consensus layer.

Based on these two features, developers can build out any application on top of the data availability layer. It’s important to note though, that execution doesn't happen on Celestia. But we will get to that later.

What's important to remember is that Celestia only focuses on doing two things (transaction ordering & making data available) in a highly scalable way.

A powerful feature is that Celestia scales with the number of light nodes in the network. The more data sampling light nodes there are, the larger the block size can be while still ensuring security & decentralization (see Fig. 14).

One very distinctive feature of Celestia is the separation of execution and consensus / DA layer. While validity rules determine which transactions are considered valid, consensus allows nodes to agree on the order of these valid transactions. While Celestia does have a consensus protocol (Tendermint) to order transactions, it doesn’t care about the validity of these transactions and isn't responsible for executing them. This separates execution and consensus, ensuring validity is a responsibility of the chains (or rollups) composed on top of Celestia, as visualized in Fig. 19 below.

Hence, rollup nodes simply monitor to identify and download the transactions that are relevant to their rollup to subsequently execute the transactions, thereby computing the rollup state. Interestingly, Celestia implements a Merkle tree sorted by namespaces. This is key in making it a rollup-agnostic DA layer.

Any rollup on Celestia can just download data relevant to their chain & ignore data for other rollups.

Rollups on Celestia can basically be broken down into two main categories:

• Ethereum-based rollups: Use Celestia as a data availability layer, while Celestia would attest to the Ethereum L1. In this case, the flow is Rollup > Celestia > Ethereum. At no point does Celestia attest the block data back to the L2, making it entirely rollup agnostic.

• Celestia-based rollups: Post data & validity proofs (if validity rollup) on Celestia. But while the data is stored on Celestia, Celestia won't verify the data's correctness. Hence, verification has to happen on a separate layer.

Celestia also makes rollups super flexible. As it doesn’t interpret any of the data it stores & leaves all the interpretation to rollup nodes, the canonical state of a rollup on Celestia is independently determined by nodes choosing to run a certain client software.

Rollups on Celestia are basically self-sovereign blockchains & nodes are free to hard/soft fork by upgrading their software & choosing to make sense of the underlying data in a different way, without significantly diluting network security.

Another core feature is Celestia’s execution-agnostic nature. Unlike Ethereum rollups, rollups on Celestia don’t necessarily have to be designed for fraud/validity proofs interpretable by the EVM, opening up a wide array of alternative VM choices to rollups. Also, unlike on Ethereum, active state growth & historical data are treated completely separately on Celestia. Celestia’s block space only stores historical rollup data measured & paid in bytes, while state execution is metered by the rollups themselves in their own units. Consequently, because the two things have separated fee markets, a spike of activity (and fees) in one place can’t deteriorate user experience in the other. But this only holds true for execution-related congestion. if a single rollup fills a full Celestia block, it raises DA costs for all rollups.

In conclusion, Celestia is a modular data availability & consensus layer achieving scalability while maintaining a high degree of decentralization by scaling block verification through a scalable network of light nodes leveraging DAS & erasure coding.

Avail

Another L1 blockchain focused on providing scalable data availability guarantees is Avail. Unlike Celestia, Avail is not built on the Cosmos SDK and is not using Tendermint BFT. However, Avail is also a PoS network, using a nominated PoS consensus algorithm.

The problem Avail tries to solve is the same as Celestia. Monolithic L1 chains face the state bloat issue (state bloat occurs as more transactions are added to a blockchain’s ledger). Consequently, the state of the blockchain (information necessary to execute transactions) grows and in turn, makes it more costly to run a full node (because of increased hardware requirements). This results in an undesirable scenario in which the number of full nodes starts to decrease and light nodes start to increase, centralizing the network around consensus nodes and making it more vulnerable to malicious full node behavior.

This is exactly why most chains limit the rate at which their state grows by enforcing a block/gas size limit. As many blockchains value decentralization among the validator set, they have an interest in enabling their community to run full nodes on consumer hardware. This tradeoff is embodied in the blockchain trilemma, which is visualized in the figure below.

Like Celestia, Avail takes a modular approach. The modular architecture includes:

• DA layer (Avail)

• Settlement, or rather proof verification/dispute layer (Ethereum)

• Execution layer (Rollup)

It’s important to recall that because light clients do not download the entire block, they can theoretically be tricked into accepting blocks whose underlying data is not available. The block producer might include a malicious transaction in a block & not reveal its entire content to the network, thereby compromising network security.

Avail addresses the DA problem by focusing only on ensuring the availability of the tx data posted & the tx ordering, instead of verifying the application state. A block that has consensus is considered valid only if the data behind the block is available. This lets light clients operate under almost the same security assumptions as full nodes (as long as there is at least one honest full node that can provide fraud proofs).

While Celestia relies on a fraud proof system to enable these security guarantees for light nodes, Avail uses a KZG (Kate, Zaverucha, and Goldberg) commitment scheme, which has three main features:

• Data redundancy (erasure coding).

• Fraud proof free guarantee of correct erasure coding.

• KZG commitments allow full nodes to prove transaction inclusion to light nodes using succinct proof.

In simple terms, the entire data in a block is arranged as a two-dimensional matrix. The data redundancy is brought in by erasure coding each column of the matrix to double the size of the original one.

To use Avail, rollups, instead of posting full data to the Ethereum base layer, can simply choose to become an Avail-based Validium, storing only proofs that the data is available (attestations) on Ethereum L1, instead of posting the full transaction data to L1 (basically the same that Celestiums do).

As we know from earlier, current Validiums on Ethereum use permissioned data availability committees (DACs), which are small groups of trusted institutions (or internal, crypto-economically secured validator subsets) that post signatures back to Ethereum L1, attesting to the availability of the data off-chain. They are in turn responsible for ensuring data is always accessible.

Avail provides a decentralized alternative to DACs (usually composed of 5-20 members): a permissionless network of hundreds of nodes, significantly strengthening network security & adding a security layer in the form of economic incentives (e.g. slashing).

Thanks to its chain-agnostic nature and scalable, modular architecture, Avail will be able to support an almost infinite number of rollups. In summary, the benefits of Avail compared to monolithic L1 infrastructure are:

• Self-Sovereign rollups

• Flexibility (arbitrary VM support)

• Efficient pricing (historical data vs execution)

• High-security guarantees for light clients

• Network scales with # light clients

Eigenlayer’s EigenDA

Another approach that has recently gained some attention is using middleware services leveraging the existing Ethereum validator set/trust network for security through a restaking mechanism. This approach is being pioneered by EigenLayer.

Eigenlayer is a series of smart contracts on Ethereum allowing users to re-stake liquid Ethereum staking derivatives. A mechanism to leverage the existing trust network to secure other networks.

When interacting with protocols (e.g. a DEX on Ethereum), users must trust Ethereum and the DEX they use, but also a number of other protocols that the DEX interfaces with. From a user perspective, this means:

• Potentially trusting oracles for price feed (depending on design)

• Trusting bridges for cross-chain transfer

• Trusting relayers for MEV Boost

• Trusting a DA provider (single provider, committee, or DA layer) if a user is on a Validium/DAC or Celestium

Where exactly does EigenLayer enter the picture? In all the above-mentioned cases users have to trust a middleware service that's not secured by the Ethereum validator set, hence residing outside of ecosystem security & potentially introducing a source of risk. What if there was a way to leverage the existing Ethereum trust network (Ethereum validator set) to do that?

That’s exactly what EigenLayer does. Through a restaking mechanism, EigenLayer lets stakers expose their staked ETH (stETH, rETH, and cbETH) to additional slashing conditions in exchange for securing middleware.

How does that fit into the context of data availability?

The idea is to use Eigenlayer’s restaking mechanism to provide crypto-economically secured DA guarantees. The restaking Ethereum validators that opt-in to provide DA guarantees would hence replace the DAC in the Validium case.

EigenLayer aims to achieve this through EigenDA, which is a network focused on ensuring DA for rollups that allows Ethereum to offload its DA within ecosystem security instead of going off-chain. Unlike Celestia or Avail, EigenDA doesn’t have to bootstrap a new validator set.

It’s still important to note that Ethereum validators can freely choose to opt in and secure middleware. Consequently, to achieve high-security guarantees, a middleware service has to convince a lot of Ethereum validators (through corresponding incentives).

Since EigenDA is only a DA layer (even consensus stripped off, as it is handled by Ethereum L1), it has a much higher throughput (up to 15MB/s) than Celestia (2MB/s initially). It is important to note though that this increased throughput does not come at the same level of security guarantees.

Comparison between Celestia, Avail, and EigenDA

To conclude this report, we want to make a technical comparison of EigenDA, Celestia, and Avail to understand how these solutions differ from each other.

Celestia and Avail generally are what we can refer to as general-purpose DA solutions. As we know from before, they are pretty similar in terms of technology. Nonetheless, there are some important distinctions between EigenDA and these independent L1 solutions.

A rollup that settles on Ethereum and uses Celestia for DA (a.k.a. Celestium). The L2 Contract on Ethereum verifies validity proof or fraud proof as usual, while DA is provided by Celestia. On Celestia and Avail, there are no smart contracts or computations. These application-specific blockchains only guarantee that data is available.

On Celestia, the transaction data is published to Celestia by the L2 sequencer and the Celestia validators sign the Merkle root of DA attestation that is subsequently sent to the DA bridge contract on Ethereum for verification & storage. When compared to storing DA on-chain, this greatly reduces the cost of having strong DA guarantees, while also providing security guarantees from Celestia (instead of a centralized DAC). The cost reduction aspect will be a game-changer for the entire rollup space, as calldata cost arising from posting data to Ethereum L1 accounts for +90% of rollup fees.

The data blobs that rollups post to Celestia (basically as raw data) are propagated through a P2P network, and consensus on data blobs is reached through Celestia’s Tendermint consensus mechanism that is run among its validators. Every Celestia full node has to download the entire data blob. This is different for light nodes that can use DAS to ensure DA (reach probabilistic security guarantees).In this case, Celestia acts as a L1, relying on broadcasting & consensus on data blobs. Consequently, it places high demands on the network’s full nodes (128 MB/s download & 12.5 MB/s upload). Still though, the network can scale throughput with increasing light node participation as an increasing number of data availability sampling light nodes enables larger block sizes while ensuring security & decentralization.

EigenLayer, on the other hand, employs a fundamentally different architecture, with no consensus of its own and no peer-to-peer network.

To begin, EigenDA nodes must restake ETH (or rather ETH staking derivatives) in the EigenLayer contracts on Ethereum L1. As we have already explored earlier, EigenDA nodes are a subset of Ethereum validators. Subsequently, after the DA buyer (e.g. a rollup, also referred to as disperser) receives the data blob, it encodes it with erasure code and generates a KZG commitment, publishes and distributes it for confirmation at the nodes.

After that, the disperser collects these signatures one by one, generates aggregated signatures, and publishes them to the EigenDA smart contract, which verifies the signatures.

But if the EigenDA node simply signs the attestation to claim that it stores the encoded data blob in this workflow and the EigenDA smart contract only validates the aggregated signature’s correctness, how can we be sure that the EigenDA nodes really store the data?

EigenDA uses the Proof of Custody method to achieve this. Let's take a step back and have a look at a scenario where this becomes important. So let's assume there are some lazy validators who do not complete the tasks assigned to them (such as making sure the data is available). Instead, they pretend to have completed the work and sign the final result (lying about data availability when it is not available).

Anyone can submit a proof (that a validator is lazy) to the EigenDA smart contract, which will be verified by the smart contract. If the verification is successful, the lazy validator is slashed (as it’s an objectively attributable fault).

That’s where we circle back to the concept of data availability sampling. Light nodes with DAS get an upgrade in security because they can verify if invalid blocks are produced if the DA layer only does consensus and data availability.

Celestia and Avail will both have data availability sampling, their light nodes will hence have trust-minimized security. This is different than Ethereum and EigenLayer: Ethereum with EIP-4844 will not have data availability sampling, so its light clients will not have trust-minimized security. Moreover, since Ethereum also has its smart contract environment, light clients would also need to verify execution (via fraud or validity proofs) to not rely on an honest majority assumption.

On EigenLayer, unless there is DAS (which is not yet on the roadmap), light clients (once they are supported) will rely on an honest majority among the restaked nodes that secure the EigenDA middleware.

Differently from Celestia and Avail, EigenDA’s security model is based primarily on a subset of the Ethereum validator set. It has its own slashing mechanism to ensure crypto-economic security for data availability within the Ethereum ecosystem. Consequently, more stakeholder participation in EigenDA means greater security. Reducing node requirements also contributes to greater decentralization.

Erasure coding extends a block by producing additional copies of the data. The additional data creates redundancy, giving stronger security guarantees for the sampling process. Proofs verify the correctness of the encoding. Ethereum, EigenLayer and Avail all use some type of mechanism to ensure blocks are encoded correctly.

• Avail: Validity Proofs

• Celestia: Fraud Proofs

• EigenDA: Validity Proofs

Using fraud proofs comes at a trade-off, as light nodes do have to wait for a short period before they can assume a block is correctly encoded, finalizing it in their view. This is nicely summarized in the comparison table below.

Conclusion

Data availability is currently the main bottleneck for rollup-based scaling on Ethereum. Calldata on Ethereum is expensive and the data capacity of Ethereum L1 puts a global limit on the data that rollups can post to L1. While there is a very promising roadmap to improve Ethereum’s DA capabilities through EIP-4844 and full danksharding, these upgrades are likely to still take some time. Additionally, for highly cost-sensitive use cases, Ethereum might still be too expensive even post-EIP-4844.

This puts alternative DA solutions in an interesting position. With the modular narrative gaining more traction and the benefits it unlocks becoming more tangible, we expect these protocols to gain a lot of traction & adoption. On the one hand as an intermediary solution for rollups that need scalable DA before Ethereum improves its capabilities, and on the other hand for the long tail of (app-specific) rollups that will emerge based on modular RaaS frameworks, that might cater to more cost-sensitive use cases and might not need Ethereum DA guarantees.

Larger ecosystems with general-purpose execution layers like Arbitrum, Optimism, zkSync, Starknet or Scroll, Ethereum will likely remain the DA solution of choice. However, there are modular blockchains that are opting for alternative DA solutions, including Manta Network (Celestia), Mantle Network (EigenDA), and Celo (EigenDA). These specialized DA layers also introduce different trust assumptions, which might be a factor shaping a project’s DA solution. Another important factor is definitely cost. However, this is difficult to assess without production data. We will be closely watching the production data though once available to see what the future of cheap blockspace looks like in terms of numbers.

Brought to you by zerokn0wledge

Thanks for reading, please give us a follow on Twitter at @Castle__Cap and visit our website to learn more about our services and get in touch.

Virtually yours,

The Castle