SUI Testnet: Wave 1 Report

9 min readJan 19, 2023

By Trace and Velvet

Figment Inc. operated a validator node for Sui Testnet Wave 1, which ran from November 10 to December 3, 2022. To better understand the network, our team closely monitored our validator and the testnet communications channel and analyzed on-chain data. On-chain data was collected by indexing the entire testnet. In this report, we summarize our key quantitative and qualitative findings from observing Testnet Wave 1.

This testnet was the first of four planned prior to Sui’s mainnet launch which is tentatively set for Q1 2023. In total, 20 validators spanning across 8 countries and 10 time zones participated in Testnet Wave 1. In one word, the purpose of Testnet Wave 1 was “stability.” At this stage, the team was not focused on maximizing network performance. Rather, Wave 1 was about ensuring the network had a solid foundation and could produce blocks under sandbox conditions with low network load. The testnet was also an opportunity for node operators to get comfortable managing a Sui validator.

Our key conclusions include:

Overall, Testnet Wave 1 was a success in achieving basic network stability.
Since developers were still advised to use devnet, testnet activity was mostly simple test transactions by a small number of Sui ecosystem teams.
92% of transactions were payments or NFT mints.
120 methods were used across 63 smart contracts. We anticipate these numbers to grow considerably once more developers begin deploying contracts in later testnet waves.
The network reached 327 TPS without faltering, though we expect future waves to see significantly higher throughput.

Sui Infrastructure

When evaluating new opportunities, Figment holistically assesses networks for long-term potential. One of the key indicators is a network’s operational complexity — in other words, how hard it is to operate the network.

We assessed three aspects of operational complexity during Testnet Wave 1: genesis, network maintenance, and monitoring overhead.

Genesis Ceremony

Genesis is a one-time event and therefore is not critical to a network’s long-term operational complexity. However, the ease of a genesis ceremony is often a leading indicator for the network’s quality.

Sui Testnet Wave 1 was launched by the 20 validators coordinating over a zoom call. Our devops team found the network launch to be simple and straightforward, assigning it a 3 out of 10 on launch complexity. There were no major issues with launching the network and it successfully began producing blocks immediately. However, a consensus bug towards the beginning of Wave 1 required the network to be restarted (more on this later). During the second genesis ceremony, validators were able to re-launch the network simply coordinating over Discord, which highlights the straightforwardness of the launch process.

Maintenance

Communication between validators and the team was held through two private Discord channels. This ensured validators had direct lines of communication with Mysten Labs and could quickly troubleshoot any issues that may have arisen, with the help of other validators. Communication through Discord worked well for Testnet Wave 1, but will likely get busier as more validators are onboarded to the network.

Mysten Labs released three software upgrades during Testnet Wave 1. Upgrades needed to be processed one validator at a time because the team did not want to stress the network. Obviously, as the validator set scales it will be harder to have such coordinated upgrades and the network will need to be robust against such stresses to be mainnet ready.

The most notable technical issue on Testnet Wave 1 was a long-standing memory leak issue in the Sui codebase. Wave 1 allowed the Mysten team to identify and prepare a fix, which will be implemented and tested during Testnet Wave 2. The other key challenge was the aforementioned consensus stall. The team was again able to identify and patch the issue, but needed to relaunch the network for other technical reasons. Interestingly, because of Sui’s dual-consensus paths, single-writer transactions that used Byzantine Consistent Broadcast were still executing when the network was in a consensus failure. However, a planned protocol upgrade will begin sending single-writer transactions through Narwhal without full consensus, preventing this scenario in future chains.

Although upgrades required careful coordination from validators, the upgrades themselves were successful and easy to implement. Validators were generally responsive and implemented protocol upgrades within 12–24 hours.

Monitoring

Ideally, validators are monitored 24/7 to immediately identify and resolve issues. During testnets where conditions are not as sensitive, validators are usually only tasked with being responsive within 6–24 hours. However, the more issues a network is having, the more closely it needs to be monitored by the node operator. Fortunately, Testnet Wave 1 had few issues. Experiencing three network upgrades over nearly 18 days meant the day-to-day overhead was relatively light.

One of the interesting aspects of Testnet Wave 1 was how the Mysten team itself was monitoring testnet. Every validator was directed to run a Grafana Agent alongside their validator. Whereas most teams request logs from specific validators whenever issues arise, the agent enabled Mysten to proactively monitor the network in real time by automatically streaming validator data directly to their engineers. Though Grafana Agents will not be run during mainnet to foster decentralization, we believe this monitoring technique will help Mysten better understand and improve the network throughout testnet. This was a first for us — we have not heard of any other team using a similar technique but immediately recognized its strategic purpose.

Network Activity

Transactions

There were 36.5M transactions on testnet. Due to low activity, the network was not processing many transactions per second (not due to an inability to process more transactions). A graph of the network’s TPS is shown below:

For the majority of the testnet, the network processed sub-100 TPS with an average TPS of 24. At the end of Testnet Wave 1, Mysten load tested the network by spiking activity to above 200 TPS, and momentarily to a high of 339 TPS. Despite warnings to validators that such a load could take down the network, testnet remained stable at this increased level of throughput without any issues.

Simulated Network Activity

Of the 36.5M transactions, 55% were payments or asset transfers, 37% were NFT mints, 2% was AMM activity, 1% was faucet activity, and 5% was from other activities. The transactions collectively called 120 methods across 63 smart contracts. We anticipate the number of contracts used and the diversity of activity to increase when developers begin using testnet in later waves.

The 36.5M transactions were executed by 2.8M different addresses. Although developers were advised to remain on devnet, a handful of ecosystem projects including wallets and apps experimented on testnet and constituted most of the activity. Mysten did not push any traffic to the testnet until their load testing at the very end. We cannot confidently assess how many of these addresses were different people, but almost all of this activity was generated by a very small number of parties.

Gas Analysis

Every transaction on Sui has three components to its transaction costs: computation cost, storage cost, and storage rebate. The computation cost is levied on users to cover the validators’ computational effort. The storage cost charges users for the real-world hardware storage cost of their data. The storage rebate is a repayment to users from Sui’s storage fund that users receive whenever their transaction removes data from Sui’s storage.

The overall cost of any transaction on Sui can be calculated as:

Total Cost = Computation Cost + Storage Cost — Storage Rebate

On average, storage costs of a transaction made up a smaller percentage of the overall transaction cost when compared to computation costs. Computation costs were ~2.4x greater than storage costs during Testnet Wave 1, however this ratio varies significantly by transaction type and will require further analysis in future waves.

For NFT mints, the computation costs were 28x greater than storage costs, since NFT mints are computationally intensive to create. By contrast, payments are rather cheap to compute.
PaySui transactions are near-instant, peer-to-peer transactions requiring very low computational effort. The computation costs incurred to process PaySui transactions were a fraction (~6.7%) of the computation costs required to mint an NFT.

Sui’s data model allows objects to be removed from storage. When objects are permanently deleted from storage, the current object owner receives a rebate. The rebate payment received for deleting objects from storage is a large percentage of the storage cost paid by the original object creator.

An interesting but inconsequential implication of Sui’s storage rebate model is that some transaction fees on Sui can hypothetically be profitable. If a transaction receives more in storage rebate than it pays in computation and storage, the user ends up with more Sui tokens than it started with. This usually occurs when performing an action that is computationally cheap but deletes many items. A common example is when using the PayAllSui function. The function takes multiple Sui tokens, sums their total value, merges them into a single object equal to the total, and sends it to an address. One PayAllSui transaction on testnet combined 3,447 individual Sui objects into a single object, with the following gas cost:

Computation Cost: 30,752
Storage Cost: 32
Storage Rebate: 55,184

This single transaction resulted in a profitable transaction with the user earning 24,400 gas units.

While it’s still too early to reach definitive conclusions from our initial results, we will continue to monitor how the cost structure of various transaction types evolves over the four testnet waves.

The Next Wave and Beyond

Overall, Testnet Wave 1 was a success. The Sui Network successfully launched and maintained stability and liveness with a set of third party node operators while processing low levels of transaction activity. Over the course of the next few weeks, the Mysten Labs team will analyze the Testnet’s data, refine Sui’s infrastructure, apply fixes to identified issues, and prepare for Testnet Wave 2.

Testnet Wave 2 is planned for January 2023. This second phase will focus on fundamental operations and functionality including epoch management, tokenomics, and stake delegation. We also anticipate that third party application developers will be encouraged to experiment on Testnet. We expect to observe significant increases in transaction load and more diverse activity during this second wave.

Beyond continuing to analyze the metrics we have already observed, we’ll be seeking to better understand how Mysten Labs and Sui will answer the following questions in Testnet Waves 2–4:

Will the aforementioned memory leak issue be completely fixed upon launch or will the problem persist in Testnet Wave 2?
How does network stability / performance vary between different validator sets?
What is the network’s validator performance throughout the span of the network?
How does validator performance correlate with transaction volume?
Will any acute changes in network performance be experienced / observed during epoch changes?
Will operation and maintenance requirements increase in Testnet Wave 2 relative to Wave 1?
Will the core development team make any changes to the network’s gas model?
What adjustments to individual costs (computation cost, storage cost, storage rebate) have we observed for specific transaction types? Have we observed any wholesale changes when the total gas cost breakdown is compared with Testnet Wave 1?
What are the statistics around reference gas prices set by validators across epochs?
In Testnet Wave 1, each validator had an equal delegation stake. Will Testnet Wave 2 run tests with unequally weighted delegation stake? How will unequal token delegation distributions impact network performance? How will Figment’s validator perform under different scenarios?
How does the team manage communications in Testnet Wave 2 given the larger validator set? Will coordination continue as smoothly despite having more validators, including less experienced operators?
What is the transaction latency for single-writer and multi-writer object transactions?
What does the validator storage used look like, graphed over time?
Can we map the network’s storage cost to data sizes? What is the proportion between bytes and gas units?

Figment Capital is eagerly anticipating Sui’s second wave of testnet. Our team looks forward to sharing our insights from Sui Testnet Wave 2 as we gain a deeper understanding of the network.

Feel free to download and share our report here.

If you have any questions regarding Sui’s Testnet or if you would like to connect with our team, feel free to reach out.