Building a Better, Faster Polkadot

This article has been reproduced and adapted from this post published on Grill, a revolutionary social platform that allowed bloggers and their followers to earn together. It is an application built by the Polkadot parachain, Subsocial.

State of the Ecosystem

I have to say I am impressed how quickly the ecosystem on Polkadot is maturing. Not long ago, when I tried doing basic things on parachains, I just failed. Nothing really worked. A few months later, things don't only work, but are actually a pleasant and more importantly an exciting user experience!

I am truly proud. With easy to use DEXes, it became astonishingly easy to get hold of tokens for any parachain and just use it right away. It's not really any harder and definitely more satisfying than setting up Web2 user accounts for a service: you just get some tokens and you are good to go. I love it!

With that out of the way, what are we at Parity up to on the node side?

Agile Coretime, Mythical Games

Right now the Parachains and Node SDK team are busy building elastic scaling, and pushing asynchronous backing and Agile Coretime into production, to name a few highlights.

Additionally, we are expecting a high-volume launch of the Mythos chain from Mythical Games later this summer. This is a real-life example that showcases how Polkadot can scale. It is cool if the tech works, but if you can show it with real applications, that's a totally different thing. That's why we are really excited to do our best to make this a huge success.

For those not familiar with some of these terms: asynchronous backing enables parachains to use up to two seconds of execution in each block and will halve their block time from 12s to 6s, giving them lower latency and eight times the throughput (700% increase). Together with another feature, “clawback”, we are actually achieving 10 times the throughput (a 900% increase).

Agile Coretime allows more efficient allocation of blockspace or coretime for parachains, improving allocation efficiency and lowering the barrier to entry for teams. And finally, elastic scaling allows fast collators to use multiple cores on Polkadot simultaneously, with the effect that their blocks will be validated in parallel, allowing even higher throughput and even lower block times. Fast and optimized collators should be able to produce three blocks, each with 2s worth of execution, every 6s Relay Chain block. Therefore the effective block time for that Parachain would be just 2s, with another 3x increased throughput.

Concrete Status: Agile Coretimes is already live on Kusama, asynchronous backing is already fully live on both Polkadot and Kusama. Agile Coretime and a first version of elastic scaling are both expected to be ready for Polkadot later this summer.

On-demand Coretime

An easily overlooked feature of Agile Coretime is on-demand coretime. It’s a feature that seems quite important though, for developer experience and low barrier to entry. What on-demand coretime allows for, is low-latency + low-throughput parachains. Put differently, it's designed for early-stage teams to offer their users a superb user experience with low latency, and thus attracting more users, until eventually utilization will be high enough that buying bulk coretime becomes sensible.

Let me explain this a bit more: I, as a user, want to try out a new cool parachain, but it just launched and also it's the middle of the night, so literally nobody is using that parachain right now. To ensure a good user experience, that parachain would traditionally have to build a block or at least buy the possibility of authoring one, for every Relay Chain block, despite the fact that most of that coretime is not needed and any produced blocks would be empty.

With on-demand, where a user is sending some tokens to the chain to use it, the chain will notice this and order a core, build a block and the user is ready to go: for this process, latency should be smaller than 12s. And if someone use the chain, does a few things on it, before sending funds back to the origin chain, the chain can order a core, build a block, and provide a service to the user.

It is also possible to imagine hybrid models: an early-stage chain orders bulk, but only wants to buy one block per minute. Now they can offer their users two options: fast processing (several seconds), with higher fees, or lower fees with a wait of around a minute for those who have plenty of time to kill.

Get faster and scale more!

Thanks to Aaro Altonen we have a new networking library litep2p and he also ported our networking stack to use it, enhancing our networking performance significantly!

Why is this important? One of the biggest complaints about blockchains has always been: "It doesn't scale!" And for many, even popular projects, this is just true. Polkadot can scale by design, but we also need to prove it can. People want to build on a platform that fulfills their needs today, but they also want to be confident that it will do so tomorrow. Hence, I believe it is important to show improvements on a regular basis, to give confidence in the project. Also it is just cool to make things work at scale ;-)

Those networking improvements have already been merged to master and will be tested on some validators soon. If all goes well, we will encourage more validators to enable it and eventually make it the default.

We also have other improvements; some of these are ready and are just waiting to be rolled out and others are in the pipeline, such as systemic chunk recovery, and improving performance on data availability.

Outlook for Q3 and Q4

For the second half of 2024, I want us to go even further in terms of developer experience and scaling.

One major component I want to see is making on-demand coretime easy to use. We must have ready-to-use integration into node side + pallets, so you can just plug it in and use on-demand coretime to offer the best possible user experience.

Another crucial aspect with regards to developer experience: we will also make interfaces for builders more stable, so upgrading a parachain node should finally become easy. Ideally we will have a full blown omni-node ready by the end of the year.

It is always hard to predict how much an improvement to scalability each update brings, but I see huge potential in optimizing our approval subsystems. I don't find it too unrealistic that with additional work on networking, on top of the improvements already mentioned above, to get to 1,000 validators and at least 100 parachains, all with superb security properties.

Obviously security is a main focus all the time anyway and we do plan further improvements with regards to DoS resilience for the second half of 2024 as well.

Further Outlook: 2025 onwards

With our ecosystem maturing further and the number of messages exchanged, increasing off-chain XCMP will become more and more important. Therefore I expect this project to gain traction next year.

Thanks again to everybody building on Polkadot! Without you, all our work would be pointless. I am really glad to see everyone making incredible use of it. Thank you for being with us on this exciting journey.

About the Author

I am eskimor, lead developer here at Parity. I am leading the Node SDK, the Parachains and the Zombienet teams, building the best possible tech with those amazing teams.

The views expressed in this post are my own and don't necessarily reflect the ones of Parity Technologies.