TPS isn’t Nonsense - We’re Just Using it Wrong

When blockchain enthusiasts debate the pros and cons of different chains, the transactions per second (TPS) comparison discussion inevitably starts. Then (that's a general rule), a tech guy appears out of nowhere and says TPS is nonsense.

More often than I would have liked, I found myself in situations where that tech guy was me. That karma backfired when I became involved in TPS benchmarking for different chains in the Polkadot ecosystem. Did that change my mind? Sort of. Let's get into it. 

Once again, why nonsense?

So let's reiterate why we, tech professionals, do not favor TPS as a metric. First of all, it is most often used to compare apples to oranges. In that "transactions-per-second" formula, we all know what the "second" is, but what is the transaction?

Surprisingly, sometimes that question seems more philosophical than technical. Is a batch transaction, containing 1,000 balance transfers, still considered a single transaction, or can we easily multiply our TPS by 1,000? Depends on how you look at the problem.

Secondly, TPS is considered nonsense because it provides virtually no value to the end user. Okay, in some extreme cases, like the highly overloaded and slow Bitcoin network, end users may not be happy because of low TPS, but generally, an average network user does not send thousands of transactions every minute, and what they are interested in is how much time will pass from the moment they press the "send" button to the moment the receiver sees the updated balance in their wallet.

That is called end-to-end latency and has little to do with TPS.

And last but not least, the TPS metric is easily abused. We all remember the notorious Volkswagen incident known as "Dieselgate"; the TPS metric also offers similar loopholes waiting to be exploited.

Engineers can exploit the batching mentioned above, create a special, super-light transaction type, adjust transaction weights, or employ numerous other techniques to multiply TPS by orders of magnitude.

Of course, it's a blockchain, and everything happens in plain sight... However, the voices of tech experts who understand what was abused this time are rarely heard and will likely be drowned out by the Twitter hype around a large TPS number beating competitors.

People rejoice, tech guys are facepalming, nothing unusual.

Note the gulf between the max recorded TPS and theoretical TPS. Which figure is used in marketing materials?!

So it's always nonsense technology-wise?

I used to think so myself. But my experience with TPS benchmarking convinced me it's not that simple. Yes, it doesn't make any technical sense to compare different chains by this metric. But what if we compare a single chain to itself?

For instance, we have a high-throughput application that serves a large number of users who expect their transactions to process quickly. Moreover, that network is prone to spikes: sometimes an event can triple the number of transactions per second.

In Polkadot, Elastic Scaling technology, together with Agile Coretime, makes a perfect couple for handling spikes by buying additional cores and assigning them to a chain on demand. But how do we know how many cores we even need? That's where TPS comes in.

With a predictable spike profile, we can numerically measure the TPS increase required to maintain, rather than degrade, end-to-end transaction latency, and calculate the exact number of cores needed. Users don't notice the degradation, and the network doesn't incur additional costs for cores it doesn't need.

Did we just feed all the wolves without harming the sheep?

Closely related, just looking at the other side of the problem space, are networks that don't require high throughput but do need low-latency transactions. TPS and latency are not directly connected. A high-TPS network may have high latency, accumulating transactions and sending them out when 10,000 have accumulated; or it may have low latency, sending them right away, one by one, very quickly. In Polkadot, a 0.5-second block time is achievable!

Still, a low TPS network is always high-latency in the long run. Having 1 TPS, you cannot achieve a latency of less than 1 second, obviously. Therefore, if you want low latency, one key thing to do is maintain a high TPS. But how can the TPS metric help both of the mentioned network types beyond simply calculating the number of cores, which is just basic arithmetic?

Well, the networks evolve and change their code frequently (Polkadot is not paying me extra for native advertising, so I will not mention seamless runtime logic upgrades that can be performed as often as you want). We know errors can happen sometimes. The networks wouldn't want to degrade their throughput due to those errors, so what could they integrate into their CI pipelines to ensure throughput isn't worse than in the previous version? Wait, does it sound like TPS?

Bringing substance and not hype to business'

Blockchains are businesses. The subtle difference is that blockchains are (mostly) not privately owned; in some sense, they are public companies. It just so happens that they are even more public than those "public companies" we know. So they're businesses, and what's the purpose of a business? That's easy: businesses are for profit. They sell products or services, earn revenue, and make a profit.

And I know some of my fellow tech guys will hate me for what I'm about to say, but the ultimate goal of a sound engineer is not to find beautiful technical solutions, but to help a business make a profit. It's about delivering products that are not flawless, but that a company can sell, preferably earlier, better, and cheaper than its competitors. A beautiful technical solution that doesn't sell may be an art, but not a product. Good for bragging, bad for business.

Blockchains MegaPixel problem 

Now, let's recall for a moment what our customers buy. When it comes to digital cameras, people buy megapixels. The more megapixels, the better. A niche producer making 0.3 Mpx old-school cameras would still find its audience, but would never be as profitable as a modern startup claiming it reached 108 Mpx (even though you can't always tell the difference looking at the images).

And the camera producers are not absurd; they know megapixels are nonsense. But they need them to sell. They need a screaming number on the box to incentivise a buyer to read further, and then they catch him with other killer features. They always have them, but buyers wouldn't even consider that information unless they had a "required" number of megapixels.

So, I personally hate the situation described, but yes, we do need TPS for marketing. The network must prosper, and for that, it requires users, and to attract users, it needs high TPS, sad but true. As engineers, we strive to do our best to help the network thrive.

Nonsense or not nonsense? That is the question.

To conclude, whether the TPS is nonsense or not is a matter of opinion. For a spherical engineer in a vacuum, it's definitely nonsense. For a real-life guy, be it an engineer, a product owner, or a marketer, it's definitely not: it helps us to maintain the chain's performance over time; it allows us to forecast the resource usage and make plans, including financial ones; and, finally, it creates a marketing advantage, albeit through sometimes not well-justified hype.

As a final thought, I wanted to emphasize that we in Web3 are competing with traditional payment systems, which are decades old, well-established, highly profitable, and powerful businesses, and even they are TPS-measuring with each other.

So what we must do is to attract our customers' attention with our high TPS numbers, and then persuade them with our killer features, like being permissionless and trustless, and not stealing and selling their personal data, and tens of other things we do better, but first let us show them our incredibly large TPS.

By the way, Polkadot has the ability to scale your TPS to very high levels, if you want to put a strong number on a box for your product.