Adjustable Timeouts and Empty Blocks

20 December 2017



One of the features of existing public blockchain solutions is statistically permanent intervals of time within which a new block must be generated. Thus, in Bitcoin it is supposed that new blocks should be generated in average every 10 minutes (every 2016 blocks the difficulty of mining task is adjusted according to the network hashrate). During development of Ethereum an interval of 12 seconds was chosen based on Bitcoin statistical data.

In private blockchains as well the intervals between blocks are often kept permanent, although this limitation can be eliminated.

In Exonum 0.2 release an opportunity to use adjustable timeouts for generating blocks was introduced. In this article we will speak about the advantages they provide.

What Is Bad about Static Timeouts for Block Generation?

Generation of blocks in permanent intervals of time, although being a simple solution, conceals a number of disadvantages as often the blockchain network workload is uneven. For example, if people (not computers) use blockchain, they generate blocks only during a certain day period (during the daytime).

  • If blocks are generated frequently, then blockchain is capable of managing peek loads, but during low network loads empty or almost empty blocks will be generated that leads to ineffective use of storage space.
  • If blocks are generated occasionally, the time of transactions acceptance will be long. Bitcoin here is an extreme case as it cannot be used for swift micro-payments.

Frequent block generation also leads to another serious problem: update time for lagging/new nodes in the network (the time that is spent for a node to catch up with most of the nodes) may be large. In Exonum almost all the time is spent on checking digital signatures and application of transactions to the storage state (execute method). This means that if blocks are generated without any additional delays in the consensus algorithm (without artificial timeouts) in the network with closely situated validator nodes (network delays are small), then the update speed will be almost equal to the block generation speed. Potentially, this can result in that lagging/new nodes will not be able to catch up with the blockchain at all. Thus, in Ethereum it is recommended that full nodes should be launched on SSD computers as HDD speed is practically not enough to catch up with the network.

Why Do We Need Empty Blocks?

With the apparent uselessness of empty blocks, they play an important role in the blockchain. Suppose that during a day in a certain instance of the blockchain there were no transactions and, accordingly, no blocks were generated. The user who requested the current network status will receive a block that was created a day ago in response, which de facto is correct in this situation. However, this behavior cannot be distinguished from collusion of the nodes that intentionally send out old blocks in order to mislead the user.

Creation of empty blocks is necessary to confirm the fact that nothing has changed in the state of the blockchain.

Exonum Approach to Adjustable Block Generation

Exonum implements the block generation method, which adjusts to the network workload (source code). This is done by managing the timeout of the first Propose message at each height of the consensus algorithm.

In the Exonum model, it is assumed that the network participants may be asynchronous in time. In particular, this is the reason block timestamps are absent in the consensus algorithm. Nevertheless, from the point of view of reducing the block acceptance time, it is desirable to make the applied timeouts identical for all validator nodes. In order this could be implemented, the data they depend on must be consistent (that is, the same for all network members). Thus, the size of the pool of unconfirmed transactions cannot be used to calculate the next timeout, because it can be different for different nodes. At the same time, any data stored in the blockchain is automatically consistent due to finality of the Exonum consensus algorithm (absence of fork in the Exonum blockchain).

Examples of consistent data include blockchain configuration: it is in it that you can specify the constants that the blockchain applies. However, adjustable timeouts cannot be implemented through configuration, since it changes seldom and only with the direct intervention of the validator nodes administrators.

Consistent data, which can give an idea of the network workload, is fullness of blocks – a ratio of the number of transactions included into a block to the maximum amount that can be included therein. Small number of transactions in the block implies small network workload and vice versa.

In Exonum 0.4 three general variants of timeout management are implemented: Constant (static), Dynamic and MovingAverage (adjustable).

Constant Timeout

Selecting this option, one timeout constant is specified in configuration - the timeout value within which the first offer is generated. This option should be used in networks with practically permanent workload.

Dynamic Timeout

Selecting this option, three values are specified in configuration: min, max - the values of the minimum and maximum timeouts, and also threshold - the threshold of the block fill (a number from 0 to 1). If the block is filled less than the threshold value, then the timeout for the next height is set to max, otherwise it is set to min.

This option is suitable for networks with clearly identified periods of enhanced workload.

Note that the minimum timeout value directly affects the maximum network throughput. If the value is too high, the network may not be able to handle the workload.

MovingAverage Timeout

Selecting this option, the following values are specified in configuration:

  • min and max - the minimum and maximum timeout values respectively
  • adjustment_speed - the rate of adjustment
  • optimal_block_load - optimal filling of the block (a number from 0 to 1)

And also previous_timeout is used - the timeout value at the previous height.

Below is a (C style) pseudo-code that implements the MovingAverage timeout logic:

float optimal_load = block_size * optimal_block_load;  
float load_percent = current_load / optimal_load;

/// Calculate target timeout for current load
float target;  
if (current_load < optimal_load) {  
    target = max - (max - previous_timeout) * load_percent;
} else {
    target = previous_timeout - (previous_timeout - min) * (load_percent - 1.) / (1. / optimal_block_load - 1.);

/// Calculate timeout
float timeout = target * adjustment_speed + previous_timeout * (1. - adjustment_speed);  

Thus, in fact, the exponential moving average scheme with adjustment_speed weight is realized. The larger adjustment_speed is, the faster the network will adapt to the current workload; however, the stronger Byzantine behavior of nodes (in this case, creation of empty blocks) can slow down creation of blocks; and vice versa.

Optimal_block_load number has a great significance: the network selects such a timeout (if possible), at which blocks are approximately filled at optimal_block_load. Choosing optimal_block_load = 1 is not always good: if the workload on the network rises sharply, there will be no space in the blocks to immediately include incoming transactions. That is, for some time the network will be "gaining speed" (the speed of block acceptance will gradually increase), and during this time the pool of unconfirmed transactions will be increasing. In case of optimal_block_load <1, the validators have a "safety margin": if network workload increases, you can include additional block_size * (1 - optimal_block_load) transactions in each block. However, too small values of optimal_block_load lead to generation of sparsely filled blocks and that increases their number.

The optimal values for adjustment_speed and optimal_block_load parameters depend on the network workload dynamics, but we recommend that the work of the blockchain network should be launched with the values within the following ranges:

0.75 < adjustment_speed < 0.95  
0.5 < optimal_block_load < 0.9  

Having analyzed the network workload (dependence of the transactions flow on time), you can not only choose the optimal parameters for the built-in timeout estimation algorithms, but also, using various methods of constructing adjustable meshes, to develop your own algorithm tailored to the task.

Adjustable block generation allows to avoid large intervals for updating lagging/new nodes and excessive use of disk space, while at the same time preserving the ability of the blockchain to work under high workload.