

Design Overview¶

This page describes the core design decisions of the Exonum framework: transaction processing, network structure, consensus algorithm, data storage organization, services, and cryptography used in Exonum.

Tip

The architecture guide in the Exonum core repository contains a more technical outlook on the Exonum architecture.

Transaction Processing¶

Tip

See the Transactions article for more details.

For an outer application, Exonum blockchain represents a key-value storage and an online transaction processing facility managing this storage. Its core functions are processing transactions, persisting data, and responding to read queries from external clients.

Transactions are the main entity Exonum works with. A transaction represents an atomic patch that should be applied to the key-value storage. Transactions are authenticated with the help of public-key digital signatures. Transactions need to be verified and ordered before they are considered accepted / committed. Ordering is performed by the consensus algorithm; the algorithm also ensures that only successfully verified transactions are committed.

Transactions are templated; each transaction template has a set of variable parameters, which influence the transaction execution and are used to serialize transactions for network transmission and persistence. (Hence, transactions could be compared to stored procedures in RDBMSs.) Transaction templates and the processing rules for each template are defined by services. In particular, services define verification rules for transactions and the way transactions are applied to the key-value storage.

All data in the Exonum blockchain is divided into two parts:

Data storage, which contains data structured into objects. Objects represent high-level wrappers over the key-value store
Transaction log, i.e., the complete history of all transactions ever applied to the data storage.

As transactions include operations on the key-value storage such as creating a new value, or updating already saved values, the actual data storage state can be restored completely from the transaction log. When a new node in the Exonum network appears, it loads already generated blocks and applies their transactions to the data storage one by one. Such approach allows seeing the whole history of any data chunk and simplifies auditing.

By using a transaction log, Exonum implements state machine replication. It guarantees agreement of data storage states among nodes in the network. The same approach is often used by non-blockchain distributed DBs, such as MongoDB or PostgreSQL.

Blocks¶

Exonum gathers transactions into blocks; the whole block is approved atomically. If a transaction has not been written to any block yet, it is not regarded as accepted. After a block is approved, every transaction in it is executed sequentially, with changes applied to the data storage.

Exonum blocks contain the following parts:

The hash of the previous Exonum block
The list of the approved transactions. When the nodes execute the block, they execute every transaction in the given order and apply changes to their data storages. Every transaction type is executed by the appropriate Exonum service
The hash of the new data storage state. The state itself is not included; however, transactions are applied deterministically and unequivocally. The agreement on the hash of data storage is a part of the Exonum consensus algorithm, so the hash is guaranteed to coincide for all validators.

As every block includes the hash of the previous block, it is impossible to change one block without the appropriate changes to each of the following blocks. This ensures immutability of the transaction log; once a transaction is committed, it cannot be retroactively modified or evicted from the log. Similarly, it is impossible to insert a transaction in the middle of the log.

Note

The agreement on the hash of the data storage means that not only full nodes execute transactions in the same order; they also must execute all transactions in exactly the same way. This protects against a scenario where execution results differ for the nodes in the network (e.g., because of non-deterministic instructions in the transaction execution code), which may lead to all sorts of trouble.

Network Structure¶

Tip

See separate articles for more details: Network, Clients.

The Exonum network consists of full nodes connected via peer-to-peer connections and light clients.

Full Nodes¶

Full nodes replicate the entire contents of the blockchain and correspond to replicas in distributed databases. All the full nodes are authenticated with public-key cryptography. Full nodes are further subdivided into 2 categories:

Auditors replicate the entire content of the blockchain. They can generate new transactions, but cannot choose which transactions should be committed (i.e., cannot generate new blocks)
Validators provide the network liveness. Only validators can generate new blocks by using a Byzantine fault tolerant consensus algorithm. Validators receive transactions, verify them, and include into a new block. The list of the validators is restricted by network maintainers, and normally should consist of 4–15 nodes

Light Clients¶

Light clients represent clients in the client-server paradigm; they connect to full nodes to retrieve information from the blockchain they are interested in and to send transactions. Exonum provides a “proofs mechanism”, based on cryptographic commitments via Merkle / Merkle Patricia trees. This mechanism allows verifying that a response from the full node has been really authorized by a supermajority of validators.

Consensus¶

Tip

See separate articles for more details: Consensus, Consensus spec, Requests.

Exonum uses a custom modification of Byzantine fault tolerant consensus (similar to PBFT) to guarantee that in any time there is one agreed version of the blockchain. It is assumed that the environment is decentralized, i.e., any node is allowed to fail or be compromised. Consensus is authenticated; consensus participants (i.e., validators) are identified with the help of public-key cryptography.

To generate a new block and vote upon it, a 3-phase approach is used.

The consensus algorithm is divided into rounds, the beginning of which is determined by each validator based on its local clock. For every round, there is a predefined leader validator, which is determined based on the round number, blockchain height and other information from the blockchain state. The leader creates a block proposal and sends it to other validators
Other validators check the proposal, and if it is correct, vote for it by broadcasting prevote messages to the validators
If a validator collects prevote messages for the same proposal from a supermajority of validators, it executes transactions in the proposal, creates a precommit message with the resulting data storage state and broadcasts it to the validators
Finally, if a validator receives precommits from a supermajority of validators for the same proposal, the proposal becomes a new block and is committed to the local storage of the validator.

Note

A block can be committed at different times for different validators. The consensus algorithm guarantees that validators cannot commit different blocks at the same height (see the safety property below).

If a validator does not receive a correct block proposal in a particular round, it eventually moves to the next round by a timeout and is ready to review proposals from the leader in the new round.

The consensus algorithm can withstand up to 1/3 of the validators acting maliciously, being switched off or isolated from the rest of the network. This is the best possible amount under the conditions in which the Exonum consensus operates (partial synchrony, which can be roughly summarized as the absence of reference time in the system). For example, a leader may not generate a proposal in time, or send different proposals to different validators; eventually, all honestly acting validators will agree on the same new block.

Validators can be changed during the blockchain operation by updating the global blockchain configuration. This mechanism can be used to rotate validators’ keys, and to add, replace or remove validator nodes without having to start a blockchain anew.

Safety and Liveness¶

Technically speaking, the consensus algorithm used in Exonum guarantees 2 basic properties:

Safety means that once a single correctly operating validator commits a block, all other correctly operating validators will eventually commit the same block at the same height; in other words, the blockchain cannot split
Liveness means that correctly operating validators continue committing blocks from time to time.

These properties are formally proven to hold for the consensus algorithm in a partially synchronous network with up to 1/3 of validator nodes being compromised or non-responsive. If the network is asynchronous (i.e., there are arbitrary high connection latencies among validators), the algorithm guarantees safety, but may lose liveness. The same happens in most scenarios in which more than 1/3 (but less than 2/3) of the validators are compromised.

Data Storage¶

Tip

See the Data Storage article for more details.

MerkleDB and RocksDB¶

Exonum uses MerkleDB as a storage framework. MerkleDB is an object database. It represents a high-level wrapper over the key-value store.

RocksDB is used for low-level key-value storage engine in Exonum. This storage engine provides high efficiency and minimal storage overhead.

Object Types¶

Exonum MerkleDB supports several types of data objects. All objects fall into blobs and root objects. Root objects represent typed data stores (lists, sets, maps and entries):

ListIndex implements an array list
MapIndex represents a map / key-value storage
SparseListIndex represents a ListIndex that may contain "gaps". It provides the possibility to delete items not only from the end of the list, but from any part thereof. Such deletions do not break the order of the indices inside the list
ProofListIndex is an enhanced version of an array store. It implements a balanced (but not necessarily full) binary Merkle tree. Leaves of the tree keep the actual array items, while the intermediate nodes keep the hashes from concatenated children data. ProofListIndex only allows to append the data or update the already stored items
ProofMapIndex extends the map. It is based on a Merkle Patricia tree, implemented as a binary tree. Leaves of the tree keep the actual values from the map. Intermediate nodes consist of the following four parts:
- Hash of the left child value
- Hash of the right child value
- Key for the left child node
- Key for the right child node
ValueSetIndex and KeySetIndex both implement sets, and both reduce them to maps (as it is commonly done in programming languages). ValueSetIndex maps hashes of the set items to the items themselves, while KeySetIndex maps set items to null. Thus, KeySetIndex is preferable when set items have relatively short serialization and need to be iterated in a deterministic order. ValueSetIndex is a better match for complex items or items that need to be looked up by a hash.
Entry and ProofEntry represent an optional single item (i.e., Option<T> in Rust terms).

Both ListIndex and ProofListIndex support updating by index, extending and truncating. MapIndex and ProofMapIndex allow inserting, updating or deleting key-value pairs. KeySetIndex and ValueSetIndex support adding and removing elements from the set. Finally, all collections support iterations over items (or keys, values, and key-value pairs in the case of maps).

Proofs¶

ProofListIndex and ProofMapIndex allow efficiently creating a proof that specific values are saved under particular keys. To prove that, it is sufficient to return a list of hashes from the tree root to a particular cell (a Merkle path). Merkle Patricia objects also allow to generate proofs that there is no data in the database with a specific key.

When a full node communicates with a light client, proofs are returned together with the requested data. This allows to prove data authenticity efficiently.

Modularity and Services¶

Tip

See the Services article for more details.

Besides the core, Exonum includes the framework for building services. While the core is responsible for consensus, and provides middleware functionality for sending and receiving transactions and blocks, services implement all business logic of the blockchain and are the main point to extend Exonum functionality.

Exonum services can be written in Rust or Java. To implement services in Java, you should use Exonum Java Binding tool.

Exonum services interact with the external world with the help of interfaces. All services may define transactions, which correspond to POST / PUT methods of REST web services. Transactions transform the blockchain state. All transactions within the blockchain are completely ordered as described above, and the result of their execution is agreed among the full nodes in the blockchain network.

Depending on the runtime enclosing the service, it may also have other interfaces. In particular, both Rust and Java runtimes allow services to define HTTP APIs, which can be categorized into two groups:

Read requests correspond to GET methods of web services. They retrieve information from the blockchain, possibly together with proofs. Read requests are executed locally, are not globally ordered, and cannot modify the blockchain state
Private endpoints provide an administrative interface to the local instance of the service. They could be used to adjust local service configuration, e.g., manage secret keys specific to the service. Private endpoints are executed locally, are not globally ordered, and cannot modify the blockchain state directly (although they can generate transactions and push them to the network).

You may use open source services already written by the community, or open your service for other uses.

Smart Contracting¶

Interfaces defined by services fulfill the same role as smart contracts in other blockchain platforms. They define business logic of the blockchain, allow to retrieve data from the blockchain, and can be reused across different projects. Partial analogies for this execution model are endpoints of REST web services and stored procedures for database management systems.

The key points differentiating Exonum smart contracts from other models used in blockchains are as follows:

Evolution as first-class concern. Exonum supports complete service lifecycle, including service upgrades, asynchronous data migrations (following the basic workflow for RDBMS migrations), stopping a service temporarily, etc. The core ensures invariants during lifecycle events, making service updates easy and safe.
Restricted execution environment. With the default configuration, Exonum executes only predefined request types, not allowing to execute untrusted code received from a client. This results in a more controlled environment, and makes it easier to argue about smart contract safety. At the same time, the logic to update the blockchain contracts is encapsulated in the supervisor service and is fully customizable. Thus, it is possible to deploy client code on an Exonum blockchain, provided that it matches restrictions set by the blockchain config.
Flexible API support. It is possible to add support of entirely new APIs in runtimes without the core needing to support it (or even know about it!).
Flexible runtime isolation. Runtimes are not isolated from the system core, but isolation can be achieved within the runtime. (For example, via the use of a virtual machine, as in the Java runtime.) This makes it possible to have both high-performance business logic and execute contracts in the sandbox. Since multiple runtimes may be present on the same blockchain, Exonum makes it possible to have both at the same time!
Local state. Exonum services may define a local state, which is specific to the node on which the service is running. The local state can be used to manage secret information (e.g., private keys). The local state may be managed by private service endpoints. By utilizing the local state, services can be more proactive than their counterparts in other blockchains. For example, the anchoring service uses the local state to fully automate anchoring transaction signing.

Existing Services¶

Supervisor Service¶

Tip

See the Supervisor article for more details.

Supervisor is a service controlling the service lifecycle. Supervisor actions are generally authorized by the supermajority of the administrators of the blockchain framework.

More precisely, the supervisor service is controlling the following activities:

Service artifact deployment
Service instances creation
Changing consensus configuration
Changing service instances configuration
Migrating service data

Anchoring Service¶

Tip

See the Anchoring Service article for more details.

The anchoring service writes the hash of the current Exonum blockchain state to the Bitcoin blockchain with a certain time interval. The anchored data is authenticated by a supermajority of validators using digital signature tools available in Bitcoin.

Anchoring increases security; even if a malefactor takes control over every validator or all validators collude, it’s impossible to change the transaction log unnoticeably. After any change, retroactively modified block hashes would differ from the one recorded on the Bitcoin blockchain. To change the data on the Exonum blockchain retroactively, an attacker would need to compromise the Bitcoin blockchain too. The cost of such an attack would measure in billions of US dollars.

Additionally, the anchored data together with proofs remains verifiable even if the underlying Exonum blockchain would become inaccessible for some reason. This property could be used to provide durable electronic receipts.

Cryptography¶

Hashing¶

Exonum uses SHA-256 for all hash operations, including creating transaction and block identifiers, computing Merkle and Merkle Patricia trees, and mapping keys for Merkle Patricia trees to fixed-length byte buffers.

Public-key Cryptography and Key Management¶

Both transactions and consensus messages are authenticated with the help of Ed25519 digital signatures implemented using sodiumoxide (a libsodium wrapper for Rust). In most cases, transactions are created by the external entities (such as light clients); these entities are assumed to manage the corresponding signing keys. Keys can also be managed by full nodes themselves. In this case, a private key is stored in the local configuration of the node, does not enter the blockchain and is specific to a particular node. It is a good practice to manage such keys locally via private APIs of the corresponding service.

The Exonum core defines two pairs of Ed25519 keys for full nodes:

Consensus key is used for signing consensus messages (for validators) and signing network messages (for validators and auditors).
Service key may be used by the service code to sign transactions. The semantics of such a transaction is that it is endorsed by the node or the node owner. Such authorization may be used in blockchain administration or in oracle services.

Services may utilize additional key pairs, including from other cryptosystems. For example, the anchoring service defines an additional secp256k1 key pair for signing anchoring transactions in Bitcoin.