"Parallel-Committees": A Novelle Secure and High-Performance Distributed Database Architecture

In my PhD thesis, I proposed a novel fault-tolerant, self-configurable, scalable, secure, decentralized, and high-performance distributed database replication architecture, named “Parallel Committees”.

I utilized an innovative sharding technique to enable the use of Byzantine Fault Tolerance (BFT) consensus mechanisms in very large-scale networks.

With this innovative full sharding approach supporting both processing sharding and storage sharding, as more processors and replicas join the network, the system computing power and storage capacity increase unlimitedly, while a classic BFT consensus is utilized.

My approach also allows an unlimited number of clients to join the system simultaneously without reducing system performance and transactional throughput.

I introduced several innovative techniques: for distributing nodes between shards, processing transactions across shards, improving security and scalability of the system, proactively circulating committee members, and forming new committees automatically.

I introduced an innovative and novel approach to distributing nodes between shards, using a public key generation process, called “KeyChallenge”, that simultaneously mitigates Sybil attacks and serves as a proof-of-work. The “KeyChallenge” idea is published in the peer-reviewed conference proceedings of ACM ICCTA 2024, Vienna, Austria.

In this regard, I proved that it is not straightforward for an attacker to generate a public key so that all characters of the key match the ranges set by the system.I explained how to automatically form new committees based on the rate of candidate processor nodes.

The purpose of this technique is to optimally use all network capacity so that inactive surplus processors in the queue of a committee that were not active are employed in the new committee and play an effective role in increasing the throughput and the efficiency of the system.

This technique leads to the maximum utilization of processor nodes and the capacity of computation and storage of the network to increase both processing sharding and storage sharding as much as possible.

In the proposed architecture, members of each committee are proactively and alternately replaced with backup processors. This technique of proactively circulating committee members has three main results:

• (a) preventing a committee from being occupied by a group of processor nodes for a long time period, in particular, Byzantine and faulty processors,
• (b) preventing committees from growing too much, which could lead to scalability issues and latency in processing the clients’ requests,
• (c) due to the proactive circulation of committee members, over a given time-frame, there exists a probability that several faulty nodes are excluded from the committee and placed in the committee queue. Consequently, during this time-frame, the faulty nodes in the committee queue do not impact the consensus process.

This procedure can improve and enhance the fault tolerance threshold of the consensus mechanism.I also elucidated strategies to thwart the malicious action of “Key-Withholding”, where previously generated public keys are prevented from future shard access. The approach involves periodically altering the acceptable ranges for each character of the public key. The proposed architecture effectively reduces the number of undesirable cross-shard transactions that are more complex and costly to process than intra-shard transactions.

I compared the proposed idea with other sharding-based data replication systems and mentioned the main differences, which are detailed in Section 4.7 of my dissertation.

The proposed architecture not only opens the door to a new world for further research in this field but also represents a significant step forward in enhancing distributed databases and data replication systems.

The proposed idea has been published in the peer-reviewed conference proceedings of IEEE BCCA 2023.

Additionally, I provided an explanation for the decision not to employ a blockchain structure in the proposed architecture, an issue that is discussed in great detail in Chapter 5 of my dissertation.

The complete version of my dissertation is accessible via the following link: https://www.researchgate.net/publication/379148513_Novel_Fault-Tolerant_Self-Configurable_Scalable_Secure_Decentralized_and_High-Performance_Distributed_Database_Replication_Architecture_Using_Innovative_Sharding_to_Enable_the_Use_of_BFT_Consensus_Mec

I compared my proposed database architecture with various distributed databases and data replication systems in Section 4.7 of my dissertation. This comparison included Apache Cassandra, Amazon DynamoDB, Google Bigtable, Google Spanner, and ScyllaDB. I strongly recommend reviewing that section for better clarity and understanding.

I would greatly appreciate feedback and comments on the distributed database architecture proposed in my PhD dissertation. Your insights and opinions are invaluable, so please feel free to share them without hesitation.