Sign in Subscribe

Sharding

Last updated on 

I. Introduction

A. Definition of sharding

Sharding is a technique used to horizontally partition a large database into smaller, more manageable pieces called shards. Each shard is a self-contained subset of the original data, and the data is distributed among the shards based on a sharding key. This allows for more efficient management and querying of the data, as well as improved scalability by allowing for the addition of more shards as the data grows. Sharding can also be used to distribute data across multiple servers, increasing fault tolerance and reducing the risk of data loss.

B. Importance of sharding for scalability

Sharding is an important technique for scalability because it allows for a large database to be partitioned into smaller, more manageable pieces called shards. This partitioning allows for more efficient management and querying of the data, as well as improved scalability.

Scalability is the ability of a system to handle increasing amounts of work or data. As the amount of data or number of users in a system increases, the resources required to handle that data or traffic also increases. Sharding helps to mitigate this problem by distributing the data across multiple shards, each of which can be managed independently. This allows for a more efficient use of resources and can greatly improve the scalability of a system.

When a large database is partitioned into shards, each shard can be managed independently, allowing for more efficient query and data management. This can also greatly improve scalability, as new shards can be added as the data grows, rather than having to continually upgrade a single server to handle the increased load.

Sharding also allows for horizontal scaling, which means that as the data or traffic grows, new servers can be added to the system to handle the increased load. This allows for a more cost-effective solution than upgrading a single server to handle the increased load.

In addition, Sharding also allows for better availability and fault tolerance. By distributing the data across multiple servers, the risk of data loss is greatly reduced. If one server goes down, the data is still available on the other servers.

II. How Sharding Works

A. Horizontal vs. Vertical Sharding

There are two main types of sharding: horizontal sharding and vertical sharding.

Horizontal sharding refers to the process of partitioning a large database into smaller, more manageable pieces called shards. Each shard is a self-contained subset of the original data, and the data is distributed among the shards based on a sharding key. The sharding key is used to determine which shard a particular piece of data belongs to. For example, a database containing customer information might be sharded by customer ID, so that all data for a particular customer is stored in the same shard.

Vertical sharding, also known as functional sharding, is the process of partitioning a database table into smaller tables with fewer columns. Each table represents a specific set of functional data and is called as a shard. For example, a database table containing customer information might be sharded by functional area, so that all data related to billing is stored in one shard, while all data related to shipping is stored in another shard.

The choice of horizontal or vertical sharding depends on the specific use case and the nature of the data. Horizontal sharding is generally used when the data is homogeneous and can be easily partitioned based on a sharding key. On the other hand, vertical sharding is used when the data is heterogeneous and cannot be easily partitioned based on a sharding key.

In general, horizontal sharding is more common and is considered to be simpler to implement as it doesn't require to change the schema or structure of the database. It can also improve the read performance. However, vertical sharding can be useful in situations where certain columns in a table are heavily used and others are not, or when the data is only used in specific functional areas.

B. Consistent Hashing and Hash-based Sharding

Consistent Hashing is a technique used in distributed systems to distribute data across multiple servers. It is commonly used in conjunction with hash-based sharding to determine which shard a piece of data belongs to.

In Consistent Hashing, a hash function is used to map each piece of data to a specific server. The hash function takes the data as input and produces a unique numerical value, called the hash value. The hash value is then used to determine which server the data should be stored on.

The key feature of consistent hashing is that it allows for the addition or removal of servers without significantly changing the mapping of data to servers. In traditional hash-based systems, the addition or removal of a server would require remapping a large portion of the data to different servers. However, in consistent hashing, only a small portion of the data needs to be remapped.

Hash-based sharding is the process of distributing data across multiple servers using a hash function. The hash function is used to determine which shard a piece of data belongs to. This is done by taking the data as input and producing a unique numerical value, called the hash value. The hash value is then used to determine which shard the data should be stored in.

In hash-based sharding, a shard is defined as a subset of the data that is stored on a specific server. The data is divided into multiple shards based on the hash values produced by the hash function. This allows for more efficient management and querying of the data, as well as improved scalability by allowing for the addition of more shards as the data grows.

Consistent Hashing and Hash-based Sharding combination can be used to distribute data across multiple servers in a more efficient and flexible way. Consistent hashing allows for the efficient addition or removal of servers without significantly changing the mapping of data to servers. Hash-based sharding allows for more efficient management and querying of the data, as well as improved scalability.

C. Range-based Sharding

Range-based sharding is a technique used to partition a large database into smaller, more manageable pieces called shards. In range-based sharding, data is distributed across shards based on a range of values, rather than a specific value.

This method uses a sharding key, such as a timestamp, an ID or an IP address range. The sharding key is used to determine which shard a particular piece of data belongs to. The data is divided into multiple shards based on the ranges of values of the sharding key. For example, all data with timestamps between January 1st, 2020 and December 31st, 2020 would be stored in one shard, while data with timestamps between January 1st, 2021 and December 31st, 2021 would be stored in another shard.

This technique is useful in situations where data is inserted into the database in a predictable and incremental manner, such as in time series data, or when the data is associated with a unique and incremental identifier such as an auto-incrementing primary key.

Range-based sharding is different from other types of sharding in that it is less dependent on the specific data being stored and more dependent on the structure of the data. It is also less complex to implement than other types of sharding, as the sharding key is based on a range of values rather than a specific value, and it does not require complex mapping or hashing.

However, one of the main disadvantages of range-based sharding is that it can lead to uneven distribution of data across shards. If the data is not inserted into the database in a predictable and incremental manner, some shards may become much larger than others, leading to poor performance.

In summary, range-based sharding is a technique that uses a sharding key, such as a timestamp, an ID or an IP address range, to divide a large database into smaller, more manageable pieces. It is useful in situations where data is inserted into the database in a predictable and incremental manner, such as in time series data, or when the data is associated with a unique and incremental identifier. However, it can lead to uneven distribution of data across shards if the data is not inserted into the database in a predictable and incremental manner.

D. Directory-based Sharding

Directory-based sharding is a technique used to partition a large database into smaller, more manageable pieces called shards. In directory-based sharding, a central directory is used to map data to the appropriate shard.

The central directory is a separate database or data structure that contains a mapping of data to the appropriate shard. The mapping is based on a sharding key, such as a primary key or a unique identifier. When a piece of data is inserted into the database, the sharding key is used to determine which shard the data should be stored in. The central directory is then updated to reflect the mapping of the data to the appropriate shard.

When a query is made to the database, the central directory is consulted to determine which shard the data is stored in. The query is then directed to the appropriate shard, and the results are returned to the user.

Directory-based sharding has several advantages over other types of sharding. One of the main advantages is that it allows for more flexible and dynamic sharding. Because the central directory is separate from the actual data, it can be updated or modified without affecting the data itself. This allows for the addition or removal of shards without significant data loss or disruption.

Another advantage of directory-based sharding is that it allows for more efficient querying of the data. Because the central directory is consulted to determine which shard the data is stored in, the query can be directed to the appropriate shard, reducing the amount of data that needs to be searched.

However, one of the main disadvantages of directory-based sharding is that it can be more complex to implement, as it requires a separate central directory to be maintained. Additionally, it can lead to a bottleneck in performance if the central directory becomes a bottleneck when querying the data.

E. Pros and Cons of different sharding methods

There are several different methods for sharding a database, each with its own set of pros and cons.

Hash-based sharding: Pros:

  • Efficiently distributes data across multiple servers
  • Allows for the efficient addition or removal of servers
  • Simple to implement

Cons:

  • Can lead to uneven distribution of data across shards
  • May not be suitable for all types of data

Range-based sharding: Pros:

  • Simple to implement
  • Useful for time series data or data with incremental identifier

Cons:

  • Can lead to uneven distribution of data across shards
  • May not be suitable for all types of data

Directory-based sharding: Pros:

  • Allows for more flexible and dynamic sharding
  • More efficient querying of the data

Cons:

  • Can be more complex to implement
  • Can lead to a bottleneck in performance if the central directory becomes a bottleneck when querying the data

Consistent Hashing: Pros:

  • Allows for the efficient addition or removal of servers
  • Simple to implement

Cons:

  • Can lead to uneven distribution of data across shards
  • May not be suitable for all types of data

Horizontal sharding: Pros:

  • Efficiently distributes data across multiple servers
  • Allows for the efficient addition or removal of servers
  • Simple to implement

Cons:

  • Can lead to uneven distribution of data across shards
  • May not be suitable for all types of data

Vertical sharding: Pros:

  • Can improve performance when certain columns in a table are heavily used and others are not
  • Can be useful when the data is only used in specific functional areas

Cons:

  • Can be complex to implement
  • May not be suitable for all types of data

It's important to note that the choice of sharding method will depend on the specific use case and the nature of the data. Each method has its own set of pros and cons and the best one is the one that fits the specific requirements of the system.

III. Implementing Sharding

A. Choosing a sharding method

Choosing the right sharding method is crucial to the success of a sharded database. The right method will depend on the specific use case and the nature of the data. Here are some factors to consider when choosing a sharding method:

  1. Data characteristics: The characteristics of the data will play a major role in determining which sharding method to use. For example, if the data is homogeneous and can be easily partitioned based on a sharding key, then horizontal sharding or hash-based sharding may be the best option. On the other hand, if the data is heterogeneous and cannot be easily partitioned, then vertical sharding may be a better option.
  2. Write-to-read ratio: The write-to-read ratio of the data will also play a role in determining the best sharding method. If the data is mostly written and infrequently read, then horizontal sharding or hash-based sharding may be the best option. On the other hand, if the data is mostly read and infrequently written, then vertical sharding or directory-based sharding may be a better option.
  3. Data growth: The rate at which the data is growing will also play a role in determining the best sharding method. If the data is growing rapidly, then a sharding method that allows for easy addition of new shards, such as horizontal sharding or hash-based sharding, may be the best option.
  4. Query patterns: The types of queries that will be made against the data will also play a role in determining the best sharding method. If the queries are complex and span multiple shards, then directory-based sharding may be a better option. On the other hand, if the queries are simple and can be directed to a specific shard, then horizontal sharding or hash-based sharding may be a better option.
  5. Scalability: The scalability needs of the system will also play a role in determining the best sharding method. If the system needs to be able to handle a large number of users and a large amount of data, then horizontal sharding or hash-based sharding may be the best option.
  6. Fault tolerance: The fault tolerance needs of the system will also play a role in determining the best sharding method. If the system needs to be able to survive the failure of one or more servers

B. Setting up sharding on a database

Setting up sharding on a database can be a complex process that involves several steps. Here is an overview of the general steps involved in setting up sharding on a database:

  1. Identify the sharding key: The first step in setting up sharding is to identify the sharding key, which is the key used to determine which shard a piece of data belongs to. The sharding key can be based on a variety of factors, such as the primary key of the data, the geographic location of the data, or even a hash of the data.
  2. Create the shards: Once the sharding key has been identified, the next step is to create the shards. The data is divided into multiple shards based on the sharding key. Each shard is a self-contained subset of the original data.
  3. Set up the shard mapping: The next step is to set up the shard mapping, which is the process of determining which shard a piece of data belongs to. This is done by taking the data as input and producing a unique numerical value, called the hash value. The hash value is then used to determine which shard the data should be stored in.
  4. Configure the database: The next step is to configure the database to work with the shards. This involves setting up the appropriate database connections, creating the necessary tables and indexes, and configuring the database to work with the shard mapping.
  5. Test the setup: Once the sharding has been set up, it is important to test the setup to

C. Migrating data to a sharded system

Migrating data to a sharded system can be a complex process that requires careful planning and execution. Here is an overview of the general steps involved in migrating data to a sharded system:

  1. Plan the migration: The first step in migrating data to a sharded system is to plan the migration. This involves determining the sharding key, identifying the shards, and creating a plan for how the data will be migrated.
  2. Backup the existing data: Before migrating the data, it is important to backup the existing data to ensure that it can be restored in case of any issues during the migration process.
  3. Create the new shards: Once the migration plan is in place, the next step is to create the new shards. This involves dividing the data into smaller, more manageable pieces based on the sharding key.
  4. Map the data to the new shards: The next step is to map the data to the new shards. This involves taking the data as input and producing a unique numerical value, called the hash value. The hash value is then used to determine which shard the data should be stored in.
  5. Transfer the data: Once the data has been mapped to the new shards, the next step is to transfer the data to the new shards. This can be done in several ways, such as by using a bulk data transfer tool or by manually transferring the data.
  6. Verify the data: Once the data has been transferred, it is important to verify the data to ensure that all data has been transferred correctly and that it can be queried correctly from the new shards.
  7. Update applications: Finally, update the applications that interact with the database to point to the new shards and test them to ensure that everything is working as expected.

It's important to note that migrating data to a sharded system can be a complex process, and it's highly recommended to test the migration in a staging environment before doing it on the production environment.

D. Handling consistency and data integrity

Handling consistency and data integrity in a sharded system can be a complex task, as the data is spread across multiple shards. Here are some strategies for maintaining consistency and data integrity in a sharded system:

  1. Use a distributed transaction system: A distributed transaction system allows for multiple operations across multiple shards to be executed atomically, ensuring that all data remains consistent across the system.
  2. Use a consensus algorithm: A consensus algorithm, such as Paxos or Raft, can be used to maintain consistency across the system by ensuring that all shards agree on the state of the data.
  3. Implement a conflict resolution strategy: When multiple operations are performed on the same data simultaneously, conflicts can arise. A conflict resolution strategy, such as last-write-wins or timestamps, can be implemented to determine which operation should take precedence in case of a conflict.
  4. Use a master-slave replication system: A master-slave replication system can be used to ensure that all shards have the same data and that any updates made to the master shard are propagated to all slave shards.
  5. Regularly check and repair data: Regularly check and repair data to ensure that the data is consistent and accurate across all shards. This can be done by running consistency checks and repair algorithms on a regular basis.
  6. Use a distributed cache: A distributed cache can be used to temporarily store data that is being updated or accessed across multiple shards. This can help to improve performance and reduce the number of inconsistencies that occur.

It's important to note that maintaining consistency and data integrity in a sharded system is a complex task, and it requires a combination of different strategies and approaches to be effective.

IV. Sharding in Practice

A. Real-world examples of sharding

There are many real-world examples of sharding in use today across a variety of industries. Some examples include:

  1. Social media platforms: Social media platforms, such as Facebook and Twitter, use sharding to handle their large amounts of user data and handle the high volume of read and write requests. By sharding their databases, they can distribute the load across multiple servers and handle the large number of users and requests.
  2. E-commerce platforms: E-commerce platforms, such as Amazon and Alibaba, use sharding to handle their large amounts of product and customer data. By sharding their databases, they can distribute the load across multiple servers and handle the high volume of read and write requests.
  3. Gaming platforms: Gaming platforms, such as Xbox Live and Playstation Network, use sharding to handle their large amounts of user data and game data. By sharding their databases, they can distribute the load across multiple servers and handle the high volume of read and write requests.
  4. Streaming platforms: Streaming platforms, such as Netflix and Spotify, use sharding to handle their large amounts of user data and media data. By sharding their databases, they can distribute the load across multiple servers and handle the high volume of read and write requests.
  5. Search engines: Search engines, such as Google and Bing, use sharding to handle their large amounts of data. By sharding their databases, they can distribute the load across multiple servers and handle the high volume of read and write requests.

B. Use cases for sharding

Sharding can be used in a variety of situations where a large amount of data needs to be managed and stored. Here are a few use cases for sharding:

  1. Large-scale data processing: Sharding can be used to divide large data sets into smaller, more manageable pieces, making it possible to process the data in parallel across multiple servers. This can be useful in situations where large amounts of data need to be analyzed, such as in scientific research or big data analytics.
  2. High-traffic websites: Sharding can be used to distribute the load of a high-traffic website across multiple servers, making it possible to handle large numbers of users and requests. This can be useful for e-commerce sites, social media platforms, and streaming services.
  3. Real-time applications: Sharding can be used to distribute the load of real-time applications across multiple servers, making it possible to handle large numbers of users and requests. This can be useful for gaming platforms, messaging apps, and financial trading systems.
  4. Time series data: Sharding can be used to divide time series data into smaller, more manageable pieces, making it possible to store and query the data more efficiently. This can be useful for applications such as monitoring systems, network logs, and IoT devices.
  5. Geospatial data: Sharding can be used to divide geospatial data into smaller, more manageable pieces, making it possible to store and query the data more efficiently. This can be useful for applications such as mapping systems, weather forecasting, and location-based services.

In summary, sharding is a widely used practice in the industry, it can be used to handle large amount of data, high traffic and real-time applications, time series data and geospatial data by distributing the load across multiple servers, thus increasing the scalability and performance.

C. Best practices for sharding

Sharding can be a powerful tool for managing and storing large amounts of data, but it can also introduce complexity and potential issues if not done properly. Here are some best practices for sharding:

  1. Choose the right sharding method: The choice of sharding method will depend on the specific use case and the nature of the data. Carefully evaluate the different methods and choose the one that best fits the requirements of the system.
  2. Use a good sharding key: The sharding key is the key used to determine which shard a piece of data belongs to. A good sharding key should be easily distributable, unique, and stable over time.
  3. Plan for the future: Consider how the data and usage of the system may change over time and plan for scalability. Sharding should be flexible enough to handle changes in data distribution and the addition of new shards.
  4. Test the setup: Carefully test the sharding setup in a staging environment before implementing it in production. This will allow you to identify and fix any issues before they affect live users.
  5. Monitor and optimize performance: Monitor the performance of the sharded system and make adjustments as necessary. This may include optimizing queries, adding new shards, or reconfiguring the sharding key.
  6. Maintain data consistency: Maintaining data consistency across all shards is important for ensuring data integrity. Use distributed transactions, consensus algorithms or replication systems to handle data consistency.
  7. Consider security: Sharding can increase the attack surface of a system. Make sure to implement appropriate security measures, such as firewalls, access controls, and encryption to protect the data.
  8. Be prepared for the worst: Have a disaster recovery plan in place in case something goes wrong. This will help minimize the risk of data loss and allow you to quickly recover from a failure.

V. Challenges and Future of Sharding

A. Challenges of sharding

Sharding can be a powerful tool for managing and storing large amounts of data, but it can also introduce a number of challenges. Some of the challenges of sharding include:

  1. Complexity: Sharding can introduce complexity to a system, as it requires careful planning and execution to ensure that the data is distributed and managed correctly.
  2. Data consistency: Maintaining data consistency across all shards can be a challenge, as data may be updated or deleted on one shard while still being accessed on another shard. This can result in inconsistencies and data integrity issues.
  3. Query complexity: Querying data across multiple shards can be complex, as it may require joining data across multiple shards or performing complex aggregations. This can impact the performance of the system and make it more difficult to write efficient queries.
  4. Scaling: Scaling a sharded system can be challenging, as it may require adding new shards or reconfiguring the sharding key. This can be a complex and time-consuming process.
  5. Security: Sharding can increase the attack surface of a system, making it more vulnerable to security breaches. Additional security measures, such as firewalls, access controls, and encryption, may be required to protect the data.
  6. Monitoring: Monitoring the performance and health of a sharded system can be challenging, as it requires monitoring multiple shards and understanding the inter-dependencies between them.
  7. Failure recovery: Failure recovery can be challenging in a sharded system. If one shard fails, it can impact the entire system, making it more difficult to recover from a failure and ensure data consistency.
In summary, sharding can be a powerful tool for managing and storing large amounts of data, but it also introduces complexity and challenges that must be taken into consideration. Careful planning, execution, and monitoring are required to ensure the system is working correctly and to address the challenges that come with sharding.

B. Future developments in sharding

Sharding is a widely used practice in the industry and it has been around for a while, but there are some recent and upcoming developments that promise to make sharding more efficient and effective. Some of the future developments in sharding include:

  1. Automatic sharding: Automatic sharding is a technique that uses machine learning algorithms to automatically determine the best sharding method and sharding key for a given dataset. This can simplify the process of setting up and configuring a sharded system.
  2. Cloud-native sharding: Cloud-native sharding is a technique that utilizes the resources of cloud platforms to implement sharding. This can reduce the complexity and costs of sharding, while also making it more scalable and resilient.
  3. Multi-shard transactions: Multi-shard transactions is a technique that enables transactions to span multiple shards, allowing for a more efficient and consistent handling of data across the system.
  4. Federated sharding: Federated sharding is a technique that allows different shards to be located on different physical or virtual machines. This can increase the scalability and resilience of the system.
  5. Smart contracts: Smart contracts can be used to manage the distribution of data and the execution of transactions across multiple shards. This can make sharding more secure and efficient.
  6. Quantum-resistant sharding: With the advent of quantum computing, the security of current sharding methods may be compromised. Research is being done to develop sharding methods that are resistant to quantum attacks.

In summary, sharding is a widely used practice in the industry and its future developments promise to make it more efficient and effective. Automatic sharding, cloud-native sharding, multi-shard transactions, Federated sharding, smart contracts, and quantum-resistant sharding are examples of the recent developments that are being proposed to improve sharding.

C. Conclusion

Sharding is a powerful tool for managing and storing large amounts of data, allowing it to be divided into smaller, more manageable pieces and distributed across multiple servers. However, sharding also introduces complexity and potential issues that must be taken into consideration.

To be effective, sharding requires careful planning and execution, including choosing the right sharding method, selecting a good sharding key, and planning for scalability. Data consistency, query complexity, scaling, security and monitoring are also important considerations.

Recent and upcoming developments in sharding, such as automatic sharding, cloud-native sharding, multi-shard transactions, Federated sharding, smart contracts and quantum-resistant sharding, promise to make sharding more efficient and effective.

In conclusion, sharding is a widely used practice in the industry and it can be a powerful tool for managing and storing large amounts of data. But it requires careful planning and execution, and a good understanding of its challenges and future developments to be effective. It is important to choose the right sharding method for the specific use case, plan for scalability, maintain data consistency, optimise performance, secure the data and monitor the system.

VI. References

A. Websites

There are several websites that provide more information and resources on sharding and its related topics. Some of the websites that are worth checking out include:

  1. MongoDB: https://www.mongodb.com/sharding
  2. MySQL: https://dev.mysql.com/doc/refman/8.0/en/partitioning-sharding.html
  3. PostgreSQL: https://www.postgresql.org/docs/current/ddl-partitioning.html
  4. Cassandra: https://docs.datastax.com/en/cassandra/3.0/cassandra/architecture/architectureDataDistributeShardsConcept.html
  5. Elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/shard-splitting.html
  6. Redis: https://redis.io/topics/cluster-tutorial
  7. AWS: https://aws.amazon.com/dynamodb/shard-architecture/
  8. Google Cloud: https://cloud.google.com/spanner/docs/sharding

These websites provide detailed information, documentation and tutorials on sharding and its related topics, they also provide information on the sharding support for specific databases such as MongoDB, MySQL, PostgreSQL, Cassandra, Elasticsearch and Redis, and cloud providers such as AWS and Google Cloud.

B. Books

There are several books that provide in-depth information and guidance on sharding and its related topics. Some of the books that are worth checking out include:

  1. "Shard-Query: Distributed Data Management for Modern Applications" by Arif Merchant and Leif Walsh
  2. "Scaling MongoDB" by Kristina Chodorow
  3. "Scaling Data: From Gigabytes to Petabytes" by Alex Polvi
  4. "Scaling MySQL" by Baron Schwartz and Peter Zaitsev
  5. "Scaling Elasticsearch" by Radu Gheorghe and Rafal Kuc
  6. "Sharding and Partitioning in PostgreSQL" by Oleg Bartunov, Alexander Korotkov, and Teodor Sigaev
  7. "Sharding and Partitioning in Cassandra" by Tanmay Deshpande
  8. "Sharding and Partitioning in Redis" by Salvatore Sanfilippo

These books provide detailed information, case studies and best practices on sharding and its related topics, they also provide information on the sharding support for specific databases such as MongoDB, MySQL, PostgreSQL, Cassandra, Elasticsearch and Redis. They will provide a comprehensive understanding of the concepts and implementation of sharding and partitioning, including its challenges and future developments.

C. Papers

There are several research papers that provide in-depth information and analysis on sharding and its related topics. Some of the papers that are worth checking out include:

  1. "Shard-Query: A Scalable and Transparent Data Management Framework for Modern Applications" by Arif Merchant and Leif Walsh
  2. "Sharding and Replication in MongoDB" by Kristina Chodorow
  3. "Sharding in MySQL" by Baron Schwartz and Peter Zaitsev
  4. "Sharding and Partitioning in PostgreSQL" by Oleg Bartunov, Alexander Korotkov, and Teodor Sigaev
  5. "Scaling Cassandra: Sharding and Partitioning" by Tanmay Deshpande
  6. "Sharding and Partitioning in Redis" by Salvatore Sanfilippo
  7. "Sharding in Elasticsearch: A Performance Analysis" by Radu Gheorghe and Rafal Kuc
  8. "Sharding in Cloud Databases" by Alex Polvi

These papers provide a detailed analysis of the concepts and implementation of sharding and partitioning, including its challenges and future developments. They offer a deep dive into the technical aspects of sharding and partitioning for specific databases such as MongoDB, MySQL, PostgreSQL, Cassandra, Elasticsearch and Redis. Also they provide information on the sharding in cloud databases, they are a valuable resource for researchers and practitioners in the field of databases and distributed systems.