There are 2 broad approaches for database scaling: Vertical Scaling and Horizontal Scaling.

Vertical scaling is done by adding more power to an existing machine. But in this case, there is grater risk of single point of failures. And the overall cost of vertical scaling is high. Powerful servers are much more expensive.

Horizontal scaling is also called as Sharding in which we scale by adding more servers. Sharding separates large databases into smaller, more easily managed parts called shards. Each shard shares same schema though actual data on each shard is unique. We can use relational database and at the same time some of the non-relational functionalities are moved to NOSQL data store to reduce database load.

The most important factor to consider when implementing a sharding strategy is choice of sharding key. It consists of one or more columns that determine how data is distributed. It allows you to retrieve and modify data efficiently by routing database queries to the correct shard database.

Sharding is a great technique to scale the database but it introduces complexities and new challenges to the systems as mentioned below:

1. Resharding data: When a single shard could no longer hold more data due to rapid growth, it requires updating the sharding function and moving data around. Consistent Hashing is commonly used to silve this problem.
2. Celebrity problem: Also called as, Hotspot key problem. Excessive access to a specific shard can cause server overload. Imagine data for Arijit Singh, Sonu Nigam, and Kishor Kumar all kept on same shard. To overcome exhaustion of shard, we need to allocate a shard for each celebrity.
3. Join and de-normalization: Once database is sharded, it is hard to perform join operations across database shards. For this, we can de-normalize the database so that queries can be performed in a single table.

…Next : Caching