Do you know how to connect to a sharded Redis/Valkey cluster?


Do you know how to connect to a sharded Redis/Valkey cluster?

Do you know how to connect to a sharded Redis/Valkey cluster?

It is not as simple as you might think.

Perhaps you have used Redis, Valkey or one of the other popular value key stores in the past and learned the basics of how to interact with them, but have you figured out how to interact with them when they are in cluster mode?

Recently I designed a complex search engine and used Valkey running on Elasticache in Cluster Mode to allow for sharded / horizontal scalability and I wanted to give you some beginner tips for connecting to the cluster while they are still fresh in my mind.

Connecting:

Connecting is pretty straight forward. You may use redis-cli like normal to connect but then when you run your first get xyz you might get a message like as follows:

(error) MOVED 6918 172.18.0.3:6379

This might look like an error but it is really a “redirect”. According to the Redis docs redis doesn’t proxy commands from one node to another. Instead they forward the redis-cli or SDK of your choice to the node that contains the data you are looking for.

I will not go into crazy detail into how/why this is necessary here but if you need a refresher check out my YouTube Video on Redis Sharding.

Short synopsis:

Let say for example that we have 3 shards each taking 1/3 of the allocated slots(For simplicity sake we will just say there are 1k slots). Shard 1 has 1-333, Shard 2 has 334-666 and Shard 3 has 667-1000.

Again this is a gross simplification but what happens is when you send a node a get command it first does a CLUSTER KEYSLOT on the key to figure out what SLOT the key will be stored in.

The node then checks to see which node is in charge of storing that slot. If it happens to be the node you are already communicating with then it just does the get or whatever operation you sent it.

If it is NOT the node you are talking to it will do a redirect to the appropriate node which will do the same check and handling of the operation you requested.

How Do I Get Rid Of That Error:

Via the command line you need to add the -c cli arg. This will enable cluster mode and allow the Redis CLI tool to open up a new connection with the IP address or hostname supplied by the MOVED error we saw above.

Please note that you will lose the connection to the first instance you connected to and stay connected to the new instance you were redirected to until you either close the connection or are redirected to another node.

Complications:

Perhaps you are going through a bastion to connect to the cluster running on ElastiCache or you are trying to connect to one running in Docker locally. These redirects can be a bit tricky. In a post coming up I will give you some tips on how to work around that with the ANNOUNCE functionality.

Related Information:

Another fun thing to keep in mind when designing these crazy scalable systems using Redis or one of its forks it to avoid multi-key operations on a sharded cluster but I did a post on that a while ago so I won’t dive too deep into that one here.

Help!

If you are stuck with this don’t stress. Designing large scalable cost efficient infrastructure is what I do. Feel free to reach out to me and lets get you up and running.