Distributed System Lock Implementation using Redis and JAVA
The purpose of a lock is to ensure that among several application nodes that might try to do the same piece of work, only one actually does it (at least only one at a time).
In current days I worked on Redis lock-in distributed system. In distributed system Locking/concurrency management is a very important thing. Without prior knowledge, many unwanted problems may occur.
What we will learn here
- Create a simple wallet backend with buggy architecture and code
- Create the Race condition and identify the problem
- Solve the problem using new architecture and implementation
- Share the codebase with file descriptions
Here I am sharing an application architecture. Suppose this is a bank wallet application backend. Its architecture is very simple. It has a single app server, single app node, single Redis server, and a single database
Here, Clients request directly to come to the application server (Tomcat), which passes to the application. Here we don't have a load balancer and our application has a single instance. This application has a single Redis server and a single database.
For this tutorial, you must have knowledge of spring boot’s basic architecture. To achieve this you can follow my bellow tutorial
Spring data JPA : A to Z
According to Spring documentation Spring Data JPA, part of the larger Spring Data family, makes it easy to easily…
In a traditional wallet system, for fund transfer feature requires one sender, one receiver, and transaction amount. For example, the client’s request is :
Here, sender wallet no is 100010 and receiver wallet number is 10020 and transaction amount : 40$
- Concurrent send money not possible for the same sender
- For concurrent request, first request will execute and the second one will get an exception
To gain this we can follow bellow procedure
Step 2 : We are checking is sender has pending fund transfer or previous fund transfer is running using sender walletNo ? If running the throwing with proper error message.
Step 4 & 5 : Here fetching some data from transactional database for data and limit validation. After that it is making sender account busy so that same sender may not come to do fund transfer again in this moment.
Step 6 & 7: In this stage it is building some database objects and saving in database.
This architecture is looking good and it’s implementation is easy.
Its implementation is given bellow
Code for Redis service layer
The Redis service layer is calling from the application service layer.
isTransactionRunningForSender : it will send true if key is making transaction
makeSenderBusyForTransactionNormal : Here, in this method key is setting the value. That means It is making busy of that sender via redis
Code for application service layer
Here, Redis key is like TRANSACTION-100010
The current architecture will work fine in a single server single app node-based system. It will fail in a distributed system where more than one app node handles requests via load balancer/HA proxy. That means it will fail in a scalable distributed system. Suppose our new distributed system architecture is given below:
Load balancer: all requests are coming into the Load balancer, it implements a round-robin scheduling algorithm. So it will pass each request in it’s connected app nodes
App Node: Here two-node for our wallet backend is connected with our load balancer, each node sharing a single Redis and a single database server.
In the flowchart, in step 2 we are checking does sender wallet making transactions, and in step 5 we are making busy of that wallet, and in step 7 we are making free of that account. Between steps 2 and 5 we are doing some DB queries and computation. It may take some time.
Consider a scenario:
Two requests come to load balancer at a time for that sender with the same data. This scenario may generate many ways. Load balancer passes the first request to App node 1 and the second request to App node 2. Both requests will hit steps 2 at a time. Both requests will found false from method isTransactionRunningForSender() and both will create transactions at a time.
Now how to solve this problem? What is the optimum solution?
After redis version 2.6.12, Redis set command was expanded to avoid the two issues above. The parameters of the new Redis set command are as follows:
public String set(final String key, final String value, final String nxxx, final String expx,
final int time)
final String nxxx = The value of the key is set only if the key does not exist. That means if in redis exists same key it will not allow to set key and value. If key not exists then it will set that key and value
final int time = Lock timeout in miliseconds. If application not release the lock in this provided time, Redis automatically release this lock.
Our new working algorithm will be like bellow flowchart
Step 2: In this step, we are trying to acquire the lock. If we failed then we will throw an exception. Here failed means, this sender is already making transactions in another node. If success then we will complete the transaction and finally we will release this this lock.
Service Layer Code
acquireLock: It will try to acquire the lock for provided key. It will return true if acquire done otherwise false. If the key already exists it can’t acquire again until release.
releaseLock: It will release the lock for provided key
You can find the whole code in this Github repository.
Controller Name : RedisTestController
createTransactionNormal: For old-style buggy code
createTransactionLock: For new-style final working code
Service class name: RedisTestServiceImpl
Redis Service class: RedisCacheRepository
Swagger Url: http://localhost:8081/springreadyapp/swagger-ui.html#/RedisTestController
Thanks in advance. Happy coding :)