Redis Performance Enhancement
Recently we find our Redis service is under heavy load, very high CPU(80% – 90%), and QPS(100k counts/second).
Altough the number of clients and users are increasing but the pace is not as high as the load.
After check the slow log and monitor graph, we find several causes:
Usage of bad performance commands in production environment, such as KEYS, ZRANGE, SMEMBERS
Complicated tasks in Lua script
High QPS for EXISTS command
For the first cause, we replace those commands with SCAN like commands, such as SCAN, ZSCAN, and SSCAN.
As for Lua script, the only reason we use it is to keep commands in one transaction. But if the logic is very complicated and even involves the bad performance commands, it slows the performance badly. The solution is to separate the big script to more small scripts or even commands.
And for EXISTS command, we use is as ID check in many cases. So we solve it by keeping a copy of these IDs in memory and syncing with Redis periodically.
After review of the causes of bad performance, all causes are very low level. The reason we don't avoid them is that we pay few attention on performance other than business logic. But with the increase of clients, the problem will be more and more worse. So better solve at the beginning, design phase, not with a hurry only for work done.