I recently setup
redis for the application I'm working on. Because database records could take multiple hours to complete, the user has access to past requests by a unique identifier and can get subsequent results in seconds vs. hours.
The redis server setup was done with an ansible yum module, which installed an older version 3.2.12, but it was adequate for my needs so I pinned it at that version.
Environment:
CentOS Linux release 7.3.1611 (Core)
node v8.16.0
redis server 3.2.12
redis 3.0.2 (client)
I verified with redis-cli that the redis server was installed appropriately:
redis-cli ping (should return PONG)
redis-cli --version (should return redis-cli 3.2.12)
keys * (gets all keys)
set hello world
get hello
I realized for my use case it would be advantageous to set an expire time/time to live for the keys. You can practice this on command line first:
pttl hello (should return (integer) -1 since expire time/time to live wasn't set)
set foo bar ex 10 (expires in 10 seconds)
get foo (before expired)
pttl foo (should return a non-negative integer since expire time/time to live set)
get foo (after expired will return (nil))
del foo
Then I worked on client-side code to support this. This is the client module that any other module can import and use.
cacheClient.js module
const redis = require("redis");
const client = redis.createClient();
// Node Redis currently doesn't natively support promises, however can wrap the methods with
// promises using the built-in Node.js
const { promisify } = require("util");
const getCacheAsync = promisify(client.get).bind(client);
const DEFAULT_TTL_SECONDS = 60*60*24*5; // 5 days time to live
/**
* Client will emit error when encountering an error connecting to the Redis server or when any
* other in Node Redis occurs. This is the only event type that the tool asks you to provide a
* listener for.
*/
client.on('error', function(error) {
console.error(`Redis client error - ${error}`);
});
/**
* Adds key with value string to cache, wiith number of seconds to live.
* @param {String} key
* @param {String} value
* @param {int} seconds: default is 5 days
*/
function addToCache(key, value, seconds = DEFAULT_TTL_SECONDS) {
console.log(`Cache add hash[${key}]`);
// EX sets the specified expire time, in seconds.
client.set(key, value, 'EX', seconds);
}
/**
* Retrieves value with key.
* @param {String} key
* @returns {String}
*/
async function getFromCache(key) {
const val = await getCacheAsync(key);
if (val) {
console.log(`Cache retrieve hash[${key}]`);
}
return val;
}
module.exports = {
addToCache,
getFromCache,
};
Then I tested locally.
1. I made the max size of the cache very small (2MB), and I added two keys with two separate unique requests with a time to live of 5 days. I then added the 3rd one, and I verified the older one was deleted to fit the new one
- backup /etc/redis.conf
- edit /etc/redis.conf maxmemory 2mb
- restart - sudo systemctl restart redis
- check maxmemory took by issuing the following in redis-cli
- config get maxmemory
- check time to live for keys by issuing the following in redis-cli
- pttl <key>
- add keys with requests
- check memory information by issuing the following in redis-cli
- info memory
2. I had several keys in the cache, and we went on spring break, by the time I got back to work 5 days had gone by and my keys had been expired and no longer in the cache
I then went back to custom configure with ansible what I needed for redis, which were the following redis.conf changes.
1. Ensured /etc/redis dir exists in order to copy the default template /etc/redis.conf to it
2. Copied template config to /etc/redis/redis.conf
3. For the greatest level of data safety, run both persistence methods, so needed to enable append-only logs of all write operations performed by the redis server (AOF)
4. Enable daemonize so redis server will keep running in the background
5. Configure cache max memory (default out-of-the-box is no max)
6. Configure cache eviction policy (default out-of-the-box is no eviction)
7. Updated systemd to use custom config file and reloaded systemd
Number 5 and 6 together were key; if these two are not custom configured, it's possible to hit a RAM max and error out in your application!!
More commands that are useful:
info keyspace (summary of num keys, num keys with expire set, average ttl)
flushall (deletes all keys from existing database)
redis-cli --scan --pattern CS_* | xargs redis-cli del (flush only keys that start with)
redis-cli --scan | xargs -L1 redis-cli persist (make all keys persistent)
redis-cli --bigkeys (sample Redis keys looking for big keys)
