Using Redis with Ruby on Rails

Thursday, 24 March 2011

Buy The Redis Book!

Written by Redis creator, Salvatore Sanfilippo, and key contributor, Pieter Noordhuis, the Redis Book will show you how to work with different data structures, how to handle memory, replication, and the cache itself, and how to implement messaging, amongst other things! Buy the book

Redis is an extremely fast, atomic key-value store. It allows the storage of strings, sets, sorted sets, lists and hashes. Redis keeps all the data in RAM, much like Memcached but unlike Memcached, Redis periodically writes to disk, giving it persistence.

Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.
You can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set.
In order to achieve its outstanding performance, Redis works with an in-memory dataset. Depending on your use case, you can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log.

The above quote was taken from the official Introduction to Redis page.

Redis Data Types
Installing Redis
Starting Redis Server
Redis and Rails
Example Uses in Rails
Using Redis As Your Rails Cache Store
Monitoring Redis
Redis and Unix Sockets
Redis Related Gems

Redis Data Types

Below is a general overview of the data types available to you in Redis:

Data Type	Description
String	A string of characters. The most basic data type. Can also be used as a counter, using `incr`, `incrby`, `decr` and `decrby`
Set	Sets are unsorted collections of strings with no duplicate members.
Sorted Set	Sorted sets (`zset`) are like sets in that each member has to be unqiue. Unlike sets, sorted sets have a score associated with each member
List	Lists are sorted collections of strings, essentially like an `Array` in ruby. You can `push` and `pop` on either end, making them useful for queues.
Hash	Hashes are what you’d expect: keys and string values.

As well as those basic types, there is also the ability to do PubSub with Redis. A full list of Redis commands is available on the official site.

Installing Redis

Installation is simple:

curl -O http://redis.googlecode.com/files/redis-2.2.2.tar.gz
tar xzf redis-2.2.2.tar.gz
cd redis-2.2.2
make
cp src/redis-server src/redis-cli /usr/bin

Starting Redis Server

If you’re using redis-server locally, you can get away with just running the following, which will use the default Redis config file:

redis-server

If you’re running it on a server you’ll probably want to use your own config file:

redis-server /path/to/redis.conf

The default redis.conf looks like this.

Redis and Rails

Now that we have Redis installed and running, we need to get it playing nice with Rails. Enter redis-rb, created by Ezra Zygmuntowicz:

https://github.com/ezmobius/redis-rb

First step, add redis-rb to your Gemfile:

gem 'redis', '2.1.1'

Then install the gem via Bundler:

bundle install

Lastly, create an initializer in config/initializers/redis.rb and add the following:

$redis = Redis.new(:host => 'localhost', :port => 6379)

This will create a new instance of the Redis client, connected to localhost:6379 (the default), and store it in the global variable $redis.

Let’s check that everything is working by firing up rails console:

> $redis
=> #<Redis client v2.1.1 connected to redis://localhost:6379/0 (Redis v2.2.2)> 
> $redis.set('chunky', 'bacon')
=> "OK" 
> $redis.get('chunky')
=> "bacon"

Example Uses in Rails

User Friendships

The first example I’ll show you is modelling friendships using ActiveRecord and Redis. We’ll be using sets to store the friendship data. Here is our User model:

class User < ActiveRecord::Base
  # follow a user
  def follow!(user)
    $redis.multi do
      $redis.sadd(self.redis_key(:following), user.id)
      $redis.sadd(user.redis_key(:followers), self.id)
    end
  end
  
  # unfollow a user
  def unfollow!(user)
    $redis.multi do
      $redis.srem(self.redis_key(:following), user.id)
      $redis.srem(user.redis_key(:followers), self.id)
    end
  end
  
  # users that self follows
  def followers
    user_ids = $redis.smembers(self.redis_key(:followers))
    User.where(:id => user_ids)
  end

  # users that follow self
  def following
    user_ids = $redis.smembers(self.redis_key(:following))
    User.where(:id => user_ids)
  end

  # users who follow and are being followed by self
  def friends
    user_ids = $redis.sinter(self.redis_key(:following), self.redis_key(:followers))
    User.where(:id => user_ids)
  end

  # does the user follow self
  def followed_by?(user)
    $redis.sismember(self.redis_key(:followers), user.id)
  end
  
  # does self follow user
  def following?(user)
    $redis.sismember(self.redis_key(:following), user.id)
  end

  # number of followers
  def followers_count
    $redis.scard(self.redis_key(:followers))
  end

  # number of users being followed
  def following_count
    $redis.scard(self.redis_key(:following))
  end
  
  # helper method to generate redis keys
  def redis_key(str)
    "user:#{self.id}:#{str}"
  end
end

Some sample useage:

> %w[Alfred Bob].each{|name| User.create(:name => name)}
=> ['Alfred', 'Bob']
> a, b = User.all
=> [#<User id: 1, name: "Alfred">, #<User id: 2, name: "Bob">] 
> a.follow!(b)
=> [1, 1] 
> a.following?(b)
=> true 
> b.followed_by?(a)
=> true 
> a.following
=> [#<User id: 2, name: "Bob">] 
> b.followers
=> [#<User id: 1, name: "Alfred">]
> a.friends
=> [] 
> b.follow!(a)
=> [1, 1] 
> a.friends
=> [#<User id: 2, name: "Bob">] 
> b.friends
=> [#<User id: 1, name: "Alfred">]

It is all pretty simple. We’re using the following set commands: sadd, srem, smembers, sinter, scard, sismember and the multi command. An overview of the commands is provided below:

Command	Overview
sadd	Add member to the set stored at key. If member is already a member of this set, no operation is performed. If key does not exist, a new set is created with member as its sole member.
srem	Remove member from the set stored at key. If member is not a member of this set, no operation is performed.
smembers	Returns all the members of the set value stored at key.
sinter	Returns the members of the set resulting from the intersection of all the given sets.
scard	Returns the set cardinality (number of elements) of the set stored at key.
sismember	Returns if member is a member of the set stored at key.
multi	Marks the start of a transaction block. Subsequent commands will be queued for atomic execution using EXEC.

High Score Table

We can construct a simple high score table using Redis sorted sets:

class User < ActiveRecord::Base
  # log high score
  def scored(score)
    if score > self.high_score
      $redis.zadd("highscores", score, self.id)
    end
  end
  
  # table rank
  def rank
    $redis.zrevrank("highscores", self.id) + 1
  end
  
  # high score
  def high_score
    $redis.zscore("highscores", self.id).to_i
  end
  
  # load top 3 users
  def self.top_3
    $redis.zrevrange("highscores", 0, 2).map{|id| User.find(id)}
  end
end

Sample useage:

> a, b, c, d = User.limit(4)
=> [#<User id: 1, name: "Alfred">, #<User id: 2, name: "Bob">, #<User id: 3, name: "Charlie">, #<User id: 4, name: "Derek"">] 
> a.scored 100
=> true 
> b.scored 500
=> true 
> c.scored 25
=> true 
> d.scored 10000
 => true 
> d.high_score
 => 10000 
> d.rank
=> 1 
> c.rank
=> 4 
> c.scored 5000000
=> false 
> c.high_score
=> 5000000 
> c.rank
=> 1 
> User.top_3
=> [#<User id: 3, name: "Charlie">, #<User id: 4, name: "Derek">, #<User id: 2, name: "Bob">]

We’re using the following set commands: zadd, zrevrank, zrevrange and the zscore commands. An overview of the commands is provided below:

Command	Overview
zadd	Adds the member with the specified score to the sorted set stored at key. If member is already a member of the sorted set, the score is updated and the element reinserted at the right position to ensure the correct ordering. If key does not exist, a new sorted set with the specified member as sole member is created.
zrevrank	Returns the rank of member in the sorted set stored at key, with the scores ordered from high to low. The rank (or index) is 0-based, which means that the member with the highest score has rank 0.
zrevrange	Returns the specified range of elements in the sorted set stored at key. The elements are considered to be ordered from the highest to the lowest score. Descending lexicographical order is used for elements with equal score.
zscore	Returns the score of member in the sorted set at key.

More Redis Use Cases

Using Redis As Your Rails Cache Store

If you want to use Redis as your cache store, then look no further than Redis Store by Luca Guidi. Redis Store provides namespaced Rack::Session, Rack::Cache, I18n and cache Redis stores for Ruby web frameworks. It allows you to use Redis as your Rails cache store, session store and integrates well with Rack::Cache.

https://github.com/jodosha/redis-store

First, let’s add redis-store to the Gemfile:

gem 'redis-store', '1.0.0.beta4'

Then install the gem via Bundler:

bundle install

Then update your config/environments/production.rb:

config.cache_store = :redis_store

Then you’re good to go. Your cache store should now be using Redis.

Monitoring Redis

If you’re using Redis on your server and you have daemonize set to yes in your redis.conf, you need to monitor it. Below are an example Monit and God script:

Monit

check process redis
  start program = "/usr/bin/redis-server /etc/redis/redis.conf"
  stop program = "/usr/bin/redis-cli -p 6379 shutdown"
  with pidfile /var/run/redis.pid
  group redis

God

Adapted from this Gist:

%w{6379}.each do |port|
  God.watch do |w| 
    w.name = "redis" 
    w.interval = 30.seconds 
  
    w.start = "/usr/bin/redis-server /etc/redis/redis.conf" 
    w.stop = "/usr/bin/redis-cli -p 6379 shutdown" 
    w.restart = "#{w.stop} && #{w.start}" 
  
    w.start_grace = 10.seconds 
    w.restart_grace = 10.seconds 
  
    w.start_if do |start| 
      start.condition(:process_running) do |c| 
        c.interval = 5.seconds 
        c.running = false 
      end 
    end 
  end
end

Redis and Unix Sockets

Redis 2.2 introduced the ability to connect to Redis via unix sockets. To allow this functionality you need to uncomment the following line in your redis.conf:

unixsocket /tmp/redis.sock

Then you can connect to Redis in your app, change the contents of your config/initializers/redis.rb, to this:

$redis = Redis.new(:path => "/tmp/redis.sock")

Below are some rough benchmarks from my MacBook Pro:

~ redis-benchmark -q
PING (inline): 54347.82 requests per second
PING: 52083.33 requests per second
MSET (10 keys): 28571.43 requests per second
SET: 48309.18 requests per second
GET: 49504.95 requests per second
INCR: 47846.89 requests per second
LPUSH: 48309.18 requests per second
LPOP: 49751.24 requests per second
SADD: 48543.69 requests per second
SPOP: 51282.05 requests per second
LPUSH (again, in order to bench LRANGE): 48076.92 requests per second
LRANGE (first 100 elements): 34129.69 requests per second
LRANGE (first 300 elements): 17064.85 requests per second
LRANGE (first 450 elements): 13661.20 requests per second
LRANGE (first 600 elements): 11074.20 requests per second

~ redis-benchmark -q -s /tmp/redis.sock
PING (inline): 80000.00 requests per second
PING: 82644.62 requests per second
MSET (10 keys): 41841.00 requests per second
SET: 72463.77 requests per second
GET: 81300.81 requests per second
INCR: 63694.27 requests per second
LPUSH: 70422.53 requests per second
LPOP: 80000.00 requests per second
SADD: 72463.77 requests per second
SPOP: 77519.38 requests per second
LPUSH (again, in order to bench LRANGE): 80645.16 requests per second
LRANGE (first 100 elements): 42372.88 requests per second
LRANGE (first 300 elements): 22779.04 requests per second
LRANGE (first 450 elements): 16891.89 requests per second
LRANGE (first 600 elements): 13458.95 requests per second

Redis Related Gems

Redis Objects

Redis objects maps redis data types to ruby objects. Created as a compliment for current ORMs instead of a replacement.

https://github.com/nateware/redis-objects

Resque

Resque (pronounced like “rescue”) is a Redis-backed library for creating background jobs, placing those jobs on multiple queues, and processing them later. It was written by GitHub and is used in production there, as well as on many other apps. Read their blog post: Introducing Resque

https://github.com/defunkt/resque

Jim Neath

Manchester based Ruby on Rails & Facebook App Developer