Recently, at work, we started using MiniRedis as a lightweight store for some job queuing on Kiln's backend. We had originally planned on using the full Redis, but because we have to deploy licensed Kiln on Windows, we had to come up with our own solution.
So far, it's been working very well for us. The Redis command set is pretty small and very straightforward, which makes it easy to clone. The only annoyance we've run into is that the stable command line interface (CLI) for Redis only speaks the 1.x protocol, while MiniRedis only speaks the 2.0 version. The redis CLI also does not run on Windows. This is a bit of a problem, since we're still working on our queuing system and we need to do testing.
To get around not having a client to use, Ben would telnet in to the MiniRedis port and type out the commands manually. It ended up looking a bit like this:
I, on the other hand, would fire up Python and, using the redis-py library (which is a very nice client library), issue commands directly from there. Neither option was very convenient.
So one day, tired of having to do all the imports and set up the connection, I decided to put together a CLI using Python.
import cmd
class RedisCli(cmd.Cmd, object):
pass
if __name__ == '__main__':
RedisCli().cmdloop()
The basic CLI is very simple. Python's cmd module takes care of all the hard parts for you. If you run this script, you'll get a prompt (Cmd) that supports one command: help. Unfortunately, that's all you have. You can't even exit gracefully. So that's the first thing I added:
class RedisCli(cmd.Cmd, object):
def do_exit(self, line):
return True
do_EOF = do_exit
The cmd module lets you define commands by writing a do_foo() method which takes the line the user typed in. The above code gives you the exit command, and also makes EOF (Unix: Ctrl-D, Windows: Ctrl-Z) exit for you. That's helpful, but it doesn't really add much in terms of actual functionality. For that, we import the Redis client libraries
from redis import Redis
initialize the connection
class RedisCli(cmd.Cmd, object):
def __init__(self, host, port):
self.redis = Redis(host=host, port=port)
and add some of the Redis commands
def do_get(self, line):
print self.redis.get(line)
def do_set(self, line):
key, value = line.split()
print self.redis.set(key, value)
This is nice, because help will now list get and set. But Redis has many more commands available. One option would be to continue adding all the Redis commands until you had the full set specified and properly parsing the command line. That's pretty time consuming, brittle, and just plain boring. Personally, I don't have the patience.
Fortunately, the cmd module has a default() method which is called for any unrecognized functions. That means we can get rid of do_get and do_set and replace them with:
def default(self, line):
parts = line.split()
print getattr(self.redis, parts[0])(*parts[1:])
Huh? Let me explain: getattr() takes an object and an attribute name, and returns that attribute if it exists. To illustrate, calling getattr(obj, 'foo') is the same as calling obj.foo. In this case, we're assuming the user typed in one of the functions defined in the Redis object. We use getattr to get that function, and then we pass the rest of the arguments to it using Python's *args syntax. This sidesteps the problem of not knowing how many arguments the commands take.
Unfortunately, this approach is prone to errors. For example, the user can type in __init__, which will call the Redis object's constructor again, overwriting our connection. Someone could also try to call foobarbazbat, which does not exist in the Redis object and will throw an error. Lastly, we've also lost our list of commands when you type help.
To fix this, we're going to have to do some spelunking in the Redis object. Fortunately, Python's dir() function returns all of the Redis object's attributes. We can then iterate over them, filter out any that start with an underscore (Python's convention for private attributes) and make sure they're callable. We then use setattr to create a function in our own class that calls into the Redis object and prints the result.
def __init__(self, host, port):
super(RedisCli, self).__init__()
self.redis = Redis(host=host, port=port)
for name in dir(self.redis):
if not name.startswith('_'):
attr = getattr(self.redis, name)
if callable(attr):
setattr(self.__class__, 'do_' + name, self._make_cmd(name))
@staticmethod
def _make_cmd(name):
def handler(self, line):
parts = line.split()
print getattr(self.redis, name)(*parts)
return handler
Notice here that _make_cmd is creating a new function inside of it, and returning that function so we can set the do_foo of our own class to the function that calls self.redis.foo(). Likewise for any callable function in the Redis class.
Now if we type help on our command line, we'll get a list of all functions in the Redis object. Also, if we try to access anything private, like __init__, we'll be told that syntax is unknown. This also means that our default() method is no longer necessary, since we've already enumerated everything that we could possibly call on the Redis object.
You'll notice, however, that help lists all of the functions as "Undocumented". It would be really nice if we could also get documentation for each of these commands. Now that we can easily list all of the available commands, we could write the documentation ourselves by specifying a help_foo() function for each command. However, this is boring and like I said before, I don't have that kind of patience. It also turns out that writing our own documentation would be redundant, as the authors of our Redis client library have done a good job documenting each function in the form of docstrings:
def get(self, name):
"""
Return the value at key ``name``, or None of the key doesn't exist
"""
Python takes these docstrings pretty seriously. In fact, they become an attribute of the function itself, called __doc__. This is great for us, because it means we can pull those docstrings into our CLI and make them documentation for our commands. We use the same method as before to dynamically add help_foo() methods to our own class for every function that has a docstring:
doc = (getattr(func, '__doc__', '') or '').strip()
if doc: # Not everything has a docstring
setattr(self.__class__, 'help_' + name, self._make_help(doc))
@staticmethod
def _make_help(self, doc):
def help(self):
print doc
return help
So now if we type help, we'll get a list of "Documented" and "Undocumented" commands. If we type help get, it'll tell us "Return the value at key ``name``, or None of the key doesn't exist." Awesome!
This is getting to be a useful little CLI. In addition to having a documented list of all commands available, we also get tab completion for our commands, so if we type l<TAB>, we get the list of all list commands. (On Windows, you'll need the pyreadline module installed for this to work.) But what if we could also autocomplete our keys? Some of our keys get pretty long, and typing them out is a pain. We could define a complete_foo() method for each of our functions, but all we're ever going to be completing are the keys, so we can just use the completedefault, which is a catchall completion, to grab our keys for us.
def completedefault(self, text, line, start, end):
return self.redis.keys(text + '*').split()
Once we've added this, we can type get bar<TAB> and we'll get all of the keys that start with bar.
We're in the home stretch now. Just to make things a little nicer, let's modify the prompt and the intro message so the user knows they're in the Redis CLI:
def __init__(self, host, port):
...
self.prompt = '(Redis) '
self.intro = '\nConnected to Redis on %s:%d' % (host, port)
And finally, because this is a tool we want to be able to use with different servers, let's add the ability to specify a host (-h or --host) and port (-p or --port).
import getopt
if __name__ == '__main__':
opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])[0])
host = opts.get('-h', None) or opts.get('--host', 'localhost')
port = int(opts.get('-p', None) or opts.get('--port', 12345))
RedisCli(host=host, port=port).cmdloop()
To finish, we just add some error checking so we don't get bailed out with an exception if we happen to make a typo. Here's the final script:
import cmd
import getopt
import sys
from redis import Redis
from redis.exceptions import ConnectionError, ResponseError
class RedisCli(cmd.Cmd, object):
def __init__(self, host, port):
super(RedisCli, self).__init__()
self.redis = Redis(host=host, port=port)
self.prompt = '(Redis) '
self.intro = '\nConnected to Redis on %s:%d' % (host, port)
for name in dir(self.redis):
if not name.startswith('_'):
attr = getattr(self.redis, name)
if callable(attr):
setattr(self.__class__, 'do_' + name, self._make_cmd(name))
doc = (getattr(attr, '__doc__', '') or '').strip()
if doc:
doc = (' ' * 8) + doc # Fix up the indentation
setattr(self.__class__, 'help_' + name, self._make_help(doc))
try:
# Test the connection. It doesn't matter if 'a' exists or not.
self.redis.get('a')
except ConnectionError, e:
print e
sys.exit(1)
@staticmethod
def _make_cmd(name):
def handler(self, line):
parts = line.split()
try:
print getattr(self.redis, name)(*parts)
except Exception, e:
print 'Error:', e
return handler
@staticmethod
def _make_help(doc):
def help(self):
print doc
return help
def completedefault(self, text, line, start, end):
return self.redis.keys(text + '*').split()
def do_exit(self, line):
return True
do_EOF = do_exit
def emptyline(self):
pass # By default, cmd repeats the command. We don't want to do that.
if __name__ == '__main__':
opts = dict(getopt.getopt(sys.argv[1:], 'h:p:', ['host=', 'port='])[0])
host = opts.get('-h', None) or opts.get('--host', 'localhost')
port = int(opts.get('-p', None) or opts.get('--port', 12345))
RedisCli(host=host, port=port).cmdloop()
And there we have it. A full-featured, robust, well-documented Redis CLI in about 60 lines of code. For comparison, the C version is over 500 lines of code, and has no help documentation or code completion.
The best part, though, is that this doesn't really know anything about Redis at all. The only parts that are aware of Redis are __init__(), which sets up the Redis object, and completedefault(), which gets our keys. That means that you could easily adapt this script to be a CLI on top of any client library you have.
Cross posted from http://hicks-wright.net/blog/cheeky-python-a-redis-cli/.
