Monday, November 30, 2009

You Built a Metaclass for *what*?

Recently I had a bit of an interesting problem, I needed to define a way to represent a C++ API in Python. So, I figured the best way to represent that was one class in Python for each class in C++, with a functions dictionary to track each of the methods on each class. Seems simple enough right, do something like this:

class String(object):
functions = {
"size": Function(Integer, []),

We've got a String class with a functions dictionary that maps method names to Function objects. The Function constructor takes a return type and a list of arguments. Unfortunately we run into a problem when we want to do something like this:

class String(object):
functions = {
"size": Function(Integer, []),
"append": Function(None, [String])

If we try to run this code we're going to get a NameError, String isn't defined yet. Django models have a similar issue, with recursive foreign keys. Django's solution is to use the placeholder string "self", and have a metaclass translate it into the right class. Also having a slightly more declarative API might be nice, so something like this:

class String(DeclarativeObject):
size = Function(Integer, [])
append = Function(None, ["self"])

So now that we have a nice pretty API we need our metaclass to make it happen:


class DeclarativeObjectMetaclass(type):
def __new__(cls, name, bases, attrs):
functions = dict([(n, attr) for n, attr in attrs.iteritems()
if isinstance(attr, Function)])
for attr in functions:
new_cls = super(DeclarativeObjectMetaclass, cls).__new__(cls, name, bases, attrs)
new_cls.functions = {}
for name, function in functions.iteritems():
if function.return_type == RECURSIVE_TYPE_CONSTANT:
function.return_type = new_cls
for i, argument in enumerate(function.arguments):
function.arguments[i] = new_cls
new_cls.functions[name] = function
return new_cls

class DeclarativeObject(object):
__metaclass__ = DeclarativeObjectMetaclass

And that's all their is to it. We take each of the functions on the class out of the attributes, create a normal class instance without the functions, and then we do the replacements on the function objects and stick them in a functions dictionary.

Simple patterns like this can be used to build beautiful APIs, as is seen in Django with the models and forms API.


  1. You wouldn't necessarily need to use a metaclass here, you could make Function a descriptor that does the right thing.

  2. class String(object):
    functions = {}

    String.functions["size"] = Function(Integer, [])
    String.functions["append"] = Function(None, [String])

  3. I've done a pretty similar thing for my fbuild build system's config system. You can see an example usage here, and the metaclass stuff here.

  4. nitpick:

    "And that's all their is to it."

    should be:

    "And that's all there is to it."

  5. @Anonymous that was of tremendous value.


Note: Only a member of this blog may post a comment.