Friday, November 6, 2009

Towards a Better Template Tag Definition Syntax

Eric Holscher has blogged a few times this month about various template tag definition syntax ideas. In particular he's looked at a system based on Surlex (which is essentially an alternate syntax for certain parts of regular expressions), and a system based on keywords. I highly recommend giving his posts a read as they explain the ideas he's looked at in far better detail than I could. However, I wasn't particularly satisfied with either of these solution, I love Django's use of regular expressions for URL resolving, however, for whatever reason I don't really like the look of using regular expressions (or an alternate syntax like Surlex) for template tag parsing. Instead I've been thinking about an object based parsing syntax, similar to PyParsing.

This is an idea I've been thinking about for several months now, but Eric's posts finally gave me the kick in the pants I needed to do the work. Therefore, I'm pleased to announce that I've released django-kickass-templatetags. Yes, I'm looking for a better name, it's already been pointed out to me that a name like that won't fly in the US government, or most corporate environments. This library is essentially me putting to code everything I've been thinking about, but enough talking let's take a look at what the template tag definition syntax:

@tag(register, [Constant("for"), Variable(), Optional([Constant("as"), Name()])]):
def example_tag(context, val, asvar=None):

As you can see it's a purely object based syntax, with different classes for different components of a template tag. For example this would parse something like:

{% example_tag for variable %}
{% example_tag for variable as new_var %}

It's probably clear that this is significantly less code than the manual parsing, manual node construction, and manual resolving of variable that you would have needed to do with a raw templatetag definition. Then the function you have gets the resolved values for each of its parameters, and at that point it's basically the same as Node.render, it is expected to either return a string to be inserted into the template or alter the context. I'm looking forward to never writing manual template parsing again. However, there are still a few scenarios it doens't handle, it won't handle something like the logic in the {% if %} tag, and it won't handle tags with {% end %} type tags. I feel like these should both be solvable problems, but since it's a bolt-on addition to the existing tools it ultimately doesn't have to cover every use case, just the common ones (when's the last time you wrote your own implementation of the {% if %} or {% for %} tags).

It's my hope that something like this becomes popular, as a) developers will be happier, b) moving towards a community standard is the first step towards including a solution out of the box. The pain and boilerplate of defining templatetags has long been a complain about Django's template language (especially because the language itself limits your ability to perform any sort of advanced logic), therefore making it as painless as possible ultimately helps make the case for the philosophy of the language itself (which I very much agree with it).

In keeping with my promise I'm giving an overview of what my next post will be, and this time I'm giving a full 3-day forecast :). Tommorow I'm going to blog about pip, virtualenv, and my development workflow. Sunday's post will cover a new optimization that just landed in Unladen Swallow. And finally Monday's post will contain a strange metaphor, and I'm not saying any more :P. Go checkout the code and enjoy.

Edit: Since this article was published the name of the library was changed to be: django-templatetag-sugar. I've updated all the links in this post.


  1. Great app! I'd like to hack on it to implement parsing until the {% end %} tag sometime.

  2. Nice !
    Reading it takes some getting used to, but it sure shortens time needed to write a tag.

  3. too bad you changed the awesome project title

  4. Well, I'm disappointed that pyparsing didn't make the cut for your application. But let me put in a plug for using operator overloading in your grammar definition. It's not overly difficult, and imagine how your sample would look if you could support something like this:

    @tag(register + "for" + Variable() + Optional("as" + Name()))

    Then add in support for something like results names to simplify the access to the pieces of the parsed phrase (much better than trying to index into a list of matched strings, especially in the presence of optional elements). Good luck with your parser plug-in!

  5. Actually, most of the reason I used surlex (and not plain regex) in my tag_utils (mentioned on eric's blog) library was to get richer information about the kind of variables being passed to the template -- being able to automatically cast to a certain type if the regex matched (thereby avoiding the odd situation where you have a templatetag that wants an int, but gets that int as a unicode string). Plus, since surlex's macro's are extensible, adding support for different types is a snap -- as a theoretical example, being able to pass in datetimes as component pieces or entire datetime objects to a templatetag.

    I feel like the two major trends in redesigning templatetags fall into two camps though -- introducing a string-based DSL, like regex or surlex, to parse tags, or a more object based DSL like the templatetag-sugar proposed here. I'll definitely be checking out your implementation :)

  6. Wow. This is supercool. I would love to see something like this in Django 1.2.


Note: Only a member of this blog may post a comment.