2.1   Model-View separation

So, with a template engine the programmer should separate the presentation (=view) from the logic (=model). It is very important to strictly separate these two parts! Otherwise these parts won't be independent anymore, and you will lose all advantages of a template engine!

Unfortunately, the template engine itself cannot completely enforce this separation without some serious restrictions on the view (see below). And -- at least in my opinion -- the template-engine should not restrict the design of the view or even make some designs impossible.

Since the separation cannot be done automatically, here are some rules:

• Always cleanly separate model and view.

  Don't put parts of the view into the model, don't put parts of the model into the view. Not even temporarily.
• Always push the data from the model to the view.

  Accessing/modifying the model from the view is forbidden. Pulling data is forbidden.

  So, the model may only provide a bunch of values, and the view then uses these values.

• All data from the model must be display-/layout-independent. Display/layout-information in the data is forbidden.

• Make all computations of the model in the model.

• The view must only perform computations for view-specific and model-independent "things".

2.2   Different approaches

But in reality, there are different kinds of template-engines, with different approaches. I'll group them into 4 categories (although some of the existing template-engines are somewhere between these categories):

1. string-interpolation (substitution only):

   The template only contains placeholders which are replaced by data. No loops, no conditions etc. are possible in the template. This is the simplest but also least powerful kind of templates.

   Examples: printf-formatstring, python's string.Template, Templayer.

   It should be clear that this is insufficient for most cases.

2. template logic (substitution + if-defined + loop-over-data + recursive macros):

   The template can contain:

   • placeholders (for string-substitution)
   • if-defined-conditionals (to test presence/absence of some data)
   • foreach-data-loops (for multi-valued data/lists)
   • recursive macros (to walk recursive data-structures)

   This is powerful enough for some websites. And it cuts down the power of the template to prevent that anything which belongs to the model can be done in the template/view.

   This approach is described in detail in an interesting paper, named Enforcing Strict Model-View Separation in Template Engines. Although I don't agree to many conclusions of the author (and think that these templates are definitely not powerful enough), it may be worth reading.

   BUT: This approach puts some serious restrictions on the view: Some designs simply can't be done (see Why if/for/...) -- or can only be done if you put parts of the presentation into the model code, which is a no-go. This often makes this approach unusable.

   Examples: StringTemplate (the template of the above paper), ...

3. embedded code (unrestricted embedded statements and expressions):

   Since the above is not powerful enough, many template-engines simply include a full-powered programming language without any restrictions into the template. But this has the drawback that even the model can be modified in the template, which can completely countermine the model-view-separation. Additionally, template-code can don anything, including e.g. deleting files on the harddisk.

   Examples: Mako, Cheetah, TemplateToolkit, EmPy, ...

4. template logic + embedded expressions (substitution + conditionals + loops + macros + "embedded expressions" with restricted access):

   Since (2) is not powerful enough, but (3) is too powerful and messy, the best solutions is somewhere between them.

   In addition to (2), the template-language also contains free loops, tests of variable-values, and "expressions of a embedded language" (e.g. for calculations, formatting etc.). But this "embedded language" can only access a restricted set of variables/functions; it especially cannot access or modify the model.

   This is powerful enough for all cases, permits a clean template-design and prevents direct access and modification of the model in the template. It can't prevent that tests, calculations etc. which belong to the model are done in the template -- this is in the responsibility of the programmer/designer. But that's the price for a full flexible, unrestricted view.

   Examples: unfortunately not many, but e.g. my pyratemp

2.3   Why if/for/calculations etc. are necessary in the View

There are many people who think that only allowing "template-logic" (see above "category 2") in templates is sufficient. Unfortunately, this is not the case. There are a lot of things which clearly belong to the view, but are not possible -- or really difficult -- without variable-testing, for-loops, calculations, formatting etc.

This can be demonstrated best with a few examples. Note that these examples are not constructed, but really occurred in existing templates:

• free for-loops:

  • Imagine a fixed form, with a table which should always have 10 rows, no matter how much data exists. If there is less data, the table will have some empty rows. If there is more data, there will be 2 (or more) tables.

    This probably isn't needed very often in websites, but it definitely occurs in fixed formats, e.g. in LaTeX.

  • Or simply take an enumerated list like this:

    	Value
    1	foo
    2	bar
    3	baz
    4

    The enumeration and enumeration-style here belongs to the view.

    (Note: This is possible in some "template-logic-template-engines", but only due to some "magic-variables" like loop-counters. But it then only works for "1,2,3,..." and not for "10,20,30,..." or "I,II,III,...".)

• conditionals:

  • Imagnine a table, where some entries should be highlighted (like e.g. in the example from my template-benchmark below). Then, the template has to check if and how an entry should be highlighted. But this is difficult without conditionals (if/elif/else) and embedded expressions.

• formatting:

  • How should a date look like? CCYY-MM-DD or DD. Month CCYY or MM/DD/YY?
  • How many digits of a float should be displayed? One output may e.g. require 2, an other 3. Some other needs a "+" in front of a positive number...

• calculations:

  Imagine a bill (e.g. in LaTeX) with a variable number of items, which may be broken into several sheets of paper. The calculation of a subtotal at the end of each page clearly belongs to the view.

All these examples clearly belong to the view, but are not possible with the pure "category 2" approach. And these are only some examples -- there are a lot more...

This is why such "pure template-logic" (category 2)-templates often get extended bit by bit, until they eventually allow to embed code (category 3). But they then are messy, complex and badly-designed for a categeory-3-template.

2.4   What a template-engine needs

Here's a short description of how a template-engine should look like in my opinion:

• stand-alone (i.e. work without a webserver)

• result-independent: able to create any kind of textual data, e.g. HTML/XML (both static and dynamic), LaTeX, PostScript, email, ...

  (Contrary to most XML-template-engines, which can only create XML-documents.)

• simple, small, lightweight, easy to use, well-documented and extensible

• clear template syntax ("There should be one -- and preferably only one -- obvious way to do it." [2])

• completely Unicode

• good syntax-error-checking and good error-messages, to make the writing and debugging of templates easy

• secure, so an "untrusted template" cannot compromise your system.

  (think of a sandbox, prevention of infinite loops, etc.)

• fast

• basic functionality (from "category 4" above):

  • placeholders/string-substitution
  • conditionals
  • loops
  • macros (with parameters)
  • restricted "embedded expressions" (formatting, arithmetics etc.)

• additional functionality:

  • inclusion of other templates (for reusability and "things" which are used in several templates)
  • escaping-mechanism (e.g. for html: escape < > & ' ")
  • (MAYBE) definition of the result-encoding
  • (MAYBE) set template-variables

• complain about undefined/non-existing variables in the template by default

  (Most template-engines silently ignore undefined variables in the template. But silently ignoring errors is bad, and wrong data is much worse than no data and an error-message.)

Since I unfortunately didn't find any existing template-engine for Python which fitted these needs, I created an own template-engine which accomplishes most of the above: pyratemp

3.1   Benchsimple

"Benchsimple" creates a simple html-page with a navigation-bar and a table with 50 entries (5 rows * 10 columns) and some colors. It only uses a few template-features that are supported by all templates (escaped substitution, conditionals, loops), and so there's no Unicode, no included templates, (nearly) no arithmetics, no macros, no template-defined formatting etc.

In addition to benchmarking, the benchmark-code is useful to see how different template-engines look like (and how easy or difficult it is to make them do what you want), since the code includes templates for the exactly same task for different template-engines.

But note that the following results can only give you a hint how fast a specific template-engine might be.

Results (benchsimple 2013):

Notes:

• run on an AMD Athlon 64 3200+, 2.0 GHz, Debian Squeeze
• all times are in "ms"
• the times are pure "template-execution"-times without the time needed to start the Python-interpreter, load the files and compile the code
• "import": time to import the module (on 2nd/3rd run)
• "complete": complete time to parse and render the template
• "parse only": time to parse (+maybe compile) a template
• "re-render": time to re-render an already parsed/compiled template after it has been rendered the first time

4.1   Overview

• http://en.wikipedia.org/wiki/Template_engine_(web)
• http://en.wikipedia.org/wiki/Template_processor
• http://wiki.python.org/moin/Templating
• http://de.wikipedia.org/wiki/Template_Engine (in German)
• http://wiki.python.de/Template-Engines (in German)

Additionally, I've created the same html-page using different template-engines in Benchsimple, so look there to get a first impression how these template-engines look like. You can also look into the footnotes below the Benchsimple-results-table to see some problems of some template-engines.

As a general rule-of-thumb: XML-specific-template-engines often make it much harder for the user create templates that do what they should (e.g. see Benchsimple-examples). So in my opinion, even for XML-documents, it's much easier to use a general-purpose template-engine and ensure XML-validity manually during the template design.

Since I haven't tested every template-engine in detail, some of the fields below are empty, and I probably haven't found some hidden features of some template-engines. So, please tell me if anything is wrong.

lines-of-code:

Number of "code"-lines as reported by pylint, without docstrings, comments and "empty" lines; but it may contain lines of tests and supplementary code.

Unicode:

A template-engine should accept both templates and data in Unicode, but some of them don't and some only do optionally.

error-msg.:

How good are the error-messages for template-syntax-errors etc.? Good error-messages are extremely useful when developing or "debugging" a template, and save a lot of time.

++: good error-messages, including line- and column-number of the error and the names of undefined variables

+: good error-messages, including line- of the error and the names of undefined variables

o: only partially the position of an error or name of an undefined variable is reported

--: error-messages do not tell where the error occurred, or errors are silently ignored and a wrong result is created.

escaping:

Ideally, escaping should be easy to use in the template (so you don't accidently forget it somewhere), should escape <>&"' for HTML/XML, and be configurable for other escaping.

+: escapes <>&"'

o: only escapes <>&

--: no escaping

undef. vars:

What should be done if the template uses a variable (or other object) which does not exist? Some of the template-engines ignore undefined variables, some complain (and exit), and some are configurable.

embedded code:

Can (Python-)statements be included in the template?

In contrast to expressions (see below), statements do not have a value but "do something" (e.g. if/for, print, raise, return, import etc.). In my opinion, templates should not directly include statements from a programming-language but define a small set of needed "statements" themselves.

(see also: http://en.wikipedia.org/wiki/Statement_%28programming%29)

embedded expr.:

Can (Python-)expressions be included in the template?

An expression is everything which evaluates to a value, e.g. variables, arithmetics, comparisons, boolean expressions, function/method calls, list comprehensions etc.

(see also: http://en.wikipedia.org/wiki/Expression_%28programming%29)

sandbox:

Are the embedded expressions/statements sandboxed/restricted, or can they do anything? "Anything" of course includes things like open("/etc/passwd") and worse.

Note that these sandboxes do not limit the memory- and CPU-usage nor prevent infinite loops; if you want to limit these, use ulimit.

+: sandbox, where I don't know any way to break out (if you know any, please let me know!)

--: no sandbox

4.2   pyratemp

pyratemp is my own template-engine. It's probably (one of) the smallest complete template-engines [14], and it uses a very small set of special syntax in the templates. Both is good, because it reduces complexity and the probability of bugs and leads to an easy-to-use and intuitive user-interface.

Additionally, it is quite fast (although not optimized for speed), uses a (pseudo-)sandbox and produces exceptional good error-messages (e.g. pyratemp.TemplateSyntaxError: line 18, col 1: invalid keyword 'fo'), which is extremely useful.

pyratemp was inspired by cubictemp. Since there were some essential features missing in cubictemp (see below), after trying to extend cubictemp, I decided to write my own template-engine from scratch, which is nearly as small and simple as cubictemp, but without its weaknesses.

More details can be found on my pyratemp-page.

4.3   cubictemp

cubictemp is probably the smallest of all template-engines. I completely agree to a statement on its homepage:

There are many large, over-designed Python templating systems out there.

Unfortunately, there are some essential features missing in cubictemp, like Unicode-support, inclusion of other templates, good/helpful error-messages, and some kind of a sandbox. Additionally, its conditionals (if/else) are weak, since they don't have a "elif" and can't contain template-syntax in its branches.

Now, cubictemp seems to be abandoned and it seems that the author has deleted it from his homepage.

4.4   Jinja

Jinja 0.7 was probably the best template-engine without embedded Python-expressions. And it included a sandbox, so a template could not do "bad things". But its embedded expressions were not powerful enough for many purposes, and so embedded Python-expressions were added in later releases. (Note that this is exactly what I predicted at the end of Why if/for/calculations etc. are necessary in the View.) And Jinja 0.7 doesn't give you any usable error-messages, so you had to manually delete and re-add template-code (!) to find errors in the template. Additionally, Jinja 0.7 didn't escape ``"``, so you had to be very careful...

But then, Jinja 0.7 was replaced by Jinja 1.x, which then was replaced by Jinja2, which at least partly fixed the bugs above. Unfortunately, Jinja 0.7, Jinja 1.x and Jinja2 are incompatible, and nearly all information about the old versions vanished from the net.

But if you look for a template-engine with a sandbox, Jinja 2.0 may be worth a look.

4.5   evoque

Some time after I wrote pyratemp, I found evoque, and was surprised that it has many things in common with pyratemp, and nearly has everything I described a template-engine should have (see What a template-engine needs).

But after reading parts of its documentation and a short test, I also found a few things I don't like -- especially in comparison with pyratemp:

• It's really counterintuitive to generate a document with an exact amount of whitespace. You can find an example in my benchsimple-code, where you can compare the whitespace in the template with the ones in the result. (Note that in the benchsimple-example, the whitespace before "</ul>" is ignored, and instead the whitespace before "$rof" of the for-loop above is used for "</ul>"!)

  Although that's probably not a problem when creating HTML/XML, it is when creating other documents.

• "sandbox": By default, the sandbox is disabled, and I don't know if this sandbox is secure or not.

  But a look into the sourcecode showed that the programmer forbids known-unsafe functions instead of explicitly allowing only known-safe functions, which is definitely the wrong way and leaves a bad impression.

• And it's code base is larger and worse documented than pyratemp, and it's slower. ;)