1Twig Internals 2============== 3 4Twig is very extensible and you can easily hack it. Keep in mind that you 5should probably try to create an extension before hacking the core, as most 6features and enhancements can be handled with extensions. This chapter is also 7useful for people who want to understand how Twig works under the hood. 8 9How does Twig work? 10------------------- 11 12The rendering of a Twig template can be summarized into four key steps: 13 14* **Load** the template: If the template is already compiled, load it and go 15 to the *evaluation* step, otherwise: 16 17 * First, the **lexer** tokenizes the template source code into small pieces 18 for easier processing; 19 * Then, the **parser** converts the token stream into a meaningful tree 20 of nodes (the Abstract Syntax Tree); 21 * Eventually, the *compiler* transforms the AST into PHP code. 22 23* **Evaluate** the template: It basically means calling the ``display()`` 24 method of the compiled template and passing it the context. 25 26The Lexer 27--------- 28 29The lexer tokenizes a template source code into a token stream (each token is 30an instance of ``\Twig\Token``, and the stream is an instance of 31``\Twig\TokenStream``). The default lexer recognizes 13 different token types: 32 33* ``\Twig\Token::BLOCK_START_TYPE``, ``\Twig\Token::BLOCK_END_TYPE``: Delimiters for blocks (``{% %}``) 34* ``\Twig\Token::VAR_START_TYPE``, ``\Twig\Token::VAR_END_TYPE``: Delimiters for variables (``{{ }}``) 35* ``\Twig\Token::TEXT_TYPE``: A text outside an expression; 36* ``\Twig\Token::NAME_TYPE``: A name in an expression; 37* ``\Twig\Token::NUMBER_TYPE``: A number in an expression; 38* ``\Twig\Token::STRING_TYPE``: A string in an expression; 39* ``\Twig\Token::OPERATOR_TYPE``: An operator; 40* ``\Twig\Token::PUNCTUATION_TYPE``: A punctuation sign; 41* ``\Twig\Token::INTERPOLATION_START_TYPE``, ``\Twig\Token::INTERPOLATION_END_TYPE`` (as of Twig 1.5): Delimiters for string interpolation; 42* ``\Twig\Token::EOF_TYPE``: Ends of template. 43 44You can manually convert a source code into a token stream by calling the 45``tokenize()`` method of an environment:: 46 47 $stream = $twig->tokenize(new \Twig\Source($source, $identifier)); 48 49.. versionadded:: 1.27 50 ``\Twig\Source`` was introduced in version 1.27, pass the source and the 51 identifier directly on previous versions. 52 53As the stream has a ``__toString()`` method, you can have a textual 54representation of it by echoing the object:: 55 56 echo $stream."\n"; 57 58Here is the output for the ``Hello {{ name }}`` template: 59 60.. code-block:: text 61 62 TEXT_TYPE(Hello ) 63 VAR_START_TYPE() 64 NAME_TYPE(name) 65 VAR_END_TYPE() 66 EOF_TYPE() 67 68.. note:: 69 70 The default lexer (``\Twig\Lexer``) can be changed by calling 71 the ``setLexer()`` method:: 72 73 $twig->setLexer($lexer); 74 75The Parser 76---------- 77 78The parser converts the token stream into an AST (Abstract Syntax Tree), or a 79node tree (an instance of ``\Twig\Node\ModuleNode``). The core extension defines 80the basic nodes like: ``for``, ``if``, ... and the expression nodes. 81 82You can manually convert a token stream into a node tree by calling the 83``parse()`` method of an environment:: 84 85 $nodes = $twig->parse($stream); 86 87Echoing the node object gives you a nice representation of the tree:: 88 89 echo $nodes."\n"; 90 91Here is the output for the ``Hello {{ name }}`` template: 92 93.. code-block:: text 94 95 \Twig\Node\ModuleNode( 96 \Twig\Node\TextNode(Hello ) 97 \Twig\Node\PrintNode( 98 \Twig\Node\Expression\NameExpression(name) 99 ) 100 ) 101 102.. note:: 103 104 The default parser (``\Twig\TokenParser\AbstractTokenParser``) can be changed by calling the 105 ``setParser()`` method:: 106 107 $twig->setParser($parser); 108 109The Compiler 110------------ 111 112The last step is done by the compiler. It takes a node tree as an input and 113generates PHP code usable for runtime execution of the template. 114 115You can manually compile a node tree to PHP code with the ``compile()`` method 116of an environment:: 117 118 $php = $twig->compile($nodes); 119 120The generated template for a ``Hello {{ name }}`` template reads as follows 121(the actual output can differ depending on the version of Twig you are 122using):: 123 124 /* Hello {{ name }} */ 125 class __TwigTemplate_1121b6f109fe93ebe8c6e22e3712bceb extends \Twig\Template 126 { 127 protected function doDisplay(array $context, array $blocks = []) 128 { 129 // line 1 130 echo "Hello "; 131 echo twig_escape_filter($this->env, (isset($context["name"]) ? $context["name"] : null), "html", null, true); 132 } 133 134 // some more code 135 } 136 137.. note:: 138 139 The default compiler (``\Twig\Compiler``) can be changed by calling the 140 ``setCompiler()`` method:: 141 142 $twig->setCompiler($compiler); 143