Merge branch '3.x' into 4.x

* 3.x:
  Update CHANGELOG
  Fix accessing arrays with stringable objects as key
  Update inky_to_html.rst: Updating link
  Update replace.rst
  [Doc] Tweaks in the escaping article
  Compile 'index' with repr (not string) in EmbedNode
  Introduce registerUndefinedTestCallback
  Fix intl test
  Bump minimum Commonmark requirement
  Support two words test guard
  Bump version
  Improve documentation examples for `enum` and `enum_cases`
  Avoid errors when failing to guess the template info for an error
  Add note to format_datetime explaining how to install required extensions
  Fix compatibility layer

doc/filters/escape.rst

@@ -53,8 +53,9 @@ documents:
 * ``url``: escapes a string for the **URI or parameter** contexts. This should
   not be used to escape an entire URI; only a subcomponent being inserted.
 
-* ``html_attr``: escapes a string for the **HTML attribute** context,
-  **without quotes** around HTML attribute values.
+* ``html_attr``: escapes a string when used as an **HTML attribute** name, and
+  also when used as the value of an HTML attribute **without quotes**
+  (e.g. ``data-attribute={{ some_value }}``).
 
 Note that doing contextual escaping in HTML documents is hard and choosing the
 right escaping strategy depends on a lot of factors. Please, read related
@@ -96,23 +97,22 @@ to learn more about this topic.
 
 .. tip::
 
-    The ``html_attr`` escaping strategy can be useful when you need to 
-    escape a **dynamic HTML attribute name**:
+    The ``html_attr`` escaping strategy can be useful when you need to escape a
+    **dynamic HTML attribute name**:
 
     .. code-block:: html+twig
 
         <p {{ your_html_attr|e('html_attr') }}="attribute value">
     
-    It can also be used for escaping a **dynamic HTML attribute value** 
-    if it is not quoted, but this is **less performant**. 
-    Instead, it is recommended to quote the HTML attribute value and use 
-    the ``html`` escaping strategy:
+    It can also be used for escaping a **dynamic HTML attribute value** if it is
+    not quoted, but this is **less performant**. Instead, it is recommended to
+    quote the HTML attribute value and use the ``html`` escaping strategy:
 
     .. code-block:: html+twig
 
         <p data-content="{{ content|e('html') }}">
 
-        {# is equivalent to, but is less performant #}
+        {# this is equivalent, but less performant #}
         <p data-content={{ content|e('html_attr') }}>
 
 Custom Escapers

doc/filters/format_datetime.rst

@@ -8,6 +8,29 @@ The ``format_datetime`` filter formats a date time:
     {# Aug 7, 2019, 11:39:12 PM #}
     {{ '2019-08-07 23:39:12'|format_datetime() }}
 
+.. note::
+
+    The ``format_datetime`` filter is part of the ``IntlExtension`` which is not
+    installed by default. Install it first:
+
+    .. code-block:: bash
+
+        $ composer require twig/intl-extra
+
+    Then, on Symfony projects, install the ``twig/extra-bundle``:
+
+    .. code-block:: bash
+
+        $ composer require twig/extra-bundle
+
+    Otherwise, add the extension explicitly on the Twig environment::
+
+        use Twig\Extra\Intl\IntlExtension;
+
+        $twig = new \Twig\Environment(...);
+        $twig->addExtension(new IntlExtension());
+
+
 Format
 ------
 

doc/filters/inky_to_html.rst

@@ -2,7 +2,7 @@
 ================
 
 The ``inky_to_html`` filter processes an `inky email template
-<https://github.com/zurb/inky>`_:
+<https://github.com/foundation/inky>`_:
 
 .. code-block:: html+twig
 

doc/filters/replace.rst

@@ -6,6 +6,8 @@ format is free-form):
 
 .. code-block:: twig
 
+    {% set fruit = 'apples' %}
+
     {{ "I like %this% and %that%."|replace({'%this%': fruit, '%that%': "oranges"}) }}
     {# if the "fruit" variable is set to "apples", #}
     {# it outputs "I like apples and oranges" #}

doc/functions/enum.rst

@@ -6,15 +6,21 @@
 .. code-block:: twig
 
     {# display one specific case of a backed enum #}
-    {{ enum('App\\MyEnum').SomeCase.value }}
+    {{ enum('App\\CardSuite').Clubs.value }} {# "clubs" #}
 
     {# get all cases of an enum #}
-    {% for case in enum('App\\MyEnum').cases %}
+    {% for case in enum('App\\CardSuite').cases %}
         {{ case.value }}
     {% endfor %}
+    {# "clubs", "spades", "hearts", "diamonds" #}
+
+    {# get a specific case of an enum by value #}
+    {% set card_suite = enum('App\\CardSuite').from('hearts') %}
+    {{ card_suite.name }} {# "Hearts" #}
+    {{ card_suite.value }} {# "hearts" #}
 
     {# call any methods of the enum class #}
-    {{ enum('App\\MyEnum').someMethod() }}
+    {{ enum('App\\CardSuite').someMethod() }}
 
 When using a string literal for the ``enum`` argument, it will be validated
 during compile time to be a valid enum name.

doc/functions/enum_cases.rst

@@ -5,9 +5,10 @@
 
 .. code-block:: twig
 
-    {% for case in enum_cases('App\\MyEnum') %}
+    {% for case in enum_cases('App\\CardSuite') %}
         {{ case.value }}
     {% endfor %}
+    {# "clubs", "spades", "hearts", "diamonds" #}
 
 When using a string literal for the ``enum`` argument, it will be validated during compile time to be a valid enum name.
 

doc/recipes.rst

@@ -226,13 +226,14 @@ thanks to the magic ``__get()`` method; you need to also implement the
 Defining undefined Functions, Filters, and Tags on the Fly
 ----------------------------------------------------------
 
-When a function/filter/tag is not defined, Twig defaults to throw a
+When a function/filter/test/tag is not defined, Twig defaults to throw a
 ``\Twig\Error\SyntaxError`` exception. However, it can also call a `callback`_
-(any valid PHP callable) which should return a function/filter/tag.
+(any valid PHP callable) which should return a function/filter/test/tag.
 
 For tags, register callbacks with ``registerUndefinedTokenParserCallback()``.
 For filters, register callbacks with ``registerUndefinedFilterCallback()``.
-For functions, use ``registerUndefinedFunctionCallback()``::
+For functions, use ``registerUndefinedFunctionCallback()``.
+For tests, use ``registerUndefinedTestCallback()``::
 
     // auto-register all native PHP functions as Twig functions
     // NEVER do this in a project as it's NOT secure
@@ -244,7 +245,7 @@ For functions, use ``registerUndefinedFunctionCallback()``::
         return false;
     });
 
-If the callable is not able to return a valid function/filter/tag, it must
+If the callable is not able to return a valid function/filter/test/tag, it must
 return ``false``.
 
 If you register more than one callback, Twig will call them in turn until one
@@ -252,7 +253,7 @@ does not return ``false``.
 
 .. tip::
 
-    As the resolution of functions/filters/tags is done during compilation,
+    As the resolution of functions/filters/tests/tags is done during compilation,
     there is no overhead when registering these callbacks.
 
 .. warning::

extra/intl-extra/Tests/Fixtures/country_timezones.test

@@ -3,10 +3,10 @@
 --TEMPLATE--
 {{ country_timezones('UNKNOWN')|length }}
 {{ country_timezones('FR')|join(', ') }}
-{{ country_timezones('US')|join(', ') }}
+{{ country_timezones('US')[0:2]|join(', ') }}
 --DATA--
 return [];
 --EXPECT--
 0
 Europe/Paris
-America/Adak, America/Anchorage, America/Boise, America/Chicago, America/Denver, America/Detroit, America/Indiana/Knox, America/Indiana/Marengo, America/Indiana/Petersburg, America/Indiana/Tell_City, America/Indiana/Vevay, America/Indiana/Vincennes, America/Indiana/Winamac, America/Indianapolis, America/Juneau, America/Kentucky/Monticello, America/Los_Angeles, America/Louisville, America/Menominee, America/Metlakatla, America/New_York, America/Nome, America/North_Dakota/Beulah, America/North_Dakota/Center, America/North_Dakota/New_Salem, America/Phoenix, America/Sitka, America/Yakutat, Pacific/Honolulu
+America/Adak, America/Anchorage

extra/markdown-extra/composer.json

@@ -21,7 +21,7 @@
     },
     "require-dev": {
         "erusev/parsedown": "dev-master as 1.x-dev",
-        "league/commonmark": "^1.0|^2.0",
+        "league/commonmark": "^2.7",
         "league/html-to-markdown": "^4.8|^5.0",
         "michelf/php-markdown": "^1.8|^2.0",
         "symfony/phpunit-bridge": "^6.4|^7.0"

extra/twig-extra-bundle/composer.json

@@ -21,7 +21,7 @@
         "twig/twig": "^3.2|^4.0"
     },
     "require-dev": {
-        "league/commonmark": "^1.0|^2.0",
+        "league/commonmark": "^2.7",
         "twig/cache-extra": "^3.0",
         "twig/cssinliner-extra": "^3.0",
         "twig/html-extra": "^3.0",

src/Environment.php

@@ -809,6 +809,14 @@ class Environment
         return $this->extensionSet->getTest($name);
     }
 
+    /**
+     * @param callable(string): (TwigTest|false) $callable
+     */
+    public function registerUndefinedTestCallback(callable $callable): void
+    {
+        $this->extensionSet->registerUndefinedTestCallback($callable);
+    }
+
     /**
      * @return void
      */

src/Error/Error.php

@@ -148,6 +148,10 @@ class Error extends \Exception
             }
         }
 
+        if (null === $template) {
+            return; // Impossible to guess the info as the template was not found in the backtrace
+        }
+
         $r = new \ReflectionObject($template);
         $file = $r->getFileName();
 

src/Extension/CoreExtension.php

@@ -1627,7 +1627,7 @@ final class CoreExtension extends AbstractExtension
             }
 
             if (match (true) {
-                \is_array($object) => \array_key_exists($arrayItem, $object),
+                \is_array($object) => \array_key_exists($arrayItem = (string) $arrayItem, $object),
                 $object instanceof \ArrayAccess => $object->offsetExists($arrayItem),
                 default => false,
             }) {
@@ -1648,9 +1648,13 @@ final class CoreExtension extends AbstractExtension
                 }
 
                 if ($object instanceof \ArrayAccess) {
-                    $message = \sprintf('Key "%s" in object with ArrayAccess of class "%s" does not exist.', $arrayItem, $object::class);
+                    if (\is_object($arrayItem) || \is_array($arrayItem)) {
+                        $message = \sprintf('Key of type "%s" does not exist in ArrayAccess-able object of class "%s".', get_debug_type($arrayItem), get_debug_type($object));
+                    } else {
+                        $message = \sprintf('Key "%s" does not exist in ArrayAccess-able object of class "%s".', $arrayItem, get_debug_type($object));
+                    }
                 } elseif (\is_object($object)) {
-                    $message = \sprintf('Impossible to access a key "%s" on an object of class "%s" that does not implement ArrayAccess interface.', $item, $object::class);
+                    $message = \sprintf('Impossible to access a key "%s" on an object of class "%s" that does not implement ArrayAccess interface.', $item, get_debug_type($object));
                 } elseif (\is_array($object)) {
                     if (!$object) {
                         $message = \sprintf('Key "%s" does not exist as the sequence/mapping is empty.', $arrayItem);

src/ExtensionSet.php

@@ -57,6 +57,8 @@ final class ExtensionSet
     private array $functionCallbacks = [];
     /** @var array<callable(string): (TwigFilter|false)> */
     private array $filterCallbacks = [];
+    /** @var array<callable(string): (TwigTest|false)> */
+    private array $testCallbacks = [];
     /** @var array<callable(string): (TokenParserInterface|false)> */
     private array $parserCallbacks = [];
     private int $lastModified = 0;
@@ -408,9 +410,23 @@ final class ExtensionSet
             }
         }
 
+        foreach ($this->testCallbacks as $callback) {
+            if (false !== $test = $callback($name)) {
+                return $test;
+            }
+        }
+
         return null;
     }
 
+    /**
+     * @param callable(string): (TwigTest|false) $callable
+     */
+    public function registerUndefinedTestCallback(callable $callable): void
+    {
+        $this->testCallbacks[] = $callable;
+    }
+
     public function getExpressionParsers(): ExpressionParsers
     {
         if (!$this->initialized) {

src/Node/EmbedNode.php

@@ -41,7 +41,7 @@ class EmbedNode extends IncludeNode
             ->raw(', ')
             ->repr($this->getTemplateLine())
             ->raw(', ')
-            ->string($this->getAttribute('index'))
+            ->repr($this->getAttribute('index'))
             ->raw(')')
         ;
         if ($this->getAttribute('ignore_missing')) {

src/Parser.php

@@ -438,11 +438,26 @@ class Parser
             // try 2-words tests
             $name = $name.' '.$this->getCurrentToken()->getValue();
 
-            if ($test = $this->env->getTest($name)) {
-                $this->stream->next();
+            try {
+                $test = $this->env->getTest($name);
+            } catch (SyntaxError $e) {
+                if (!$this->shouldIgnoreUnknownTwigCallables()) {
+                    throw $e;
+                }
+
+                $test = null;
             }
+            $this->stream->next();
         } else {
-            $test = $this->env->getTest($name);
+            try {
+                $test = $this->env->getTest($name);
+            } catch (SyntaxError $e) {
+                if (!$this->shouldIgnoreUnknownTwigCallables()) {
+                    throw $e;
+                }
+
+                $test = null;
+            }
         }
 
         if (!$test) {

src/Test/IntegrationTestCase.php

@@ -91,6 +91,14 @@ abstract class IntegrationTestCase extends TestCase
         return [];
     }
 
+    /**
+     * @return array<callable(string): (TwigTest|false)>
+     */
+    protected function getUndefinedTestCallbacks(): array
+    {
+        return [];
+    }
+
     /**
      * @return array<callable(string): (TokenParserInterface|false)>
      */
@@ -236,6 +244,10 @@ abstract class IntegrationTestCase extends TestCase
                 $twig->registerUndefinedFunctionCallback($callback);
             }
 
+            foreach ($this->getUndefinedTestCallbacks() as $callback) {
+                $twig->registerUndefinedTestCallback($callback);
+            }
+
             foreach ($this->getUndefinedTokenParserCallbacks() as $callback) {
                 $twig->registerUndefinedTokenParserCallback($callback);
             }

src/TokenParser/GuardTokenParser.php

@@ -32,9 +32,15 @@ final class GuardTokenParser extends AbstractTokenParser
         $method = 'get'.$typeToken->getValue();
 
         $nameToken = $stream->expect(Token::NAME_TYPE);
+        $name = $nameToken->getValue();
+        if ('test' === $typeToken->getValue() && $stream->test(Token::NAME_TYPE)) {
+            // try 2-words tests
+            $name .= ' '.$stream->getCurrent()->getValue();
+            $stream->next();
+        }
 
         try {
-            $exists = null !== $this->parser->getEnvironment()->$method($nameToken->getValue());
+            $exists = null !== $this->parser->getEnvironment()->$method($name);
         } catch (SyntaxError) {
             $exists = false;
         }

tests/EnvironmentTest.php

@@ -427,6 +427,22 @@ class EnvironmentTest extends TestCase
         $this->assertSame('dynamic', $filter->getName());
     }
 
+    public function testUndefinedTestCallback()
+    {
+        $twig = new Environment(new ArrayLoader());
+        $twig->registerUndefinedTestCallback(function (string $name) {
+            if ('dynamic' === $name) {
+                return new TwigTest('dynamic', function () { return 'dynamic'; });
+            }
+
+            return false;
+        });
+
+        $this->assertNull($twig->getTest('does_not_exist'));
+        $this->assertInstanceOf(TwigTest::class, $test = $twig->getTest('dynamic'));
+        $this->assertSame('dynamic', $test->getName());
+    }
+
     public function testUndefinedTokenParserCallback()
     {
         $twig = new Environment(new ArrayLoader());

tests/Fixtures/regression/4701-block-inheritance-issue.test

@@ -0,0 +1,21 @@
+--TEST--
+#4701 Accessing arrays with stringable objects as key
+--TEMPLATE--
+{% set hash = {
+  'foo': 'FOO',
+  'bar': 'BAR',
+} %}
+
+{{ hash[key] }}
+--DATA--
+class MyObj {
+    public function __toString() {
+        return 'foo';
+    }
+}
+
+return [
+    'key' => new MyObj(),
+];
+--EXPECT--
+FOO

tests/Fixtures/tags/guard/basic.test

@@ -13,9 +13,26 @@
 {% else %}
     NEVER
 {% endguard %}
+
+{% guard test foobar %}
+    NEVER
+    {{ 'a'|foobar }}
+{% else -%}
+    The foobar test doesn't exist
+{% endguard %}
+
+{% guard test divisible by -%}
+    The divisible by function does exist
+{% else %}
+    NEVER
+{% endguard %}
 --DATA--
 return []
 --EXPECT--
 The foobar filter doesn't exist
 
 The constant function does exist
+
+The foobar test doesn't exist
+
+The divisible by function does exist

tests/Fixtures/tags/guard/throwing_handler.test

@@ -14,9 +14,27 @@
 {% else -%}
     The throwing_undefined_function function doesn't exist
 {% endguard %}
+
+{% guard test throwing_undefined_test -%}
+    NEVER
+    {% if 'a' is throwing_undefined_test('b') %}{% endif %}
+{% else -%}
+    The throwing_undefined_test test doesn't exist
+{% endguard %}
+
+{% guard test throwing_undefined_two words_test -%}
+    NEVER
+    {% if 'a' is throwing_undefined_test words_test('b') %}{% endif %}
+{% else -%}
+    The throwing_undefined_two words_test test doesn't exist
+{% endguard %}
 --DATA--
 return []
 --EXPECT--
 The throwing_undefined_filter filter doesn't exist
 
 The throwing_undefined_function function doesn't exist
+
+The throwing_undefined_test test doesn't exist
+
+The throwing_undefined_two words_test test doesn't exist

tests/IntegrationTest.php

@@ -72,7 +72,28 @@ class IntegrationTest extends IntegrationTestCase
         ];
     }
 
-    protected function getUndefinedTokenParserCallbacks(): array
+    protected function getUndefinedTestCallbacks(): array
+    {
+        return [
+            static function (string $name) {
+                if ('throwing_undefined_test' === $name) {
+                    throw new SyntaxError('This test is undefined in the tests.');
+                }
+                if ('throwing_undefined_two words_test' === $name) {
+                    throw new SyntaxError('This test is undefined in the tests.');
+                }
+
+                // Ensure this does not conflict with `divisible by` and `same as`.
+                if (\in_array($name, ['divisible', 'same'], true)) {
+                    return new TwigTest($name, fn () => '');
+                }
+
+                return false;
+            },
+        ];
+    }
+
+    protected function getUndefinedFilterCallbacks(): array
     {
         return [
             static function (string $name) {

tests/Node/EmbedTest.php

@@ -0,0 +1,91 @@
+<?php
+
+/*
+ * This file is part of Twig.
+ *
+ * (c) Fabien Potencier
+ *
+ * For the full copyright and license information, please view the LICENSE
+ * file that was distributed with this source code.
+ */
+
+namespace Twig\Tests\Node;
+
+use Twig\Node\EmbedNode;
+use Twig\Node\Expression\ArrayExpression;
+use Twig\Node\Expression\ConstantExpression;
+use Twig\Test\NodeTestCase;
+
+class EmbedTest extends NodeTestCase
+{
+    public function testConstructor()
+    {
+        $node = new EmbedNode('foo.twig', 0, null, false, false, 1);
+
+        $this->assertFalse($node->hasNode('variables'));
+        $this->assertEquals('foo.twig', $node->getAttribute('name'));
+        $this->assertEquals(0, $node->getAttribute('index'));
+        $this->assertFalse($node->getAttribute('only'));
+        $this->assertFalse($node->getAttribute('ignore_missing'));
+
+        $vars = new ArrayExpression([new ConstantExpression('foo', 1), new ConstantExpression(true, 1)], 1);
+        $node = new EmbedNode('bar.twig', 1, $vars, true, false, 1);
+        $this->assertEquals($vars, $node->getNode('variables'));
+        $this->assertTrue($node->getAttribute('only'));
+        $this->assertEquals('bar.twig', $node->getAttribute('name'));
+        $this->assertEquals(1, $node->getAttribute('index'));
+    }
+
+    public static function provideTests(): iterable
+    {
+        $tests = [];
+
+        $node = new EmbedNode('foo.twig', 0, null, false, false, 1);
+        $tests[] = [$node, <<<'EOF'
+// line 1
+yield from $this->load("foo.twig", 1, 0)->unwrap()->yield($context);
+EOF
+        ];
+
+        $node = new EmbedNode('foo.twig', 1, null, false, false, 1);
+        $tests[] = [$node, <<<'EOF'
+// line 1
+yield from $this->load("foo.twig", 1, 1)->unwrap()->yield($context);
+EOF
+        ];
+
+        $vars = new ArrayExpression([new ConstantExpression('foo', 1), new ConstantExpression(true, 1)], 1);
+        $node = new EmbedNode('foo.twig', 0, $vars, false, false, 1);
+        $tests[] = [$node, <<<'EOF'
+// line 1
+yield from $this->load("foo.twig", 1, 0)->unwrap()->yield(CoreExtension::merge($context, ["foo" => true]));
+EOF
+        ];
+
+        $node = new EmbedNode('foo.twig', 0, $vars, true, false, 1);
+        $tests[] = [$node, <<<'EOF'
+// line 1
+yield from $this->load("foo.twig", 1, 0)->unwrap()->yield(CoreExtension::toArray(["foo" => true]));
+EOF
+        ];
+
+        $node = new EmbedNode('foo.twig', 2, $vars, true, true, 1);
+        $tests[] = [$node, <<<EOF
+// line 1
+try {
+    \$_v0 = \$this->load("foo.twig", 1, 2);
+    \$_v0->getParent(\$context);
+;
+} catch (LoaderError \$e) {
+    // ignore missing template
+    \$_v0 = null;
+}
+if (\$_v0) {
+    yield from \$_v0->unwrap()->yield(CoreExtension::toArray(["foo" => true]));
+}
+EOF
+        ];
+
+        return $tests;
+    }
+}

tests/TemplateTest.php

@@ -76,7 +76,7 @@ class TemplateTest extends TestCase
             ['{{ null["a"] }}', 'Impossible to access a key ("a") on a null variable in "%s" at line 1.'],
             ['{{ empty_array["a"] }}', 'Key "a" does not exist as the sequence/mapping is empty in "%s" at line 1.'],
             ['{{ array["a"] }}', 'Key "a" for sequence/mapping with keys "foo" does not exist in "%s" at line 1.'],
-            ['{{ array_access["a"] }}', 'Key "a" in object with ArrayAccess of class "Twig\Tests\TemplateArrayAccessObject" does not exist in "%s" at line 1.'],
+            ['{{ array_access["a"] }}', 'Key "a" does not exist in ArrayAccess-able object of class "Twig\Tests\TemplateArrayAccessObject" in "%s" at line 1.'],
             ['{{ string.a }}', 'Impossible to access an attribute ("a") on a string variable ("foo") in "%s" at line 1.'],
             ['{{ string.a() }}', 'Impossible to invoke a method ("a") on a string variable ("foo") in "%s" at line 1.'],
             ['{{ null.a }}', 'Impossible to access an attribute ("a") on a null variable in "%s" at line 1.'],