This PR was squashed before being merged into the 3.x branch.
Discussion
----------
Fix the empty comment "{##}" being lexed as a documentation comment opening
Since #4871, the empty comment `{##}` fails to lex: the source is `{#` immediately followed by the closing `#}`, but the lexer now reads it as the documentation comment opening `{##`, which steals the `#` belonging to the closing tag. The remaining input never contains a closer, so lexing fails with `Unclosed comment`, and in a larger template everything up to the next comment gets swallowed, which surfaces as confusing errors like `Expected endblock for block "form_row_render" (but "submit_row" given)`.
Minimal reproducer:
```php
(new \Twig\Environment(new \Twig\Loader\ArrayLoader()))->createTemplate('{##}')->render();
// Twig\Error\SyntaxError: Unclosed comment at line 1.
```
This is currently breaking the symfony/symfony 8.2 CI on every new run: the Twig bridge form themes use `{##}` as a line-joining trick (e.g. `bootstrap_3_horizontal_layout.html.twig`), so all form layout tests fail to compile ([example run](https://github.com/symfony/symfony/actions/runs/32267973483/job/96117211040), 547 errors).
The fix adds a negative lookahead to the documentation comment opening so that the exact sequence `{##}` keeps lexing as a regular (empty) comment. The lookahead is only built when the comment closing tag starts with `#`, so custom delimiters where the ambiguity cannot exist (and where an empty documentation comment would legitimately match the lookahead) are unaffected. `{###}` (an empty documentation comment) and `{##-#}` keep working.
Commits
-------
c9c3b23a77 Fix the empty comment "{##}" being lexed as a documentation comment opening
This PR was squashed before being merged into the 3.x branch.
Discussion
----------
Attach documentation comments to nodes
Alternatives to #4870
To avoid BC breaks, I have another idea, using `##` as a new syntax, a bit like `/** */` in PHP vs `/* */`.
"Documentation" is attached as metadata to the next relevant node:
```twig
{## The main content displayed on the page #}
{% block content %}
...
{% endblock %}
```
Documentation comments can also describe variables declared with the `types` tag:
```twig
{% types {
## The unique identifier of the article
id: 'string',
## Whether the article should be highlighted
featured?: 'boolean',
} %}
```
Node visitors can access this metadata through `Node::getDocumentation()`, allowing IDEs, static analyzers, and documentation generators to consume it without affecting template rendering. Documentation is preserved when visitors or optimizations replace nodes.
Closes#4768Closes#4870
Commits
-------
6806e30474 Attach documentation comments to nodes
* 3.x:
Add support for tempest/markdown in markdown-extra
Add the include_only function to render a template without access to the current context
Clarify duplicate macro deprecation message
Deduplicate template error handling
# Conflicts:
# .gitignore
# CHANGELOG
# extra/markdown-extra/composer.json
# src/Extension/CoreExtension.php
# src/Parser.php
# src/Template.php
# tests/Fixtures/macros/duplicate_definition.legacy.test
This PR was merged into the 3.x branch.
Discussion
----------
Add the include_only function
Commits
-------
dc8f96df3e Add the include_only function to render a template without access to the current context
This PR was merged into the 3.x branch.
Discussion
----------
Add support for tempest/markdown in markdown-extra
Adds `TempestMarkdown`, an adapter for [`tempest/markdown`](https://github.com/tempestphp/markdown), alongside the existing `LeagueMarkdown`, `MichelfMarkdown` and `ErusevMarkdown` implementations. It follows the same pattern as the others and accepts a pre-configured `Tempest\Markdown\Markdown` in its constructor, so rules and the highlighter can be customized.
It is also appended as the **last** branch of `DefaultMarkdown`'s discovery chain, so projects that already have another library installed keep resolving to it exactly as before.
### PHP requirement
Every published version of `tempest/markdown` requires PHP `^8.5`, while `twig/markdown-extra` supports `>=8.1`. So:
- it is declared in `require-dev` only;
- a CI step removes it before `composer install` on PHP < 8.5, mirroring the existing conditional step used for `twig-extra-bundle`;
- `FunctionalTest` only adds it to the converter matrix when `Tempest\Markdown\Markdown` exists.
The suite passes both with and without the library installed.
### Test data change
Three cases in `getMarkdownTests()` used Setext headings (`Hello` underlined with `=====`). `tempest/markdown` only implements ATX headings, so those were switched to `# Hello`. Those cases exercise the filter plumbing (`{% apply %}`, indentation stripping, `include()|markdown_to_html`) rather than the Markdown dialect, so no coverage is lost.
Two patterns were also relaxed for the same reason: `<h1[^>]*>` because Tempest emits auto heading ids, and `<p>…\s*</p>` because it keeps the source's trailing newline inside the final paragraph. Both remain accurate for the other converters.
These differences, plus the fact that front matter is parsed out rather than rendered, are documented in a note in `doc/filters/markdown_to_html.rst`.
### Unrelated one-liner
The last commit also adds `.php-cs-fixer.cache` to `.gitignore` — it is generated by the project's own `php-cs-fixer` dev dependency and was showing up as untracked. Happy to split it out if you'd rather keep this PR to a single concern.
Commits
-------
aa17f59877 Add support for tempest/markdown in markdown-extra
This PR was squashed before being merged into the 3.x branch.
Discussion
----------
Nested macro imports
Closes#4879
Replaces #4880
Commits
-------
b53e100444 Nested macro imports
* 3.x:
Deprecate using parentheses when testing a macro with the defined test
# Conflicts:
# CHANGELOG
# doc/deprecated.rst
# src/ExpressionParser/Infix/DotExpressionParser.php
# src/ExpressionParser/Infix/FunctionExpressionParser.php
# src/ExpressionParser/Infix/IsExpressionParser.php
# tests/Fixtures/macros/call_without_parentheses.legacy.test
# tests/ParserTest.php
This PR was squashed before being merged into the 3.x branch.
Discussion
----------
Deprecate using parentheses when testing a macro with the defined test
Commits
-------
34d9c67d38 Deprecate using parentheses when testing a macro with the defined test
This PR was squashed before being merged into the 3.x branch.
Discussion
----------
Redesign macro calls and argument handling
This is my attempt to make macros "better". It uses modern PHP features that didn't exist when I designed macros a long time ago.
The first objective is to close the gap between their behavior and the behavior of Twig callables: functions, filters, and tests.
Here are some important changes:
* Calling a macro without passing a value for an argument that has no default value is deprecated; it is currently silently passed as `null`.
* Passing extra positional arguments or unknown named arguments to a macro without an explicit variadic argument is deprecated; these arguments are currently silently accepted through the implicit `varargs` variable.
* Explicit variadic macro arguments are now supported with `...name`.
* Macros are compiled as closures stored in the macro registry, instead of public generated `macro_*` methods.
The refactor introduces `TwigMacro` and `MacroArgument` to represent template-defined macros with an explicit signature, similar to the existing Twig callable model.
Commits
-------
d7f8b4eb1c Redesign macro calls and argument handling
* 3.x:
Make the sandbox a first-class citizen with a dedicated Sandbox class
# Conflicts:
# CHANGELOG
# doc/deprecated.rst
# doc/tags/sandbox.rst
# phpstan-baseline.neon
# src/Extension/CoreExtension.php
# src/Extension/SandboxExtension.php
# src/Sandbox/SecurityPolicy.php
This PR was squashed before being merged into the 3.x branch.
Discussion
----------
Make the sandbox a first-class citizen with a dedicated Sandbox class
I've been thinking about making the sabdbox feature as a first class citizen for years. With all the work that has been done recently on security issues, I spent some time on it again. Here is the result.
The main ideas:
* Currently, the sandbox is thigtly coupled to the "main" environment: `SandboxExtension` is registered on the environmen directly, so it instruments all compiled template, and adds runtime checks to all renders, trusted or not. As recommended in the docs, you should have a dedicated environment for sandboxes, different from the main one, but it's not really "enforced" nor natural to do.
* As a consequence, we store some state via `enableSandbox()`/`disableSandbox()` with try/finally patterns scattered across the codebase to support rendering sandboxed and non-sandboxed templates from a environment.
* When using one environment, a sandboxed template can `include` anything the loader can load, sees every application global, and inherits all extensions, this is a footgun (again, already not recommended in the docs).
* There are too maybe "knobs": global mode, `enableSandbox()`, `{% include(..., sandboxed: true) %}`, and `{% sandbox %}`.
The new `Twig\Sandbox\Sandbox` class renders untrusted templates through a dedicated, always-sandboxed environment crafted by the developer. Taht way, there is no state to toggle and nothing leaks between the main environment and the sandbox, in either direction.
Commits
-------
b762bc94b9 Make the sandbox a first-class citizen with a dedicated Sandbox class
This PR was merged into the 3.x branch.
Discussion
----------
Deprecate macro calls without parentheses
Commits
-------
ad305b414e Deprecate macro calls without parentheses
This PR was merged into the 3.x branch.
Discussion
----------
Rename macro variable AST nodes
The current name are just wrong as these classes are only used in the context of macros. They were confusing.
Commits
-------
be36fee09e Rename macro variable AST nodes
This PR was merged into the 3.x branch.
Discussion
----------
Clarify the security scope for untrusted templates
Commits
-------
222a7f3f9a Clarify the security scope for untrusted templates
Nicolas Grekas
Fabien Potencier
Pascal CESCON - Amoifr
Romain Monteil