Its defaults are to unescape within single quotes, and escape within double quotes and here-docs. Right now, it leaves everything as-is, which means our code is inconsistent and need not be. Further, although dealing with complex regular expressions will never be easy, I find it much easier to figure out what's going on when superfluous back-slashes are removed.
These changes were all made automatically using the "fix" script, so should be reliable. They, of course, pass all unit tests.
The native preg functions (preg_match, preg_replace, etc.) often require us to add a lot of useless boilerplate code to satisfy Phpstan, Scrutinizer, etc. Composer/Pcre offers us a way to remove that boilerplate, thereby giving us a cleaner codebase. I decided to try it on a few modules, and saw a result that clearly demonstrated the usefulness of doing this (aside from the cleaner codebase).
Sample 22_Reader_issue1767 reads an Xlsx spreadsheet with complex sheet names used in defined names, and writes it to Xlsx and Xls output files. When I changed Writer/Xls/Parser to use Composer/Pcre, this sample failed in many different places writing the Xls file. It turns out that some regexes were failing not because the string didn't match, but because the regex encountered "catastrophic backtracing". Composer/Pcre throws an exception when this happens; the native preg_match does return false, but we were not checking for that. The regexes in question are now changed to something which works, and formal unit tests are added for them. Finding this previously undetected error indicates that we should proceed with this change.
An alternative to using Composer/Pcre would be to test for false after all the preg calls. I have done this in the two samples changed with this PR. That seems adequate for a small number of changes, but it really just makes for more clutter considering the large number of regexps that we use in our code. I think Composer/Pcre is a better choice.
It isn't quite transparent. Composer forces all regexps to use PREG_UNMATCHED_AS_NULL, so some match fields will now be null instead of null-string (or non-existent if the unmatched field comes at the end). Our test suite doesn't report any problem (yet) due to this change, although Phpstan is sensitive to it. Several Phpstan annotations were eliminated due to this change, but some others are now needed.
It is not necessary to do this all at once. This PR addresses all the calls in Writer. I intend to address other components in several tickets.
With PhpUnit 10 came the ability to use Php attributes rather than doc-block annotations for things like "data provider". PhpUnit 11 deprecates the use of annotations, and PhpUnit 12 will not not permit their use. Since PhpUnit 11 requires Php8.2+, we cannot adopt it as long as we support Php8.1, which will continue to be the case for some time. However, there is no penalty for early adoption.
Php-cs-fixer can use:
```
'php_unit_attributes' => ['keep_annotations' => false],
```
This allows us to run `composer fix` to automate all the needed changes. No manual changes were needed for any of the test members.
With this change, PhpUnit 9 can no longer be used with the test suite. File composer.json is updated to reflect that reality, and phpunit9.xml.dist, which has been supplied in case anyone needed to use PhpUnit 9, is no longer required, and is thus deleted. For now, PhpUnit 11 is not being added as a possibility.
No source code is changed in this PR.
PHPCompatibility erroneously flags the use of $this in enumerations.
Commit 5e248cf moved disabling of this error check to .github/workflows/main.yml
Hence, `composer versions` and `composer check` should also exclude this check.
Dependabot suggested some changes this month which required an unusual effort to implement successfully. With the elimination of Php 8.0 as a supported environment, it became possible to use Phpunit 10 rather than 9. Among other considerations, the configuration file for Phpunit is changed. I preserved the Phpunit 9 version under a different name. Aside from the configuration change, several other changes needed to be made to accommodate the change:
- Fix#3993. A peculiar problem indeed. One of the reporters said it had something to do with mocking, but I couldn't duplicate it. But Phpunit 10 revealed the problem in one test (Reader/Xlsx/AutoFilterTest), and that was sufficient for me to apply the trivial source code change to Worksheet/Worksheet.
- Some extra stringency on data providers required extra work in Calculation/CalculationFunctionListTest.
- There was a misplaced label in Calculation/ParseFormulaTest. Likewise in the data member CellGetRangeBoundaries.
- More stringency required changes to data members Shared/Trend/ExponentialBestFit and Shared/Trend/LinearBestFit.
- Issue3982Test testLoadAllRows seemed to go into a memory-acquiring loop in 10 that was not evident in 9. This particular test uses a lot of memory by design, but was included only to establish a base level for the other tests in that member. I feel it is acceptable to skip it for 10.
Phpstan errors with a new release are not unusual. One of the problems this time around was, however, unusual - it is fixed when Phpstan runs under Php8.3, but not for earlier Php releases. Our tools currently use Php 8.1. It is on my to-do list to get Phpstan and other test tools running under 8.3 before 8.1 goes EOL. There are also some configuration file changes needed for Phpstan.
Php-cs-fixer has been taking an increasingly long time to run. They've added an experimental option to permit it to run its checks in parallel. I've changed its configuration to use that option.
This is according to our formal, published, policy to only support
EOL PHP after 6 months.
See https://phpspreadsheet.readthedocs.io/en/latest/#php-version-support
Also consolidate documentation in a single place, not in the README,
because it's a pain to maintain almost-duplicated-but-not-really
documentation.
Fix#3954. In response to a Security Incident, a package was added to the project to sanitize the Html for comments attached to a cell. We have tried at least 2 different packages for this purpose, and users have raised legitimate concerns about both.
I believe that adding a sanitizer package, although it addresses the problem, was overkill. Cell comments are a RichText object, and Html Writer already handled RichText cell *values*. Values did not figure in the Incident, and they surely would have done so if they were a problem because it is a lot easier to set up test cases for values than for comments. RichText values were not a problem because they were sanitized with `htmlspecialchars`; if comments were to use the same code that RichText was already using, it would likewise be safely sanitized. As an added bonus, the existing code for comments only uses the plaintext value, but the values code would allow the comments to be styled, just as they are for Xlsx.
This is a breaking change - I don't think it will affect a lot of users, but there may be some. It is worth noting that the comment block before function `writeComment` has a link explaining what is being done. That link mentions only styling elements, not other possibilities like hyperlinks. At any rate, if people are putting styling tags (e.g. `<b>`) in their comments for this purpose, those will no longer work; the styling needs to be applied to the RichText elements. That will keep the user code the same regardless of the format of the intended output, which is certainly a good thing, but it is a break. If people are trying to insert non-styling tage (e.g. `<a>`), those will no longer work, just as there is no way to do it for Xlsx (although Excel itself may convert the raw text to a hyperlink, but that's out of scope, at least for now). I also note that, even with the current code, I haven't yet found a way to keep the Html comment in place long enough to actually click on any hyperlinks.
The main package being dropped brings several other packages along with it, so the project as a whole will have a slightly lighter footprint than before.
The existing `XssVulnerability` test cases are all preserved. Although the final result of the sanitizing changes, it can easily be seen that the results are all harmless.
So that composer will not update our dependencies to something that
would not work with PHP 8.0. And so that PHPStan analyzes assuming we
are running 8.0
PHPDoc types can sometimes be entirely expressed as PHP native types. It
is better because it avoids code duplication and enables type runtime
check.
This will help us slowly migrate away from PHPDoc typing to PHP native typing.
This is according to our formal, published, policy to only support
EOL PHP after 6 months.
See https://phpspreadsheet.readthedocs.io/en/latest/#php-version-support
Also share the exact same dev deps across all PHP version for GitHub
Actions so runs are faster, much most importantly they are stable and
predictable. And we decide manually when we want to migrate to PHPUnit
10.
Fixes#3634Closes#3710
They have made some changes at my request, the major effect of which is that it will now work with 33_Chart_create_bar_stacked. This is a departure for them in that they have changed the functionality of jpgraph, not merely made sure that it is compatible with new Php releases.
Zipstream's newest production release requires Php8.1. This is not necessarily a problem for us as we are locked to an earlier release. However, psr/simple-cache became a real mess for us as people wanted to install PhpSpreadsheet alongside other products which required a version of simple-cache that could not run in some of the versions of Php which we support, and I would like to avoid having to go through that again. This ticket is a contingency to put us ahead of the curve in case such a problem should arise with Zipstream. There is reason to believe that Zipstream is not so widely used as simple-cache, so we do not need to rush this change in. However, should that belief prove incorrect, the change should be ready to go when needed.
* PhpUnit 10 Compatibility Part 3 (Last)
Final changes for PhpUnit 10, including enabling it for testing. This finishes the work of PR #3523 and PR #3526.
The major remaining problem with PhpUnit 10 is that earlier releases converted notices and warnings to exceptions, and 10 does not. Not having the information provided by the messages seems risky to me. Fortunately, it appears that you can add an error handler to the test bootstrap for 10 and make it act like earlier versions; I have done so. In order to demonstrate the effectiveness of this handler, a new otherwise unused class Helper/Handler and tests for that class are added.
As part of the testing of this change, it became apparent that the fopen in OLE::getStream attempts to create a dynamic property $context in Shared/OLE/ChainedBlockStream, and that action is deprecated in Php8.2. Adding the property to the class eliminates that problem. No executable code is added, and this is the only change to source code.
There also seems to have been a change in assertXmlStringEqualsXmlString in PhpUnit 10. The only test which uses that method is Chart/Issue589Test, and both the places which use that method could just as easily and effectively use assertSame. They are changed to do so.
* Remove Phpunit Verbose Option
Not supported in PhpUnit 10. I'm not at all sure that this is the correct solution for this problem.
* Try Changing Phpunit Command for Different Php Releases
Not sure how to test this locally. We'll see if it works on github.
* Another main.yml attempt
We shall see.
* Eliminate One Test
Not sure why testDeprecated is not working in Github; it works locally. Disable it for now and continue to research.
* Show Incomplete and Other Messages in PhpUnit 10
6 new command line args to replace the 1 they got rid of.
* Restore Disabled Test
Deprecated messages are suppressed by default setting for error_reporting. Switch that to E_ALL in bootstrap and restore original test.
* Add Deprecation Tests for PhpUnit 9-
Default configuration option caused deprecation messages to be suppressed. Change the option.
* More Carets in composer.json
Following up on PR #3086, there are 3 additional items in the require-dev section that should have carets. I probably accidentally removed them for tcpdf and mitoteam, which point to the current release anyhow. Dependabot appears to be responsible for mpdf, which is not pointing to the current release, but I have tested with current successfully.
* Minor Fix for Change Made Earlier Today
"Scrutinizer Tweak" causes a Phpstan error. Fix baseline to correct it.
They have just issued a new release. Using that should eliminate the last of our Php8.2 unit test problems, so we will be able to change that to non-experimental when it is convenient for us.
No source code changes. Mitoteam added a change to better accomodate Cygwin; pick up their new release. While testing, discovered one test that was already skipped on Windows and needs to be skipped on Cygwin as well.
* Memory Leak in Sample35
All but 6 chart samples can be rendered by Sample35. Of those 6, 3 of the problems are because the script runs out of memory processing them. Adopting a suggestion from @MAKS-dev in issue #2092, adding a call to gc_collect_cycles after the charts from each spreadsheet is rendered appears to make it possible to include those 3 spreadsheets in Sample35 after all.
Also take advantage of this opportunity to correct a number (hopefully all) of Scrutinizer problems with JpgraphRendererBases.
* Minor Fix
Problem running 8.1 unit tests.
* Resolve Problems with Pie 3D Charts
Minor fix, leaving only one spreadsheet unusable in Sample35. The reasons for its unusability are now documented in the code.
* Mitoteam Made Changes
Discussing this problem with them, they decided they should make a change for Pie3D rather than forcing us to use the workaround pushed earlier. Change to require mitoteam 10.2.3, revert workaround.
They just released a Php8.2-compatible version. We should use that version going forward. Some tests had been disabled in 8.2 due to the problems which the new release fixes; these are now restored.
* Upgrade Dev TCPDF to 6.5
Implementation of https://github.com/tecnickcom/TCPDF/pull/467, which is available in just-released Tcpdf 6.5, will improve look of Tcpdf rendering for PhpSpreadsheet. Fix#1164.
One test had been suppressed for Tcpdf, ostensibly because it was not compatible with Php8. As it turns out, the PhpSpreadsheet code which invokes Tcpdf was (harmlessly) incorrect, so the Php8 issue was actually with PhpSpreadsheet, not Tcpdf. That code is corrected, and the test is no longer suppressed.
* Update Change Log
Pick up some earlier changes as well as this one, and deprecations which had been omitted from the 1.24 change log.
* Replace Dev jpgraph/jpgraph with mitoteam/jpgraph
PR #2979 added support for mitoteam/jpgraph as an alternative to jpgraph/jpgraph. The package jpgraph/jpgraph is abandoned in composer, and the version loaded with composer has been unusable for some time. This PR removes the dev requirement for jpgraph/jpgraph, and adds a dev requirement for mitoteam/jpgraph in its place.
With a usable graph library, a number of tests and samples that had been disabled are now re-enabled. A lot of new functionality has been added to Charts recently. Some of that new code has exposed bugs in JpgraphRendererBase. I have fixed those where I could. A handful of exceptions remain; I will investigate, and hopefully fix, those over time, but I don't feel it is necessary to fix them all before installing this PR - we are already way ahead of the game with the graphs that are working.
Three members had been ignoring code coverage in whole or in part because of the unavailability of a usable graph libray. Code coverage is restored in them. I am relieved to report that, although they aren't completely covered, adding them did not reduce code coverage by much - it is still over 90.4%.
I took a look at JpgraphRendererBase and Phpstan. Phpstan reports 128 problems. When I added some docblocks to correct some of those, the number increased to 284. Sigh. I will investigate over time, but, for now, we will still suppress Phpstan for JpgraphRendererBase.
I do not find a License file for mitoteam. However, there also wasn't one for jpgraph in the first place. Based on that and the discussion in #2996 (mitoteam will be used in exactly the same manner as mpdf), I don't think this is a problem. IANAL.
* PHP 8.2 Problems
Tons of "cannot create dynamic property" deprecations in jpgraph. Disable the test with most of those for now; leave the two with only a handful of messages enabled.
* Correct Failures in 2 Stock Charts
Down to 6 templates on which Render fails.
Per https://getcomposer.org/doc/06-config.md#allow-plugins, Composer has added an extra layer of security in V2.2, and will start enforcing it in July of this year. Phpcodesniffer uses a plugin and will be affected if composer.json isn't updated before that date.
Co-authored-by: Mark Baker <mark@lange.demon.co.uk>
oleibman
m7913d
dependabot[bot]
Adrien Crivelli
dmytromikhieiev1985
MarkBaker
BrokenSourceCode
MarkBaker
Kevin Bond