We currently use Phpstan Release 1. Release 2 has been available for some time, and we would like to stay current if possible. In order to get to Release 2, we first have to pass the "bleeding edge" tests for Release 1. I will do this by generating a new baseline using bleeding edge, then fix the baseline errors over several different PR's. This is the first of those.
Its defaults are to unescape within single quotes, and escape within double quotes and here-docs. Right now, it leaves everything as-is, which means our code is inconsistent and need not be. Further, although dealing with complex regular expressions will never be easy, I find it much easier to figure out what's going on when superfluous back-slashes are removed.
These changes were all made automatically using the "fix" script, so should be reliable. They, of course, pass all unit tests.
It currently parsed by byte, which is not a good thing in a UTF-8 system. See the discussion at the bottom of PR #4203. That turns out to not be the change which led to this problem; that would have been PR #4323. That change came about because using Composer/Pcre revealed bugs in several regular expressions used in Writer/Xls/Parser. This ticket comes about because more bugs were revealed in the same module. The problem is that the `advance` method needs to process formulas character by character, but is instead doing it byte by byte. It is changed to advance by characters, and tests for non-ASCII characters are added.
Fix#4092. Change default value for Csv Reader autodetect line endings. Prior behavior can be enabled via `setTestAutodetect(true)`.
Change default value for Html Writer "better boolean" logic. Prior behavior can be explicitly enabled via `setBetterBoolean(false)`.
Change default for Xlsx Writer forceFullCalc option. Prior behavior can be explicitly enabled via `setForceFullCalc(null)`.
Fix#4331. Parser treats TRUE/FALSE as functions rather than constants, which is nominally harmless, but it then expects an argument count of 0 and instead sees null-string. Changed to recognize this situation and leave TRUE/FALSE/TRUE()/FALSE() unchanged.
The native preg functions (preg_match, preg_replace, etc.) often require us to add a lot of useless boilerplate code to satisfy Phpstan, Scrutinizer, etc. Composer/Pcre offers us a way to remove that boilerplate, thereby giving us a cleaner codebase. I decided to try it on a few modules, and saw a result that clearly demonstrated the usefulness of doing this (aside from the cleaner codebase).
Sample 22_Reader_issue1767 reads an Xlsx spreadsheet with complex sheet names used in defined names, and writes it to Xlsx and Xls output files. When I changed Writer/Xls/Parser to use Composer/Pcre, this sample failed in many different places writing the Xls file. It turns out that some regexes were failing not because the string didn't match, but because the regex encountered "catastrophic backtracing". Composer/Pcre throws an exception when this happens; the native preg_match does return false, but we were not checking for that. The regexes in question are now changed to something which works, and formal unit tests are added for them. Finding this previously undetected error indicates that we should proceed with this change.
An alternative to using Composer/Pcre would be to test for false after all the preg calls. I have done this in the two samples changed with this PR. That seems adequate for a small number of changes, but it really just makes for more clutter considering the large number of regexps that we use in our code. I think Composer/Pcre is a better choice.
It isn't quite transparent. Composer forces all regexps to use PREG_UNMATCHED_AS_NULL, so some match fields will now be null instead of null-string (or non-existent if the unmatched field comes at the end). Our test suite doesn't report any problem (yet) due to this change, although Phpstan is sensitive to it. Several Phpstan annotations were eliminated due to this change, but some others are now needed.
It is not necessary to do this all at once. This PR addresses all the calls in Writer. I intend to address other components in several tickets.
Fix#4319. PR #3495 allowed Mpdf and Tcpdf to handle hidden rows and columns. The handling is a little off when merged cells are part of the output. The fix for Mpdf is pretty straightforward. Tcpdf is a little more convoluted. Dompdf and Html do not require a change. Unit tests are added for the same scenario for all of Mpdf, Tcpdf, Dompdf, and Html.
Fix#4311. Period is a valid character in a sheet name. When a sheet with such a name is referenced in Ods format, the sheet name must be enclosed in apostrophes, because Ods uses period to separate sheet name from cell address. (Excel uses exclamation point so doesn't necessarily need to enclose the sheet name in apostrophes.) This causes a problem for the Ods Reader whenever it tries to parse such an address; however, the problem showed up specifically for auto filters, because the Ods xml for those specifies *'sheetname'.startcell:'sheetname'.endcell* (Excel omits sheetname).
Ods Reader translates these addresses in 2 different methods in FormulaTranslator. I had a relatively elegant method for handling this situation in convertToExcelAddressValue, but I could not make it work in convertToExcelFormulaValue. A kludgier method works for Formula, and also for Address. I decided it's better to be consistent, so I'm going with the kludgier method for both.
It would not surprise me in the least if there are similar problems lying in wait for other special characters in sheet names, and for other formats besides Ods. For now, I will limit myself to fixing the known problem.
Fix#4269. In response to issue #456 PR #515, `forceFullCalc` was added to workbook.xml whenever `preCalculateFormulas` was set to false. It is not clear why this should have been needed; attempts to reproduce the error in the original 6.5-year-old issue are unable to reproduce it today. Nevertheless, it is, or was, there for a reason.
Today, the forceFullCalc option sets an option where formulas *might* not be recalculated when a cell used in the formula changes. I have not succeeded in finding a situation where it doesn't automatically recalculate, but it probably exists on complicated spreadsheets. To overcome this possibility, Excel offers a button which can be used to recalculate on demand. By itself, this might not be a terrible problem. However, it seems to come with the strange property that any spreadsheets opened at the same time as a forceFullCalc spreadsheet operate as if they too specified forceFullCalc. That *is* a problem, especially since users when closing a spreasheet affected in this way will be prompted to save it even when they haven't changed anything.
I am not willing to make a BC change at this time, although I might consider it in future (PR #4240). For now, I am adding a new property `forceFullCalc` with setter (no getter needed) to Xlsx Writer. That property can be `null` (default, in which case the Xml attribute is set as today), or `false` or `true` (in which case the Xml attribute will be set to the Writer attribute). I think that, when `preCalculateFormulas` is set to false, the calling application should give consideration to setting `forceFullCalc` to false as well. All other situations should just use the default.
Fix#4261. Ods does nothing with the `indent` property of Alignment. The fix is easy, but not as easy as it should be. Excel treats indent as an int in some unspecified unit. Ods, on the other hand, specifies it as a float with unit (inches, ems, etc.). Since MS does not reveal what an indent unit is, I resorted to some experimentation. It appears that 1 unit is equal to 0.1043 inches. That is not a guaranteed relationship - the conversion might be non-linear, or it might be affected by external factors like font size. I think multiplying by 0.1043 is adequate for now, and unquestionably better than what we're currently doing (ignoring the property). If it breaks down at some point, we'll look at it again. BTW, Shared\Drawing uses some conversion values of 9525, which, reciprocating and sliding the decimal point along yields a value of 0.10498..., which is close but not close enough for me to use.
The other property used in conjunction with indent is `text-align`, and here we have not been doing the right thing. If that property has the default value (`General`), it is treated as `start` (i.e. `left` on LTR docs and presumably `right` on RTL). This will align numbers as if they were text, which does no harm (they're still usable as numbers), but is not how LibreOffice handles them by default. Ods writer is changed to omit `text-align` when it is set to `General`, and let LibreOffice choose the appropriate alignment.
These changes are for Writer only. Ods Reader support for styles remains severely lacking.
When Html Writer outputs a cell with a boolean value, the result will either be 1 or null-string; neither of these is optimal for anyone looking at the resulting html. Html Reader already has the ability to recognize data types using the html `data-type` attribute, but Html Writer doesn't use it. This PR adds the ability to generate that attribute for booleans. It will generate a string value appropriate for the locale when it encounters a boolean. Html Reader, when it encounters `data-type="b"`, will interpret the result as true if the value is 1 or a string value recognized as true in any locale; it will interpret the result as false if the value is 0, null-string, null, or a string value recognized as false in any locale; if none of the above, it will leave the value as an unchanged string. So, Reader will wind up with the correct result even if its locale is different than what Writer used.
Because this is a breaking change, it is opt-in. You need to call `Writer::setBetterBoolean(true)` in order for it take effect. The current default value for that property is false. When it is time to introduce breaking changes (see PR #4240), the default will be changed to true.
With PhpUnit 10 came the ability to use Php attributes rather than doc-block annotations for things like "data provider". PhpUnit 11 deprecates the use of annotations, and PhpUnit 12 will not not permit their use. Since PhpUnit 11 requires Php8.2+, we cannot adopt it as long as we support Php8.1, which will continue to be the case for some time. However, there is no penalty for early adoption.
Php-cs-fixer can use:
```
'php_unit_attributes' => ['keep_annotations' => false],
```
This allows us to run `composer fix` to automate all the needed changes. No manual changes were needed for any of the test members.
With this change, PhpUnit 9 can no longer be used with the test suite. File composer.json is updated to reflect that reality, and phpunit9.xml.dist, which has been supplied in case anyone needed to use PhpUnit 9, is no longer required, and is thus deleted. For now, PhpUnit 11 is not being added as a possibility.
No source code is changed in this PR.
Fix#4200. Fix#4145. Although the Xml is valid, Excel insists that worksheet.xml specifies `ignoredErrors` (introduced with 1.29.0) before `legacyDrawing` or `drawing`.
Fix#4197. Overlooked in the introduction of Dynamic Arrays, Data Validation can specify a list to be a result of the spill operator, which is implemented via the ANCHORARRAY function.
It appears that function `DataValidator::isValid` will not work if the list is specified in this manner, nor if it is specified as a defined name. Fixing those situations will be difficult (defined names probably easier than ANCHORARRAY), and there is no reason to delay this change waiting for those to be fixed. I will open a new issue when this PR is merged.
Fix#4179. Sheet has a background image, and the code is not properly accounting for the image's extension already being defined in ContentTypes, resulting in what Excel thinks is a corrupt file.
oleibman