When Html Writer outputs a cell with a boolean value, the result will either be 1 or null-string; neither of these is optimal for anyone looking at the resulting html. Html Reader already has the ability to recognize data types using the html `data-type` attribute, but Html Writer doesn't use it. This PR adds the ability to generate that attribute for booleans. It will generate a string value appropriate for the locale when it encounters a boolean. Html Reader, when it encounters `data-type="b"`, will interpret the result as true if the value is 1 or a string value recognized as true in any locale; it will interpret the result as false if the value is 0, null-string, null, or a string value recognized as false in any locale; if none of the above, it will leave the value as an unchanged string. So, Reader will wind up with the correct result even if its locale is different than what Writer used.
Because this is a breaking change, it is opt-in. You need to call `Writer::setBetterBoolean(true)` in order for it take effect. The current default value for that property is false. When it is time to introduce breaking changes (see PR #4240), the default will be changed to true.
This has been requested a few times, most recently issue #4092. Because it's a breaking change, I haven't proceeded with it. But, because I have a breaking change PR #4240 already in the queue, this gives a plan for getting where we want to go (under the extremely likely assumption that most users don't deal with Csv files with Mac line endings). This PR doesn't change the current behavior, but it gets us to a state where a single-line change will be sufficient when the time comes for a new major release.
Fix#4248. PhpSpreadsheet has used what appear to be default attributes and tags when they are missing from Fill patterns and colors. However, Excel handles their absence a little differently from what the "default" would require. PhpSpreadsheet is changed to omit the attributes and tags in question when missing. This change is mostly targeted towards Xlsx read and write, but minor changes for Xls and Html write are also included.
This seems like it could be a breaking change, but I don't think it is. One test (DefaultFillTest introduced by PR #2050) must change, but the change is internal - loading and then saving the spreadsheet used in that change will appear the same after this change as it did before. Other differences are very likely to be bug fixes rather than breaks.
This commit fix two issues that happened when the a cell was formatted as text
* When the cell contains a number prefixed with dollar sign, this number is getting replaced with 0. The replacement happens due to the preg_replace function.
* When the cell contains quotes, the quote would be removed.
Fix#4241. Some security batches caused a minor break in Drawings, forcing `setWorksheet` to come after `setPath`. Although the problem is easily fixed in user code, this was not an intended change. Some slight recoding restores the earlier functionality where the order of calls was not important, without sacrificing the security gains. This change will be back-ported to the other active branches to which the security patch had been applied.
With PhpUnit 10 came the ability to use Php attributes rather than doc-block annotations for things like "data provider". PhpUnit 11 deprecates the use of annotations, and PhpUnit 12 will not not permit their use. Since PhpUnit 11 requires Php8.2+, we cannot adopt it as long as we support Php8.1, which will continue to be the case for some time. However, there is no penalty for early adoption.
Php-cs-fixer can use:
```
'php_unit_attributes' => ['keep_annotations' => false],
```
This allows us to run `composer fix` to automate all the needed changes. No manual changes were needed for any of the test members.
With this change, PhpUnit 9 can no longer be used with the test suite. File composer.json is updated to reflect that reality, and phpunit9.xml.dist, which has been supplied in case anyone needed to use PhpUnit 9, is no longer required, and is thus deleted. For now, PhpUnit 11 is not being added as a possibility.
No source code is changed in this PR.
Having addressed several security advisories, one evident *theoretical* problem remains. This is an attempt to future-proof our code against similar vulnerabilities. It all begins with our implementation of libXmlLoaderOptions, which uses as a default LIBXML_DTDLOAD. This unfortunate choice opens us to XXE problems, many recently solved. I do not believe that there is a legitimate use case for allowing this, and will therefore ignore and deprecate that option.
Although this might seem to be a breaking change, it is not. The setting is used only after the Xml has been subject to a security scan, and the security scan throws an exception if it detects the use of `<!DOCTYPE` within the Xml. Therefore, the setting will be effective only on Xml which does not contain that tag, and will consequently have no effect on most Xml. The only exception would be Xml which has been crafted to avoid detection by the security scanner in a manner which has not been disclosed to us. Although we hope that we've now blocked all such avenues, this provides additional protection just in case.
With this change in place, we could relax certain restrictions, e.g. the use of EBCDIC or even UTF-7. For now, these will remain in place. I will need to be convinced that there is a legitimate use case for easing the restrictions before doing so. We might even consider the elimination of the Security Scanner altogether. However, it does allow for early detection, and, in any case, provides a method to correct Xml which most Xml readers would fail but which Excel accepts.
My plan is to merge this within the next few days, and tag a new release immediately after. It will also be backported to all active branches.
* Security Patch
* Throw Exception for EBCDIC Encoding
* Mixed UTF-8 and UTF-16
Further mischief. I don't know if the examples truly are valid Xml, but PhpSpreadsheet is letting them sneak through.
Fix#4213. Early versions of Php 8.4 caused problems for some Excel functions, among them ROUNDDOWN and ROUNDUP. New code was added which seemed to work for 8.4 and all prior releases (back to 7.4). However, as the issue shows, there is a problem with the new logic that had not been a problem for prior releases (PhpSpreadsheet 1.29.1 with Php 8.3 or earlier).
As it happens, some time after 8.4 broke the existing functionality, new features were added to it which enabled much cleaner logic. The code will now take this cleaner path when available, and revert to the older logic when not. This appears to work for all relevant Php releases, at least till the next non-match between PhpSpreadsheet and Excel is reported.
And, having made the change to ROUNDDOWN, it occurred to me that ROUNDDOWN always returns the same result as TRUNC (they differ only in number of required arguments). So I changed TRUNC to call ROUNDDOWN, and that didn't break anything. So I will keep that change. This probably means I should change Calculation so that ROUNDDOWN is called automatically, and deprecate TRUNC. That can happen at a later time.
Fix#4200. Fix#4145. Although the Xml is valid, Excel insists that worksheet.xml specifies `ignoredErrors` (introduced with 1.29.0) before `legacyDrawing` or `drawing`.
Fix#4192. Although that issue can be dealt with by changing user code, it would be better to fix it within PhpSpreadsheet. A cloned worksheet may have a pointer to a spreadsheet to which it is not attached. Code can assume it does belong to the spreadsheet, and throw an exception when the spreadsheet cannot find the worksheet in question. It may also not throw an exception when it should.
In my comments to the issue, I was concerned that adding in the needed protection would add overhead to an extremely common situation (setting a cell's value) in order to avoid a pretty rare problem. However, there are problems with both the accuracy and efficiency of the existing code, and I think any performance losses caused by the additional checks will be offset by the performance gains and accuracy of the new code.
Spreadsheet `getIndex` attempts to find the index of a worksheet within its spreadsheet collection. It does so by comparing the hash codes of each sheet in its collection with the hash code of the sheet it is looking for. Its major problem problem is performance-related, namely that it recomputes the hash code of the target sheet with each iteration.
A more severe problem is the accuracy of the hash code. It generates this by hashing together the sheet title, the string range of its auto-filter, and a character representation of whether sheet protection is enabled. Title should definitely be part of the calculation (it must be unique for all sheets attached to a spreadsheet), but it is not clear why this subset of the other properties of Worksheet is used. It tries to save some cycles by using a `dirty` property to indicate whether re-hashing is necessary. It sets that property whenever the title changes, or when `setProtection` is called. So, it doesn't set it when auto-filter changes, and you can easily bypass `setProtection` when changing any of the `Protection` properties. Not to mention the many other properties of worksheet that can be changed. Additionally, if you clone a worksheet, the clone and the original will have the same hash code, which can lead to problems:
```php
$clone = clone $original;
$spreadsheet->getSheet($spreadsheet->getIndex($clone))
->setCellValue('A1', 100);
```
That code will change the value of A1 in the original, not the clone.
The `hash` property in Worksheet will now be calculated immediately when the object is constructed or cloned or unserialized. It will not be recalculated, and there is no longer a need for the `dirty` property, which is removed. Hash will be generated by spl_object_id, which was designed for this purpose. (So was spl_object_hash, but many online references suggest that \_id performs much better than \_hash.) Our problem example above will now throw an Exception, as it should, rather than changing the wrong cell. `setValueExplicit`, the problem in the original issue, will now test that the worksheet is attached to the spreadsheet before doing any style manipulation. In order that this not be a breaking change, `getHashCode` will continue to return string, but it is deprecated in favor of `getHashInt`, and Worksheet will no longer implement IComparable to facilitate the deprecation.
I had a vague hope that this change might help with issue #641. It doesn't.
Fix#4197. Overlooked in the introduction of Dynamic Arrays, Data Validation can specify a list to be a result of the spill operator, which is implemented via the ANCHORARRAY function.
It appears that function `DataValidator::isValid` will not work if the list is specified in this manner, nor if it is specified as a defined name. Fixing those situations will be difficult (defined names probably easier than ANCHORARRAY), and there is no reason to delay this change waiting for those to be fixed. I will open a new issue when this PR is merged.
Fix#4201. Although that issue can be dealt with without any change to PhpSpreadsheet, it is pretty clear that `setLabelFont` has been accidentally omitted from Chart/Layout. Add it now. Xlsx Chart Writer is changed to use font name from labelFont if latin, eastAsian, or complexScript is uninitialized. Finally, chart label font size is multiplied by 100 in Xlsx Writer, as it is in Excel, but the corresponding division by 100 has been omitted from Xlsx Chart Reader - add that now.
See #4161. The best way to future-proof is to set the escape character to null string, and set testAutoDetect to false before reading. However, depending on the file being read, this may lead to different results than expected. This will be unavoidable because Php itself will change. This PR adds a new static method `affectedByPhp9` to Csv Reader. This can be used to identify in advance whether an input file will be affected by the changes. This will allow users to identify problems in advance, and prepare for how they might be handled.
Fix#2157. Excel 2003 format allows some things which a real Xml parser would reject. In particular, it permits Html entities, and leading whitespace. Change Xml Reader to do likewise.
oleibman
Everton Barbosa
m7913d