Html Reader recognizes a `data-format` attribute for storing the number format associated with a cell. However, Html Writer does not set this attribute. This PR offers a new Writer property `preserveFormatAndValue` and setter; when `true`, for any cell which uses a number format other than the default `General` the writer will generate a `data-format` attribute and, for strings or numbers, it will also generate a `data-value` attribute. This will increase the accuracy of the spreadsheet when reading/writing Html.
To make this change less abstract, assume a cell which has a value of `1234` and a number format of `$#,###.00`. The cell displayed in the generated Html will appear, regardless of the setting of the new property, as `$1,234.00`. However, the Reader currently treats it as a string with that value. If the html has been generated with the new property set to `true`, the Reader will treat it as a numeric cell with a value of `1234` and a style of `#,###.00`. This permits a more accurate reproduction of the original spreadsheet.
I suspect that setting the new property, aside from causing a slight increase in file size, will not cause any significant breaks. However, I'm not totally convinced, so the property will default to `false` (no change in behavior). The default may change in a future breaking release.
Researching a security advisory, the code for handling text colors in Html Writer when the cell is non-numeric and the color is hard-coded in the Number Format looked a bit suspicious. As it turns out, the code was *not* subject to a security exploit. However, the color for the Number Format was ignored. This PR corrects that omission.
Loading external images from a spreadsheet can subject the caller to security exploits. For those who think they need something between not loading any external images (the default and our strong recommendation) or loading them all unconditionally (the current alternative), a callback can now be provided to indicate whether a specific image is okay to load. We continue to urge caution in using this option.
This will improve round-tripping from Xlsx to Html and back again. The calculated result will still be what is displayed when viewing the result in a browser, but, using the new option, the original formula can be reconstructed when PhpSpreadsheet reads the Html file. The ability to recognize this situation is automatically enabled in Html Reader, however, to avoid BC issues, it needs to be explicitly enabled for Html Writer. A new boolean property `dataFormula` and setter are added to Writer; it will default to current behavior `false` for now, but my plan is to change the default in the next breaking release. I cannot think of a decent use case for a getter, so have not added one.
Fix#647, which went stale in 2019 and is now reopened.
Fix#849, which went stale in 2019 and is now reopened.
Html Reader currently echoes any warning messages from Dom::loadHtml when it parses Html input. These are generally not needed, and may cause PhpSpreadsheet to fail needlessly. A new property `$suppressLoadWarnings` is added. It will additionally default to `null`, which will avoid any BC breaks. However, the default will be changed to `true` in a future release. Should the caller be interested in any messages which are suppressed when `true` is in effect, these are available through new method `getLibxmlMessages` after the Html has been loaded.
The tests stay; they just no longer run in separate processes. I think that only 2 tests still require separate processes - Helper/SampleTest and Tcpdf/NoDieTest. Aside from a new internal method in IOFactory, all the changes here are to test members.
Add to all readers the option to allow or forbid fetching external images. This is unconditionally allowed now. The default will be set to "allow", so no code changes are necessary. However, we are giving consideration to changing the default.
Fix#1104, which went stale over 6 years ago, and is now reopened. This PR addresses the alignment of the table, not alignment of text. Support is added for the following:
- Html. Full support.
- Mpdf. Full support, except that, when mixed LTR and RTL worksheets are output, all tables will be on the left of the page (but will be properly aligned).
- Tcpdf. Full support when all worksheets being output are RTL; no support when mixed LTR and RTL worksheets are output.
- Dompdf. No support.
For the second time in recent months, some tests are failing/erring because https file_get_contents is not working on github (cannot reproduce locally on Windows or Linux). Filed an issue with Php when this first happened, and their suggested code change worked till now. If they come up with another successful code change, I will implement it and restore these tests.
With PhpUnit 10 came the ability to use Php attributes rather than doc-block annotations for things like "data provider". PhpUnit 11 deprecates the use of annotations, and PhpUnit 12 will not not permit their use. Since PhpUnit 11 requires Php8.2+, we cannot adopt it as long as we support Php8.1, which will continue to be the case for some time. However, there is no penalty for early adoption.
Php-cs-fixer can use:
```
'php_unit_attributes' => ['keep_annotations' => false],
```
This allows us to run `composer fix` to automate all the needed changes. No manual changes were needed for any of the test members.
With this change, PhpUnit 9 can no longer be used with the test suite. File composer.json is updated to reflect that reality, and phpunit9.xml.dist, which has been supplied in case anyone needed to use PhpUnit 9, is no longer required, and is thus deleted. For now, PhpUnit 11 is not being added as a possibility.
No source code is changed in this PR.
Having addressed several security advisories, one evident *theoretical* problem remains. This is an attempt to future-proof our code against similar vulnerabilities. It all begins with our implementation of libXmlLoaderOptions, which uses as a default LIBXML_DTDLOAD. This unfortunate choice opens us to XXE problems, many recently solved. I do not believe that there is a legitimate use case for allowing this, and will therefore ignore and deprecate that option.
Although this might seem to be a breaking change, it is not. The setting is used only after the Xml has been subject to a security scan, and the security scan throws an exception if it detects the use of `<!DOCTYPE` within the Xml. Therefore, the setting will be effective only on Xml which does not contain that tag, and will consequently have no effect on most Xml. The only exception would be Xml which has been crafted to avoid detection by the security scanner in a manner which has not been disclosed to us. Although we hope that we've now blocked all such avenues, this provides additional protection just in case.
With this change in place, we could relax certain restrictions, e.g. the use of EBCDIC or even UTF-7. For now, these will remain in place. I will need to be convinced that there is a legitimate use case for easing the restrictions before doing so. We might even consider the elimination of the Security Scanner altogether. However, it does allow for early detection, and, in any case, provides a method to correct Xml which most Xml readers would fail but which Excel accepts.
My plan is to merge this within the next few days, and tag a new release immediately after. It will also be backported to all active branches.
Fix#1395, a 2020 issue which had been marked stale and is now re-opened. Static valueBinder property of Cell isn't ideal. It would be more flexible to make it a dynamic property of the spreadsheet. Static property will continue to be used, but dynamic property will be used first if it is set. Readers will also be changed to add a valueBinder property which they pass to the spreadsheet; however, it will make a difference only for Csv/Html/Slk, since the other readers use setValueExplicit which ignores valueBinder.
Documentation is updated in several places to note that dynamic property is preferred over static.
Fix#1284, which was closed as stale in 2019, but which I will now reopen. Html Reader converts *Unicode* whitespace characters in a DOM text node to space. However, Html treats only space, tab, CR, LF, vertical tab, and form-feed as whitespace. Using a regular expression with the `u` (Unicode) modifier causes a number of other characters to be converted to space inappropriately. The issue mentions "ideographic space" in particular, stating that it is used for formatting and should be preserved. "Non-breaking space" is also used in the same way and should also be preserved. An exception is made for a text node consisting of a single non-breaking space, since that is used as a placeholder by Html Writer; my own guess is that this is the reason why the Unicode modifier was used in the first place.
Fix#3995. Fix#866. Fix#1681. Php DOM loadhtml defaults to character set ISO-8859-1, but our data is UTF-8. So Html Reader alters its html so that loadhtml will not misinterpret characters outside the ASCII range. This works for UTF-8, but breaks other charsets. However, loadhtml uses the correct non-default charset when charset is specified in a meta tag, or when the html starts with a BOM. So, it is sufficient for us to alter the non-ASCII characters only when (a) the data does not start with a BOM, and (b) there is no charset tag.
This will allow us to use:
- UTF-8 files or snippets without BOM, with or without charset
- UTF-8 files with BOM (charset should not be specified and will be ignored if it is)
- UTF-16 files with BOM (charset should not be specified and will be ignored if it is)
- all charsets which are ASCII-compatible for 0x00-0x7f when the charset is declared. This applies to ASCII itself, many Windows and Mac charsets, all of ISO-8859, and most CJK and other-language-specific charsets.
We cannot use:
- UTF-16BE or UTF-16LE declared in a meta tag
- UTF-32, with or without a BOM (browser recommendation is to not support UTF-32, and most browsers do not support it)
- unknown (to loadhtml) or non-ASCII-compatible charsets (EBCDIC?)
I will note that the way I detect the `charset` attribute is imperfect (e.g. might find it in text rather than a meta tag). I think we'd need to write a browser to get it perfect. Anyhow, it is about the same as XmlScanner's attempt to find the `encoding` attribute, and, if it's good enough there, it ought to be good enough here.
* ListWorksheetInfo/Names for Html/Csv/Slk
Fix#3706. ListWorksheetInfo is implemented for all Readers except Html. For most (not all), ListWorksheetInfo is more efficient than reading the spreadsheet. I can't think of a way to make that so for Html, but that shouldn't be a reason to leave it unimplemented.
ListWorksheetNames is not implemented for Html, Csv, or Slk. It isn't terribly useful for those formats, but that isn't a reason to omit it. The requester's use case consists of using IOFactory to create a reader for a file of unknown format and determining the first sheet name. That seems legitimate, but it is currently not possible without extra user code if the file is Html, Csv, or Slk; this PR will make it possible.
When Excel opens a Slk or Csv file, the sheet name is based on the file name. PhpSpreadsheet does this for Slk, but it uses a default name for Csv. I am not interested in creating a break for that behavior, but I have added a new boolean property `sheetNameIsFileName` with a setter to Csv Reader. The requester actually mentioned that possibility in our discussion, although it is not essential to the request.
As an adjunct to the issue, the requester wishes to use the worksheet name in `setLoadSheetsOnly`. That is already possible for Html, Csv, and Slk, but that particular property is ignored for those formats. I do not see a reason to change that behavior. This treatment is now explicitly noted in the documentation for property `loadSheetsOnly`.
There had been no tests for what happens when `loadSheetsOnly` is specified but no sheets match the criteria for the formats for which this makes sense (Xlsx, Xls, Ods, Gnumeric, Xml). The behavior was not consistent - some formats threw an Exception while others continued with a single empty worksheet. All cases attempt to set the active sheet, and they will now all throw identical Exceptions when they attempt to do so in this situation. Tests are added for each.
There also had been no tests for `loadSheetsOnly` returning more than one sheet. One is added.
* Update LoadSheetsOnlyTest.php
Add strict types to this new test, consistent with work being done in PR #3718.
* Update LoadSheetsOnlyTest.php
Add strict types to this new test, consistent with work being done in PR #3718.
While we might never be able to have 100% of our code strict, we can at
the very least do it for all of our tests. This ensures that our tests
are using our API with the types as intended by the test author, and not
silently be cast to what our API requires.
And quite a bit more manual changes. The idea is that typing of our
tests can be a bit more loose, so we assume PHPDoc is mostly correct. If
that happens to be wrong, it should be caught by the tests themselves.
oleibman
Adrien Crivelli