Its defaults are to unescape within single quotes, and escape within double quotes and here-docs. Right now, it leaves everything as-is, which means our code is inconsistent and need not be. Further, although dealing with complex regular expressions will never be easy, I find it much easier to figure out what's going on when superfluous back-slashes are removed.
These changes were all made automatically using the "fix" script, so should be reliable. They, of course, pass all unit tests.
Fix#4092. Change default value for Csv Reader autodetect line endings. Prior behavior can be enabled via `setTestAutodetect(true)`.
Change default value for Html Writer "better boolean" logic. Prior behavior can be explicitly enabled via `setBetterBoolean(false)`.
Change default for Xlsx Writer forceFullCalc option. Prior behavior can be explicitly enabled via `setForceFullCalc(null)`.
Fix#4319. PR #3495 allowed Mpdf and Tcpdf to handle hidden rows and columns. The handling is a little off when merged cells are part of the output. The fix for Mpdf is pretty straightforward. Tcpdf is a little more convoluted. Dompdf and Html do not require a change. Unit tests are added for the same scenario for all of Mpdf, Tcpdf, Dompdf, and Html.
When Html Writer outputs a cell with a boolean value, the result will either be 1 or null-string; neither of these is optimal for anyone looking at the resulting html. Html Reader already has the ability to recognize data types using the html `data-type` attribute, but Html Writer doesn't use it. This PR adds the ability to generate that attribute for booleans. It will generate a string value appropriate for the locale when it encounters a boolean. Html Reader, when it encounters `data-type="b"`, will interpret the result as true if the value is 1 or a string value recognized as true in any locale; it will interpret the result as false if the value is 0, null-string, null, or a string value recognized as false in any locale; if none of the above, it will leave the value as an unchanged string. So, Reader will wind up with the correct result even if its locale is different than what Writer used.
Because this is a breaking change, it is opt-in. You need to call `Writer::setBetterBoolean(true)` in order for it take effect. The current default value for that property is false. When it is time to introduce breaking changes (see PR #4240), the default will be changed to true.
With PhpUnit 10 came the ability to use Php attributes rather than doc-block annotations for things like "data provider". PhpUnit 11 deprecates the use of annotations, and PhpUnit 12 will not not permit their use. Since PhpUnit 11 requires Php8.2+, we cannot adopt it as long as we support Php8.1, which will continue to be the case for some time. However, there is no penalty for early adoption.
Php-cs-fixer can use:
```
'php_unit_attributes' => ['keep_annotations' => false],
```
This allows us to run `composer fix` to automate all the needed changes. No manual changes were needed for any of the test members.
With this change, PhpUnit 9 can no longer be used with the test suite. File composer.json is updated to reflect that reality, and phpunit9.xml.dist, which has been supplied in case anyone needed to use PhpUnit 9, is no longer required, and is thus deleted. For now, PhpUnit 11 is not being added as a possibility.
No source code is changed in this PR.
* Validate Mime Type for Images
* Extend Change to Xlsx Reader
* Attach Drawing to Sheet Only If Valid
* Suppress Some Theoretical Warning Messages
* Minor Tweak
See discussion #4117. This PR implements image transparency, for Xlsx Reader and Writer, Html Reader, and Html/Dompdf/Mpdf Writer. (Mpdf treats 100% opacity as if it were zero, but it's otherwise okay and that would be an unusual choice anyhow.) A new property `opacity` with getter/setter is added to BaseDrawing (and therefore Drawing and MemoryDrawing). Although the Excel UI lets you set the image *transparency*, the value stored in the Xml is actually its *opacity* (expressed as an integer between 0 and 100,000). Likewise Html/Css lets you set opacity (as a float between 0 and 1). PhpSpreadsheet lets you set it as it is stored in Xml - so a value of 40,000 would indicate 40% opacity = 60% transparency.
In the course of testing, some problems with Html Reader presented themselves. They are corrected as part of this ticket:
- Web page title will no longer be "Untitled Spreadsheet" if a title tag is present in the header section of the html.
- If a class attribute is used with the table tag representing a worksheet, showGridlines and printGridlines will be set according to whether gridlines and/or gridlinesp are among the class names.
- Reader has been performing urldecode on the src attribute of the img tag. This breaks data url's by converting plus signs to spaces. It will now perform urldecode only for non-data url's.
- If height and width are not used as attributes on the img tag, they will be extracted from the style attribute if they are present there.
Till now we have used a static variable/getter/setter to decide what type of result should be returned when a formula is evaluated and an array is the result. This is messy; it would be much better to use an instance variable instead. We cannot eliminate `setArrayReturnType` and `getArrayReturnType` because that would be a BC break. I am considering whether they should be deprecated. In the meantime, I have added a new instance property `instanceArrayReturnType` with getter and setter methods. The property is initially null, and, if it remains so when needed, the static property will be used instead. However, if it is set, its value will be used.
While researching issue #1551, I came across some minor problems.
When a spreadsheet does not have a title, which is often the case for spreadsheets created with Excel (note that this is not the case for spreadsheets created with PhpSpreadsheet), if you try to save it as Html, it throws an exception. It will now use the sheet title of the active sheet as a title in this case.
When writing an Html spreadsheet using `useInlineCss(true)`, gridlines are not handled properly. This is addressed by adding `class=gridlines gridlinesp` to the cell's `td` tag, and by suppressing any border attributes which would be styled as `none #000000`. It would be unusual to turn off gridlines for specific cells, but that can still be accomplished by using `Border::BORDER_NONE` in conjunction with any color other than `#000000` - see new test `testHideSomeGridlines`.
The CONCATENATE function has been treated as equivalent to CONCAT. This is not how it is treated in Excel; it is closer to (and probably identical to) the ampersand concatenate operator. The difference manifests itself when any of the arguments is an array (typically a cell range). Code is added to support this difference.
Support for array results is added to Csv Writer, Html Writer, and Ods Reader and Writer. I have not figured out how to get it to work with Xls.
Following up from PR #4006. There is an additional RTL property available. It controls the placement of bidirectional neutral characters (mainly punctuation), as opposed to strong (alphabetic characters) or weak (numeric characters), especially at the beginning or end of a line. The new Comment property textboxDirection will be used for that purpose.
In a discussion in issue #4004 following the implementation of the PR, the comment was mixed RTL and LTR, and this led to some formatting problems. The user was able to overcome these with the timely insertion of Unicode directional control characters, but it would be preferable to have it happen automatically, which this change will permit. However, the use of these control characters cannot be entirely done away with. In the new test case, if one of the all-English lines ended with, say, a colon, it would not display correctly; LRM (left-to-right mark) after the colon would be needed. Likewise, one or two of the comment lines with mixed RTL and LTR (discussed in the issue) is not formatted correctly, and might require LRO/PDF or equivalent.
Fix#3954. In response to a Security Incident, a package was added to the project to sanitize the Html for comments attached to a cell. We have tried at least 2 different packages for this purpose, and users have raised legitimate concerns about both.
I believe that adding a sanitizer package, although it addresses the problem, was overkill. Cell comments are a RichText object, and Html Writer already handled RichText cell *values*. Values did not figure in the Incident, and they surely would have done so if they were a problem because it is a lot easier to set up test cases for values than for comments. RichText values were not a problem because they were sanitized with `htmlspecialchars`; if comments were to use the same code that RichText was already using, it would likewise be safely sanitized. As an added bonus, the existing code for comments only uses the plaintext value, but the values code would allow the comments to be styled, just as they are for Xlsx.
This is a breaking change - I don't think it will affect a lot of users, but there may be some. It is worth noting that the comment block before function `writeComment` has a link explaining what is being done. That link mentions only styling elements, not other possibilities like hyperlinks. At any rate, if people are putting styling tags (e.g. `<b>`) in their comments for this purpose, those will no longer work; the styling needs to be applied to the RichText elements. That will keep the user code the same regardless of the format of the intended output, which is certainly a good thing, but it is a break. If people are trying to insert non-styling tage (e.g. `<a>`), those will no longer work, just as there is no way to do it for Xlsx (although Excel itself may convert the raw text to a hyperlink, but that's out of scope, at least for now). I also note that, even with the current code, I haven't yet found a way to keep the Html comment in place long enough to actually click on any hyperlinks.
The main package being dropped brings several other packages along with it, so the project as a whole will have a slightly lighter footprint than before.
The existing `XssVulnerability` test cases are all preserved. Although the final result of the sanitizing changes, it can easily be seen that the results are all harmless.
Tcpdf issues warnings when processing hidden rows. Some time ago, Mpdf was having problems in the same situation; this was resolved by not writing out hidden rows in the html which Mpdf uses to generate its file. The same solution can be easily applied to Tcpdf.
Neither Tcpdf nor Mpdf handles hidden columns. Dompdf doesn't have a problem with either rows or columns. At any rate, the solution for Tcpdf/Mpdf is to not write out the data in the hidden columns. It is more difficult to implement this for columns than for rows, but this PR should do the trick.
Html writer generally uses `display:none` for the data in suppressed columns. This works, but is technically incorrect - the "approved" method is `visibility:collapse` on the col element. However, Firefox doesn't handle that correctly (open bug was filed over a decade ago), and, since all browsers seem to handle the existing implementation, it is left alone.
Among other considerations, this PR is a necessary precursor for supporting printArea in Html/Pdf should we decide to do that (issue #3941).
Writer/Html protected property `$isMPdf` is deprecated with this PR in favor of testing for `instanceof Mpdf`.
Change "mixed" declarations to more accurate types in test members; in particular, change those that would be flagged if we were to run Phpstan at level 9 (we currently run level 8). I may or may not follow up with source code (over 700 level-9 problems remain for src), but, as with strict typing, there is no reason to avoid the effort for test members.
It was necessary to update some doc blocks in src to accommodate this change. However, no executable code is touched.
There are different rules for Title between Excel and Html, in particular maximum length and which characters are valid. Html Reader can throw an exception because of this difference. This PR allows it to tolerate an invalid title, retaining the default title if the Html suggests an invalid one.
This was suggested by the investigation of issue #3811. No fix is necessary for the issue. However, two possible code solutions (Php setlocale, which comes with certain design flaws, and StringHelper set(Decimal/Thousands)Separator were suggested, and neither is adequately tested. This PR adds such tests.
Unusually, getting StringHelper Decimal Separator, Thousands Separator, and Currency Code can result in a change to those properties. So, the existing design in several tests where those properties are captured in Setup and restored in Teardown do not work quite as designed. Instead, the ability to set those properties to their default value (null) is added, and the tests re-done to restore the default in Teardown.
The two methods yield the same results when parsing input. However, they diverge when examining output fields through `getFormattedValue`. Such output is currently correct (usually) when using setlocale, but not when using StringHelper. The former works through the 'trick' of using `sprintf(%f)`, which generates a locale-aware string. However, using non-locale-aware `sprintf(%F)` followed by `str_replace` will produce the correct result for both setlocale and StringHelper. One place in the code uses a cast to string, which is incorrect for both methods. Following that up with the same str_replace makes it correct for both. These changes permit, but do not require, the user to avoid setlocale altogether.
It remains an open question whether Settings/Calculation::setLocale should set DecimalSeparator, CurrencySeparator, and CurrencyCode. That makes logical sense, but it would be a breaking change, and having to explicitly set those values when using setLocale does not seem especially burdensome. For now, such a change will not be made.
* Sheet Background Images
Fix#1649, a 3-year-old issue long marked "stale". Excel supports background images on sheets; now PhpSpreadsheet will as well. Support is limited to Xlsx (read and write) and Html (write only). As far as I can tell, Excel Xml and Gnumeric do not support this, nor, of course, do Csv and Slk; Excel Xls does, but, as usual, how to handle it in BIFF format is a mystery; LibreOffice ODS supports it differently than Excel, and this is just another of many ODS style properties not currently supported by PhpSpreadsheet.
* Update CHANGELOG.md
Because they tend to clutter our code a lot and unfortunately, there are
lots of false positives. Instead, it would probably be better to deal
with false positives out of band, via the Scrutinizer web UI.
While we might never be able to have 100% of our code strict, we can at
the very least do it for all of our tests. This ensures that our tests
are using our API with the types as intended by the test author, and not
silently be cast to what our API requires.
oleibman
Mikko
Adrien Crivelli
dmytromikhieiev1985