PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the second of three PRs to replace that one. This accomodates the use of slash as a delimiter in column names in Tables and Structured References. The source changes are very simple. Additional tests exercise all the source changes.
There is also a preg_quote call when a table is renamed. I have also changed it to accomodate slash, because it's the right thing to do. But ... I can't think how to test it. PhpSpreadsheet will not allow you to set a table name to a string containing a slash (a test is added to confirm), and, if I manually update the Xml in an Xlsx spreadsheet so that the name does contain a slash, Excel will, understandably, complain that the file is corrupt.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the first of three PRs to replace that one. This accomodates the use of slash as a delimiter in functions TEXTAFTER, TEXTBEFORE, TEXTSPLIT, and NUMBERVALUE. The source changes are very simple. Additional tests exercise all the source changes.
* Redo Calculation of Color Tinting
Fix#3550. Some colors are specified in Excel by specifying a theme color to which a tint is applied. The original PHPExcel algorithm for doing this was developed by trial and error, and is good enough a lot of the time. However, for the issue at hand, the resulting color is detectably different from the calculation that Excel makes. Searching the web, I found https://gist.github.com/Mike-Honey/b36e651e9a7f1d2e1d60ce1c63b9b633 which comes much closer for the case in hand, and for all the other cases that I've looked at. That code depends on Python colorsys package; I have adapted the code from the Python gist and package into a new Php class. This doesn't agree perfectly with Excel. However, if each of the red, green, and blue components (each a value between 0 and 255 inclusive) agree within plus or minus 3 (arbitrary choice) of Excel's result, I think that is good enough.
I have added a new test member which reads from a spreadsheet with Xml altered by hand to set up several theme/tint cells. These tests use the plus-or-minus-3 criterion. They result in 100% code coverage of the new class.
Unsuprisingly, some existing tests failed with the new code. Issue2387Test reads a theme/tint font color, and is changed to use the plus-or-minus-3 criterion, comparing against the color as Excel shows it.
ColorChangeBrightness showed 9 failures with the new code. It consists of calculations not involving a spreadsheet. For that reason, I felt it was sufficient to just do an exact match test, changing the 9 old results for new results confirmed with the Python code. I also added one new test case, the one that kicked off this entire PR.
* Scrutinizer Being Stupid
It strikes again.
* Changes to NUMBERVALUE, VALUE, DATEVALUE, TIMEVALUE
Fix#3574. Reporter received deprecation notice for NUMBERVALUE function with invalid arguments. In fact, the arguments turn out to be valid after all; NUMBERVALUE treats a null-string or an all-blank-string in the first argument as if it were 0. Fixed this, and added several test cases suggested by it.
VALUE had been parsing its argument the same way as NUMBERVALUE. However, VALUE does not substitute 0 for null-string or all-blank-string. Coded up the difference between the two, and added the same tests for VALUE as for NUMBERVALUE.
VALUE can also pass its argument to DATEVALUE or TIMEVALUE. It is currently over-permissive about that, because Php is over-permissive, e.g. `new DateTime('q')` will return a DateTime object with the current date and time with a timezone of 'q'. Excel will, naturally, return `#VALUE!` for `DATEVALUE('q')`. I don't know that we can ever match Excel's (AFAIK not formally documented) decisions here 100%, but we can get a lot closer by parsing the date string if and only if it contains at least one digit. Code to enforce that is added to DATEVALUE and TIMEVALUE, and appropriate tests are added.
* Failure for Php 7.4 Linux
After not seeing any such problem for many months, this is the second day in a row where result is different on Windows than Linux with no apparent reason to think why that should be the case. At any rate, easily solved.
Zipstream's newest production release requires Php8.1. This is not necessarily a problem for us as we are locked to an earlier release. However, psr/simple-cache became a real mess for us as people wanted to install PhpSpreadsheet alongside other products which required a version of simple-cache that could not run in some of the versions of Php which we support, and I would like to avoid having to go through that again. This ticket is a contingency to put us ahead of the curve in case such a problem should arise with Zipstream. There is reason to believe that Zipstream is not so widely used as simple-cache, so we do not need to rush this change in. However, should that belief prove incorrect, the change should be ready to go when needed.
* Recognize 'Hidden' Attribute in Xml Spreadsheet
Fix#3566; PhpSpreadsheet should now handle Hidden attribute for rows and columns in Xml spreadsheets. Also added the ability to load Xml spreadsheet from string rather than file, as can be done for Csv and Html.
* Add Support for Top Left Cell
Another missing piece.
* Font Bold/Italic
Was always setting to true rather than checking value for 0/1.
* Active Sheet Index
Had always been set to hard-coded 0. Read it from XML if available.
* Selected Cells
Get Selected Cells from XML when available.
* Worksheet Protection and Style Protection
Add support for those to Xml Reader.
* Column Spans
Not really an essential part of Excel, used in Xml to reduce file size.
Fix#3561. PhpSpreadsheet VLOOKUP/HLOOKUP allow the specification of index_number (third parameter) as an array when lookup_value (first parameter) is supplied as an array. Change to permit index_number as an array even when lookup_value is not an array, as Excel does.
* Update Html.php
Add position attr for image whitch driving by MemoryDrawing
* add unit test for pr#3535
* Fix code styles
* Fix code styles
* use a local image instead of web image
* add image for pr#3535
* code style fix
Fix#3015. Fix#2766. Both issues requested a way to set Mpdf configuration parameters. I explained this could be done by extending class Mpdf and overriding protected function createExternalWriterInstance. So, no change is needed to PhpSpreadsheet. The parameters they wanted to override didn't seem interesting from a PhpSpreadsheet perspective. However, the configuration parameters include the ability to add fonts on the fly to those distributed by Mpdf, and that does sound useful. Normally, Mpdf (and Dompdf and Tcpdf) support a limited number of fonts, and will substitute for fonts they don't know (most fonts, even commonly used one, wind up using a DejaVu variant). Using these configuration options can lead to a more faithful Pdf representation of the spreadsheet.
In order to demonstrate this in action, I've added a new sample, a new class extending Mpdf, and a distinctive small font file from Google fonts with its (SIL Open Font) license. IANAL, but I can't see how this wouldn't be permitted use. There are no source code changes.
An alternate approach would be to add a callback to Writer/Pdf/Mpdf. It seems that this would be at least as much effort for the end-user as extending the class, and more effort for us.
It appears that Dompdf and Tcpdf need prep work to achieve the same end, and are thus less suitable for demonstrating via an example. I will continue to research.
Fix#3552. Inserting rows or columns can cause page break arrays to be out of order. Sort them in Worksheet functions getBreaks, getRowBreaks, and getColumnBreaks before returning to caller.
* Not Finding Rels File
Fix#3553. Read of `\xl\worksheets\_rels\sheet1.xml.rels` fails on the new test worksheet (generated by 3rd party product), but read of `xl\worksheets\_rels\sheet1.xml.rels` (same name without leading slash) succeeds. I'm not sure why (over 400 tests succeed with the slash), so will probably delay this a bit while I do more research.
* Move Fix
Move to more logical place.
* Changes for Chart Rendering Samples
PR #3521 added chart rendering to the 33* samples. Some of those samples had rendering problems; this PR fixes some of the errors by changing some of the code in JpGraphRendererBase. I am discussing approaches to some other problems with mitoteam.
The changes in 3521 applied only when running the samples as web pages. A new member 35_Chart_render33 is added to do the same when running the samples from the command line. Because of 3521, 35_Chart_render and new 35_Chart_render33 (in web page mode) display rendering which is already displayed by other samples and so are superfluous as web pages, so they are no longer presented as options.
The samples in 35_Chart_render properly accounted for multiple charts on a single worksheet, but did not properly account for charts on multiple worksheets. This problem is fixed.
33_Chart_create_stock2 was added using a base from before 3521, but did not incorporate the 3521 changes. It now does so.
32_Chart_read_write_HTML and 32_Chart_read_write_PDF are made consistent with the 32* Xlsx samples by allowing the download of the generated file when run as web pages. Helper/Downloader needed a minor change to support Html. Some of the changes in PR #3522 require the Downloader change to work properly. Some samples which were not changed by 3522 are changed in this PR to give a more uniform look and feel to the web samples.
* Scrutinizer
Fix class name conflict. Ignore "complexity" problem.
* Improve Sample 26
The UTF8 sample 26 includes CJK characters. These are generated in Xls, Xlsx, Html, and Csv (with BOM) correctly. Csv without BOM has a problem, but that's Microsoft's problem, not ours. That leaves PDF, which currently does not display the CJK characters (nor a Latin extension character). Mpdf, in combination with editHtmlCallback (and earlier commits in this PR), can support them; change the sample to do so. Dompdf, which had been used for the sample, apparently supported Web Fonts at one time, which might also have solved this problem, but no longer seems to do so.
* Fix Style Problem
Need blank line.
* Change Some Spacing in Source Code
Make Header PSR12-compliant.
* PhpUnit 10 Compatibility Part 3 (Last)
Final changes for PhpUnit 10, including enabling it for testing. This finishes the work of PR #3523 and PR #3526.
The major remaining problem with PhpUnit 10 is that earlier releases converted notices and warnings to exceptions, and 10 does not. Not having the information provided by the messages seems risky to me. Fortunately, it appears that you can add an error handler to the test bootstrap for 10 and make it act like earlier versions; I have done so. In order to demonstrate the effectiveness of this handler, a new otherwise unused class Helper/Handler and tests for that class are added.
As part of the testing of this change, it became apparent that the fopen in OLE::getStream attempts to create a dynamic property $context in Shared/OLE/ChainedBlockStream, and that action is deprecated in Php8.2. Adding the property to the class eliminates that problem. No executable code is added, and this is the only change to source code.
There also seems to have been a change in assertXmlStringEqualsXmlString in PhpUnit 10. The only test which uses that method is Chart/Issue589Test, and both the places which use that method could just as easily and effectively use assertSame. They are changed to do so.
* Remove Phpunit Verbose Option
Not supported in PhpUnit 10. I'm not at all sure that this is the correct solution for this problem.
* Try Changing Phpunit Command for Different Php Releases
Not sure how to test this locally. We'll see if it works on github.
* Another main.yml attempt
We shall see.
* Eliminate One Test
Not sure why testDeprecated is not working in Github; it works locally. Disable it for now and continue to research.
* Show Incomplete and Other Messages in PhpUnit 10
6 new command line args to replace the 1 they got rid of.
* Restore Disabled Test
Deprecated messages are suppressed by default setting for error_reporting. Switch that to E_ALL in bootstrap and restore original test.
* Add Deprecation Tests for PhpUnit 9-
Default configuration option caused deprecation messages to be suppressed. Change the option.
Successor to PR #3523. There are 494 single-line changes (`public function provider` to `public static function provider`) in this PR. None of these were made manually; they were all created with the following script (adapted from
https://stackoverflow.com/questions/25909820/how-to-recursively-iterate-through-files-in-php):
```php
$dir = 'C:/git/unit10prep2/tests/PhpSpreadsheetTests';
$it = new RecursiveDirectoryIterator($dir);
// Loop through files
foreach(new RecursiveIteratorIterator($it) as $file) {
if ($file->getExtension() === 'php') {
$contents = file_get_contents($file);
$new = preg_replace('/public function (\\w*)([Pp])rovider/', 'public static function $1$2rovider', $contents);
if ($new !== $contents) {
echo "changing $file\n";
file_put_contents($file, $new);
}
}
}
```
After this PR, there will be one more, with a small number of test changes, and enabling PhpUnit 10 for Php 8.1+.
* PhpUnit 10 Compatibility Part 1
This is not a change to move to PhpUnit 10. There is no compelling reason to do so at this time, although it is bound to happen eventually. There are a staggering number of problems (somewhere around 3,000) with the current test suite under PhpUnit 10; this is an attempt to get ahead of the curve by addressing them now.
Method `setOutputCallback` has gone away. This affects only Helper/SampleTest. It appears that `ob_start` and its allies provide an effective equivalent. FWIW, the absence of `setOutputCallback` is a good indication of whether or not PhpUnit 10 is in use, and I will use that fact in a few tests.
Class `ComplexAssert` with no constructor, and always used with `new ComplexAssert()`, extends `TestCase`. Apparently, the constructor for TestCase requires an argument, and PhpUnit 10 complains about not supplying one. Adding an empty constructor to ComplexAssert avoids this problem.
There are two very minor source changes, to Calculation/Calculation and Reader/Xlsx, where problems were exposed with PhpUnit 10 that had not been previously been exposed. AFAIK, these are the only source changes required; the rest of the changes are to test members.
The bulk of the problems are because PhpUnit 10 insists that provider methods be static. Most of those can be changed by a script without any further action; those changes will constitute the Part 2 counterpart of this PR. In this PR you will find the exceptional cases that can't be automated for one reason or another. The tests for Database functions have mild complications that are easily handled. Most of the other provider changes in this PR are because the method names didn't follow an established pattern ('provider' isn't part of the method name); those are also easily handled manually. Modifying the following tests provided significant challenges:
- Writer/Xls/WorkbookTest testAddColor
- Worksheet/Table/TableTest testSetRangeValidRange
The handling of warning messages issued by the code differs in PhpUnit 10. According to the change log, "This means that using PHP functionality which triggers E_DEPRECATED, E_NOTICE, E_STRICT, or E_WARNING or calling code which triggers E_USER_DEPRECATED, E_USER_NOTICE, or E_USER_WARNING can no longer hide a bug in your code." To me, the effect of that change seems to be exactly the opposite - such messages were available to the test with PhpUnit 9 (so we could test for them), and are no longer available (so we can't). I haven't even succeeded with a custom error message handler as part of the script. I will continue to investigate, but, for now, will skip some tests under PhpUnit 10 for the following:
- Shared/OleTest testChainedWriteMode and testChainedBadPath
- Reader/Html/HtmlLoadStringTest testLoadInvalidString
- Reader/Html/HtmlTest testBadHtml
* Scrutinize, and Parent Construct
Parent construct suggested by @MarkBaker.
* Redo Tests Dependent on Warning Messages
Warning (and other) messages are handled differently in PhpUnit 10 than in earlier versions.
Stock charts currently ignore upDownBars tag and its subsidiary gapWidth, upBars, and downBars tags when reading, and hard-codes those tags on write. As a result, the sample reproductions of stock charts in the 32* series aren't faithful to the originals. This PR fixes samples 1, 2, and 5. Samples 3 and 4 are reproduced better, but they require currently unsupported secondary axes (issue #560, issue #1072, and PR #1073 were closed as stale; see also https://github.com/PHPOffice/PHPExcel/pull/1037). I will start to look at those, but it could take a while, and I don't think there's a reason to delay this in the meantime.
Charts which depended on the hard-coded values written by the Xlsx Chart writer will be slightly different as a result of this change. To restore the hard-coded behavior:
```php
$plotArea->setGapWidth(300);
$plotArea->setUseUpBars(true);
$plotArea->setUseDownBars(true);
```
The new behavior is demonstrated in 33_Chart_create_stock. The old behavior is demonstrated (with the code above) in new 33_Chart_create_stock2.
Wanting a sanity test for ExactFontTest, I used the MD5 of a known file. However, there are various versions of the file in the wild, and use of these variants cause test to be skipped. Allow known variations to satisfy MD5 check.
* Add Ability to Ignore Cell Errors in Excel
Fix#1141, which had been closed as stale, but which I have reopened. Excel will show cells with certain "errors" with a green triangle in the upper left. The suggestion in the issue to use quotePrefix to suppress the numberStoredAsText error is ineffective. In Excel, the user can turn this indicator off for individual cells. Cells where this is turned off can be detected at read time, and PhpSpreadsheet will now process those. In addition, the user can explicitly set the ignored error as in Excel.
```php
$cell->setIgnoredErrorNumberStoredAsText(true);
```
There are a number of different errors that can be ignored in this fashion. This PR implements `numberStoredAsText` (which is likely to be by far the most useful one), `formula`, `twoDigitTextYear`, and `evalError`, all of which are demonstrated in the new test spreadsheet. There are several others for which I am not able to create good examples; I have not implemented those, but they can be easily added if needed (`calculatedColumn`, `emptyCellReference`, `formulaRange`, `listDataValidation`, and `unlockedFormula`).
* Scrutinizer
A new change, a new Scrutinizer false positive.
* Move Ignored Errors to Own Class
In response to comments from @MarkBaker, implement ignoredError as a new class. This simplifies Cell by requiring only 1 new method, rather than 8+. This requires a slightly more complicated syntax.
```php
$cell->getIgnoredErrors()->setNumberScoredAsText(true);
```
Mark had also suggested that there might be a pre-existing regexp for processing the cells/cellranges when reading the sqref attribute. Those in Calculation are too complicated (read "non-performant") for this piece of code; the one in Coordinates is slightly less complicated than Calculation, but still more complicated than the one I'm using, and doesn't handle ranges.
Fix#3495. This seems to be a bug in Excel, one which it manages to cover up but PhpSpreadsheet is affected. User enters a formula preceded by an apostrophe into a cell. Excel turns on `quotePrefix` style and stores the data as a string rather than a formula. User now enters a formula not preceded by an apostrophe into the same cell. Excel stores it is a formula but does not turn `quotePrefix` off. When the spreadsheet is saved, the cell's style specifies `quotePrefix`, but the cell's content indicates it's a formula. Till now, PhpSpreadsheet sees that quotePrefix is set, and therefore treats the cell's contents as a string rather than a formula. This PR will change that behavior so that quotePrefix is automatically turned off when Xlsx Reader sees that the cell indicates that it is a formula.
oleibman
dubox
dependabot[bot]
Mikhail Oleynik
MarkBaker