PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the first of three PRs to replace that one. This PR also allows the use of slash as a thousands separator or decimal separator or currency symbol. These are, of course, very unusual situations; the main reason to support them is so that PhpSpreadsheet code will not crash when users set those options. New Calculation/Engine and AdvancedValueBinder tests confirm these. While making these changes, a few errors were found in AdvancedValueBinder and Calculation/Engine/FormattedNumber, e.g. currencies weren't parsed correctly when period was used as the group separator and comma as the decimal separator. Tests have been added for those situations.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the second of three PRs to replace that one. This accomodates the use of slash as a delimiter in column names in Tables and Structured References. The source changes are very simple. Additional tests exercise all the source changes.
There is also a preg_quote call when a table is renamed. I have also changed it to accomodate slash, because it's the right thing to do. But ... I can't think how to test it. PhpSpreadsheet will not allow you to set a table name to a string containing a slash (a test is added to confirm), and, if I manually update the Xml in an Xlsx spreadsheet so that the name does contain a slash, Excel will, understandably, complain that the file is corrupt.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the first of three PRs to replace that one. This accomodates the use of slash as a delimiter in functions TEXTAFTER, TEXTBEFORE, TEXTSPLIT, and NUMBERVALUE. The source changes are very simple. Additional tests exercise all the source changes.
* Redo Calculation of Color Tinting
Fix#3550. Some colors are specified in Excel by specifying a theme color to which a tint is applied. The original PHPExcel algorithm for doing this was developed by trial and error, and is good enough a lot of the time. However, for the issue at hand, the resulting color is detectably different from the calculation that Excel makes. Searching the web, I found https://gist.github.com/Mike-Honey/b36e651e9a7f1d2e1d60ce1c63b9b633 which comes much closer for the case in hand, and for all the other cases that I've looked at. That code depends on Python colorsys package; I have adapted the code from the Python gist and package into a new Php class. This doesn't agree perfectly with Excel. However, if each of the red, green, and blue components (each a value between 0 and 255 inclusive) agree within plus or minus 3 (arbitrary choice) of Excel's result, I think that is good enough.
I have added a new test member which reads from a spreadsheet with Xml altered by hand to set up several theme/tint cells. These tests use the plus-or-minus-3 criterion. They result in 100% code coverage of the new class.
Unsuprisingly, some existing tests failed with the new code. Issue2387Test reads a theme/tint font color, and is changed to use the plus-or-minus-3 criterion, comparing against the color as Excel shows it.
ColorChangeBrightness showed 9 failures with the new code. It consists of calculations not involving a spreadsheet. For that reason, I felt it was sufficient to just do an exact match test, changing the 9 old results for new results confirmed with the Python code. I also added one new test case, the one that kicked off this entire PR.
* Scrutinizer Being Stupid
It strikes again.
* Changes to NUMBERVALUE, VALUE, DATEVALUE, TIMEVALUE
Fix#3574. Reporter received deprecation notice for NUMBERVALUE function with invalid arguments. In fact, the arguments turn out to be valid after all; NUMBERVALUE treats a null-string or an all-blank-string in the first argument as if it were 0. Fixed this, and added several test cases suggested by it.
VALUE had been parsing its argument the same way as NUMBERVALUE. However, VALUE does not substitute 0 for null-string or all-blank-string. Coded up the difference between the two, and added the same tests for VALUE as for NUMBERVALUE.
VALUE can also pass its argument to DATEVALUE or TIMEVALUE. It is currently over-permissive about that, because Php is over-permissive, e.g. `new DateTime('q')` will return a DateTime object with the current date and time with a timezone of 'q'. Excel will, naturally, return `#VALUE!` for `DATEVALUE('q')`. I don't know that we can ever match Excel's (AFAIK not formally documented) decisions here 100%, but we can get a lot closer by parsing the date string if and only if it contains at least one digit. Code to enforce that is added to DATEVALUE and TIMEVALUE, and appropriate tests are added.
* Failure for Php 7.4 Linux
After not seeing any such problem for many months, this is the second day in a row where result is different on Windows than Linux with no apparent reason to think why that should be the case. At any rate, easily solved.
* Recognize 'Hidden' Attribute in Xml Spreadsheet
Fix#3566; PhpSpreadsheet should now handle Hidden attribute for rows and columns in Xml spreadsheets. Also added the ability to load Xml spreadsheet from string rather than file, as can be done for Csv and Html.
* Add Support for Top Left Cell
Another missing piece.
* Font Bold/Italic
Was always setting to true rather than checking value for 0/1.
* Active Sheet Index
Had always been set to hard-coded 0. Read it from XML if available.
* Selected Cells
Get Selected Cells from XML when available.
* Worksheet Protection and Style Protection
Add support for those to Xml Reader.
* Column Spans
Not really an essential part of Excel, used in Xml to reduce file size.
Fix#3561. PhpSpreadsheet VLOOKUP/HLOOKUP allow the specification of index_number (third parameter) as an array when lookup_value (first parameter) is supplied as an array. Change to permit index_number as an array even when lookup_value is not an array, as Excel does.
* Update Html.php
Add position attr for image whitch driving by MemoryDrawing
* add unit test for pr#3535
* Fix code styles
* Fix code styles
* use a local image instead of web image
* add image for pr#3535
* code style fix
Fix#3552. Inserting rows or columns can cause page break arrays to be out of order. Sort them in Worksheet functions getBreaks, getRowBreaks, and getColumnBreaks before returning to caller.
* Not Finding Rels File
Fix#3553. Read of `\xl\worksheets\_rels\sheet1.xml.rels` fails on the new test worksheet (generated by 3rd party product), but read of `xl\worksheets\_rels\sheet1.xml.rels` (same name without leading slash) succeeds. I'm not sure why (over 400 tests succeed with the slash), so will probably delay this a bit while I do more research.
* Move Fix
Move to more logical place.
* PhpUnit 10 Compatibility Part 3 (Last)
Final changes for PhpUnit 10, including enabling it for testing. This finishes the work of PR #3523 and PR #3526.
The major remaining problem with PhpUnit 10 is that earlier releases converted notices and warnings to exceptions, and 10 does not. Not having the information provided by the messages seems risky to me. Fortunately, it appears that you can add an error handler to the test bootstrap for 10 and make it act like earlier versions; I have done so. In order to demonstrate the effectiveness of this handler, a new otherwise unused class Helper/Handler and tests for that class are added.
As part of the testing of this change, it became apparent that the fopen in OLE::getStream attempts to create a dynamic property $context in Shared/OLE/ChainedBlockStream, and that action is deprecated in Php8.2. Adding the property to the class eliminates that problem. No executable code is added, and this is the only change to source code.
There also seems to have been a change in assertXmlStringEqualsXmlString in PhpUnit 10. The only test which uses that method is Chart/Issue589Test, and both the places which use that method could just as easily and effectively use assertSame. They are changed to do so.
* Remove Phpunit Verbose Option
Not supported in PhpUnit 10. I'm not at all sure that this is the correct solution for this problem.
* Try Changing Phpunit Command for Different Php Releases
Not sure how to test this locally. We'll see if it works on github.
* Another main.yml attempt
We shall see.
* Eliminate One Test
Not sure why testDeprecated is not working in Github; it works locally. Disable it for now and continue to research.
* Show Incomplete and Other Messages in PhpUnit 10
6 new command line args to replace the 1 they got rid of.
* Restore Disabled Test
Deprecated messages are suppressed by default setting for error_reporting. Switch that to E_ALL in bootstrap and restore original test.
* Add Deprecation Tests for PhpUnit 9-
Default configuration option caused deprecation messages to be suppressed. Change the option.
Successor to PR #3523. There are 494 single-line changes (`public function provider` to `public static function provider`) in this PR. None of these were made manually; they were all created with the following script (adapted from
https://stackoverflow.com/questions/25909820/how-to-recursively-iterate-through-files-in-php):
```php
$dir = 'C:/git/unit10prep2/tests/PhpSpreadsheetTests';
$it = new RecursiveDirectoryIterator($dir);
// Loop through files
foreach(new RecursiveIteratorIterator($it) as $file) {
if ($file->getExtension() === 'php') {
$contents = file_get_contents($file);
$new = preg_replace('/public function (\\w*)([Pp])rovider/', 'public static function $1$2rovider', $contents);
if ($new !== $contents) {
echo "changing $file\n";
file_put_contents($file, $new);
}
}
}
```
After this PR, there will be one more, with a small number of test changes, and enabling PhpUnit 10 for Php 8.1+.
* PhpUnit 10 Compatibility Part 1
This is not a change to move to PhpUnit 10. There is no compelling reason to do so at this time, although it is bound to happen eventually. There are a staggering number of problems (somewhere around 3,000) with the current test suite under PhpUnit 10; this is an attempt to get ahead of the curve by addressing them now.
Method `setOutputCallback` has gone away. This affects only Helper/SampleTest. It appears that `ob_start` and its allies provide an effective equivalent. FWIW, the absence of `setOutputCallback` is a good indication of whether or not PhpUnit 10 is in use, and I will use that fact in a few tests.
Class `ComplexAssert` with no constructor, and always used with `new ComplexAssert()`, extends `TestCase`. Apparently, the constructor for TestCase requires an argument, and PhpUnit 10 complains about not supplying one. Adding an empty constructor to ComplexAssert avoids this problem.
There are two very minor source changes, to Calculation/Calculation and Reader/Xlsx, where problems were exposed with PhpUnit 10 that had not been previously been exposed. AFAIK, these are the only source changes required; the rest of the changes are to test members.
The bulk of the problems are because PhpUnit 10 insists that provider methods be static. Most of those can be changed by a script without any further action; those changes will constitute the Part 2 counterpart of this PR. In this PR you will find the exceptional cases that can't be automated for one reason or another. The tests for Database functions have mild complications that are easily handled. Most of the other provider changes in this PR are because the method names didn't follow an established pattern ('provider' isn't part of the method name); those are also easily handled manually. Modifying the following tests provided significant challenges:
- Writer/Xls/WorkbookTest testAddColor
- Worksheet/Table/TableTest testSetRangeValidRange
The handling of warning messages issued by the code differs in PhpUnit 10. According to the change log, "This means that using PHP functionality which triggers E_DEPRECATED, E_NOTICE, E_STRICT, or E_WARNING or calling code which triggers E_USER_DEPRECATED, E_USER_NOTICE, or E_USER_WARNING can no longer hide a bug in your code." To me, the effect of that change seems to be exactly the opposite - such messages were available to the test with PhpUnit 9 (so we could test for them), and are no longer available (so we can't). I haven't even succeeded with a custom error message handler as part of the script. I will continue to investigate, but, for now, will skip some tests under PhpUnit 10 for the following:
- Shared/OleTest testChainedWriteMode and testChainedBadPath
- Reader/Html/HtmlLoadStringTest testLoadInvalidString
- Reader/Html/HtmlTest testBadHtml
* Scrutinize, and Parent Construct
Parent construct suggested by @MarkBaker.
* Redo Tests Dependent on Warning Messages
Warning (and other) messages are handled differently in PhpUnit 10 than in earlier versions.
Wanting a sanity test for ExactFontTest, I used the MD5 of a known file. However, there are various versions of the file in the wild, and use of these variants cause test to be skipped. Allow known variations to satisfy MD5 check.
* Add Ability to Ignore Cell Errors in Excel
Fix#1141, which had been closed as stale, but which I have reopened. Excel will show cells with certain "errors" with a green triangle in the upper left. The suggestion in the issue to use quotePrefix to suppress the numberStoredAsText error is ineffective. In Excel, the user can turn this indicator off for individual cells. Cells where this is turned off can be detected at read time, and PhpSpreadsheet will now process those. In addition, the user can explicitly set the ignored error as in Excel.
```php
$cell->setIgnoredErrorNumberStoredAsText(true);
```
There are a number of different errors that can be ignored in this fashion. This PR implements `numberStoredAsText` (which is likely to be by far the most useful one), `formula`, `twoDigitTextYear`, and `evalError`, all of which are demonstrated in the new test spreadsheet. There are several others for which I am not able to create good examples; I have not implemented those, but they can be easily added if needed (`calculatedColumn`, `emptyCellReference`, `formulaRange`, `listDataValidation`, and `unlockedFormula`).
* Scrutinizer
A new change, a new Scrutinizer false positive.
* Move Ignored Errors to Own Class
In response to comments from @MarkBaker, implement ignoredError as a new class. This simplifies Cell by requiring only 1 new method, rather than 8+. This requires a slightly more complicated syntax.
```php
$cell->getIgnoredErrors()->setNumberScoredAsText(true);
```
Mark had also suggested that there might be a pre-existing regexp for processing the cells/cellranges when reading the sqref attribute. Those in Calculation are too complicated (read "non-performant") for this piece of code; the one in Coordinates is slightly less complicated than Calculation, but still more complicated than the one I'm using, and doesn't handle ranges.
Fix#3495. This seems to be a bug in Excel, one which it manages to cover up but PhpSpreadsheet is affected. User enters a formula preceded by an apostrophe into a cell. Excel turns on `quotePrefix` style and stores the data as a string rather than a formula. User now enters a formula not preceded by an apostrophe into the same cell. Excel stores it is a formula but does not turn `quotePrefix` off. When the spreadsheet is saved, the cell's style specifies `quotePrefix`, but the cell's content indicates it's a formula. Till now, PhpSpreadsheet sees that quotePrefix is set, and therefore treats the cell's contents as a string rather than a formula. This PR will change that behavior so that quotePrefix is automatically turned off when Xlsx Reader sees that the cell indicates that it is a formula.
* Font Themes
It isn't a feature of Excel that I've made any use of, but PR #3476 added better Theme support for colors, and it is relatively easy to add Theme support for Fonts on top of that.
Excel assigns two theme fonts to its spreadsheets, one for Headings (major), and one for Body (minor). If the body theme is Calibri, when you choose a font for a cell in Excel, you can choose 'Calibri (Body)' from the Theme Fonts section at the top of the Font dropdown, or 'Calibri' from the 'All Fonts' section. If you choose the former, the cell will be automatically restyled if you change the Theme Fonts (via Page Layout, Themes, Fonts). The relationship to the theme fonts is recorded in the XML via a `scheme` tag (descending from `font`) whose `val` attribute can be either `major` or `minor`. Accordingly, this PR, in addition to defining the Theme Font properties, adds a `scheme` property, with getter and setter, to Style/Font.
The main benefit of this PR is that you can now load and save a spreadsheet preserving the connections to the Theme Fonts, without having to take any additional action.
A secondary benefit arises from the following difference. Empty cells in Excel will use the spreadsheet's default font name when they are filled in; but, in Google Sheets, they will use the Theme Minor Font name. By setting the `scheme` property in the default style, the resulting spreadsheet will behave the same in both Excel and Google.
I will note that Excel's font themes specify a Latin font, an East Asian font, a Complex Scripts font, and a set of font substitutions for various languages. PhpSpreadsheet will preserve all of these, and allow them to be changed. However, although it is easy to imagine how the non-Latin options might work, I have not yet been able to come up with an example where Excel uses any of them. In particular, if I use a theme font which does not support language X, and I use Language X in a cell bound to the theme, Excel will use a substitution font which does support it, but the font which it uses does not seem to be chosen from the alternatives supplied in the theme.
* Scrutinizer
The sun rises in the east, and Scrutinizer issues more false positives.
* Font and Effects Support for Chart Data Labels and Axis
Addresses some remaining issues with 32readwriteLineChart5 (see issue #1797). Font size is covered. So are effects, although the results are a bit odd. For the new spreadsheet 32readwriteLineChart6, the Axis labels have a yellow-ish glow, but reading and writing the spreadsheet in PhpSpreadsheet gives them a purple-ish glow. Nevertheless, the new test shows that the output file uses schemeClr accent4, as does the input file. So the effect is handled correctly, but it seems there is likely to be a difference between theme colors (Writer/Xlsx/Theme appears to write hard-coded color schemes, and, in any case, Reader/Xlsx does not appear to handle schemeClr). Fixing that will be a great deal more difficult, with a large chance of regression, and will need to happen in a separate PR (one that I am not currently investigating, but I will open a new issue). Effects using srgbClr (and probably sysclr) should be okay.
* Better Theme Support
When reading Xlsx, the theme colors will now also be used for writing. This means that a file can be loaded and saved and its chart colors will now be preserved. If the spreadsheet is created new, Excel 2007-2010 colors are used. The writer is currently hard-coded to use them, so this avoids making this a breaking change. The theme colors can be explicitly changed if desired, and Excel 2013+ colors can be introduced very easily.
```php
$spreadsheet->getTheme()
->setThemeColorName(Theme::COLOR_SCHEME_2013_PLUS_NAME);
```
Likewise, if the old behavior of changing to the 2007-2010 scheme rather than using the input values is desired, that is easy to achieve after the load has taken place.
```php
$spreadsheet->getTheme()
->setThemeColorName(Theme::COLOR_SCHEME_2007_2010_NAME);
```
The new Theme class introduced by this change can easily be extended to include Fonts and Effects. Unlike Colors, I am unsure what the practical effects of changing those to, say, the 2013+ defaults would be.
* Scrutinizer
Use an alias in a use statement.
* Update Change Log
Due to potential behavior change.
Fix#3453. User sets a valid formula (e.g. `=SUM(Sheet2!B1:Sheet2!B3)`), and then does something to invalidate the formula (e.g. delete Sheet2). Excel changes the formula to `SUM(#REF!:#REF!)` when the spreadsheet is saved; apparently someone thought this was a good idea. But PhpSpreadsheet (a) used to throw an Exception when it evaluated the formula, and (b) now gives a result of `0` when evaluating the formula. Neither is ideal. It would be better to propagate the `#REF!` error.
It is likely that more tests are needed, which is why I will keep this in draft status for a bit.
* Support Border for Charts
All chart linestyles will be supported for Chart Border. Also add fill color for Chart. Also 'nofill' for Axis (allows suppressing of vertical axis line). These are demonstrated in sample 32_readwriteChartLine5, and a new unit test member is added. This addresses some vague problems added to issue #1797 over 8 months after it was closed; there is still at least one problem, much more complicated than the 2 being addressed in this PR.
* Handle Legend Borders in Same Way as Chart Borders
Consistency is good, and the implementation of Chart Borders offers more possibilities than the implementation of Legend Borders had. Since Legend Borders hasn't made it to a release yet, there is no need for deprecations.
* Xlsx Writer Honor Alignment in Default Font
Fix#3443. A mysterious implementation by Excel. The style tags have an attribute applyAlignment, which nominally says whether or not the style should use its own Alignment. Except ... Excel ignores that attribute and uses the alignment tag if it is supplied ... and, another mystery, uses not the default style for the spreadsheet if not supplied, but rather uses the default alignment style for all spreadsheets even if the spreadsheet's default style uses a non-default alignment. I am changing Xlsx Writer to generate alignment tag unless the alignment matches both the default alignment for the spreadsheet and the default alignment for all spreadsheets (which I expect to happen most of the time).
* Improve Performance
Also don't change horizontal/vertical on Xlsx Read if they aren't explicitly set.
* Specify data type in html tags using attributes #3444
* Set data types using attributes in flushCell function and add some unit tests
* Check TYPE_INLINE in flushCell and some changes in test cases
* More Coverage In Unit Tests
Minimal source code changes.
* Scrutinizer
One legitimate complaint and one from out of left field.
* Scrutinizer Lunacy
If this doesn't stop its complaint, I give up.
* Scrutinizer - Whatever
Try again.
* Glutton For Punishment
Try again.
No source changes. Act on some items that have come up in recent discussions.
- Sample 33_Chart_create_line creates a stacked line chart. According to @MarkBaker, the stacking is done on the wrong variable, and, even were that not the case, stacking is unusual for line charts. Since this is our primary sample showing how to create a line chart, remove the stacking. Another sample, with a more appropriate choice of chart (33_Chart_create_bar_stacked), still shows how to create a stacked chart.
- The test for reading a styled cell from Html is flawed. It sets a date format for a string date/time, but the format is applied only to numeric data, so the format, although set correctly, is ineffective. Keep the test, but add some explanation in the assertion, and add some new more effective tests, also with explanations in the assertions.
- Wrong namespace used for Writer/Xlsx/ConditionalFillTest.
Stop a few of scrutinizers complaints about 100% valid use of PHP variadics.
Hopefully, this issue will cease to be an issue when we can specify mixed datatype for variadic arguments in the tests
oleibman
dubox
Mark Baker
anhoder
Pouria Seyfi