Fix#642. Opened over 5 years ago, probably the oldest problem I've worked on. And https://github.com/PHPOffice/PHPExcel/issues/1320, opened a year before that. And https://github.com/SpartnerNL/Laravel-Excel/issues/1521.
Shared/StringHelper::UTF8toBIFF8UnicodeLong calculates incorrect length for strings when they contain characters outside Unicode BMP. Xls uses UTF-16 to encode its strings, and characters outside BMP require a surrogate pair to encode. PhpSpreadsheet (and PhpExcel before it) have been counting these as a single character, but Excel counts them as 2. Change to compute the length as half the number of bytes in the UTF-16 string, as Excel does.
A formal test is added, but it's a bit difficult to follow. So I aso added a non-BMP emoji to 27template.xls, which will cause it to be both read by Xls reader and written by Xls writer. This would previously have created a corrupt worksheet. The emoji is now handled correctly.
* Correct Re-computation of Relative Addresses in Defined Names
Fix#3661. Insertion or deletion of rows or columns can cause changes to the ranges for Defined Names. In fact, only the absolute parts of such ranges should be adjusted, while the relative parts should be left alone. Otherwise, as the original issue documents, the adjustment to the relative portion winds up being double-counted when the Defined Name is referenced in a formula. The major part of this change is to ReferenceHelper and CellReferenceHelper to not adjust relative addresses for Defined Names. An additional small change is needed in the Calculation engine to `recursiveCalculationCell` when a Defined Formula is being calculated.
In a sense, this is a breaking change, but for an obscure use case which (a) was wrong, and (b) is unlikely to be of importance. Some of the tests in ReferenceHelperTest were wrong and are now corrected, with the results being cross-checked against Excel.
When a Defined Name using relative addressing is defined in Excel, the result is treated as relative to the active cell on the sheet in which the name is defined. PhpSpreadsheet treats it as relative to cell A1. I think that is a reasonable treatment, and will not change its behavior to match Excel's - that would definitely be a breaking change of some consequence.
An interesting use of relative address in a defined name is demonstrated at https://excelguru.ca/always-refer-to-the-cell-above/. Note that the steps there involve setting the selected cell to A2 before defining the name. When that spreadsheet is stored, the actual definition of the range is `A1048576`. Likewise, adding a defined name for the cell to the left would be stored as `XFD1`. This seems a little fragile, but Ods, which I believe does not have the same row and column limits as Excel, certainly treats these values the same as Excel. This particular construction is formally unit-tested. Note, however, that although using these Defined Names as a formula on their own works just fine, a construction like `=SUM(A1:CellAbove)`, as suggested in the article, seems to put PhpSpreadsheet calculation engine in a loop. In the likely event that I can't solve that before I merge this change, I will open a new issue to that effect when I do merge it. Note that this can be handled without defined names as `=SUM(A$1:INDIRECT(ADDRESS(ROW()-1,COLUMN())))`. PhpSpreadsheet will handle this as a cell formula, but not yet as a Named Formula.
The tests show a breakdown evaluating `=ProductTotal` (product of 2 formulas using defined names with relative addresses) on the sheet on which it is defined, but it works from a different sheet. The usual debugging techniques show me why this is happening, but I can't see how to overcome it. As above, if I can't solve it before I merge, I will open a new issue.
For those situations where I intend to open a new issue, tests are added but are marked Incomplete. Because of those, I will leave this PR in draft status for 2 weeks before moving forward with it.
* Fix productTotal Problem
Need to restore current cell after evaluating defined name.
* Fix Several Problems in a Very Complicated Spreadsheet
Fix#3679. That issue was opened for a problem which was already solved by PR #3659, however there were additional problems with the spreadsheet.
The main problem is that Data Validations and Conditional Styles can each be supplied in the Xml in either "external" or "internal" formats. The code for each to handle "external" assumes that each is the only "external" item on the worksheet in the Xml, but some of the worksheets in the sample spreadsheet provide both as "external" on some sheets. The code to fix this is verified against the supplied sample, however no formal test has been added for it. The sample is much too large and complicated to be added to the test suite - it takes several minutes to read, and even longer to write (`setPreCalculateFormulas(false)` is highly recommended). I will leave this ticket open for a few days to see if I can hand-craft a suitable test case, but I am not hopeful.
A second problem is that something in the Xlsx Reader `$xmlSheetNS->sheetData->row` loop breaks the selected cell for the worksheet. This is easily fixed and verified by eye (and with the supplied sample), but, again, no explicit test case is added.
A third problem is that drawings which are part of the supplied sample use `srcRect` tags in the Xml to effectively produce a cropped version of the image. This tag has hitherto been ignored. It is now supported in Xlsx Reader, Xlsx Writer, and Worksheet/BaseDrawing object. This is again verified with the supplied sample; unlike the other parts, it was easy to add a new formal test case for this part of the fix.
* Scrutinizer False Positives
Fix#3658. Readers will call `setCalculatedValue` to save the original calculated value for a cell that contains a formula; this affects the result of `getOldCalculatedValue`. The set routine automatically casts numeric values to float, which is not appropriate when the calculated value is explicitly set to a string type. An optional parameter is added to the set routine to indicate whether a cast is appropriate (default is true so userland uses of this method will remain unaffected). The readers for Xlsx, Xls, Xml, Ods, and Slk are all changed to set the new parameter appropriately. Gnumeric saves the formula, but does not appear to save the result, so no change is needed there. Html and Csv save the result but not the formula (for preCalculateFormulas true - they save the formula but not the result for false), so no change is needed for them.
It is not clear what the use case is for oldCalculatedValue, but it does date back to PhpExcel. A possible use case is demonstrated in the new tests - Excel function INFO is not implemented, so getCalculatedValue will generally return null on such a cell, but will return oldCalculatedValue when it sees that the function is not implemented but oldCalculatedValue has been set (typically by loading an existing spreadsheet), so the result that is on the spreadsheet will be available to the program which loaded it.
Reading a spreadsheet which has been created with preCalculateFormulas false will, or at least should, result in a null oldCalculatedValue. This was true for Xls, Xml, and Slk. Xlsx was actually storing 0 in this situation; that wasn't precisely a problem, but it wasn't necessary and seems misleading. Xlsx Writer is changed to no longer set a value in this situation. This is technically a break, but the existing code was wrong; one unit test needed a very minor modification as a result of this change.
This is a replacement for PR #2404 which has been open for almost 2 years, and which I will close now. As submitted, it broke many unit tests, and no attempt was made to fix those and add others. However, while reviewing it, I found that, among all the tests which it accidentally broke, there were tests which it "broke" (in ExplicitDateTest) which were actually wrong in the first place. That seemed a good enough reason to investigate further.
The original PR suggested the change was needed because "there is now support for microseconds when reading Datetime cells". I'm not sure that's true. Time of day is stored as a fraction of a day, and nothing prevents microseconds from being part of that fraction. It is true that Excel does not allow you to format a date/time cell to display more than 3 decimal positions for the seconds value, even if the value turns out to be accurate to the microsecond, and that has not changed. It is also true that Php supports microsecond accuracy in its DateTime objects, and it behooves PhpSpreadsheet to accommodate that.
PhpSpreadsheet, like Excel, nominally supported the use of one, two, or three decimals when displaying seconds. However, it did not do it correctly, and there had been no tests of this using a value where the decimals were anything other than 0. One existing test, in NumberFormatDates, was wrong. It is fixed and new tests added.
* Check For and Eliminate Octal Literals
Php8.1 introduced a new form of octal literal (0o123); the old form (0123) continues to be supported, but can certainly be misleading. There are exactly 6 uses of the old form in the code base (5 in tests, 1 in infra). 4 of these are clearly unintended (02 or 03), and the leading 0 should just be dropped for those. The other 2 are for file permissions and are easily replaced.
* Minor Changes to LocaleGenerator
Testing the permissions change exposed some problems when the locale files don't exist. This never shows up as an issue when the files do exist, which is pretty much all the time, but correct the theoretical exposures anyhow.
* Populate bg and en_uk Locale Files from Spreadsheet
Also rename locale generator so that it is executed first. This is when it should happen because other tests depend on its results.
* Some Bulgarian Function Omissions
Figured out a reasonable way to compare old Bulgarian function file to new; this identified a handful of now-corrected omissions and errors. Note that JIS function, in old list, is not on spreadsheet, but the Bulgarian translation is the same as English (so the translation isn't needed), and the function has been replaced by DBCS, which is also not on the spreadsheet.
* Fix Xlsx Read Ignoring Comments
Fix#3654. Several places in Reader Xlsx make a truthy test for `$zip->locateName()`. However, zero is a legitimate result which should not be treated as false. Change all existing tests in this form to test for (in)equality to false. For the record, there is no existing exposure of this kind in Reader Ods.
* Absolute Path In Printer Settings File
Another unexpected use of an absolute path in a rels file. Easily fixed, but the test file is 7+ MB. I have asked the problem reporter to try to find a smaller file.
* Reduced Size of Test File
It was 7.8MB. I manipulated the file to replace all the comment backgrounds in column AN with a single image, much smaller than the existing ones. This reduced the file size by over 6MB. It's still larger than I'd like, but might be acceptable. (AN121 uses a different background than the others just so that I could check that they would all be copied faithfully.)
* Html Writer Styles when Using Inline Css
Fix#3678. Problem introduced by PR #3016. Combining `td` and `th` styles into a single declaration greatly reduces file size when `useInlineCss` is false, which is the default. However, generating code with the non-default option was not changed to use the combined declaration, so styling was lost. This PR rectifies that error.
* Apostrophe Rather Than Quote
* fix: Allow to read data from a table located in a different sheet
* Add Test Case, Correct Calculation
Calculate correct result, at least for calculations with a single answer. For an array of answers, result will be similar to other functions which return an array (dynamic arrays not yet fully supported).
* Rename test class
* Allow Appropriate Use of Table as DefinedName in Calculations
See Issue3569Test, which behaves like Excel.
* Remove Dead Code
Scrutinizer is correct here.
* Memory Leak
Worksheet points to Table, Table points back to Worksheet. Circular reference prevents garbage collection. Remove Table collection in Worksheet at destruct time to avoid this problem.
* PhpUnit10 Failure
Avoiding memory leak triggered segfault in Phpunit 10. Research and fix later.
* Add toString to StructuredReference
Absence of such a method seems to cause problems for untaken IF branches in Calculation.
* Remove Dead Assignments
Correctly detected by Scrutinizer.
---------
Co-authored-by: Kevin Verschaeve <kevin.verschaeve@exotec.com>
Co-authored-by: oleibman <10341515+oleibman@users.noreply.github.com>
* Read Code Page for Xls ListWorksheetInfo/Names for BIFF5
Fix#3671. Xls reader was not processing Code Page as part of functions ListWorksheetInfo/Names, which was causing them to fail for for BIFF5 (and BIFF7); this was not a problem for BIFF8. There were no unit tests for these functions for either BIFF5 or BIFF8. There are now.
* Add getVersion and getCodePage Methods
These came about because test file for non-standard codepage was supposed to be BIFF5, but turned out to be BIFF8 using UTF-16 with some string data otherwise encoded. Add a BIFF5 equivalent (some hex editing was required), and the means to distinguish one from the other.
* Found MACCENTRALEUROPE Text in BIFF8
It was used for 'Last Modified By' property, even though bulk of spreadsheet uses UTF-16LE. Add a test.
* Fixed inconsistent string handling in SUM() implementations
* Use assertEqual instead of assertSame to allow ints and floats with the
same value
* Removewd special handling of empty strings and use default casts
* Eliminate Phpstan errors
* More Test Cases
* More New Tests
---------
Co-authored-by: oleibman <10341515+oleibman@users.noreply.github.com>
Fix#3660. Code existed to read a hyperlink in a cell for Ods, but did not exist for writing. Hyperlinks pointing within a document use a different representation than for Excel (and therefore for PhpSpreadsheet); read and write will both handle the mapping from one to the other.
While researching how to write the text part of a started Xml element, it transpired that writing 2 Ods properties (Company and Category) did not escape their values properly. Confirmed and fixed problem. I do not believe that there is such an exposure for any other writer. As it turns out, Ods Reader was not processing Company or Category properly; that is fixed.
* Xlsx Reader Namespacing for Tables, AutoFilters
Fix#3665. The original issue was the use of an absolute path in the rels file pointing to the comments file. That was easy to take care of, but a bigger problem with the spreadsheet accompanying the problem report was that it used unexpected spacing for AutoFilters and Tables. AutoFilters were already known not to be covered, but Tables appeared after the namespacing changes, but without namespacing support. This PR fixes the absolute path problem and adds namespacing support for Tables and AutoFilters.
Remaining areas which are still namespace unaware, mainly because of the absence of test samples which use them with unexpected namespacing, include conditional formatting (internal or external), sheet view options, sheet protection, unparsed loaded data, data validation (internal or external), alternate content, and header/footer images.
* Mysterious Warning for Php7.4 Only
Node no longer exists, doesn't affect result. Suppress warning.
* Scrutinizer
What would a change be without some new false positives?
* Scrutinizer
A gift that keeps on giving :-( Now it's deciding that things that it didn't report in a scan from a few minutes ago are worth reporting, even with no relevant code changes.
* Scrutinizer
It is an idiot. Let me see if I can fix one false positive and succeed in guessing where it might report another one, even though it didn't do so with this last run.
* Minor Changes to Writer/Mpdf and Writer/Html
Changed a sample to illustrate how to add header/footer in Mpdf using setHtmlEditCallback. This uses custom Html tags, and, like body, it appears that these must be defined in the first writeHtml. Adjust writeMpdf to permit this by using a new constant `SIMULATED_BODY_START`, defined as an Html comment, as a delimiter.
Sample 21c_Pdf, to which the header/footer code is added, had been introduced with PR #2434 to ensure that the body tag was always in the first chunk. However, PR #3016 accidentally invalidated that test by reducing the number of style lines so that the sample now included the body tag in its first 1000 records rather than afterwards. This change puts it past record 1000 again.
Inspecting the results of all the Html/Pdf samples after this change, it turns out that sample 25_In_memory_image was accidentally broken by PR #3535 - the combination of `max-width:100%` (already present before that change) with `position:absolute` (introduced with that change) made the memory drawing disappear from the rendered html when the image occurs in a column after the last column with data in it. It appears that there is no need for max-width (drawings which are not memory drawings do not use it), so it is dropped. The sample is changed to add a second page with a memory drawing, one page with the memory drawing after the last data column, and one with it before. The Html results now reflect the Xlsx result, as they should.
* Minor Performance Improvements
Anonymous function.
* Php 8.3 Problem - RLM Added to NumberFormatter Currency - Minor Break
Fix#3571. This isn't truly a Php8.3 problem - it all depends on the version of ICU with which Php was linked. ICU 72.1 adds an RLM (right-to-left mark) character in some circumstances when creating a currency format. This broke some tests for the Currency and Accounting wizards, and can result in a difference in the appearance of some spreadsheet cells.
This PR changes code to strip out the RLM or not depending on a new property. The new property could default to true (so end-users will not see any change no matter what release of ICU is used), or false. For the latter, users might see a break, but my assumption is that the ICU developers have good reasons for their change, and it's probably best to go along with it. If users wish to retain the existing behavior, they can do so by adding the following code before setting the wizard's locale:
```php
$wizard->setStripLeadingRLM(true);
```
* Eliminate 2 Dead Statements
Correctly flagged by Scrutinizer.
Some tests were failing with Php nightly. Row values, which were expected to be int, were now being returned as string. Adding a couple of casts eliminated the problem in Php 8.3 without adverse effects in other releases.
* Preserve Transparency in Memory Drawing
Fix#3626. User felt that Exact setting of column width should not add padding. In order to avoid making this a breaking change, I am adding a new boolean property `usePaddingForExact` with setter and getter to Shared/Font; default is false (current behavior).
* Change Parameter
Instead of bool, make it null|float|int, allowing user to set whatever is deemed appropriate.
Fix#3624. Use the same logic as elsewhere in the same module to invoke `imagesavealpha` when appropriate. (I confess that I do not understand the use case where you would not use imagesavealpha.) The fix was easy; writing a test was not. Google to the rescue.
* Cleanup for Phpstan, PHP-CS-Fixer Upgrades
Dependabot wanted to upgrade, but changes resulted in new "errors". Fix them.
* Scrutinizer
It says value must be numeric; Phpstan says it might not be. Try to reconcile them.
Fix#3565. Requester suggested adding viewport as a property of Html Writer. I think it would fit better as a Document Property, so that it can be both read and written, just like other meta properties, making it easier to load an Html spreadsheet and save it as a new one preserving this property. It is an open question whether we should try to preserve this property when writing to other spreadsheet types, e.g. as a custom property for Xlsx. For now, I am not attempting that. Requester also suggested this as a boolean property, but there are other parameters and different values that might be used for viewport. So I am defining it as a string property, with a constant SUGGESTED_VIEWPORT that can be used for requester's use case.
* Split Screens
Fix#3601. Split screens are a feature that affects the display of the spreadsheet to the end user; they do not affect the data. They are conceptually similar to "freeze panes". The differences are explained in the issue. As will be explained, support is fairly full for Xlsx, and less full for Xml; no attempt is yet made to support Xls or Ods.
For freeze or split, the window can be divided into 2 horizontal panes, or 2 vertical panes, or 4 horizontal+vertical panes. In Excel, you can split or freeze on cell A1, which causes 4 panes centered at the middle of the screen. PhpSpreadsheet will not duplicate that functionality, and code is added to ignore an attempt to freeze at A1. This breaks one existing nonsensical test, which is changed to something sensible for this PR.
In the spreadsheet xml, both 'freeze' and 'split' use attributes 'xSplit' and 'ySplit' to indicate the position. Unfortunately, the attributes have different meanings for 'freeze' (*columns* from top/left) than for 'split' (*distance* from top/left). For that reason, it is difficult to change between 'freeze' and 'split', so PhpSpreadsheet will not yet support doing so.
There are 3 possible states when freeze/split is used - 'frozen', 'split', or 'frozenSplit'. All will be maintained during read/write, and the user can set them. In Excel, you get 'frozenSplit' by first splitting, then by changing to freezing. In that case, when you unfreeze, you revert to split. PhpSpreadsheet will not duplicate that functionality - when you unfreeze, the panes will go away regardless of whether you were 'frozen' or 'frozenSplit'.
Changing the selected cell will cause the 'active pane' to update when the panes are frozen. PhpSpreadsheet will not yet update the active pane when the panes are split. The programmer will, of course, have the opportunity to explicitly change the active pane in this circumstance.
I have not yet been able to figure out how Xml spreadsheets map their panes, or how to determine selected cell for the sheet as a whole or for individual panes when freeze/split is in effect. The programmer will have the opportunity to specify these explicitly. However, loading an Xml spreadsheet in PhpSpreadsheet will not fill in these values.
Each pane has its own selected cells, and you can navigate between these with the F6 key in Excel (this may work only with split but not with freeze, but Excel maintains the values for freeze and so will PhpSpreadsheet). For Xlsx, PhpSpreadsheet loads and saves these values.
* Eliminate Some Dead Code
Pointed out by Scrutinizer.
The overall problem is described in issue #3613. This PR represents only a partial solution. Very complicated formulas are resulting in calls to string functions using null arguments, which is deprecated in recent Php releases. In fact, there are many such deprecations for the spreadsheet in question. Eliminating the deprecations is easy. However, the result of the calculation is, for many cells, 0 rather than what Excel determines it should be. This can be overlooked to a certain extent, because Excel will recalculate when the spreadsheet is opened, so loading and saving the spreadsheet in question will result in a spreadsheet which looks okay when it is opened in Excel. Resolving the incorrect calculation in PhpSpreadsheet would be nice, so I'm leaving the issue open, but that looks too complicated for me to get a toehold.
Fix#3609. Reporter created a spreadsheet with non-adjacent columns having non-default widths. Ods Writer needs to generate entries for the missing columns with default width.
The fix was fairly simple. Testing it was not. Ods Reader basically ignores all styles; they are complicated, declared in (at least) 2 places (content.xml and styles.xml). This change deals only with the problem as reported, in which the missing declarations should be in content.xml. If someone reports a real-world example of this involving styles.xml, I can look at that then. In the meantime, this toehold might serve as a template for adding other style processing to Ods Reader.
Looking at other formats, processing of column widths was also missing from Html Reader, and is now added. Note that this will work only with inline Css declarations on the `col` tags, which can be generated by Html Writer using `setUseInlineCss(true)`. This creates a much larger file than one created without inline CSS.
A general problem became evident when studying the code. Worksheet `columnDimensions` is an unsorted array. This is not a problem per se, but it can easily lead to unexpected results from a `getColumnDimensions` call. The code is changed to sort the array before returning it in `getColumnDimensions`.
One existing test failed as a result of this change. It errorneously tested `getHighestColumn` instead of `getHighestDataColumn`, which caused a problem because the final column declaration included a `number-columns-repeated` attribute. The new result for `getHighestColumn` is correct, and the test is changed to use `getHighestDataColumn` instead, which was certainly its intent.
Fix#3592. Fix#3594. The original issue asked that Data Validation be added for Xml spreadsheets. However, doing so exposed some other Data Validation problems, which are corrected along with the Xml portion.
The main other problem is that the code for `Cell\DataValidator::isValid` covered almost no possibilities. It is expanded to include all possibilities, except for Custom Types, which I will continue to think about. Many tests are added.
An obscure problem is that Xlsx Reader does not quite work properly when the Data Validation cells are beyond the high-used column or row. We use the high-used values because it is completely impractical to set all the cells in a range when an entire column or row is selected. This change causes the leftmost top cell in a range to be explicitly allocated, affecting the high-used values. It is not 100% effective, but it will be much more difficult to get into the problem situation, which I will continue to think about.
Although our expectation is that we will not see any Xml Spreadsheets using non-standard namespacing, much of the Xml Reader code is changed to use proper namespace techniques. A few areas which just had nothing to do with this change continue to use less robust techniques; I plan to address those soon after implementing this change.
One of the Data Validation types is for time of day. Testing revealed that `Shared\Date::convertIsoDate` was insufficiently robust when determining whether its input consisted of a time of day without date. It is now more robust.
Fixed typo in code examples in name of variable `$columnAddress` that was wrongly shortened to `$column`. Long version of the variable name $columnAddress is used in the corresponding samples scripts so now the docs match the samples.
* Unzipped Gnumeric File
A Gnumeric file is normally a gzipped Xml file. However, the Gnumeric application will also read an unzipped Xml file. Allow PhpSpreadsheet to do the same.
* Scrutinizer (legitimate)
Fix minor problem.
* HyperlinkBase Property, and Html Handling of Properties
Fix#3573. The original issue concerned non-support of Document Properties in Xml spreadsheets. However, most of the Properties mentioned there were already supported. But the investigation revealed some gaps in Html coverage.
HyperlinkBase is the one property mentioned in the issue that was not supported for Xml, nor indeed for any other format. All the other document properties are 'meta'; but HyperlinkBase is functional - if you supply a relative address for a link, Excel will use HyperlinkBase, if supplied, to convert to an absolute address. (Default is directory where spreadsheet is located.) Here's a summary of how this PR will handle this property for various formats:
- Support is added for Xlsx read and write.
- Support is added for Xml read (there is no Xml writer). Ironically, Excel messes up this processing when reading an Xml spreadsheet; however, PhpSpreadsheet will get it right.
- HyperlinkBase is supported for Xls, but I have no idea how to read or write this property. For now, when writing hyperlinked cells, PhpSpreadsheet will be changed to convert any relative addresses that it can detect to absolute references by adding HyperlinkBase to the relative address. In a similar vein, Xls supports custom properties, but PhpSpreadsheet does not know how to read or write those.
- Gnumeric has no equivalent property, so nothing needs to be done to its reader. Since we don't have a Gnumeric writer, that's not really a problem for us.
- Odt has no equivalent property, so nothing needs to be done to its reader. The Odt writer does not have any special logic for hyperlinks, so, at least for now, will remain unchanged.
- Csv has no equivalent property, so nothing needs to be done to its reader. The Csv writer does not have any special logic for hyperlinks, so, at least for now, will remain unchanged.
- Html allows for an equivalent `base` tag in the head section. Support for this is added to Html reader and writer.
Html Writer was only handling 8 of the 11 'core' properties. Support is added for `created`, `modified`, and `lastModifiedBy`. Custom properties were not supported at all, and now are.
Html Reader did not support any properties. It will now support all of them.
* Scrutinizer
Remove one dead reference.
They have made some changes at my request, the major effect of which is that it will now work with 33_Chart_create_bar_stacked. This is a departure for them in that they have changed the functionality of jpgraph, not merely made sure that it is compatible with new Php releases.
Dependabot wanted to do this, but the changes to php-cs-fixer caused it to report errors in 7 modules, all due to spacing in comments. I hate this, but who asked me? Just fix them.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the first of three PRs to replace that one. This PR also allows the use of slash as a thousands separator or decimal separator or currency symbol. These are, of course, very unusual situations; the main reason to support them is so that PhpSpreadsheet code will not crash when users set those options. New Calculation/Engine and AdvancedValueBinder tests confirm these. While making these changes, a few errors were found in AdvancedValueBinder and Calculation/Engine/FormattedNumber, e.g. currencies weren't parsed correctly when period was used as the group separator and comma as the decimal separator. Tests have been added for those situations.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the second of three PRs to replace that one. This accomodates the use of slash as a delimiter in column names in Tables and Structured References. The source changes are very simple. Additional tests exercise all the source changes.
There is also a preg_quote call when a table is renamed. I have also changed it to accomodate slash, because it's the right thing to do. But ... I can't think how to test it. PhpSpreadsheet will not allow you to set a table name to a string containing a slash (a test is added to confirm), and, if I manually update the Xml in an Xlsx spreadsheet so that the name does contain a slash, Excel will, understandably, complain that the file is corrupt.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the first of three PRs to replace that one. This accomodates the use of slash as a delimiter in functions TEXTAFTER, TEXTBEFORE, TEXTSPLIT, and NUMBERVALUE. The source changes are very simple. Additional tests exercise all the source changes.
* Redo Calculation of Color Tinting
Fix#3550. Some colors are specified in Excel by specifying a theme color to which a tint is applied. The original PHPExcel algorithm for doing this was developed by trial and error, and is good enough a lot of the time. However, for the issue at hand, the resulting color is detectably different from the calculation that Excel makes. Searching the web, I found https://gist.github.com/Mike-Honey/b36e651e9a7f1d2e1d60ce1c63b9b633 which comes much closer for the case in hand, and for all the other cases that I've looked at. That code depends on Python colorsys package; I have adapted the code from the Python gist and package into a new Php class. This doesn't agree perfectly with Excel. However, if each of the red, green, and blue components (each a value between 0 and 255 inclusive) agree within plus or minus 3 (arbitrary choice) of Excel's result, I think that is good enough.
I have added a new test member which reads from a spreadsheet with Xml altered by hand to set up several theme/tint cells. These tests use the plus-or-minus-3 criterion. They result in 100% code coverage of the new class.
Unsuprisingly, some existing tests failed with the new code. Issue2387Test reads a theme/tint font color, and is changed to use the plus-or-minus-3 criterion, comparing against the color as Excel shows it.
ColorChangeBrightness showed 9 failures with the new code. It consists of calculations not involving a spreadsheet. For that reason, I felt it was sufficient to just do an exact match test, changing the 9 old results for new results confirmed with the Python code. I also added one new test case, the one that kicked off this entire PR.
* Scrutinizer Being Stupid
It strikes again.
* Changes to NUMBERVALUE, VALUE, DATEVALUE, TIMEVALUE
Fix#3574. Reporter received deprecation notice for NUMBERVALUE function with invalid arguments. In fact, the arguments turn out to be valid after all; NUMBERVALUE treats a null-string or an all-blank-string in the first argument as if it were 0. Fixed this, and added several test cases suggested by it.
VALUE had been parsing its argument the same way as NUMBERVALUE. However, VALUE does not substitute 0 for null-string or all-blank-string. Coded up the difference between the two, and added the same tests for VALUE as for NUMBERVALUE.
VALUE can also pass its argument to DATEVALUE or TIMEVALUE. It is currently over-permissive about that, because Php is over-permissive, e.g. `new DateTime('q')` will return a DateTime object with the current date and time with a timezone of 'q'. Excel will, naturally, return `#VALUE!` for `DATEVALUE('q')`. I don't know that we can ever match Excel's (AFAIK not formally documented) decisions here 100%, but we can get a lot closer by parsing the date string if and only if it contains at least one digit. Code to enforce that is added to DATEVALUE and TIMEVALUE, and appropriate tests are added.
* Failure for Php 7.4 Linux
After not seeing any such problem for many months, this is the second day in a row where result is different on Windows than Linux with no apparent reason to think why that should be the case. At any rate, easily solved.
Zipstream's newest production release requires Php8.1. This is not necessarily a problem for us as we are locked to an earlier release. However, psr/simple-cache became a real mess for us as people wanted to install PhpSpreadsheet alongside other products which required a version of simple-cache that could not run in some of the versions of Php which we support, and I would like to avoid having to go through that again. This ticket is a contingency to put us ahead of the curve in case such a problem should arise with Zipstream. There is reason to believe that Zipstream is not so widely used as simple-cache, so we do not need to rush this change in. However, should that belief prove incorrect, the change should be ready to go when needed.
* Recognize 'Hidden' Attribute in Xml Spreadsheet
Fix#3566; PhpSpreadsheet should now handle Hidden attribute for rows and columns in Xml spreadsheets. Also added the ability to load Xml spreadsheet from string rather than file, as can be done for Csv and Html.
* Add Support for Top Left Cell
Another missing piece.
* Font Bold/Italic
Was always setting to true rather than checking value for 0/1.
* Active Sheet Index
Had always been set to hard-coded 0. Read it from XML if available.
* Selected Cells
Get Selected Cells from XML when available.
* Worksheet Protection and Style Protection
Add support for those to Xml Reader.
* Column Spans
Not really an essential part of Excel, used in Xml to reduce file size.
Fix#3561. PhpSpreadsheet VLOOKUP/HLOOKUP allow the specification of index_number (third parameter) as an array when lookup_value (first parameter) is supplied as an array. Change to permit index_number as an array even when lookup_value is not an array, as Excel does.
oleibman
dependabot[bot]
Kevin Verschaeve
Jacek
Juan Antonio FM
MarkBaker
jpachta