* Security Patch
* Throw Exception for EBCDIC Encoding
* Mixed UTF-8 and UTF-16
Further mischief. I don't know if the examples truly are valid Xml, but PhpSpreadsheet is letting them sneak through.
Fix#4213. Early versions of Php 8.4 caused problems for some Excel functions, among them ROUNDDOWN and ROUNDUP. New code was added which seemed to work for 8.4 and all prior releases (back to 7.4). However, as the issue shows, there is a problem with the new logic that had not been a problem for prior releases (PhpSpreadsheet 1.29.1 with Php 8.3 or earlier).
As it happens, some time after 8.4 broke the existing functionality, new features were added to it which enabled much cleaner logic. The code will now take this cleaner path when available, and revert to the older logic when not. This appears to work for all relevant Php releases, at least till the next non-match between PhpSpreadsheet and Excel is reported.
And, having made the change to ROUNDDOWN, it occurred to me that ROUNDDOWN always returns the same result as TRUNC (they differ only in number of required arguments). So I changed TRUNC to call ROUNDDOWN, and that didn't break anything. So I will keep that change. This probably means I should change Calculation so that ROUNDDOWN is called automatically, and deprecate TRUNC. That can happen at a later time.
See #4161. The best way to future-proof is to set the escape character to null string, and set testAutoDetect to false before reading. However, depending on the file being read, this may lead to different results than expected. This will be unavoidable because Php itself will change. This PR adds a new static method `affectedByPhp9` to Csv Reader. This can be used to identify in advance whether an input file will be affected by the changes. This will allow users to identify problems in advance, and prepare for how they might be handled.
Fix#2157. Excel 2003 format allows some things which a real Xml parser would reject. In particular, it permits Html entities, and leading whitespace. Change Xml Reader to do likewise.
* Validate Mime Type for Images
* Extend Change to Xlsx Reader
* Attach Drawing to Sheet Only If Valid
* Suppress Some Theoretical Warning Messages
* Minor Tweak
This supersedes PR #607 by @christian-forgacs, who deserves all the credit for reporting the problem and devising the solution. The PR went stale in 2018, and it is just easier to resubmit a clean version rather than clean up the old one. Among the suggestions in the PR was that you should try to create a spreadsheet from scratch to demonstrate the problem rather than supply one. However, my attempts to match the failing spreadsheet do not have a problem when they are read. So, a supplied spreadsheet it is.
Fix#1570. No sample spreadsheet was supplied with that issue, but I am almost certain that this is another example of the same problem. I am removing the stale label from that issue; it will be closed properly when this PR is merged.
Fix#4125. Currency and Accounting Wizards generate styles for ISO codes, but these are incorrect and cause a problem when Excel tries to open a spreadsheet containing these styles. Debugging that problem, other problems with Wizards came to light:
- Currency Wizard should permit four different styles for negative numbers (as Excel does) - minus sign, minus sign and red font, paretheses, and parenthese and red font. It currently uses only minus sign.
- Accounting Wizard should use parentheses for negative numbers (as Excel does). It currently uses minus sign.
- Accounting Wizard should always use SYMBOL_WITH_SPACING (as Excel does). It currently permits the use of SYMBOL_WITHOUT_SPACING. What WITH_SPACING really does is to ensure decimal-point alignment among adjacent cells in a column with the same format.
- Currency Wizard should always use SYMBOL_WITHOUT_SPACING (as Excel does). It currently permits the use of SYMBOL_WITH_SPACING.
I am correcting these problems by:
- renaming Currency Wizard to CurrencyBase
- adding a `negative` property with setter to it and its constructor.
- adding a new Currency which extends CurrencyBase, always using SYMBOL_WITHOUT_SPACING when formatting.
- having Accounting extend CurrencyBase rather than Currency, always using SYMBOL_WITH_SPACING and NEGATIVE_PARENS when formatting.
- CurrencyBase can be used if the restrictions on Currency and Accounting are not desired (e.g. the suggested accounting constant from [this unimplemented PR](https://github.com/PHPOffice/PhpSpreadsheet/pull/1576)).
Excel does some funny stuff with these formats. In particular, it might try to guess if you have a particular Accounting format in mind. So the Accounting wizard for dollar sign generates a format which (a) matches FORMAT_ACCOUNTING_USD, and (b) Excel (correctly) interprets as an Accounting format for symbol $. On the other hand, the Accounting wizard for euro sign generates a format which (a) matches FORMAT_ACCOUNTING_EUR, but (b) Excel interprets as a custom code rather than an Accounting format. This in itself is not a particularly big deal, but it has made it impossible for me to see exactly what format Excel uses for trailing currency symbols for negative numbers. I can't get them to decimal-point align with positive numbers if I put any kind of space between the trailing parenthesis and the currency symbol, so I omit that. It doesn't look terrible, and it keeps everything aligned, but it might not be what people are used to.
I've also changed the formatting to use spaces rather than non-breaking spaces. They seem to work just fine, and the constants mentioned above use them rather than nbsp.
Fix#4124. Currency formats that contain an ISO currency code which contains one of the characters used to recognize a date format (hmsdy), e.g. [$HUF], are being formatted by PhpSpreadsheet as dates rather than currencies. Code is changed to recognize open bracket followed by dollar sign followed by 3 Latin alphabetic characters followed by close bracket as a non-date.
Prevent XEE by hiding custom entities by using single quote to
declare a non-UTF-8 encoding.
XML standard, https://www.w3.org/TR/xml/#NT-EncodingDecl, allows single
quote to declare encoding, but we did not support it. Instead, we
incorrectly fell back on the default of UTF-8. That incorrectly kept the
XML as non-UTF-8, and thus prevented our regexp-based custom entity
detection mechanism to work.
Fix#4113. TRUNC isn't always producing the expected result. There was a promising algorithm at https://stackoverflow.com/questions/4668628/truncate-float-numbers-with-php from user Juan. It works through Php8.3, but failed in Php8.4 (more on this later). User Savageman on the same page has a solution that needs work, but, once the work had taken place, it works on Php8.1-8.4.
The ROUNDUP and ROUNDDOWN functions were adversely affected by Php8.4, probably for the same reasons as Juan's TRUNC suggestion. I put a kludge in place for them some time ago, but I wasn't happy with it. The solution used for TRUNC here suggested a change to the ROUNDUP and ROUNDDOWN code that would no longer require the kludge. The change to those functions now works more cleanly on Php8.1-8.4.
Suggested by PR #848 from @markkimsal. The RATE calculation had already been corrected, so that part of the PR was unnecessary, however one of the tests included a floating point value for Number of Periods, which Excel permits. PhpSpreadsheet till now expected that parameter to be an integer. This is trivially changed with some tests added.
Fix#4099. Ods Reader was expecting there to always be `header-style` and `footer-style` tags when `page-layout` tag is present, but these need not exist. It seemed like there might be other exposures along this line in `readPageSettingStyles`; rather than waiting for a problem report to show up for each, the code is updated to use `->item(0)` in place of `[0]` when appropriate, and make use of the nullsafe `?->` operator introduced with Php8.
Fix#476. Another in the "better late than never" series, closed as stale in June 2018. Xlsx Writer expects cells containing RichText to have DataType `TYPE_INLINE`; but the spreadsheet associated with the issue has the cell defined as `TYPE_STRING`. Change Writer to handle RichText TYPE_STRING appropriately.
Fix#460. Another in the "better late than never" series, closed as stale in June 2018. Ods Writer and Ods Reader handle booleans differently; what is worse, neither of them do it correctly. They will now match the behavior of LibreOffice. Reporter said that part of the xml would vary depending on locale; I believe that part is never actually used, but I do emulate that behavior.
PR #4073 adds parsing of formulas for `$cell->setValue()`, but the spill operator won't parse correctly, breaking some of this PR. Add code to avoid that problem.
Fix#64 (really!), closed as stale in December 2017, another in our "better late than never" series. Excel's INDEX function doesn't really behave quite as described. If a single row is used as an argument, either in literal form `{item1, item2, item3}` or expressed as a range `A1:A6`, INDEX is happy to evaluate the array as if each entry were a row rather than a single item. PhpSpreadsheet is changed to do likewise.
INDEX also returned `#REF!` when it would normally return an array (which would often be reduced to its leftmost topmost entry later). This code is deleted, invalidating one existing test, and INDEX will now operate like other functions which can return arrays.
Fix#804, opened in Dec. 2018, and closed as stale in Feb. 2019, and which I have re-opened to be closed properly by this PR. Better late than never, I suppose. A third party generated an ODS spreadsheet which PhpSpreadsheet could not read. By way of explanation, the xml in the file contained lots of whitespace between tags, which is wonderful for those humans among us who have to analyze it; but PhpSpreadsheet was not prepared for it. It is now.
Fix#4081. Ods Reader was not reading entire contents of comment. On further inspection, Ods Writer also was not handling comments completely correctly. Ods comments are recorded as `text:p` children of `office:annotation` elements. A newline is inserted between successive `text:p` elements. The `text:p` element itself can have as descendants (at least):
- raw text
- `text:span` elements
- `text:line-break` elements, which also causes the insertion of a newline
Ods Writer is changed to use a single `text:p` with multiple span/linebreak elements. Ods Reader is changed to process in their entirety either that form, or multiple `text:p` elements. Styling of the individual elements of the comment is permitted in Ods. That has not been supported till now by PhpSpreadsheet, and this PR will not address that situation - Ods Reader hast little style support, and this would hardly be the most urgent case where it is missing.
Fix#2581 (not obvious - see next paragraph for explanation). This continues the work of PR #2902 (and also PR #3467) to have errors propagated through function calculations rather than treating them as strings. All text functions, and the concatenation operator, are addressed in this PR.
In the original issue, the spreadsheet being loaded uses the result of an unimplemented function as an argument to another function. When `getCalculatedValue` is used on the cell in question, the result is returned as `#VALUE!`. If the cell had just contained a function call to the unimplemented function, getCalculatedValue would have recognized the situation and returned oldCalculatedValue as the result. Not perfect, but good enough most of the time. User would like oldCalculatedValue returned here as well, which seems like a reasonable request.
PhpSpreadsheet always returns `#Not Yet Implemented` as the result for a function which it knows about but which is not yet implemented. That is the key to the `Cell` class being able to substitute oldCalculatedValue in the first place. However, in order to do that for the issue in question, that result has to be propagated to any functions for which the result is an argument. I don't want to add unimplemented to the list of known error codes, but I am willing to add a parameter to `ErrorValue::isError` to indicate whether that value should be considered an error (default is "no").
The first use of that new parameter would be by the text functions. They go through a common Helper routine, so it is pretty easily implemented. And, as it turns out, most of the text functions do not currently propagate errors, e.g. if A1 results in a value error, `=LEFT(A1,2)` will result in `#V` rather than `#VALUE!`. With this PR, they will now be handled correctly.
Fix#1310, which was closed as stale in 2020, but which I will now reopen. Supersedes PR #1311 (@jaiminmoslake7020), from which I will remove the stale label but leave closed. The issue and the PR were too limited - they detected that the use of two equal signs at the start of a string made for an invalid formula, but there are variations, trivial and otherwise, which might also be detected. Using `setValue` with a string which starts with an equal sign will now attempt to parse (not evaluate) the formula; for certain situations in which the parser throws an exception, the string will be treated as a string rather than a formula. An example where it will still be treated as a formula is a 3D range reference, where the problem is not that it can't be parsed, but rather that the formula isn't supported (see unit test Calculation/Engine/RangeTest::test3dRangeEvaluation). Allowing such a formula might cause problems later on, but that is already what happens.
A string beginning with an equal sign but which isn't treated as a formula will automatically set the `quotePrefix` attribute to `true`; all other `setValue` attempts will set it to `false`. This avoids the problem of a lingering value causing problems later on.
It has long been a matter of discontent that setting a style can change the selected cells. A new method is added to `Worksheet`:
```php
applyStylesFromArray(string $coordinate, array $styleArray)
```
This will attempt to guarantee that the active sheet in the current spreadsheet, and the selected cells in the current worksheet, remain undisturbed after the call. The setting of `quotePrefix` above is the first use of the new method.
This change is extracted from PR #2787 by @MarkBaker. That change mostly deals with array functions, and that part will be superseded by PR #3962. However, this part of 2787 is not included in 3962.
Fix#1036 (closed as stale in 2019 and just reopened). Excel spreadsheets can have either of 2 base dates, 1900 or 1904, and the numeric value of any date cells will vary depending on which base date is in use. PhpSpreadsheet has, till now, handled that as a static property of Shared/Date. This does not work well if two spreadsheets with different base dates are open simultaneously. The code is changed to store the base date as a property of the spreadsheet when an Xls/Xlsx spreadsheet is loaded, and use that property when saving an Xls/Xlsx spreadsheet. Any call to `getCalculatedValue` or `getFormattedValue` will temporarily set the Shared/Date value to that of the spreadsheet, and restore it at completion. In order to avoid a BC break, the Xls and Xlsx readers will continue to populate the Shared/Date value as before.
SINGLE function can be used to return first value from a dynamic array result, or to return the value of the cell which matches the current row (VALUE error if not match) for a range. Excel allows you to specify an at-sign unary operator rather than SINGLE function; this PR does not permit that.
Add support for reading CSE array functions for Gnumeric.
Throw an exception if setValueExplicit Formula is invalid (not a string, or doesn't begin with equal sign. This is equivalent to what happens when setValueExplicit Numeric specifies a non-numeric value.
Added a number of tests from PR #2787.
Implement SPILL for dynamic arrays. Calculating a dynamic array function will result in a SPILL error if it attempts to overlay a non-null cell which was not part of its previous calculation. Furthermore, it will set to null all cells which were part of its previous calculation but which are not part of the current one (i.e. one or both of the dimensions of the calculation is smaller than it had been); this should also apply for spills (whose result is reduced to 1*1).
Excel will stop you from changing the value in any cell in a dynamic array except the formula cell itself. I have not built this particular aspect into PhpSpreadsheet.
As usual, MS has taken some unusual steps here. If the result of a dynamic array calculation is #SPILL!, it will nevertheless be written to the xml as #VALUE!. It recognizes this situation by adding a new `vm` attribute to the cell, and expanding metadata.xml to recognize this.
A new optional parameter `$reduceArrays` is added to `toArray` and related functions. This will reduce a dynamic array to its first cell, which seems more useful than outputing it as an array (default).
The CONCATENATE function has been treated as equivalent to CONCAT. This is not how it is treated in Excel; it is closer to (and probably identical to) the ampersand concatenate operator. The difference manifests itself when any of the arguments is an array (typically a cell range). Code is added to support this difference.
Support for array results is added to Csv Writer, Html Writer, and Ods Reader and Writer. I have not figured out how to get it to work with Xls.
See issue #4062. When calculating an array formula, populate all the cells associated with the result. This is almost the same as Excel's behavior. As yet, there is no attempt to create a #SPILL error, so cells may be inappropriately overwritten. Also, if the array size shrinks (e.g. there are fewer unique values than before), no attempt is made to unpopulate the cells which were in range but are now outside the new dimensions. Spill and unpopulation are somewhat related, and will probably be handled at the same time, but their time has not yet come.
UNIQUE, at least for rows, was treating all cell (calculated) values as strings. This is not the same behavior as Excel, which will preserve datatypes, and treat int 3 and string 3 as unique values. Excel will, however, treat int 3 and float 3.0 as non-unique. Within UNIQUE, private function uniqueByRow is changed to try to preserve the the datatype when executing (it will probably treat 3.0 as int - I don't know how I can, or even if I should attempt to, do better - but no int nor float should be treated as a string).
oleibman
Adrien Crivelli