Excel supports the following notations for cell styles:
- `[$-F800]` and `[$-x-sysdate]` will format the date according to what appears to be the user's system long date format.
- `[$-F400]` and `[$-x-systime]`, will format the time according to what appears to be the user's system long time format.
- Builtin style 14 will format date according to what appears to be the user's system short date format.
- Builtin style 22 will format date and time according to the user's preference. It appears that the date portion is formatted according to the user's system short date format, but the time portion is formatted according to a format which is neither system short time format nor system long time format, so I'm not sure how this preference is set.
For F800, sysdate, F400, and systime, any other characters in the style are ignored, except that, if you have more than one of this type of block in the style, Excel will treat it as corrupt (error message on open and style changed to General).
Support is added for the new codes. In addition, note that the value displayed in the cell may differ in different environments. To give the PhpSpreadsheet programmer an opportunity to emulate what the intended audience will most often see, properties `shortDateFormat` (default value is builtin 14), `longDateFormat` (default value is `dddd, mmmm d, yyyy`), `dateTimeFormat` (defaults to builtin 22), and `timeFormat` (default is `FORMAT_DATE_TIME2`), with corresponding setters and getters, are added to Style/NumberFormat. Note that, if these properties are set to some other value in PhpSpreadsheet, it will not affect the values in the cell or the style - it is merely a convenience for the programmer. It will, however, affect column width if autosize is specified for the column. If the programmer does not alter any of the new properties, the output should be unchanged from before for builtins 14 and 22.
The new styles are also recognized by the `TEXT` function. In this case, the cell's calculated value may differ from user to user.
Note that this is a small subset of adding locale information to styles. No attempt is made to support any of the other possibilities - locale data will continue to be passed through to the spreadsheet, but PhpSpreadsheet will discard it before attempting to generate the formatted value of a cell.
While we might never be able to have 100% of our code strict, we can at
the very least do it for all of our tests. This ensures that our tests
are using our API with the types as intended by the test author, and not
silently be cast to what our API requires.
PR #3513, developed by @SaidkhojaIftikhor, has been stuck for some time awaiting tests. This is the first of three PRs to replace that one. This accomodates the use of slash as a delimiter in functions TEXTAFTER, TEXTBEFORE, TEXTSPLIT, and NUMBERVALUE. The source changes are very simple. Additional tests exercise all the source changes.
* Changes to NUMBERVALUE, VALUE, DATEVALUE, TIMEVALUE
Fix#3574. Reporter received deprecation notice for NUMBERVALUE function with invalid arguments. In fact, the arguments turn out to be valid after all; NUMBERVALUE treats a null-string or an all-blank-string in the first argument as if it were 0. Fixed this, and added several test cases suggested by it.
VALUE had been parsing its argument the same way as NUMBERVALUE. However, VALUE does not substitute 0 for null-string or all-blank-string. Coded up the difference between the two, and added the same tests for VALUE as for NUMBERVALUE.
VALUE can also pass its argument to DATEVALUE or TIMEVALUE. It is currently over-permissive about that, because Php is over-permissive, e.g. `new DateTime('q')` will return a DateTime object with the current date and time with a timezone of 'q'. Excel will, naturally, return `#VALUE!` for `DATEVALUE('q')`. I don't know that we can ever match Excel's (AFAIK not formally documented) decisions here 100%, but we can get a lot closer by parsing the date string if and only if it contains at least one digit. Code to enforce that is added to DATEVALUE and TIMEVALUE, and appropriate tests are added.
* Failure for Php 7.4 Linux
After not seeing any such problem for many months, this is the second day in a row where result is different on Windows than Linux with no apparent reason to think why that should be the case. At any rate, easily solved.
* Fix Unintential Deprecated Calls in Tests - DATABASE
I think it's best to install these before PR #3166. There are no changes to source code, only to test members which continue to inadvertently use calls to deprecated functions.
* Fix deprecation Blocks
Deprecated->deprecated, adjust see and comments
* Fix Deliberate Deprecated Tests
Add annotations.
* Change Unit Tests to Run in Spreadsheet Context
... rather than as direct calls. The major difference is that specifying an invalid column should result in an Excel error, not null. Minor code changes were needed, including to Statistical/Conditional which sometimes calls Database.
* Correct Some DocBlocks
Null is no longer a possible output for most of these functions.
* 2 Overlooked Tests
Change to run in spreadsheet context.
* T Function Should Return Null-String, not Null
Fix it.
Allow default values for TEXTJOIN() delimiter and ignore_empty... even though the official MS documentation lists them as required arguments, they are actually optional
* Keep Calculated String Results Below 32K
This is the result of an investigation into issue #2884 (see also PR #2913). It is, unfortunately, not a fix for the original problem; see the discussion in that PR for why I don't think there is a practical fix for that specific problem at this time.
Excel limits strings to 32,767 characters. We already truncate strings to that length when added to the spreadsheet. However, we have been able to exceed that length as a result of the concatenation operator (Excel truncates); as a result of the CONCATENATE or TEXTJOIN functions (Excel returns #CALC!); or as a result of the REPLACE, REPT, SUBSTITUTE functions (Excel returns #VALUE!). This PR changes PhpSpreadsheet to return the same value as Excel in these cases. Note that Excel2003 truncates in all those cases; I don't think there is a way to differentiate that behavior in PhpSpreadsheet.
However, LibreOffice and Gnumeric do not have that limit; if they have a limit at all, it is much higher. It would be fairly easy to use existing settings to differentiate between Excel and LibreOffice/Gnumeric in this respect. I have not done so in this PR because I am not sure how useful that is, and I can easily see it leading to problems (read in a LibreOffice spreadsheet with a 33K cell and then output to an Excel spreadsheet). Perhaps it should be handled with an additional opt-in setting.
I changed the maximum size from a literal to a constant in the one place where it was already being enforced (Cell/DataType). I am not sure that is the best place for it to be defined; I am open to suggestions.
* Implement Some Suggestions
... from @MarkBaker.
* PHP8.1 Deprecation Passing Null to String Function
For each of the files in this PR, one or more statements can pass a null to string functions like strlower. This is deprecated in PHP8.1, and, when deprecated messages are enabled, causes many tests to error out. In every case, use coercion to pass null string rather than null.
* TextData - Minor Changes, Test Coverage
Per agreement on a previous push, I looked into standardizing the initialization of the TextData functions (like Engineering and MathTrig), with particular regard for avoiding multiple later null coercions. This simplifies the code quite a bit. This PR also increases coverage to 100% for all TextData modules. All entries in Phpstan baseline for non-deprecated TEXTDATA functions are removed. There were some minor bugfixes.
Whereas Excel (and Gnumeric) treat booleans when supplied as strings as 'TRUE' or 'FALSE', ODS treats them as '1' or '0'. Unlike Excel, ODS generally does not allow bool for int arguments; it does, however, allow them for FIND and SEARCH. ODS allows boolean for into for SUBSTITUTE even though Excel doesn't. ODS allows bool for string for NUMBERVALUE and VALUE even though Excel doesn't. ODS accepts 0 as an argument for CHAR; Excel doesn't. Most of this seems like random decisions on the part of the developers; I've done my best to follow the products in each case. There is a new test member devoted to ODS tests.
Gnumeric has an anomaly vis-a-vis the others - if length is supplied to LEFT/MID/RIGHT as null, Gnumeric treats it as 0 rather than 1.
All tests now take place in the context of a spreadsheet ...
Except for RETURNSTRING, which is not the implementation of an Excel function, and is referred to in the rest of PhpSpreadsheet only in the unit tests for itself. It should probably be deprecated, but that is not part of this PR, just in case there is some reason for it that I couldn't discern.
I have tried to make the first line of each doc block identify the Excel function name rather than its name in PhpSpreadsheet. I think it makes things more comprehensible.
Some tests call Settings::setLocale, but there was no Settings::getLocale. At the end of the tests which do it, they invoke setLocale('EN-US'), which, in a practical sense, is sufficient. However, in theory it would be better for them to get the current locale before changing it, then changing it back to the original when the time came. I have added getLocale and made the appropriate testing change.
The CHAR function took an interesting turn. One can set the value of a cell to, say, CHAR(2), the ASCII/UTF-8 representation of a control character, which is not legal in certain contexts. The only Reader/Writer that could handle this without problems is Xls, which deals with binary data all the time. However, if you tried to write it to Xlsx, Excel would not be able to open the resulting file because of what it considers an illegal character. I changed the Xlsx writer to escape such characters when writing the value of a string function. I did not make any other changes to the Xlsx writer - it seems to me that setting a cell to CHAR(2) is legitimate, but setting it to say `"\x02"` seems less likely to be legitimate, so the latter will still fail (although `="\x02"` should work). The Xlsx reader already supports the escape mechanism that I added to the writer.
CHAR control character and Ods - not supported by either Reader or Writer. I did not attempt to add this now. There is lots still missing from ODS, and this item just can't be a high priority amongst all of those.
CHAR control character and Csv - it is supported by reader and writer if the file has a csv extension. However, trying to guess the mime type without an extension - the control character makes mime_get_type guess application/octet-stream, and PhpSpreadsheet therefore thinks that Csv can't read it.
CHAR control character and Html. Actual use of the control character in the file is subject to the same problems as Xml (i.e. Xlsx and Ods). It wasn't terribly difficult to get the Html Writer to change `"\x02"` to "``". I believe that this is technically legal; however, DOMDocument.loadHTML rejects it as an illegal entity, and I am not convinced that it is wrong to do so, so I haven't changed the Html writer.
* Scrutinizer
Correct 3 minor errors.
* First steps toward refactoring Statistical Distributions into smaller classes: BETA() and GAMMA() (and related functions) to start with... they all need a lot of tidying up, and more testing; but it's a start
* Add basic datatype validations to Beta and Gamma Excel function implementations
* Switch to using a trait with the validation methods to provide easier sharing between distribution classes
* Additional unit tests for Beta and Gamma functions, including unhappy path for validations
* Extract ChiSquared functions
* Additional argument validation checks with unit tests for Chi Squared functions
* Extract Fisher
* Move MEDIAN() and MODE() to the Averages class
* Extract filters for Median and Mode for common usage
* Problems Using Builtin PHP Functions Directly As Excel Functions
This fixes issue #1789.
As originally reported, stricter typing was causing PHP8 to throw
an exception when a non-numeric value was passed to the Round function.
Previous releases of PHP did not see this problem, however, on further
analysis, they were also incorrect in returning 0 as the result in the
erroneous situation, when they should have been returning a VALUE error.
Yet more analysis showed that other functions would also have problems,
and, in addition, might not handle invalid input (e.g. a negative length
passed to REPT) or output (e.g. NAN in the case of ACOS(2)) correctly.
The following MathTrig functions are affected:
ABS, ACOS, ACOSH, ASIN, ASINH, ATAN, ATANH,
COS, COSH, DEGREES (rad2deg), EXP, LN (log), LOG10,
RADIANS (deg2rad), REPT (str_repeat), SIN, SINH, SQRT, TAN, TANH.
One TextData function (REPT) is also affected.
This change lets PhpSpreadsheet validate the input for each of these
functions before passing control to the builtin, and handle the output
afterwards.
There were no explicit tests for any of these functions, a fact made
easy to ignore by the fact that PhpSpreadsheet delegated the heavy
lifting to PHP itself for these cases. A full suite of tests is
now added for each of the affected functions.
* Scrutinizer Recommendations
Only in 3 modules which are part of this PR.
* Improved Handling of Tan(PI/2)
Return DIV0 error for TAN when COS is very small.
* Additional Trig Tests
Results which should be infinity, i.e. DIV/0 error.
This had been intended to get 100% coverage for TextData functions, and it does that.
However, some minor bugs requiring source changes arose during testing.
- the Excel CHAR function restricts its argument to 1-255. PhpSpreadsheet CHARACTER
had been allowing 0+. Also, there is no need to test if iconv exists,
since it is part of Composer requirements.
- The DOLLAR function had been returning NUM for invalid arguments. Excel returns VALUE.
Also, negative amounts were not being handled correctly.
- The FIXEDFORMAT function had been returning NUM for invalid arguments. Excel FIXED returns VALUE.
- Fix ISFORMULA() function to work with a cell reference to another worksheet
- Added calculation engine support for the new functions that were added in MS Excel 2013 and MS Excel 2016
- Text Functions
- CONCAT() Synonym for CONCATENATE()
- NUMBERVALUE() Converts text to a number, in a locale-independent way
- UNICHAR() Synonym for CHAR() in PHPSpreadsheet, which has always used UTF-8 internally
- UNIORD() Synonym for ORD() in PHPSpreadsheet, which has always used UTF-8 internally
- TEXTJOIN() Joins together two or more text strings, separated by a delimiter
- Logical Functions
- XOR() Returns a logical Exclusive Or of all arguments
- Date/Time Functions
- ISOWEEKNUM() Returns the ISO 8601 week number of the year for a given date
- Lookup and Reference Functions
- FORMULATEXT() Returns a formula as a string
- Engineering Functions
- ERF.PRECISE() Returns the error function integrated between 0 and a supplied limit
- ERFC.PRECISE() Synonym for ERFC
- Math and Trig Functions
- SEC() Returns the secant of an angle
- SECH() Returns the hyperbolic secant of an angle
- CSC() Returns the cosecant of an angle
- CSCH() Returns the hyperbolic cosecant of an angle
- COT() Returns the cotangent of an angle
- COTH() Returns the hyperbolic cotangent of an angle
- ACOT() Returns the cotangent of an angle
- ACOTH() Returns the hyperbolic cotangent of an angle
- Financial Functions
- PDURATION() Calculates the number of periods required for an investment to reach a specified value
- RRI() Calculates the interest rate required for an investment to grow to a specified future value
By design, UTF-8 allows any byte-oriented substring searching algorithm,
since the sequence of bytes for a character cannot occur anywhere else
([source](https://en.wikipedia.org/wiki/UTF-8#Advantages_3)).
So `str_replace()` also works for UTF-8-encoded strings, assuming that
the input strings are valid UTF-8 strings. The previous implementation
of mbStrReplace() did nothing to detect invalid strings.
Also, `str_replace()` does not support [Unicode equivalence](https://en.wikipedia.org/wiki/Unicode_equivalence),
but nor do the other `mb_string` functions, and nor does `=SUBSTITUTE()` in Excel
(tested on Excel for Mac version 15.19.1, Excel 2016 for Windows and LibreOffice 5.1).
Closes#109
oleibman
Adrien Crivelli
MarkBaker
oleibman
MarkBaker
Rolands Usāns
Christian Schmidt