Fix#1324, which went stale many years ago, and is now re-opened. It can be considered a follow-on to PR #4479 which was installed in May. The user complains of a *very* small loss of precision. This can be corrected by using `StringHelper::convertToString` in lieu of a simple cast from float to string in two places. `Style\NumberFormat\Formatter` is quite straightforward. `Writer\Csv` is a bit more complicated. If you put a float with more than 15 digits precision in a Csv, on opening it Excel will give a pop-up, saying something inaccurate like "We've corrected this for you - is that okay?" Better to avoid that by making sure to avoid too much precision when writing to Csv.
In issue #4557, the user complains, with some justification, about the way Excel handles certain calculations. We are not able to help with that problem. However, the user also notes a problem in Shared/Date when `isDateTime` has to evaluate a cell whose calculated value is an array. This is solved by flattening the calculated result to a single value.
It became obvious while working on this change that the code to set `instanceArrayReturnType` was kind of awkward. Simpler methods `returnArrayAsArray` and `returnArrayAsValue` are added to `Spreadsheet`. Even these started out a bit awkward because `Spreadsheet::calculationEngine` was defined as nullable, which really isn't true. It is allocated by the constructor, and never freed except in the destructor. It is no longer nullable.
Fix#347, which had gone stale but is now reopened. PhpSpreadsheet is automatically replacing CR-LF and CR with LF. The reason for this is unknown, although I suggest a possibility in the issue. I am not willing to make this a breaking change. This PR adds a new property `preserveCr` with setter and getter to DefaultValueBinder (and, by extension, to StringValueBinder and AdvancedValueBinder). The property defaults to false, but users who wish to preserve CR-LF and CR can set it to true for a Spreadsheet or a Reader.
Dealing mostly with mixed variable type. In a great many cases, problems can be resolved replacing `getValue` with `getValueString` or `getCalculatedValue` with `getCalculatedValueString`.
We do not recommend it, but users can call the Php function `setlocale` and that might affect the character used as currency symbol. A problem arises when the caller to setlocale does not specify a character set - in that case, Php will attempt to return its `localeconv()` values in a single-byte character set rather than UTF-8. This is particularly problematic for currency symbols. PhpSpreadsheet till now has accepted such a character, and that can lead to corrupt spreadsheets. It is changed to validate the currency symbol as UTF-8, and fall back to a different choice if not (e.g. EUR rather than 0x80, which is how the euro symbol is depicted in Win-1252).
An additional problem arises because Linux systems seem to return the alternate symbol with a trailing blank, but Windows systems do not. To allow callers to get a consistent result, a parameter is added to `getCurrencyCode` which will trim or not (default) the currency code.
With PhpUnit 10 came the ability to use Php attributes rather than doc-block annotations for things like "data provider". PhpUnit 11 deprecates the use of annotations, and PhpUnit 12 will not not permit their use. Since PhpUnit 11 requires Php8.2+, we cannot adopt it as long as we support Php8.1, which will continue to be the case for some time. However, there is no penalty for early adoption.
Php-cs-fixer can use:
```
'php_unit_attributes' => ['keep_annotations' => false],
```
This allows us to run `composer fix` to automate all the needed changes. No manual changes were needed for any of the test members.
With this change, PhpUnit 9 can no longer be used with the test suite. File composer.json is updated to reflect that reality, and phpunit9.xml.dist, which has been supplied in case anyone needed to use PhpUnit 9, is no longer required, and is thus deleted. For now, PhpUnit 11 is not being added as a possibility.
No source code is changed in this PR.
The latest release seems to not want you to give a class element both a Php type and a doc-block type. I used the "fix" operand to delete the redundant doc-block declarations, with no other changes. So there should be no change to executable code.
Fix#3900. The code to adjust Decimal and Thousands separators to user-specified choices expects to convert periods to commas and vice versa - hard-coded without taking the user specification into account. This is likely to be the only significant use case, however, as the issue demonstrates, it is not the only possibility. In particular, it will mess up Csv output if decimal separator is kept as period, but thousands separator is set to null string. The code is altered to replace period with the user-selected decimal separator, and comma with the user-selected thousands separator. No existing tests broke as a result of this change in behavior.
This PR resolves about half the statements which we tell Phpstan to ignore. There will still be 23 such annotations spread over 10 source modules for various reasons (too complicated, suspect PhpSpreadsheet code, one Phpstan bug), plus some deliberate errors in the test suite.
I check from time to time. There are a number of problems now, mostly due to the elimination of Php 7.4 and replacement of doc-block typing with explicit Php typing.
- Bitwise functions were particularly affected by PR #3718 and PR #3793.
- Chart/Axis and Writer/Xlsx were amusingly affected by PR #3836, which added a scaling option which included an array indexed by the known allowable factors, one of which is 1 trillion, which cannot be represented as an integer on a 32-bit system. Issue3833Test, introduced by the same PR (and not suffering any errors) was expanded to test this value.
- Some minor changes to Reader/Xls and Shared/OLE/PPS to accommodate hex values which are negative in 32-bit but which Php-32 may wind up casting to large floating point numbers; it is not clear to me why these hadn't shown up as problems previously. Possibly this is the result of changes in the most recent Php versions.
- BitAndTest, BitOrTest, BitXorTest and Shared/DateTest were adversely affected by PR #3859 when arguments and/or expected results too large for a 32-bit integer were supplied.
- ImExpTest required a slightly reduced precision for 32-bit. No idea why this hadn't shown up earlier.
Change "mixed" declarations to more accurate types in test members; in particular, change those that would be flagged if we were to run Phpstan at level 9 (we currently run level 8). I may or may not follow up with source code (over 700 level-9 problems remain for src), but, as with strict typing, there is no reason to avoid the effort for test members.
It was necessary to update some doc blocks in src to accommodate this change. However, no executable code is touched.
Fix#2809. User felt font files should be searched recursively. The problem could be overcome merely by specifying a more precise font location. However, user has a point - the file layout on user's system is pretty common, and can be integrated into PhpSpreadsheet code easily. As a bonus, the DocBlock for setTrueTypeFontPath, which the user felt was misleading, doesn't even have to change.
This was suggested by the investigation of issue #3811. No fix is necessary for the issue. However, two possible code solutions (Php setlocale, which comes with certain design flaws, and StringHelper set(Decimal/Thousands)Separator were suggested, and neither is adequately tested. This PR adds such tests.
Unusually, getting StringHelper Decimal Separator, Thousands Separator, and Currency Code can result in a change to those properties. So, the existing design in several tests where those properties are captured in Setup and restored in Teardown do not work quite as designed. Instead, the ability to set those properties to their default value (null) is added, and the tests re-done to restore the default in Teardown.
The two methods yield the same results when parsing input. However, they diverge when examining output fields through `getFormattedValue`. Such output is currently correct (usually) when using setlocale, but not when using StringHelper. The former works through the 'trick' of using `sprintf(%f)`, which generates a locale-aware string. However, using non-locale-aware `sprintf(%F)` followed by `str_replace` will produce the correct result for both setlocale and StringHelper. One place in the code uses a cast to string, which is incorrect for both methods. Following that up with the same str_replace makes it correct for both. These changes permit, but do not require, the user to avoid setlocale altogether.
It remains an open question whether Settings/Calculation::setLocale should set DecimalSeparator, CurrencySeparator, and CurrencyCode. That makes logical sense, but it would be a breaking change, and having to explicitly set those values when using setLocale does not seem especially burdensome. For now, such a change will not be made.
While we might never be able to have 100% of our code strict, we can at
the very least do it for all of our tests. This ensures that our tests
are using our API with the types as intended by the test author, and not
silently be cast to what our API requires.
And quite a bit more manual changes. The idea is that typing of our
tests can be a bit more loose, so we assume PHPDoc is mostly correct. If
that happens to be wrong, it should be caught by the tests themselves.
* Check For and Eliminate Octal Literals
Php8.1 introduced a new form of octal literal (0o123); the old form (0123) continues to be supported, but can certainly be misleading. There are exactly 6 uses of the old form in the code base (5 in tests, 1 in infra). 4 of these are clearly unintended (02 or 03), and the leading 0 should just be dropped for those. The other 2 are for file permissions and are easily replaced.
* Minor Changes to LocaleGenerator
Testing the permissions change exposed some problems when the locale files don't exist. This never shows up as an issue when the files do exist, which is pretty much all the time, but correct the theoretical exposures anyhow.
* Populate bg and en_uk Locale Files from Spreadsheet
Also rename locale generator so that it is executed first. This is when it should happen because other tests depend on its results.
* Some Bulgarian Function Omissions
Figured out a reasonable way to compare old Bulgarian function file to new; this identified a handful of now-corrected omissions and errors. Note that JIS function, in old list, is not on spreadsheet, but the Bulgarian translation is the same as English (so the translation isn't needed), and the function has been replaced by DBCS, which is also not on the spreadsheet.
* Preserve Transparency in Memory Drawing
Fix#3626. User felt that Exact setting of column width should not add padding. In order to avoid making this a breaking change, I am adding a new boolean property `usePaddingForExact` with setter and getter to Shared/Font; default is false (current behavior).
* Change Parameter
Instead of bool, make it null|float|int, allowing user to set whatever is deemed appropriate.
Successor to PR #3523. There are 494 single-line changes (`public function provider` to `public static function provider`) in this PR. None of these were made manually; they were all created with the following script (adapted from
https://stackoverflow.com/questions/25909820/how-to-recursively-iterate-through-files-in-php):
```php
$dir = 'C:/git/unit10prep2/tests/PhpSpreadsheetTests';
$it = new RecursiveDirectoryIterator($dir);
// Loop through files
foreach(new RecursiveIteratorIterator($it) as $file) {
if ($file->getExtension() === 'php') {
$contents = file_get_contents($file);
$new = preg_replace('/public function (\\w*)([Pp])rovider/', 'public static function $1$2rovider', $contents);
if ($new !== $contents) {
echo "changing $file\n";
file_put_contents($file, $new);
}
}
}
```
After this PR, there will be one more, with a small number of test changes, and enabling PhpUnit 10 for Php 8.1+.
* PhpUnit 10 Compatibility Part 1
This is not a change to move to PhpUnit 10. There is no compelling reason to do so at this time, although it is bound to happen eventually. There are a staggering number of problems (somewhere around 3,000) with the current test suite under PhpUnit 10; this is an attempt to get ahead of the curve by addressing them now.
Method `setOutputCallback` has gone away. This affects only Helper/SampleTest. It appears that `ob_start` and its allies provide an effective equivalent. FWIW, the absence of `setOutputCallback` is a good indication of whether or not PhpUnit 10 is in use, and I will use that fact in a few tests.
Class `ComplexAssert` with no constructor, and always used with `new ComplexAssert()`, extends `TestCase`. Apparently, the constructor for TestCase requires an argument, and PhpUnit 10 complains about not supplying one. Adding an empty constructor to ComplexAssert avoids this problem.
There are two very minor source changes, to Calculation/Calculation and Reader/Xlsx, where problems were exposed with PhpUnit 10 that had not been previously been exposed. AFAIK, these are the only source changes required; the rest of the changes are to test members.
The bulk of the problems are because PhpUnit 10 insists that provider methods be static. Most of those can be changed by a script without any further action; those changes will constitute the Part 2 counterpart of this PR. In this PR you will find the exceptional cases that can't be automated for one reason or another. The tests for Database functions have mild complications that are easily handled. Most of the other provider changes in this PR are because the method names didn't follow an established pattern ('provider' isn't part of the method name); those are also easily handled manually. Modifying the following tests provided significant challenges:
- Writer/Xls/WorkbookTest testAddColor
- Worksheet/Table/TableTest testSetRangeValidRange
The handling of warning messages issued by the code differs in PhpUnit 10. According to the change log, "This means that using PHP functionality which triggers E_DEPRECATED, E_NOTICE, E_STRICT, or E_WARNING or calling code which triggers E_USER_DEPRECATED, E_USER_NOTICE, or E_USER_WARNING can no longer hide a bug in your code." To me, the effect of that change seems to be exactly the opposite - such messages were available to the test with PhpUnit 9 (so we could test for them), and are no longer available (so we can't). I haven't even succeeded with a custom error message handler as part of the script. I will continue to investigate, but, for now, will skip some tests under PhpUnit 10 for the following:
- Shared/OleTest testChainedWriteMode and testChainedBadPath
- Reader/Html/HtmlLoadStringTest testLoadInvalidString
- Reader/Html/HtmlTest testBadHtml
* Scrutinize, and Parent Construct
Parent construct suggested by @MarkBaker.
* Redo Tests Dependent on Warning Messages
Warning (and other) messages are handled differently in PhpUnit 10 than in earlier versions.
Wanting a sanity test for ExactFontTest, I used the MD5 of a known file. However, there are various versions of the file in the wild, and use of these variants cause test to be skipped. Allow known variations to satisfy MD5 check.
* More Coverage In Unit Tests
Minimal source code changes.
* Scrutinizer
One legitimate complaint and one from out of left field.
* Scrutinizer Lunacy
If this doesn't stop its complaint, I give up.
* Scrutinizer - Whatever
Try again.
* Glutton For Punishment
Try again.
Fix#3405. Autosize is definitely not working well with CJK characters (column is not wide enough). User reports a workaround using `mb_strwidth` to calculate and set the column width. PhpSpreadsheet uses `mb_strlen` for width calculations. Change it to use mb_strwidth instead. For non-CJK strings, the results will be identical (and there are already unit tests on such strings which assert the expected results, and these tests did not need to change). For CJK strings, the results will be wider. The string I'm using to test comes from the issue. It currently results in a column width of 30.564. When I open the resulting sheet in Excel and auto-fit the column width, the width winds up as 43.00. So, as long as the computed width exceeds 43.00, the spreadsheet will show the full cell. With the new calculation, the computed width is 55.2722, satisfying our condition. This is wider than expected, but that is generally true for this type of computation. For example, for 'abcdefghijklmnopqrstuvwxyz', the computed width (before and after this change) is 31.7065, but Excel auto-fit actually uses 24.73.
Disappointingly, "exact width calculation" does not solve this problem. It does seem to do a little better than "approximate" for non-CJK, but its CJK calculation is not wide enough. This might or might not indicate a bug in Php function `imagegetttfbbox`; I do not know enough about it to report a bug. Anyhow, since we're dependent on that result, there is no equivalent in this case for swapping mb_strlen out for mb_strwidth.
* Resolve Phpstan Messages - FINALE - Shared/OLE
Reduce number of Phpstan messages by addressing their issues. This is the last of a series of related tickets to achieve that end. Only a handful of messages will be left in Phpstan baseline after this change is merged.
Shared/Ole.php and Shared/Ole/ChainedBlockStream.php were completely uncovered in the test suite. So I wrote some tests and found they didn't work at all. There were 2 problems. The first was that ChainedBlockStream checks for a mode of `r`, but Ole invokes it with mode `rb`. ChainedBlockStream will now just check the first character of the mode.
The second was a more interesting problem. Ole was reading "an *unsigned* long". However, what it should have been reading was "a 32-bit *signed* long". The irony of this mismatch is that, once the mode problem was corrected, 32-bit Php handled Ole and ChainedBlockStream correctly, but 64-bit Php did not. The code is corrected to work properly for both 32- and 64-bit.
* Scrutinizer
2 dead assignments - sufficient to just make calls which are expected to fail.
* Resolve Phpstan Messages Reader/Xls Shared/Escher
Reduce number of Phpstan messages by addressing their issues.
* Scrutinizer
Hyperactivity.
* More Scrutinizer
Getting close.
* Even More Scrutinizer
Closer and closer.
oleibman
Marc Bennewitz
Adrien Crivelli
Mark Baker