We currently use Phpstan Release 1. Release 2 has been available for some time, and we would like to stay current if possible. In order to get to Release 2, we first have to pass the "bleeding edge" tests for Release 1. I will do this by generating a new baseline using bleeding edge, then fix the baseline errors over several different PR's. This is the first of those.
When the Dynamic Array PR #3962 was introduced, it left the default as Return Array as Value. At some point, the default should be changed to Return Array as Array. This would, of course, be a breaking change, one which will not be part of Release 4. However, it will possibly be part of Release 5.
Rather than relying on the default setting, this PR explicitly sets Return Array as Value when tests require that setting. This will make it easier to identify potential breaks when the default is changed. The entire test suite will now succeed with either setting as default.
In making these changes, a few minor problems were discovered with how Array as Array is handled. These are fixed with this PR.
Fix#4129. Fix#4168. Html Writer, which all the Pdf writers use, defines its charts and drawings (henceforth I will just use charts for this discussion) using position:absolute and z-index. Browsers handle this correctly, but none of the Pdf writers do, and I can't think of an alternative method of styling them. The result is that the charts take up too much or too little room on the Pdf.
I suggested in the two discussions that treating the areas covered by the charts as merged cells might mitigate the problem. I think there are too many unknowns to do so automatically (and see next paragraph). However, adding to Spreadsheet new methods `mergeChartCellsForPdf` and `mergeDrawingCellsForPdf` allows the end user to do this if desired. The new methods are exercised for charts in samples/Chart/32_Chart_read_write_PDF, the results of which are much improved as a result. New samples/Pdf/21f_Drawing_mpdf does likewise for drawings.
The new methods alter the spreadsheet they are working on, which could be a problem if you still wish to work with the spreadsheet after writing it to Pdf. In that case, making a copy of the spreadsheet, then calling the new methods on the copy, and writing the copy to Pdf is probably best.
Custom properties would be better in this context using j (no leading 0) rather than d (leading 0) for day of month.
Properties has the same problem, and also needs backslashes before p to make it part of a <sup> tag.
The native preg functions (preg_match, preg_replace, etc.) often require us to add a lot of useless boilerplate code to satisfy Phpstan, Scrutinizer, etc. Composer/Pcre offers us a way to remove that boilerplate, thereby giving us a cleaner codebase. I decided to try it on a few modules, and saw a result that clearly demonstrated the usefulness of doing this (aside from the cleaner codebase).
Sample 22_Reader_issue1767 reads an Xlsx spreadsheet with complex sheet names used in defined names, and writes it to Xlsx and Xls output files. When I changed Writer/Xls/Parser to use Composer/Pcre, this sample failed in many different places writing the Xls file. It turns out that some regexes were failing not because the string didn't match, but because the regex encountered "catastrophic backtracing". Composer/Pcre throws an exception when this happens; the native preg_match does return false, but we were not checking for that. The regexes in question are now changed to something which works, and formal unit tests are added for them. Finding this previously undetected error indicates that we should proceed with this change.
An alternative to using Composer/Pcre would be to test for false after all the preg calls. I have done this in the two samples changed with this PR. That seems adequate for a small number of changes, but it really just makes for more clutter considering the large number of regexps that we use in our code. I think Composer/Pcre is a better choice.
It isn't quite transparent. Composer forces all regexps to use PREG_UNMATCHED_AS_NULL, so some match fields will now be null instead of null-string (or non-existent if the unmatched field comes at the end). Our test suite doesn't report any problem (yet) due to this change, although Phpstan is sensitive to it. Several Phpstan annotations were eliminated due to this change, but some others are now needed.
It is not necessary to do this all at once. This PR addresses all the calls in Writer. I intend to address other components in several tickets.
* Validate Post Item in Convert-Online.php
* Correct Samples - Currency, Accounting, and Downloader
* Smarter Color Formatting for Currency
* Use helper->log Rather than echo in Convert-Online
Responding to comment from @PowerKiki about use of echo for error messages. I do not believe that an exception is warranted, but other scripts use helper->log for error messages, and now so will this one.
Fix#4241. Some security batches caused a minor break in Drawings, forcing `setWorksheet` to come after `setPath`. Although the problem is easily fixed in user code, this was not an intended change. Some slight recoding restores the earlier functionality where the order of calls was not important, without sacrificing the security gains. This change will be back-ported to the other active branches to which the security patch had been applied.
* Validate POST Input in Sample 45
Errors will result if inputs are not numeric.
* Update 45_Quadratic_equation_solver.php
---------
Co-authored-by: Owen Leibman <owenleibman@fico.com>
See discussion #4117. This PR implements image transparency, for Xlsx Reader and Writer, Html Reader, and Html/Dompdf/Mpdf Writer. (Mpdf treats 100% opacity as if it were zero, but it's otherwise okay and that would be an unusual choice anyhow.) A new property `opacity` with getter/setter is added to BaseDrawing (and therefore Drawing and MemoryDrawing). Although the Excel UI lets you set the image *transparency*, the value stored in the Xml is actually its *opacity* (expressed as an integer between 0 and 100,000). Likewise Html/Css lets you set opacity (as a float between 0 and 1). PhpSpreadsheet lets you set it as it is stored in Xml - so a value of 40,000 would indicate 40% opacity = 60% transparency.
In the course of testing, some problems with Html Reader presented themselves. They are corrected as part of this ticket:
- Web page title will no longer be "Untitled Spreadsheet" if a title tag is present in the header section of the html.
- If a class attribute is used with the table tag representing a worksheet, showGridlines and printGridlines will be set according to whether gridlines and/or gridlinesp are among the class names.
- Reader has been performing urldecode on the src attribute of the img tag. This breaks data url's by converting plus signs to spaces. It will now perform urldecode only for non-data url's.
- If height and width are not used as attributes on the img tag, they will be extracted from the style attribute if they are present there.
Fix#4125. Currency and Accounting Wizards generate styles for ISO codes, but these are incorrect and cause a problem when Excel tries to open a spreadsheet containing these styles. Debugging that problem, other problems with Wizards came to light:
- Currency Wizard should permit four different styles for negative numbers (as Excel does) - minus sign, minus sign and red font, paretheses, and parenthese and red font. It currently uses only minus sign.
- Accounting Wizard should use parentheses for negative numbers (as Excel does). It currently uses minus sign.
- Accounting Wizard should always use SYMBOL_WITH_SPACING (as Excel does). It currently permits the use of SYMBOL_WITHOUT_SPACING. What WITH_SPACING really does is to ensure decimal-point alignment among adjacent cells in a column with the same format.
- Currency Wizard should always use SYMBOL_WITHOUT_SPACING (as Excel does). It currently permits the use of SYMBOL_WITH_SPACING.
I am correcting these problems by:
- renaming Currency Wizard to CurrencyBase
- adding a `negative` property with setter to it and its constructor.
- adding a new Currency which extends CurrencyBase, always using SYMBOL_WITHOUT_SPACING when formatting.
- having Accounting extend CurrencyBase rather than Currency, always using SYMBOL_WITH_SPACING and NEGATIVE_PARENS when formatting.
- CurrencyBase can be used if the restrictions on Currency and Accounting are not desired (e.g. the suggested accounting constant from [this unimplemented PR](https://github.com/PHPOffice/PhpSpreadsheet/pull/1576)).
Excel does some funny stuff with these formats. In particular, it might try to guess if you have a particular Accounting format in mind. So the Accounting wizard for dollar sign generates a format which (a) matches FORMAT_ACCOUNTING_USD, and (b) Excel (correctly) interprets as an Accounting format for symbol $. On the other hand, the Accounting wizard for euro sign generates a format which (a) matches FORMAT_ACCOUNTING_EUR, but (b) Excel interprets as a custom code rather than an Accounting format. This in itself is not a particularly big deal, but it has made it impossible for me to see exactly what format Excel uses for trailing currency symbols for negative numbers. I can't get them to decimal-point align with positive numbers if I put any kind of space between the trailing parenthesis and the currency symbol, so I omit that. It doesn't look terrible, and it keeps everything aligned, but it might not be what people are used to.
I've also changed the formatting to use spaces rather than non-breaking spaces. They seem to work just fine, and the constants mentioned above use them rather than nbsp.
Fix#4124. Currency formats that contain an ISO currency code which contains one of the characters used to recognize a date format (hmsdy), e.g. [$HUF], are being formatted by PhpSpreadsheet as dates rather than currencies. Code is changed to recognize open bracket followed by dollar sign followed by 3 Latin alphabetic characters followed by close bracket as a non-date.
While researching another problem, I noticed that font color was not working as expected for Xls Conditional Formats, at least not when a "non-standard" color is used. In such cases, the color might wind up being rendered as black. The reason is as follows. Xls Writer includes a color palette which is dynamically generated from the (non-Conditional) styles used in the workbook. Any colors used in the workbook are indexes to this dynamic palette. However, Conditional colors use a static palette found in class ColorMap to determine the index, so the determination of index will often not find a match, and, if a match is found, it is not necessarily correct. (Also, the ColorMap method was case-sensitive and needs to be insensitive.)
In order to correct this, the `addColor` method in Xls Writer Workbook needs to be accessible to the Conditional logic which is found in Xls Writer Worksheet. This is accomplished by passing the Workbook in the Worksheet's constructor, and changing the method to public, and changing Conditional Font to use this method rather than ColorMap.
The logic for Conditional Fill colors is similarly changed. Although Xls Conditional Fill has appeared to just not work, I was finally able to figure out the problem. Excel Xls Conditional Fill with fill type Solid requires that the fill color be specified as startColor, and that endColor be omitted. Our conditional samples used endColor, and are now changed to use startColor instead; the same is true for our online documentation, and for some tests. Xlsx continues to work as expected, and now Xls does at least some of the time. If the condition is one that Excel Xls does not recognize (e.g. cell contains), it will, of course, not work. A surprising situation that also doesn't work is the use of ISODD or ISEVEN in formulas. Those are "add-in functions" which are handled differently than other functions, and I'm not sure how to support them. I will document this in issue #3403.
Samples 08_Conditional_Formatting(_2) had produced corrupt Xls versions. This turned out to be because the code was using hash codes to avoid having to write out duplicate conditionals; this is often a good idea, but not in this case. Allowing the duplicates fixes the corruption problem.
Conditional Border colors also ought to figure in this change, but the current code does not support Border colors, and I have not yet been able to figure out how to implement it (BIFF format can be very messy to figure out).
With this change, I could delete ColorMap altogether. However, it is a public class with a static public method, so maybe someone is using it for a purpose I'm not familiar with. I will just deprecate it.
This is according to our formal, published, policy to only support
EOL PHP after 6 months.
See https://phpspreadsheet.readthedocs.io/en/latest/#php-version-support
Also consolidate documentation in a single place, not in the README,
because it's a pain to maintain almost-duplicated-but-not-really
documentation.
Fix#3951. When an Excel sheet is protected, even when sorting is explicitly allowed without a password, sorts are permitted only on "protected ranges" within the sheet. PhpSpreadsheet already supports protected ranges, and only minor tinkering is necessary for that (e.g. the protected range can have, but does not require, a password). The more important part of this change is documenting the far-from-intuitive way that Excel handles this. To that end, documentation is updated, and a new sample is added.
A new class, `Worksheet\ProtectedRange` is added in place of the string array which had been used. `Worksheet::getProtectedCells` is deprecated in favor of the new `Worksheet::getProtectedCellRanges`.
The samples have become unwieldy when running them from a browser. In particular, the drop-down lists are fixed size with no scrolling, and many of them are now just too large. I have moved all the Calculation samples up a level, and broken several categories (Basic, Chart, DateTime, Engineering, Financial, and Reader) into several pieces.
Convert-Online (now found in the Engineering category) had a number of different problems which are now resolved. It is the only member with any significant code change.
A continuation of PR #3859. Change code that would be flagged if we were to run Phpstan at level 9 (we currently run level 8). I may or may not follow up with source code (454 level-9 problems remain for src), but there is no reason to avoid the effort for samples.
No changes are made to src.
Fix#3833. Among the options for chart axes in Excel are the ability to show labels as multiples of specific powers of 10, with or without an explanatory label. The Excel spec seems to indicate that that you can customize the multiples to other values, but I don't see how to do that in Excel, so that can be a project for another day if someone figures out how. The label title can also be styled; that is not part of this PR, but I will look into it in future. For now, this PR supports the use of multiples with or without labels.
The axes can also be displayed on a logarithmic scale (any base between 2 and 1000). This PR supports that.
* Let Phpstan Run on Samples
Phpstan currently analyzes all source and test members. We already run phpcs and php-cs-fixer on samples as well. I would expect that samples are often used as templates for code in userland; it behooves us to be at least as careful with those members as for the others which are already being analyzed. Aside from 1300+ messages `Variable $helper might not be defined.`, which will be suppressed in phpstan.neon.dist, there are really only a few changes needed for sample members, so that part of the code base was already in good shape, and is now even better. No annotations were needed.
* Scrutinizer 2 out of 3
1 false positive, now suppressed; fix other 2.
* Remove Dead Code
* Very Minor Changes
* Add infra
* Chart Dynamic Title and Special Font Properties
Fix#3797. Excel allows a Chart Title to be a formula, albeit a very rigidly limited one. It can only be a reference to a single cell, and the worksheet name must be specified, and the column and row must be absolute. Methods are added to Chart/Title to accommodate this (and styling for it). This will be handled for input/output for Xlsx, and for output for Html.
The sample file which was submitted with this issue demonstrated that something else was missing. When setting the font for a chart title in Excel, you can specify all-caps or small-caps, options not available for most cell formatting. These are now added.
The sample file also fell into the category of spreadsheets which lose one or more charts when converted to Html. I have redone the "extend rows and charts" logic in Html Writer. It is now clearer (I hope) and more efficient, and hopefully this problem will not arise again.
* Scrutinizer 50/50
One false positive, one correct "unused parameter".
* Two Problems with Html Chart Rendering - Minor Break
Several problems are noted in #3783. This PR addresses those problems which make the rendering unsatisfactory (see following paragraphs). It does not address some items where the rendering is IMO satisfactory although it doesn't match Excel. In particular, the use of a different color palette and the rotation of charts are not addressed. I will leave the issue open for now because of those.
As for the items which are addressed, in some cases the Html was omitting a chart altogether. This is because it had been extending the column range for charts only when it decided that extending the row range was needed. The code is changed to now extend the column range whenever the chart begins beyond the current column range of the sheet.
Also, the rendering always produced a fixed-size image, so saving as Html could result in charts overlaying each other or other parts of the spreadsheet. New properties `renderedWidth` and `renderedHeight` are added to Chart, along with setters and getters. Writer/Html is changed to set these values using the chart's top left and bottom right cells to try to determine the actual size that is needed. Users can also set these properties outside of Writer/Html if they wish. Thanks to @f1mishutka for determining the source of this problem and suggesting an approach to resolving it.
Because the size of the rendered image in Html/Pdf is changed, this could be considered a breaking change. To restore the prior behavior, do the following for all charts before saving as Html:
```php
$chart->setRenderedWidth(640.0);
$chart->setRenderedHeight(480.0);
```
* Update CHANGELOG.md
Fix#3760. That problem actually was easily fixed, by enabling calendar extension in user's environment. Astonishingly, however, this is the only use of calendar in the entire project. Since the same functionality is available in DateTime, which is used throughout the project, use that instead to eliminate the dependency on calendar.
* Added Conditional Formatting: ColorScale for Xlsx
* Add Reader Support, Tests, Sample
Also correct Phpstan and phpcs problems.
* Update cond08_colorscale.php
* Improve Coverage
* More Coverage Improvements
* Use StyleReader for Colors for ColorScale and DataBar
The implementation of DataBar looks for an rgb attribute, but the color may be provided via theme attribute instead. The initial implementation for ColorScale did the same. Change both to use the existing code in Reader\Xlsx\Styles to parse the color.
* Change Some Doc Blocks to Type Declarations
---------
Co-authored-by: oleibman <10341515+oleibman@users.noreply.github.com>
Because they tend to clutter our code a lot and unfortunately, there are
lots of false positives. Instead, it would probably be better to deal
with false positives out of band, via the Scrutinizer web UI.
Fix#3739. User is seeing an intermittent notice when running samples using a web browser. The notice is comming from ob_clean, complaining that there is no buffer to delete. I frankly do not understand what the ob_clean is supposed to be paired with, nor why I cannot duplicate this result. Possibly, a better solution would be to eliminate the ob_clean; but using ob_get_length beforehand to see if there is anything to clean up seems safer, and I can't think of a downside.
The code in question is never executed when a sample is run from the command line. Consequently, no formal unit test is possible. The user reporting the problem was asked to test the change, and confirmed that the problem went away; no new problem arose on my system.
This is according to our formal, published, policy to only support
EOL PHP after 6 months.
See https://phpspreadsheet.readthedocs.io/en/latest/#php-version-support
Also share the exact same dev deps across all PHP version for GitHub
Actions so runs are faster, much most importantly they are stable and
predictable. And we decide manually when we want to migrate to PHPUnit
10.
Fixes#3634Closes#3710
Fix#642. Opened over 5 years ago, probably the oldest problem I've worked on. And https://github.com/PHPOffice/PHPExcel/issues/1320, opened a year before that. And https://github.com/SpartnerNL/Laravel-Excel/issues/1521.
Shared/StringHelper::UTF8toBIFF8UnicodeLong calculates incorrect length for strings when they contain characters outside Unicode BMP. Xls uses UTF-16 to encode its strings, and characters outside BMP require a surrogate pair to encode. PhpSpreadsheet (and PhpExcel before it) have been counting these as a single character, but Excel counts them as 2. Change to compute the length as half the number of bytes in the UTF-16 string, as Excel does.
A formal test is added, but it's a bit difficult to follow. So I aso added a non-BMP emoji to 27template.xls, which will cause it to be both read by Xls reader and written by Xls writer. This would previously have created a corrupt worksheet. The emoji is now handled correctly.
* Minor Changes to Writer/Mpdf and Writer/Html
Changed a sample to illustrate how to add header/footer in Mpdf using setHtmlEditCallback. This uses custom Html tags, and, like body, it appears that these must be defined in the first writeHtml. Adjust writeMpdf to permit this by using a new constant `SIMULATED_BODY_START`, defined as an Html comment, as a delimiter.
Sample 21c_Pdf, to which the header/footer code is added, had been introduced with PR #2434 to ensure that the body tag was always in the first chunk. However, PR #3016 accidentally invalidated that test by reducing the number of style lines so that the sample now included the body tag in its first 1000 records rather than afterwards. This change puts it past record 1000 again.
Inspecting the results of all the Html/Pdf samples after this change, it turns out that sample 25_In_memory_image was accidentally broken by PR #3535 - the combination of `max-width:100%` (already present before that change) with `position:absolute` (introduced with that change) made the memory drawing disappear from the rendered html when the image occurs in a column after the last column with data in it. It appears that there is no need for max-width (drawings which are not memory drawings do not use it), so it is dropped. The sample is changed to add a second page with a memory drawing, one page with the memory drawing after the last data column, and one with it before. The Html results now reflect the Xlsx result, as they should.
* Minor Performance Improvements
Anonymous function.
* Cleanup for Phpstan, PHP-CS-Fixer Upgrades
Dependabot wanted to upgrade, but changes resulted in new "errors". Fix them.
* Scrutinizer
It says value must be numeric; Phpstan says it might not be. Try to reconcile them.
They have made some changes at my request, the major effect of which is that it will now work with 33_Chart_create_bar_stacked. This is a departure for them in that they have changed the functionality of jpgraph, not merely made sure that it is compatible with new Php releases.
Fix#3015. Fix#2766. Both issues requested a way to set Mpdf configuration parameters. I explained this could be done by extending class Mpdf and overriding protected function createExternalWriterInstance. So, no change is needed to PhpSpreadsheet. The parameters they wanted to override didn't seem interesting from a PhpSpreadsheet perspective. However, the configuration parameters include the ability to add fonts on the fly to those distributed by Mpdf, and that does sound useful. Normally, Mpdf (and Dompdf and Tcpdf) support a limited number of fonts, and will substitute for fonts they don't know (most fonts, even commonly used one, wind up using a DejaVu variant). Using these configuration options can lead to a more faithful Pdf representation of the spreadsheet.
In order to demonstrate this in action, I've added a new sample, a new class extending Mpdf, and a distinctive small font file from Google fonts with its (SIL Open Font) license. IANAL, but I can't see how this wouldn't be permitted use. There are no source code changes.
An alternate approach would be to add a callback to Writer/Pdf/Mpdf. It seems that this would be at least as much effort for the end-user as extending the class, and more effort for us.
It appears that Dompdf and Tcpdf need prep work to achieve the same end, and are thus less suitable for demonstrating via an example. I will continue to research.
oleibman
Christian Weiske
Caleb Mazalevskis
Adrien Crivelli
redtailmatt
MarkBaker