The native preg functions (preg_match, preg_replace, etc.) often require us to add a lot of useless boilerplate code to satisfy Phpstan, Scrutinizer, etc. Composer/Pcre offers us a way to remove that boilerplate, thereby giving us a cleaner codebase. I decided to try it on a few modules, and saw a result that clearly demonstrated the usefulness of doing this (aside from the cleaner codebase).
Sample 22_Reader_issue1767 reads an Xlsx spreadsheet with complex sheet names used in defined names, and writes it to Xlsx and Xls output files. When I changed Writer/Xls/Parser to use Composer/Pcre, this sample failed in many different places writing the Xls file. It turns out that some regexes were failing not because the string didn't match, but because the regex encountered "catastrophic backtracing". Composer/Pcre throws an exception when this happens; the native preg_match does return false, but we were not checking for that. The regexes in question are now changed to something which works, and formal unit tests are added for them. Finding this previously undetected error indicates that we should proceed with this change.
An alternative to using Composer/Pcre would be to test for false after all the preg calls. I have done this in the two samples changed with this PR. That seems adequate for a small number of changes, but it really just makes for more clutter considering the large number of regexps that we use in our code. I think Composer/Pcre is a better choice.
It isn't quite transparent. Composer forces all regexps to use PREG_UNMATCHED_AS_NULL, so some match fields will now be null instead of null-string (or non-existent if the unmatched field comes at the end). Our test suite doesn't report any problem (yet) due to this change, although Phpstan is sensitive to it. Several Phpstan annotations were eliminated due to this change, but some others are now needed.
It is not necessary to do this all at once. This PR addresses all the calls in Writer. I intend to address other components in several tickets.
Fix#4319. PR #3495 allowed Mpdf and Tcpdf to handle hidden rows and columns. The handling is a little off when merged cells are part of the output. The fix for Mpdf is pretty straightforward. Tcpdf is a little more convoluted. Dompdf and Html do not require a change. Unit tests are added for the same scenario for all of Mpdf, Tcpdf, Dompdf, and Html.
This is the second recent occasion where style for whole row or column is behaving unexpectedly (see issue #4285). For the earlier issue, a documentation update was made to show the preferred method of styling. Setting styles for row(s) or column(s) in that way will work just fine, however retrieving the style doesn't yield the expected result. Although that problem can be overcome with existing code, simpler methods are needed, and this PR adds methods getRowStyle and getColumnStyle to Worksheet. I will continue to investigate why this problem and the one from 4285 have unexpected results.
Cloning a worksheet attached to a spreadsheet creates a clone which is detached from the spreadsheet. This can have its uses, but I think it would also be useful to have the ability to duplicate the worksheet and keep the duplicate attached to the spreadsheet. You can do that in Excel and LibreOffice, and you can now do it in PhpSpreadsheet as well. The duplicated worksheet will come immediately after its source.
The worksheet being duplicated could be identified in a number of ways - by passing the worksheet itself to the new method, by passing the worksheet title, or by passing the index of the worksheet within the spreadsheet. For now, I am just implementing the one I think is most useful (title).
Fix#4311. Period is a valid character in a sheet name. When a sheet with such a name is referenced in Ods format, the sheet name must be enclosed in apostrophes, because Ods uses period to separate sheet name from cell address. (Excel uses exclamation point so doesn't necessarily need to enclose the sheet name in apostrophes.) This causes a problem for the Ods Reader whenever it tries to parse such an address; however, the problem showed up specifically for auto filters, because the Ods xml for those specifies *'sheetname'.startcell:'sheetname'.endcell* (Excel omits sheetname).
Ods Reader translates these addresses in 2 different methods in FormulaTranslator. I had a relatively elegant method for handling this situation in convertToExcelAddressValue, but I could not make it work in convertToExcelFormulaValue. A kludgier method works for Formula, and also for Address. I decided it's better to be consistent, so I'm going with the kludgier method for both.
It would not surprise me in the least if there are similar problems lying in wait for other special characters in sheet names, and for other formats besides Ods. For now, I will limit myself to fixing the known problem.
Fix#641 (marked stale in 2018, but now reopened). When a sheet's title is changed, PhpSpreadsheet updates references to the old sheet name found in formulas. Which is a good idea when the sheet is attached to the spreadsheet, but a bad idea when it isn't (often because it has been cloned without re-attaching to the spreadsheet). This PR continues to change formulas in the former case, but will no longer do so for the latter.
They have made a change at our request to help us eliminate runInSeparateProcess for one or more tests. This will be helpful when we get to PhpUnit 11 (not imminent, since it doesn't support Php8.1, but it will happen eventually).
These are 5 closely related functions for manipulating arrays. Now that dynamic arrays are part of PhpSpreadsheet, this PR implements those previously-unimplemented functions. Documentation undergoes a very minor change, since they are re-categorized as "Lookup and Reference", which is how Excel classifies them, rather than "Math and Trig".
This is a partial response to issue #4282. The actual logic to implement GROUPBY is probably very complicated. And, even worse, Excel has thrown a whole new way of (internally) specifying one of the arguments into the mix. That argument is a function name, expressed not as a mapped integer (as SUBTOTAL does), nor even as a string, but as the unquoted function name prefixed by `_xleta.`. And, unlike its `_xlfn.` and `_xlws.` predecessors, it is difficult to figure out when the new prefix needs to be added, and when it needs to be ignored. I am not even going to attempt that task with this ticket.
So, what does this change do? Like earlier attempts to introduce limited functionality (such as with form controls), it is there so that using GROUPBY can be passed through - you can load a spreadsheet that contains it, and save it to a new spreadsheet, and the function and its results are preserved. Some cautionary notes. Dynamic arrays must be enabled (the function makes no sense without doing that). Changing any of the inputs used in the function may result in internal inconsistencies between PhpSpreadsheet and Excel; this is especially so if the dimensions of the returned array change as a result of changes to the input data. The programmer can avoid some of these problems by changing the formulatAttributes of the cell where the function is used; this may be difficult to do in practice. Oh, yes, using the GROUPBY cell as an argument in another formula will probably lead to problems. Finally, I confess that part of this solution looks awfully kludgey to me.
With its limitations and those cautions, is it worth proceeding with this change? My gut feel is that it is more useful to proceed than not. However, I will give others the opportunity to weigh in. I will wait at least a couple of weeks into the new year before proceeding with this.
A solution, at least in part, for issue #4280. Xlsx Reader is not handling shared formulae correctly. As a result, some cells are treated as if they contain boolean values rather than formulae.
We do not recommend it, but users can call the Php function `setlocale` and that might affect the character used as currency symbol. A problem arises when the caller to setlocale does not specify a character set - in that case, Php will attempt to return its `localeconv()` values in a single-byte character set rather than UTF-8. This is particularly problematic for currency symbols. PhpSpreadsheet till now has accepted such a character, and that can lead to corrupt spreadsheets. It is changed to validate the currency symbol as UTF-8, and fall back to a different choice if not (e.g. EUR rather than 0x80, which is how the euro symbol is depicted in Win-1252).
An additional problem arises because Linux systems seem to return the alternate symbol with a trailing blank, but Windows systems do not. To allow callers to get a consistent result, a parameter is added to `getCurrencyCode` which will trim or not (default) the currency code.
After breaking up Xls Reader (PR #4118), it is a little easier to identify uncovered code. BIFF8 had no tests involving constant arrays. This PR adds some. Most of the work is in the tests, but some source code is modernized to use things like null coercion.
For the second time in recent months, some tests are failing/erring because https file_get_contents is not working on github (cannot reproduce locally on Windows or Linux). Filed an issue with Php when this first happened, and their suggested code change worked till now. If they come up with another successful code change, I will implement it and restore these tests.
Fix#4269. In response to issue #456 PR #515, `forceFullCalc` was added to workbook.xml whenever `preCalculateFormulas` was set to false. It is not clear why this should have been needed; attempts to reproduce the error in the original 6.5-year-old issue are unable to reproduce it today. Nevertheless, it is, or was, there for a reason.
Today, the forceFullCalc option sets an option where formulas *might* not be recalculated when a cell used in the formula changes. I have not succeeded in finding a situation where it doesn't automatically recalculate, but it probably exists on complicated spreadsheets. To overcome this possibility, Excel offers a button which can be used to recalculate on demand. By itself, this might not be a terrible problem. However, it seems to come with the strange property that any spreadsheets opened at the same time as a forceFullCalc spreadsheet operate as if they too specified forceFullCalc. That *is* a problem, especially since users when closing a spreasheet affected in this way will be prompted to save it even when they haven't changed anything.
I am not willing to make a BC change at this time, although I might consider it in future (PR #4240). For now, I am adding a new property `forceFullCalc` with setter (no getter needed) to Xlsx Writer. That property can be `null` (default, in which case the Xml attribute is set as today), or `false` or `true` (in which case the Xml attribute will be set to the Writer attribute). I think that, when `preCalculateFormulas` is set to false, the calling application should give consideration to setting `forceFullCalc` to false as well. All other situations should just use the default.
PR #2850 and PR #2851 added support for Worksheet Visibility to ODS. They work well when LibreOffice is used to open the spreadsheet. However, when Excel tries to open it, it reports corruption. Strictly speaking, this is not a PhpSpreadsheet problem, but, if we can fix it, we should.
It took a while to figure out what's bothering Excel. I'm not sure that all of what follows is necessary, but it works. It appears that it wants the content.xml `style:automatic-styles` definition for the `table` (i.e. worksheet) to include a `style:master-page-name` attribute. That attribute requires a corresponding definition in the `office:master-styles` section in styles.xml, and that attribute likewise requires a definition in `office:automatic-styles`.
The new entries in styles.xml can be used to specify things like header and footer. However, the ways that these are specified is distinctly different from what Excel (and therefore PhpSpreadsheet) does. Implementing that will be a good future project. However, for now, they will remain unsupported for Ods.
Ods Writer currently will write something like `<table:table-cell table:number-columns-repeated="1023" />` at the end of each row. This is not necessary. In eliminating that, I have made the code a bit more efficient and (hopefully) more readable.
Fix#4261. Ods does nothing with the `indent` property of Alignment. The fix is easy, but not as easy as it should be. Excel treats indent as an int in some unspecified unit. Ods, on the other hand, specifies it as a float with unit (inches, ems, etc.). Since MS does not reveal what an indent unit is, I resorted to some experimentation. It appears that 1 unit is equal to 0.1043 inches. That is not a guaranteed relationship - the conversion might be non-linear, or it might be affected by external factors like font size. I think multiplying by 0.1043 is adequate for now, and unquestionably better than what we're currently doing (ignoring the property). If it breaks down at some point, we'll look at it again. BTW, Shared\Drawing uses some conversion values of 9525, which, reciprocating and sliding the decimal point along yields a value of 0.10498..., which is close but not close enough for me to use.
The other property used in conjunction with indent is `text-align`, and here we have not been doing the right thing. If that property has the default value (`General`), it is treated as `start` (i.e. `left` on LTR docs and presumably `right` on RTL). This will align numbers as if they were text, which does no harm (they're still usable as numbers), but is not how LibreOffice handles them by default. Ods writer is changed to omit `text-align` when it is set to `General`, and let LibreOffice choose the appropriate alignment.
These changes are for Writer only. Ods Reader support for styles remains severely lacking.
oleibman