[Feature] File picker improvements (#454)

* First iteration for internationalization of the loader

* First iteration for internationalization of the loader

* Cleanup node mess

* Cleanup node mess pt2

* Additional touches

* Latest decky changed merged into i18n and updated translation.

* Styling fixes

* Initial backend hosting implementation

* Added correct url path of the loopback server.

* Added correct url path of the loopback server.

* Some better namespaced text.

* Added whitelist for locales path.

* Refactor languages and fix hooks logic bugs.

* Small typo in language translation structure.

* Working backend, automatically swtich languages with steam and language fixes.

* Fix to languages

* Key fixes

* Additional language fixes.

* Additional json changes

* Final text revision and added a vscode tasks to automatically extract text from code.

* Typo in the middleware

* Remove unused imports

* Cleanup whitespaces.

* Import changes

* Revert "Import changes"

This reverts commit 8e8231950f.

* Update index.d.ts

* Clean up unused imports

* Delete pnpm-lock.yaml

* Update rollup.config.js

* Update PluginInstallModal.tsx

* Update index.tsx

* Update plugin-loader.tsx

* Update plugin-loader.tsx

* Revert "Delete pnpm-lock.yaml"

This reverts commit 3a39f36f21.

* Additional strings reworks.

* Fixes for issues coming from github merge.

* Fixes for master

* Styling fixes

* Styling pt2

* Missed a few strings in master,

* Styling fixes

* Additional master merge fixes.

* Final cleanup and adaptation to master.

* Final empty language cleanup and few string added

* Small changes to italian translation

* Disabled translation on a few components inside plugin-loader for missing react hooks.

* Fixed passing tag to translation.

* Disable debug output for reducing console spam.

* Return correct content type

* Small italian language change

* Added support for country code

* Fixed missing translation for uninstall popup.

* Fix class name shenanigans for  toast notification

* Update dependencies

* Fixed github workflow to include the new locales folder

* Update dependencies to latest version (unless it's React) and fixed the new small errors that cropped up

* Missed a file name change

* Updated dev dependencies to latest version

* Missed a few dev dependencies

* Revert "Update dependencies to latest version (unless it's React) and fixed the new small errors that cropped up"

Messed up merge with a different main branch

* Messed up deletion of rollup config.

* Fix broken pnpm lock file

* Missed a localized string during the merge

* Fixed a parameter mistake in the uninstall text parameter

* Fix pnpm random issues

* Small italian language tweaks

* Fix wrong parameter passed to the uninstall function call

* Another fix on a wrong function parameter

* Additional translation text on the store and branch selection channels

* Changed the default type passed to map to being able to index the two arrays.

* Reverted and reworked the last changes

* Distinguish events in UI for installing vs reinstalling plugins

* Additional fixes for reinstall prompt

* Revert the use of intevalPlural since the parser doesn't seem to support that.

* Missed a routing path in the backend

* Small bugfixes

* Small fixes

* Correctly adding the parameter to the request headers.

* Refactoring of the UI popup modal

* Fix pnpm shenanigans

* Final fixes for the install UI localization

* Clean up unnedeed backend code

* Small rework on text selection.

* Cleaned up parser configuration

* Removed extracttext dependency to pnpmsetup

* Merged translation and cleaned up parser

* Fixed JSON structure after manual merge.

* Added translation to the file picker

* First iteration for merging the new filepicker.

* Revert changes to PluginInstallModal

* Reworked the text modal for the final time

* Missed the proper linted text

* Missed the backend change

* Final branch cleanup

* First iteration for porting the new file picker

* Hotfix for i18n where the detector was overriding localStorage

* Please, pnpm, cooperate

* Small fix regarding the backend getting hammered when switching to not supported languages plus a small english typo

* Initial working upstream iteration for file picker

* Typo on translation variable

* File picker final improvements

* Stylistic fixes and fix on wrong bool passed to fp

* Fixup merge from main

* Other merge errors fixed

* Minor cleanups

* Fixed missing padding under text label extension

* Implement pagination backend side

* First draft for filtering backend side

* Implemented matching on file names.

* Fix for unable to order per size on folders.

* Hard checking a return value

* Added a missing import.

* Implemented show more as a frontend button

* Whoops, python typo

* Fixed python backend

* Rendering bug fix and small qol improvement

* Added missing parameter to openFilePicker call

* Fixed path on windows and unknown error on wrong path

* Small backend fixes

* Extension fix

* Simplified extension logic

* Less string conversions.

* Optimize backend code and removed additional components.

* Take correctly into account the max value

The button will now respect the actual maximum desired number of entries.

* Bugfix for ordering logic and ignore cases during sorting

* Regex call was missing an argument

* Fixed issues with filtering extensions

* Rollback testing changes

* Minor cleanup and attempt at fixing the not updating multimodal.

* Cleanup variable types.

* Mantains the same api format from the original source code.

* Removing hardcoded paths in the code

* Additional fixes for resolving the user path

* Cleanup useless modifications

* Final fixes for avoid path hardcoding

* Update lockfile and i18next version

backend/helpers.py

@@ -159,4 +159,4 @@ async def stop_systemd_unit(unit_name: str) -> bool:
     return await localplatform.service_stop(unit_name)
 
 async def start_systemd_unit(unit_name: str) -> bool:
-    return await localplatform.service_start(unit_name)
+    return await localplatform.service_start(unit_name)

backend/locales/en-US.json

@@ -12,9 +12,37 @@
         "disabling": "Disabling React DevTools",
         "enabling": "Enabling React DevTools"
     },
+  "DropdownMultiselect": {
+    "button": {
+      "back": "Back"
+    }
+  },
+  "FilePickerError": {
+    "errors": {
+      "file_not_found": "The path specified is not valid. Please check it and reenter it correctly.",
+      "unknown": "An unknown error occurred. The raw error is: {{raw_error}}"
+    }
+  },
     "FilePickerIndex": {
-        "folder": {
-            "select": "Use this folder"
+    "files": {
+      "all_files": "All Files",
+      "file_type": "File Type",
+      "show_hidden": "Show Hidden Files"
+    },
+    "filter": {
+      "created_asce": "Created (Oldest)",
+      "created_desc": "Created (Newest)",
+      "modified_asce": "Modified (Oldest)",
+      "modified_desc": "Modified (Newest)",
+      "name_asce": "Z-A",
+      "name_desc": "A-Z",
+      "size_asce": "Size (Smallest)",
+      "size_desc": "Size (Largest)"
+    },
+      "folder": {
+      "label": "Folder",
+      "select": "Use this folder",
+      "show_more": "Show more files"
         }
     },
     "PluginView": {

backend/locales/it-IT.json

@@ -12,9 +12,37 @@
         "disabling": "Disabilito i tools di React",
         "enabling": "Abilito i tools di React"
     },
+  "DropdownMultiselect": {
+    "button": {
+      "back": "Indietro"
+    }
+  },
+  "FilePickerError": {
+    "errors": {
+      "file_not_found": "Il percorso specificato non è valido. Controllalo e prova a reinserirlo di nuovo.",
+      "unknown": "È avvenuto un'errore sconosciuto. L'errore segnalato è {{raw_error}}"
+    }
+  },
     "FilePickerIndex": {
+    "files": {
+      "all_files": "Tutti i file",
+      "file_type": "Tipo di file",
+      "show_hidden": "Mostra nascosti"
+    },
+    "filter": {
+      "created_asce": "Creazione (meno recente)",
+      "created_desc": "Creazione (più recente)",
+      "modified_asce": "Modifica (meno recente)",
+      "modified_desc": "Modifica (più recente)",
+      "name_asce": "Z-A",
+      "name_desc": "A-Z",
+      "size_asce": "Dimensione (più piccolo)",
+      "size_desc": "Dimensione (più grande)"
+    },
         "folder": {
-            "select": "Usa questa cartella"
+      "label": "Cartella",
+            "select": "Usa questa cartella",
+      "show_more": "Mostra più file"
         }
     },
     "PluginCard": {

backend/utilities.py

@@ -1,16 +1,21 @@
 import uuid
 import os
 from json.decoder import JSONDecodeError
+from os.path import splitext
+import re
 from traceback import format_exc
+from stat import FILE_ATTRIBUTE_HIDDEN
 
 from asyncio import sleep, start_server, gather, open_connection
 from aiohttp import ClientSession, web
 
 from logging import getLogger
 from injector import inject_to_tab, get_gamepadui_tab, close_old_tabs, get_tab
+from pathlib import Path
+from localplatform import ON_WINDOWS
 import helpers
 import subprocess
-from localplatform import service_stop, service_start
+from localplatform import service_stop, service_start, get_home_path, get_username
 
 class Utilities:
     def __init__(self, context) -> None:
@@ -33,7 +38,8 @@ class Utilities:
             "filepicker_ls": self.filepicker_ls,
             "disable_rdt": self.disable_rdt,
             "enable_rdt": self.enable_rdt,
-            "get_tab_id": self.get_tab_id
+            "get_tab_id": self.get_tab_id,
+            "get_user_info": self.get_user_info,
         }
 
         self.logger = getLogger("Utilities")
@@ -189,31 +195,82 @@ class Utilities:
         await service_stop(helpers.REMOTE_DEBUGGER_UNIT)
         return True
 
-    async def filepicker_ls(self, path, include_files=True):
-        # def sorter(file): # Modification time
-        #     if os.path.isdir(os.path.join(path, file)) or os.path.isfile(os.path.join(path, file)):
-        #         return os.path.getmtime(os.path.join(path, file))
-        #     return 0
-        # file_names = sorted(os.listdir(path), key=sorter, reverse=True) # TODO provide more sort options
-        file_names = sorted(os.listdir(path)) # Alphabetical
+    async def filepicker_ls(self, 
+                            path : str | None = None, 
+                            include_files: bool = True,
+                            include_folders: bool = True,
+                            include_ext: list[str] = [],
+                            include_hidden: bool = False,
+                            order_by: str = "name_asc",
+                            filter_for: str | None = None,
+                            page: int = 1,
+                            max: int = 1000):
+        
+        if path == None:
+            path = get_home_path()
 
-        files = []
+        path = Path(path).resolve()
 
-        for file in file_names:
-            full_path = os.path.join(path, file)
-            is_dir = os.path.isdir(full_path)
+        files, folders = [], []
 
-            if is_dir or include_files:
-                files.append({
-                    "isdir": is_dir,
-                    "name": file,
-                    "realpath": os.path.realpath(full_path)
-                })
+        #Resolving all files/folders in the requested directory
+        for file in path.iterdir():
+            if file.exists():
+                filest = file.stat()
+                is_hidden = file.name.startswith('.')
+                if ON_WINDOWS and not is_hidden:
+                    is_hidden = bool(filest.st_file_attributes & FILE_ATTRIBUTE_HIDDEN)
+                if include_folders and file.is_dir():
+                    if (is_hidden and include_hidden) or not is_hidden:
+                        folders.append({"file": file, "filest": filest, "is_dir": True})
+                elif include_files:
+                    # Handle requested extensions if present
+                    if 'all_files' in include_ext or splitext(file.name)[1].lstrip('.') in include_ext:
+                        if (is_hidden and include_hidden) or not is_hidden:
+                            files.append({"file": file, "filest": filest, "is_dir": False})
+        # Filter logic
+        if filter_for is not None:
+            try:
+                if re.compile(filter_for):
+                    files = filter(lambda file: re.search(filter_for, file.name) != None, files)
+            except re.error:
+                files = filter(lambda file: file.name.find(filter_for) != -1, files)
+        
+        # Ordering logic
+        ord_arg = order_by.split("_")
+        ord = ord_arg[0]
+        rev = True if ord_arg[1] == "asc" else False
+        match ord:
+            case 'name':
+                files.sort(key=lambda x: x['file'].name.casefold(), reverse = rev)
+                folders.sort(key=lambda x: x['file'].name.casefold(), reverse = rev)
+            case 'modified':
+                files.sort(key=lambda x: x['filest'].st_mtime, reverse = not rev)
+                folders.sort(key=lambda x: x['filest'].st_mtime, reverse = not rev)
+            case 'created':
+                files.sort(key=lambda x: x['filest'].st_ctime, reverse = not rev)
+                folders.sort(key=lambda x: x['filest'].st_ctime, reverse = not rev)
+            case 'size':
+                files.sort(key=lambda x: x['filest'].st_size, reverse = not rev)
+                # Folders has no file size, order by name instead
+                folders.sort(key=lambda x: x['file'].name.casefold())
+        
+        #Constructing the final file list, folders first
+        all =   [{
+                    "isdir": x['is_dir'],
+                    "name": str(x['file'].name),
+                    "realpath": str(x['file']),
+                    "size": x['filest'].st_size,
+                    "modified": x['filest'].st_mtime,
+                    "created": x['filest'].st_ctime,
+                } for x in folders + files ]
 
         return {
-            "realpath": os.path.realpath(path),
-            "files": files
+            "realpath": str(path),
+            "files": all[(page-1)*max:(page)*max],
+            "total": len(all),
         }
+        
 
     # Based on https://stackoverflow.com/a/46422554/13174603
     def start_rdt_proxy(self, ip, port):
@@ -289,5 +346,11 @@ class Utilities:
         await tab.evaluate_js("location.reload();", False, True, False)
         self.logger.info("React DevTools disabled")
 
+    async def get_user_info(self) -> dict:
+        return {
+            "username": get_username(),
+            "path": get_home_path()
+        }
+    
     async def get_tab_id(self, name):
         return (await get_tab(name)).id