- Creates scripts/dev/stop_server.sh to complement restart_server.sh
- Gracefully stops the LDR server process
- Includes force stop fallback if processes don't terminate cleanly
- Cleans up any orphaned Flask dev server processes
- Provides clear status messages and exit codes
* feat: Add pre-commit hook to enforce pathlib usage (issue #640)
- Created check-pathlib-usage.py pre-commit hook using AST parsing
- Detects os.path usage and suggests pathlib alternatives
- Fixed os.path.normpath usage in auth/routes.py to use PurePosixPath
- Added hook configuration to .pre-commit-config.yaml
The hook provides helpful suggestions for replacing os.path calls with
their pathlib equivalents for better cross-platform compatibility.
Co-Authored-By: djpetti <djpetti@users.noreply.github.com>
* feat: Add missing pathlib pre-commit hook script
Co-Authored-By: djpetti <djpetti@users.noreply.github.com>
* refactor: Migrate core src modules from os.path to pathlib
- Fixed web/app_factory.py, config/llm_config.py, metrics/token_counter.py
- Fixed utilities/es_utils.py, web/routes/benchmark_routes.py
- Fixed web/routes/settings_routes.py, web_search_engines/engines/search_engine_local.py
- Replaced os.path.join() with Path() / syntax
- Replaced os.path.exists() with Path().exists()
- Replaced os.path.basename() with Path().name
- Replaced os.path.dirname() with Path().parent
Part of the migration to modern pathlib API for better cross-platform
compatibility and cleaner code.
Co-Authored-By: djpetti <djpetti@users.noreply.github.com>
* refactor: Migrate from os.path to pathlib in src and tests (issue #640)
Replaced os.path usage with pathlib.Path throughout:
- src/local_deep_research/benchmarks: All os.path.join, exists, dirname, basename, abspath replaced
- tests directory: Complete migration of all test files
- Improved cross-platform compatibility and code readability
- Kept os.path.expandvars in env_settings.py (no pathlib equivalent)
Part of pre-commit hook enforcement for pathlib usage.
Remaining work: examples/ and scripts/ directories.
Co-Authored-By: djpetti
* fix: Complete migration from os.path to pathlib.Path (issue #640)
Completed manual migration of all os.path usage to pathlib.Path across:
- scripts/ directory (3 files)
- examples/ directory (25 files total)
- examples/benchmarks/ (8 files)
- examples/optimization/ (16 files)
- examples/show_env_vars.py
- src/local_deep_research/settings/env_settings.py
Changes made:
- Replaced os.path.join() with Path() / syntax
- Replaced os.path.exists() with Path().exists()
- Replaced os.path.dirname() with Path().parent
- Replaced os.path.basename() with Path().name or Path().stem
- Replaced os.path.abspath() with Path().resolve()
- Replaced os.makedirs() with Path().mkdir(parents=True, exist_ok=True)
- Added pathlib import where needed
Note: Kept os.path.expandvars in env_settings.py as there is no pathlib
equivalent. Added comment explaining this limitation.
This completes the pathlib migration for issue #640.
Co-Authored-By: djpetti
* fix: Allow os.path.expandvars in pathlib pre-commit hook
Updated the check-pathlib-usage.py pre-commit hook to skip checking
os.path.expandvars since it has no pathlib equivalent.
Changes:
- Added exception for expandvars in both visit_Attribute and visit_Call methods
- Added comment in equivalents dictionary noting expandvars is allowed
- This allows env_settings.py to use os.path.expandvars without failing checks
This resolves the pre-commit CI failure while maintaining the pathlib
enforcement for all other os.path methods.
Co-Authored-By: djpetti
---------
Co-authored-by: djpetti
- Fix processing time and accuracy charts to use real-time database queries
- Add sync_pending_results to save benchmark results during execution
- Fix duplicate element ID issue for estimated time display
- Pass research_context through search strategies for proper metrics
- Update all CachedSettingsManager references to SettingsManager
- Remove hardcoded paths from restart_server.sh for security
- Fix import and linting issues across codebase
BREAKING CHANGE: Data files now stored in platform-specific user directories
with SQLCipher encryption. Users must register/login to access the application.
## Major Features
### Security & Authentication
- Implemented complete multi-user authentication system with Flask-Login
- Per-user SQLCipher encrypted databases (falls back to SQLite with warnings)
- Secure session management with proper CSRF protection
- Password hashing with bcrypt for user credentials
- Complete isolation between user data - no cross-user access possible
- Thread-safe database connections with proper session management
### Database Architecture
- Migrated from single shared database to per-user encrypted databases
- Centralized auth database for user management
- User-specific databases for research data, settings, and metrics
- Automatic database initialization on user registration
- Platform-specific data directories using platformdirs library
- Removed all hardcoded paths and personal information
### User Experience
- Registration page with data privacy acknowledgment
- Login/logout functionality with session persistence
- Automatic redirect to login for unauthenticated access
- Research queue system with 3 concurrent research limit per user
- Real-time queue position updates
- Comprehensive error handling with user-friendly messages
### API & Routes
- All API endpoints now require authentication
- Updated routes: /auth/register, /auth/login, /auth/logout, /auth/check
- Protected research submission and history endpoints
- Proper JSON error responses for API routes
- CSRF token validation for state-changing operations
### Testing
- Added 53 Puppeteer tests for UI authentication flows
- Comprehensive auth integration tests (248 Python test files)
- Multi-user concurrent access testing
- Queue system testing with position tracking
- Database migration and encryption tests
### Configuration
- Single LDR_DATA_DIR environment variable for data location
- LDR_ALLOW_UNENCRYPTED environment variable for development
- Updated Docker configuration for proper volume mounting
- Removed multiple environment variables for simplicity
### Documentation
- Added DATA_MIGRATION_GUIDE.md for upgrade instructions
- Added SQLCIPHER_INSTALL.md for encryption setup
- Updated environment configuration documentation
- Professional error messages throughout
## Technical Improvements
- Replaced raw SQL with SQLAlchemy ORM throughout
- Proper database session management with context managers
- Thread-local storage for database connections
- Automatic cleanup of stale sessions
- Rate limiting infrastructure for future use
- Comprehensive logging with loguru
## Files Changed
- 322 files modified/added
- 248 Python files (core functionality and tests)
- 53 JavaScript files (Puppeteer tests)
- 6 Markdown files (documentation)
- No binary files, screenshots, or database files included
- All test credentials properly marked with pragma comments
This migration ensures each user's research data is completely isolated and
encrypted, providing enterprise-grade security for sensitive research operations.
* Clean up temporary JavaScript files and reorganize test files
- Remove temp_js_*.js files that appear to be development iterations
- Move test_simple_cost.js to tests/ui_tests/ where it belongs with other UI tests
- Move package.json to tests/ directory since it's only used for test dependencies (Puppeteer)
- These changes clean up the root directory from temporary development files
* chore: Clean up root directory and organize dev scripts
- Remove redundant app.py (was calling non-existent local_deep_research.main())
- Move development utility scripts to scripts/dev/:
- kill_servers.py: Flask server process management
- run_tests.py: Test runner with coverage
- debug_pytest.py: CI test debugging
- Fix kill_servers.py to use correct server command (local_deep_research.web.app)
* fix: Update UI tests workflow to use tests/package.json
The npm install command was looking for package.json in the root directory,
but it was moved to tests/ directory. Update the workflow to install npm
dependencies from the correct location.
* fix: Update full tests workflow to use tests/package.json
Another npm install command in the full tests workflow was looking for
package.json in the root directory. Update it to install from tests/
directory where the package.json is located.
