* Revert "Revert "fix: add SQLCipher 4.x compatibility for cipher pragma orderi…"
This reverts commit d72b7ae668.
* ci: add backwards compatibility workflow for SQLCipher encryption
Add a CI workflow that verifies database encryption backwards compatibility:
- Runs encryption constants tests on PRs touching database files
- Runs full PyPI version compatibility test on main/releases
- Triggers on schedule (weekly) to catch dependency drift
- Path-filtered to only run when relevant files change
This prevents regressions like salt or KDF parameter changes that would
break existing encrypted databases.
* ci: add backwards compatibility to security release gate
- Add workflow_call trigger to backwards-compatibility.yml
- Include backwards-compatibility check in security-release-gate.yml
- Run full PyPI compatibility test on releases (not just encryption constants)
This ensures database encryption changes can't break existing user databases
in a release.
* test: increase timeouts for PyPI backwards compatibility test
- Add pytest.mark.timeout(600) for the test function
- Increase pip install timeout from 300s to 600s
The package has many dependencies (numpy, torch, etc.) which take
significant time to install in a fresh venv.
* ci: fix shellcheck SC2129 in backwards-compatibility workflow
Group consecutive GITHUB_OUTPUT redirects into a single block
to satisfy actionlint/shellcheck SC2129 style check.
* fix: comprehensive SQLCipher security and correctness improvements
- Fix critical PRAGMA ordering: cipher_default_* before key for new DBs,
cipher_* after key for existing DBs (per Zetetic SQLCipher 4.x docs)
- Replace unbounded @cache with @lru_cache(maxsize=8) to prevent memory leaks
- Add thread safety: RLock for connections dict, Lock for cipher_default globals
- Pre-derive hex keys before closures to avoid capturing plaintext passwords
- Add connection cleanup on failure (close conn, remove partial DB files)
- Add engine.dispose() on failed open_user_database()
- Whitespace password validation (reject " " as password)
- Remove cipher_memory_security=OFF (SQLCipher >=4.5 defaults to OFF)
- CI-aware KDF minimum: 100k production, relaxed in test environments
- Add set_sqlcipher_key_from_hex(), get_sqlcipher_version() utilities
- Replace direct db_manager.connections dict access with is_user_connected()
- Update harden-runner to v2.14.1, improve CI error handling
- Fix all except Exception: pass patterns to re-raise AssertionError in tests
- Update all test files for renamed functions and correct PRAGMA ordering
* fix: address race condition in get_session() and exception swallowing in test
Move sessionmaker + session creation inside _connections_lock to prevent
race with close_user_database() disposing the engine. Also fix exception
handler in test_sqlcipher_integration.py to re-raise AssertionError.
* fix: address review issues in SQLCipher 4.x compatibility PR
- Update 3 broken web test files (test_queue_manager, test_decorators,
test_session_cleanup) to mock is_user_connected()/get_session() instead
of the removed connections.get() API
- Wire backwards-compatibility.yml into security-release-gate.yml so
encryption compat checks actually run during releases
- Add missing apply_sqlcipher_pragmas() call in _check_encryption_available()
to properly set kdf_iter after key on the test connection
- Replace generic CI/TESTING env var checks with PYTEST_CURRENT_TEST and
LDR_TEST_MODE to prevent accidental KDF weakening in production
- Add timeout-minutes to both backwards-compatibility CI jobs
- Bulk fix import paths from src.local_deep_research to local_deep_research
across 190+ test files
- Fix test_settings_routes.py: update validate_setting tests to use proper
Setting objects instead of incorrect parameter signature
- Fix test_research_service.py: remove tests for non-existent functions,
correct mimetype assertions for latex export
- Fix test_app_factory.py: use truly public IP (8.8.8.8) instead of
TEST-NET reserved range for IP validation test
- Fix test_api_functions.py: correct patch paths for get_user_db_session,
has_request_context, and get_settings_manager
- Fix test_news_api.py: correct patch paths for database session
Test results: 8592 passing, 175 failing (down from 286), 115 skipped
Remaining failures are test isolation issues that pass individually.
