LearningCircuit 65b2383eb7 Revert "Revert "fix: add SQLCipher 4.x compatibility for cipher pragma ordering"" (#1867) ...

* Revert "Revert "fix: add SQLCipher 4.x compatibility for cipher pragma orderi…"

This reverts commit d72b7ae668.

* ci: add backwards compatibility workflow for SQLCipher encryption

Add a CI workflow that verifies database encryption backwards compatibility:

- Runs encryption constants tests on PRs touching database files
- Runs full PyPI version compatibility test on main/releases
- Triggers on schedule (weekly) to catch dependency drift
- Path-filtered to only run when relevant files change

This prevents regressions like salt or KDF parameter changes that would
break existing encrypted databases.

* ci: add backwards compatibility to security release gate

- Add workflow_call trigger to backwards-compatibility.yml
- Include backwards-compatibility check in security-release-gate.yml
- Run full PyPI compatibility test on releases (not just encryption constants)

This ensures database encryption changes can't break existing user databases
in a release.

* test: increase timeouts for PyPI backwards compatibility test

- Add pytest.mark.timeout(600) for the test function
- Increase pip install timeout from 300s to 600s

The package has many dependencies (numpy, torch, etc.) which take
significant time to install in a fresh venv.

* ci: fix shellcheck SC2129 in backwards-compatibility workflow

Group consecutive GITHUB_OUTPUT redirects into a single block
to satisfy actionlint/shellcheck SC2129 style check.

* fix: comprehensive SQLCipher security and correctness improvements

- Fix critical PRAGMA ordering: cipher_default_* before key for new DBs,
  cipher_* after key for existing DBs (per Zetetic SQLCipher 4.x docs)
- Replace unbounded @cache with @lru_cache(maxsize=8) to prevent memory leaks
- Add thread safety: RLock for connections dict, Lock for cipher_default globals
- Pre-derive hex keys before closures to avoid capturing plaintext passwords
- Add connection cleanup on failure (close conn, remove partial DB files)
- Add engine.dispose() on failed open_user_database()
- Whitespace password validation (reject " " as password)
- Remove cipher_memory_security=OFF (SQLCipher >=4.5 defaults to OFF)
- CI-aware KDF minimum: 100k production, relaxed in test environments
- Add set_sqlcipher_key_from_hex(), get_sqlcipher_version() utilities
- Replace direct db_manager.connections dict access with is_user_connected()
- Update harden-runner to v2.14.1, improve CI error handling
- Fix all except Exception: pass patterns to re-raise AssertionError in tests
- Update all test files for renamed functions and correct PRAGMA ordering

* fix: address race condition in get_session() and exception swallowing in test

Move sessionmaker + session creation inside _connections_lock to prevent
race with close_user_database() disposing the engine. Also fix exception
handler in test_sqlcipher_integration.py to re-raise AssertionError.

* fix: address review issues in SQLCipher 4.x compatibility PR

- Update 3 broken web test files (test_queue_manager, test_decorators,
  test_session_cleanup) to mock is_user_connected()/get_session() instead
  of the removed connections.get() API
- Wire backwards-compatibility.yml into security-release-gate.yml so
  encryption compat checks actually run during releases
- Add missing apply_sqlcipher_pragmas() call in _check_encryption_available()
  to properly set kdf_iter after key on the test connection
- Replace generic CI/TESTING env var checks with PYTEST_CURRENT_TEST and
  LDR_TEST_MODE to prevent accidental KDF weakening in production
- Add timeout-minutes to both backwards-compatibility CI jobs

2026-02-11 02:26:58 +00:00

16 KiB

Raw Blame History

View Raw