cipher_memory_security=ON calls mlock() which fails in Docker without
IPC_LOCK, spamming logs. Defaulting to OFF because the password already
sits unprotected in Flask session, db_manager.connections, and
thread-local storage — mlock on SQLCipher's internal buffers alone
doesn't meaningfully help when the same secret is exposed elsewhere.
Users can opt back in with LDR_DB_CONFIG_CIPHER_MEMORY_SECURITY=ON
(requires IPC_LOCK capability in Docker).
Also moves the pragma outside the creation_mode guard so the env var
override works for existing databases too.
* Revert "Revert "fix: add SQLCipher 4.x compatibility for cipher pragma orderi…"
This reverts commit d72b7ae668.
* ci: add backwards compatibility workflow for SQLCipher encryption
Add a CI workflow that verifies database encryption backwards compatibility:
- Runs encryption constants tests on PRs touching database files
- Runs full PyPI version compatibility test on main/releases
- Triggers on schedule (weekly) to catch dependency drift
- Path-filtered to only run when relevant files change
This prevents regressions like salt or KDF parameter changes that would
break existing encrypted databases.
* ci: add backwards compatibility to security release gate
- Add workflow_call trigger to backwards-compatibility.yml
- Include backwards-compatibility check in security-release-gate.yml
- Run full PyPI compatibility test on releases (not just encryption constants)
This ensures database encryption changes can't break existing user databases
in a release.
* test: increase timeouts for PyPI backwards compatibility test
- Add pytest.mark.timeout(600) for the test function
- Increase pip install timeout from 300s to 600s
The package has many dependencies (numpy, torch, etc.) which take
significant time to install in a fresh venv.
* ci: fix shellcheck SC2129 in backwards-compatibility workflow
Group consecutive GITHUB_OUTPUT redirects into a single block
to satisfy actionlint/shellcheck SC2129 style check.
* fix: comprehensive SQLCipher security and correctness improvements
- Fix critical PRAGMA ordering: cipher_default_* before key for new DBs,
cipher_* after key for existing DBs (per Zetetic SQLCipher 4.x docs)
- Replace unbounded @cache with @lru_cache(maxsize=8) to prevent memory leaks
- Add thread safety: RLock for connections dict, Lock for cipher_default globals
- Pre-derive hex keys before closures to avoid capturing plaintext passwords
- Add connection cleanup on failure (close conn, remove partial DB files)
- Add engine.dispose() on failed open_user_database()
- Whitespace password validation (reject " " as password)
- Remove cipher_memory_security=OFF (SQLCipher >=4.5 defaults to OFF)
- CI-aware KDF minimum: 100k production, relaxed in test environments
- Add set_sqlcipher_key_from_hex(), get_sqlcipher_version() utilities
- Replace direct db_manager.connections dict access with is_user_connected()
- Update harden-runner to v2.14.1, improve CI error handling
- Fix all except Exception: pass patterns to re-raise AssertionError in tests
- Update all test files for renamed functions and correct PRAGMA ordering
* fix: address race condition in get_session() and exception swallowing in test
Move sessionmaker + session creation inside _connections_lock to prevent
race with close_user_database() disposing the engine. Also fix exception
handler in test_sqlcipher_integration.py to re-raise AssertionError.
* fix: address review issues in SQLCipher 4.x compatibility PR
- Update 3 broken web test files (test_queue_manager, test_decorators,
test_session_cleanup) to mock is_user_connected()/get_session() instead
of the removed connections.get() API
- Wire backwards-compatibility.yml into security-release-gate.yml so
encryption compat checks actually run during releases
- Add missing apply_sqlcipher_pragmas() call in _check_encryption_available()
to properly set kdf_iter after key on the test connection
- Replace generic CI/TESTING env var checks with PYTEST_CURRENT_TEST and
LDR_TEST_MODE to prevent accidental KDF weakening in production
- Add timeout-minutes to both backwards-compatibility CI jobs
Addresses djpetti's review comment to use the project's environment
variable setting infrastructure (EnumSetting + get_env_setting) instead
of plain os.environ.get() for cipher_memory_security.
- CRITICAL: Fix set_sqlcipher_rekey to use PBKDF2-derived key via
_get_key_from_password instead of raw UTF-8 hex encoding, which
would make databases unopenable after password change
- Replace unbounded functools.cache with TTL-bounded cachetools.TTLCache
(5min TTL, thread-safe) to prevent memory growth from cached keys
- Make cipher_memory_security configurable via LDR_DB_CIPHER_MEMORY_SECURITY
env var (default ON) instead of hardcoded OFF
- Remove password length logging in scheduler (log presence only)
- Expand documentation on static PBKDF2 salt trade-off
- Add tests for rekey fix, cipher_memory_security, and TTL cache
Integrates 67 SQLCipher integration tests with the SQLCipher 4.x
compatibility branch. Updates unit tests to reflect the new architecture:
- apply_cipher_settings_before_key(): cipher_page_size, cipher_hmac_algorithm,
cipher_kdf_algorithm (must be set BEFORE key in SQLCipher 4.x)
- apply_sqlcipher_pragmas(): only kdf_iter (set AFTER key)
Test changes:
- Update test_applies_core_pragmas to only check kdf_iter
- Rename test_key_before_cipher_pragmas to test_sqlcipher_4x_pragma_order
- Update test_all_cipher_pragmas_present to verify both functions
- Add TestApplyCipherSettingsBeforeKey test class for new function
Add real SQLCipher integration tests that verify the registration → login
flow which was previously failing in UI tests. Also add pragma order
verification tests to ensure key is set before cipher pragmas.
New tests:
- test_sqlcipher_integration.py: 8 real SQLCipher tests (not mocked)
- TestPragmaOrder class: 5 pragma order verification tests
