- Fix Ollama API key NoSettingsContextError by using empty string default
- Add ui_element fallback in thread_settings to handle custom overrides
- Pass settings_snapshot to file_write_verifier for programmatic mode
- Auto-detect ui_element type in settings_utils (checkbox/number/json)
- Add api.allow_file_output to programmatic examples
- Add programmatic API tests to api-tests.yml workflow
These fixes enable the programmatic API to work correctly without database
access, as intended for programmatic usage scenarios.
* refactor: Replace programmatic_mode setting with explicit argument (#627)
Move programmatic_mode from settings_snapshot to an explicit constructor
parameter for better API clarity and to prevent accidental modification.
Changes:
- Add programmatic_mode as explicit parameter to AdvancedSearchSystem
- Update BaseSearchEngine and RateLimitTracker to accept programmatic_mode
- Maintain backward compatibility in research_functions.py
- Update all examples to use new parameter style
- Update tests to reflect new API
This change makes the programmatic mode behavior more explicit and
prevents users from accidentally modifying this internal flag through settings.
Thanks to @djpetti for the review and suggestion in PR #616.
* feat: Complete programmatic_mode refactor as per review feedback
- Made programmatic_mode an explicit parameter with default value True in research_functions
- Added programmatic_mode parameter to create_search_engine and get_search functions
- Updated MetaSearchEngine and LocalAllSearchEngine to accept and pass programmatic_mode
- Removed extraction of programmatic_mode from settings_snapshot
This completes the refactor suggested by @djpetti, fully eliminating
programmatic_mode from settings and making it a first-class parameter.
Thanks again to @djpetti for the thorough review\!
* fix: Complete removal of programmatic_mode from settings_snapshot
- Remove programmatic_mode from all example settings_snapshot calls
- Add programmatic_mode=True as explicit parameter to all API function calls
- Update README documentation to reflect the new API pattern
- This addresses @djpetti's review comments to fully eliminate programmatic_mode from settings
As noted by @djpetti, programmatic_mode should be a normal function argument
rather than a setting, since it's an internal flag that users shouldn't modify.
All examples now properly pass it as an explicit parameter to quick_summary(),
detailed_research(), and other API functions.
* fix: Enable generate_report() to work in programmatic mode
- Fix generate_report() to properly handle settings_snapshot parameter
- Update IntegratedReportGenerator to avoid unnecessary get_llm() calls
- Add advanced_features_example.py showing report generation and analysis
- Fix all examples to use correct create_settings_snapshot() syntax with overrides parameter
- Update README to show generate_report() is now working
The main issue was:
1. generate_report() wasn't properly creating settings snapshots like other API functions
2. IntegratedReportGenerator was calling get_llm() without settings context
3. Examples were incorrectly passing settings dict as base_settings instead of overrides
* fix: Skip Puppeteer Chrome download and use Playwright's Chrome instead
- Set PUPPETEER_SKIP_DOWNLOAD and PUPPETEER_SKIP_CHROMIUM_DOWNLOAD to prevent timeout
- Configure Puppeteer to use Playwright's already installed Chrome binary
- This fixes the npm install timeout issue in CI/CD pipeline
- Remove redundant examples (example_with_ollama, quick_summary_example)
- Remove outdated auth-based examples (comprehensive_tutorial, programmatic_access_v1, retriever_usage_example)
- Add search_strategies_example.py demonstrating source-based and focused-iteration strategies
- Add hybrid_search_example.py showing multiple retrievers and hybrid search patterns
- Update README with comprehensive documentation of all examples
- Add TODO note in simple_programmatic_example about generate_report() auth requirement
This cleanup focuses the examples on demonstrating programmatic access without authentication,
making them easier to understand and use.
- Fix get_setting_from_snapshot to always extract values from dict structures
- Update simple_programmatic_example to use settings utilities
- Add workaround in search_config for dict values
- Example now uses create_settings_snapshot for proper defaults
- Add required settings_snapshot with all necessary LLM and search settings
- Pass settings_snapshot to all API function calls
- Example now runs without crashing (though returns empty results in programmatic mode)
- Preserves original example structure and API usage patterns
BREAKING CHANGE: Data files now stored in platform-specific user directories
with SQLCipher encryption. Users must register/login to access the application.
## Major Features
### Security & Authentication
- Implemented complete multi-user authentication system with Flask-Login
- Per-user SQLCipher encrypted databases (falls back to SQLite with warnings)
- Secure session management with proper CSRF protection
- Password hashing with bcrypt for user credentials
- Complete isolation between user data - no cross-user access possible
- Thread-safe database connections with proper session management
### Database Architecture
- Migrated from single shared database to per-user encrypted databases
- Centralized auth database for user management
- User-specific databases for research data, settings, and metrics
- Automatic database initialization on user registration
- Platform-specific data directories using platformdirs library
- Removed all hardcoded paths and personal information
### User Experience
- Registration page with data privacy acknowledgment
- Login/logout functionality with session persistence
- Automatic redirect to login for unauthenticated access
- Research queue system with 3 concurrent research limit per user
- Real-time queue position updates
- Comprehensive error handling with user-friendly messages
### API & Routes
- All API endpoints now require authentication
- Updated routes: /auth/register, /auth/login, /auth/logout, /auth/check
- Protected research submission and history endpoints
- Proper JSON error responses for API routes
- CSRF token validation for state-changing operations
### Testing
- Added 53 Puppeteer tests for UI authentication flows
- Comprehensive auth integration tests (248 Python test files)
- Multi-user concurrent access testing
- Queue system testing with position tracking
- Database migration and encryption tests
### Configuration
- Single LDR_DATA_DIR environment variable for data location
- LDR_ALLOW_UNENCRYPTED environment variable for development
- Updated Docker configuration for proper volume mounting
- Removed multiple environment variables for simplicity
### Documentation
- Added DATA_MIGRATION_GUIDE.md for upgrade instructions
- Added SQLCIPHER_INSTALL.md for encryption setup
- Updated environment configuration documentation
- Professional error messages throughout
## Technical Improvements
- Replaced raw SQL with SQLAlchemy ORM throughout
- Proper database session management with context managers
- Thread-local storage for database connections
- Automatic cleanup of stale sessions
- Rate limiting infrastructure for future use
- Comprehensive logging with loguru
## Files Changed
- 322 files modified/added
- 248 Python files (core functionality and tests)
- 53 JavaScript files (Puppeteer tests)
- 6 Markdown files (documentation)
- No binary files, screenshots, or database files included
- All test credentials properly marked with pragma comments
This migration ensures each user's research data is completely isolated and
encrypted, providing enterprise-grade security for sensitive research operations.
* feat: Add LangChain retriever integration for vector store support
- Add RetrieverRegistry for dynamic retriever registration
- Create RetrieverSearchEngine wrapper for LangChain BaseRetriever
- Integrate retrievers with search factory and config system
- Add retrievers parameter to all API functions
- Include comprehensive test suite and examples
- Support thread-safe operations and multiple retrievers
This allows users to pass any LangChain retriever (FAISS, Pinecone,
Vertex AI, etc.) to LDR and use it as a search engine seamlessly.
* refactor: Organize API examples into structured folders
- Create api_usage/ directory with programmatic/ and http/ subdirectories
- Move existing examples to appropriate folders
- Add comprehensive HTTP API examples (simple and advanced)
- Add curl examples for command-line usage
- Add simple programmatic example for quick start
- Include README explaining when to use each API type
* chore: Remove old example files from root examples directory
Files have been moved to examples/api_usage/programmatic/
* fix: Address PR review comments
- Replace logger.error with logger.exception for better error tracking
- Default retriever name to class name if not provided
