mirror of
https://github.com/LearningCircuit/local-deep-research.git
synced 2026-06-16 12:02:34 +03:00
6e64601769
- Replace str(e) exception exposure with generic error messages in metrics and research routes to prevent information leakage - Replace logger.error with logger.exception in exception blocks for better debugging with full tracebacks - Simplify SafeSearchSetting logic in SearXNG engine to handle int/string cases more clearly feat: implement comprehensive call stack tracking and input/output token display - Add call stack tracking to identify which code files/functions trigger LLM calls * Capture calling_file, calling_function, and full call_stack for each LLM call * Add database migration for new call stack columns * Enhanced TokenCountingCallback with inspect.stack() integration - Enhance metrics dashboard with call stack analysis * Most active files and functions making LLM calls * Recent call stack traces with performance metrics * Input/output token breakdown in all visualizations - Update research detail timeline with comprehensive token tracking * Separate input/output token summary cards and charts * Enhanced timeline chart with 5 data series (cumulative + per-call) * Call stack traces for each LLM call with context - Improve token visualization throughout UI * Consistent input/output token display across all charts * Color-coded visualization (blue=input, red=output, teal=total) * Enhanced API endpoints with prompt/completion token breakdowns feat: add research mode filtering and expandable token breakdown to metrics dashboard - Add research mode filtering (Quick Summary, Detailed, All) alongside time range filtering - Implement expandable token card with detailed breakdown (average per research vs total usage) - Enhance token_counter.py with research mode SQL filtering and token breakdown calculations - Update metrics API endpoints to accept both time period and research mode parameters - Add smooth CSS animations and toggle functionality for expandable token details - Include tooltips for improved user experience and clarity docs: update metrics dashboard enhancement status with current achievements - Document implemented features: dual filtering, expandable token breakdown, enhanced tracking - Mark completion status for time-series analytics, quality metrics, and research deep dive - Add December 2024 major release summary highlighting key achievements - Identify next priority features: cost analysis, real-time monitoring, individual research pages - Transform from proposal document to progress tracker and roadmap fix: enable search engine tracking in metrics dashboard - Set search_engine_selected from user's search engine choice in research parameters - Fix empty 'Search Engines' section by using existing search engine parameter - Enable tracking for all individual search engines (DDG, Brave, etc.) not just meta search engine - Search engine usage will now be visible in metrics dashboard for future research sessions fix: enable search engine tracking in metrics dashboard - Set search_engine_selected from user's search engine choice in research parameters - Fix empty 'Search Engines' section by using existing search engine parameter - Enable tracking for all individual search engines (DDG, Brave, etc.) not just meta search engine - Search engine usage will now be visible in metrics dashboard for future research sessions fix: remove unused imports from search tracker feat: implement comprehensive 5-star user satisfaction rating system - Add research_ratings database table for local rating storage - Create interactive star rating widget on research results page - Build comprehensive rating analytics in metrics dashboard - Add cross-navigation between results and metrics pages - Implement smart tooltip positioning to prevent screen edge cutoff - Include privacy notices emphasizing local-only data storage - Add rating distribution charts and satisfaction statistics - Enable CSRF protection handling for browser-based rating saves The rating system provides valuable user feedback while maintaining complete privacy with local-only storage and clear privacy messaging. Resolve merge conflict in llm_config.py - integrate provider availability functions with enhanced token tracking refactor: address djpetti PR feedback - replace raw SQL with ORM queries - Remove hardcoded path from migrate_call_stack_tracking.py - Move SearchCall model from search_tracker.py to db_models.py - Replace all raw SQL queries with ORM queries in search_tracker.py - Replace raw SQL queries with ORM queries in token_counter.py main methods - Create query_utils.py with common time/mode filtering to remove code duplication - Remove duplicated _get_time_filter and _get_research_mode_filter methods This addresses all the feedback from djpetti's PR review focusing on: - Using ORM instead of raw SQL for better maintainability and safety - Centralizing models in the models file - Removing code duplication - Removing hardcoded development paths fix: correct import paths from .models to .db_models - Fix import in token_counter.py - Fix import in __init__.py - Fix import in database.py These imports were pointing to the old .models module instead of .db_models after the refactoring, causing metrics functionality to break. fix: remove duplicate api_enhanced_metrics function Removed the duplicate function I accidentally added since there was already an existing enhanced metrics endpoint at line 247. This was causing a function redefinition error that crashed the server. Fix metrics dashboard JavaScript errors and add browser-based UI tests - Fix SQLAlchemy syntax error in search_tracker.py (func.case -> case) - Restore working displayEnhancedMetrics function from before ORM conversion - Fix JavaScript syntax errors that prevented metrics dashboard from loading - Add comprehensive browser-based UI testing with Puppeteer - Add tests/ui_tests/ directory with page validation tests - Include focused metrics dashboard test to prevent future regressions The metrics dashboard now loads correctly and displays: - Total tokens: 41,801 - Total researches: 11 - Enhanced metrics with search engine stats and phase breakdowns Fix database session issues and add search activity chart - Fix missing List import in search_tracker.py that caused server crashes - Fix database session parameter issues in settings_routes.py - Fix file path resolution for research reports when server runs from src/ - Add search activity over time chart to metrics dashboard - Add comprehensive UI test suite for metrics, settings, and research pages - Update metrics template with side-by-side chart layout fix: remove duplicate api_enhanced_metrics function Removed the duplicate function I accidentally added since there was already an existing enhanced metrics endpoint at line 247. This was causing a function redefinition error that crashed the server. Fix metrics dashboard JavaScript errors and add browser-based UI tests - Fix SQLAlchemy syntax error in search_tracker.py (func.case -> case) - Restore working displayEnhancedMetrics function from before ORM conversion - Fix JavaScript syntax errors that prevented metrics dashboard from loading - Add comprehensive browser-based UI testing with Puppeteer - Add tests/ui_tests/ directory with page validation tests - Include focused metrics dashboard test to prevent future regressions The metrics dashboard now loads correctly and displays: - Total tokens: 41,801 - Total researches: 11 - Enhanced metrics with search engine stats and phase breakdowns Fix database session issues and add search activity chart - Fix missing List import in search_tracker.py that caused server crashes - Fix database session parameter issues in settings_routes.py - Fix file path resolution for research reports when server runs from src/ - Add search activity over time chart to metrics dashboard - Add comprehensive UI test suite for metrics, settings, and research pages - Update metrics template with side-by-side chart layout Fix token tracking and add star reviews analytics - Remove foreign key constraints from TokenUsage and ModelUsage models to fix broken token tracking - Add star reviews analytics page with LLM model and search engine rating visualizations - Implement SQLAlchemy ORM queries for all star reviews data aggregation - Enhance metrics dashboard with user satisfaction display and navigation - Improve token metrics display with better error handling for empty data - Add UI tests for star reviews functionality - Update gitignore to exclude node_modules and test artifacts Potential fix for code scanning alert no. 98: Information exposure through an exception Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> feat: implement comprehensive LLM cost analytics and pricing system - Add complete pricing API system under metrics/pricing/ with: * PricingFetcher for LLM model pricing data * PricingCache for performance optimization * CostCalculator for token usage cost analysis - Create dedicated cost analytics page at /metrics/costs with: * Real-time cost overview cards and insights * Interactive cost trends and model comparison charts * Most expensive research sessions tracking * Cost optimization tips and recommendations - Add comprehensive tooltips and information section explaining: * How costs are calculated and tracked * Pricing sources for major LLM providers * Cost optimization strategies * Privacy and data handling information - Integrate cost data into main metrics dashboard - Add navigation links between metrics pages - Update enhanced token tracking documentation with current status - Support for all major LLM providers (OpenAI, Anthropic, Google) - Local model cost tracking (free for Ollama/self-hosted) Implement provider-aware pricing system and neutral cost dashboard - Add provider column to TokenUsage table for accurate cost tracking - Update pricing system to use provider-first logic for local models - Enhance cost calculator to handle provider information throughout - Fix cost analytics dashboard with neutral, informative messaging - Add comprehensive local model pricing disclaimers and caveats - Remove judgmental cost warnings in favor of usage pattern insights - Support provider-based cost calculations in API endpoints - Include database migration for existing token usage records Key improvements: • Local models (Ollama, LM Studio, vLLM) now correctly show $0.00 costs • Provider-aware pricing prevents local models from getting commercial rates • Dashboard provides celebratory messages for local users without shaming commercial usage • Cost estimates include proper disclaimers about quality differences and hidden costs • Neutral optimization tips focus on usage patterns rather than arbitrary spending thresholds Add comprehensive testing documentation and fast health check tools - Create organized health check tests in tests/health_check/ directory - Add three health check implementations: Python, shell, and auto-detecting runner - Fast endpoint testing (1-5 seconds) without browser automation dependencies - Test all main pages, metrics dashboards, and API endpoints concurrently - Support both curl-only and Python requests implementations - Include comprehensive testing documentation and usage examples - Organize test suite with clear directory structure and guides Fix health check endpoints to match actual application routes - Correct /research/results to /research/results/1 (requires research_id parameter) - Update /history to /research/api/history (history blueprint registered under research/api) - Fix /settings to /research/settings (settings is under research namespace) - Update settings API paths to /research/settings/api/* (correct blueprint prefix) - Remove non-existent endpoints /api/health and /metrics/api/search-activity - Health check now achieves 100% success rate with 12 working endpoints - Verified all main pages, metrics dashboards, and API endpoints are accessible Address PR feedback: improve path handling and logging - Replace repeated path resolution logic with reusable function using pathlib - Use pathlib instead of os.path for modern path handling in history routes - Replace logger.error with logger.exception for better debugging in metrics routes - Pre-calculate OUTPUT_DIR as absolute path to avoid repeated calculations - Improve code maintainability and follow Python best practices Potential fix for code scanning alert no. 101: Information exposure through an exception Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> feat: implement comprehensive test automation system - Add tests/run_all_tests.py with 5 execution profiles (unit-only, fast, standard, full, ci) - Automatic server startup/shutdown with pdm run ldr-web - Smart test orchestration with proper error handling and cleanup - Detailed test reporting with timing and success rates - Complete documentation in tests/README.md with troubleshooting guide - Support for --no-server-start flag for manual server management - Unit tests reliably working (4.4s execution time) - Integration with existing pytest, Puppeteer, and health check systems improve: optimize Puppeteer test performance and capture successful results - Reduce navigation timeout from 10s to 3s for faster execution - Fix waitForTimeout compatibility issues with setTimeout pattern - Update test results showing successful cost analytics page testing - Capture comprehensive API responses and multiple screenshot states fix: use actual quick_summary research function instead of simplified LLM call - Address djpetti's review feedback on PR #300 - Replace direct LLM call with proper programmatic access function - Ensures API uses full research capabilities with search, question generation, and document analysis - Maintains consistent API interface with proper error handling fix: update quick_summary_test endpoint to use programmatic access - Replace mock data with actual research_functions.quick_summary call - Use minimal parameters (Wikipedia search, 1 iteration) for fast testing - Ensures all API endpoints consistently use programmatic access - Maintains proper error handling and response structure feat: add comprehensive REST API test suite - Add tests/api_tests/ directory with multiple test types - test_api_basic.py: Quick verification tests (< 5s runtime) - test_api_pytest.py: pytest-compatible tests for CI/CD integration - test_rest_api_simple.py: Extended testing with minimal research queries - test_rest_api.py: Full test suite with timeout handling - README.md: Comprehensive documentation for all test types Features: - ✅ Tests health check, documentation, and error handling endpoints - ✅ Verifies programmatic access integration (no direct LLM calls) - ✅ Uses minimal queries ('cat', 'dog', 'AI') for fast execution - ✅ Validates request/response structure and error handling - ✅ Confirms djpetti's PR feedback has been addressed - ✅ Skips detailed report testing as requested All basic tests pass (6/6) confirming REST API is operational. Delete docs/PR-SUMMARY.md fix: apply URL normalization consistently in Ollama availability check Delete docs/ollama-url-parsing-fix.md Fix formatting for CI workflow files