mirror of
https://github.com/LearningCircuit/local-deep-research.git
synced 2026-06-15 19:46:56 +03:00
8f380dd101
* fix: resolve provider selection case mismatch and inconsistent defaults Fixes #3339 - LLM provider dropdown broken on fresh install due to case mismatch between stored values (OLLAMA) and dropdown option values (ollama). Also fixes corruption-fix defaults using openai instead of ollama, and inconsistent gemma:latest fallback vs gemma3:12b in default_settings.json. * revert: remove unrelated themes.css changes from PR * refactor: add normalize_provider() and enforce lowercase canonical form Replace ad-hoc .upper()/.lower() calls with a centralized normalize_provider() function in llm/providers/base.py. This establishes lowercase as the single canonical form for provider values, matching VALID_PROVIDERS and get_llm(). Key changes: - Add normalize_provider() as module-level function in base.py - Use normalize_provider() in get_llm(), research_routes, followup routes - Fix research_routes comparisons to use lowercase ("ollama", "openai_endpoint") - Lowercase all hardcoded provider fallbacks in news module - Lowercase provider tags in settings_routes model listings - Lowercase fallback model options in settings.js - Remove llm.provider from KNOWN_SELECT_ISSUES (case mismatch fixed) - Update remaining gemma:latest references to gemma3:12b in tests/examples * refactor: adopt normalize_provider() across remaining codebase Extend normalize_provider() usage to all locations that previously used ad-hoc .lower()/.upper() calls or compared provider strings without normalization. High priority (fixes silent comparison failures): - benchmark_routes.py: 12 provider comparisons now normalized - benchmark_functions.py: 6 evaluation_provider comparisons - graders.py: 1 provider comparison - api_routes.py: 2 bare .lower() calls replaced - pricing_fetcher.py: 2 bare .lower() calls (one had no None guard) - wrapper.py: 1 bare .lower() call replaced - llm_config.py: get_selected_llm_provider() uses normalize_provider() Medium priority (normalize at DB write boundaries): - news/api.py: normalize before writing to NewsSubscription - news/flask_api.py: normalize at HTTP input boundary Low priority (consistency): - settings_routes.py: dict key construction uses normalize_provider() - auto_discovery.py: register_llm() uses normalize_provider() * fix(test): update assertions to expect lowercase provider after normalize_provider() Tests were asserting uppercase provider strings (e.g., "OPENAI", "OLLAMA") but normalize_provider() now returns lowercase canonical forms. Updated expected values in 8 failing tests across 4 files while keeping test inputs uppercase to verify normalization works correctly.
Optimization Tools for Local Deep Research
This directory contains scripts for optimizing Local Deep Research's parameters.
Parameter Optimization
Optimization helps find the best settings for different use cases:
- Balanced: Optimizes for a good balance of speed and quality
- Speed-focused: Prioritizes faster responses
- Quality-focused: Prioritizes more accurate, comprehensive answers
- Efficiency: Balances quality, speed, and resource usage
Available Scripts
Main Optimization Runner
run_optimization.py provides a command-line interface for running different types of optimization:
python run_optimization.py "What are the latest developments in fusion energy?" --mode quality --trials 20
Options:
query: The research query to use for optimization--output-dir: Directory to save results (default: "optimization_results")--search-tool: Search tool to use (default: "searxng")--model: Model name for the LLM (e.g., 'claude-3-sonnet-20240229')--provider: Provider for the LLM (e.g., 'anthropic', 'openai', 'openai_endpoint')--endpoint-url: Custom endpoint URL (e.g., 'https://openrouter.ai/api/v1' for OpenRouter)--api-key: API key for the LLM provider--temperature: Temperature for the LLM (default: 0.7)--trials: Number of parameter combinations to try (default: 30)--mode: Optimization mode ("balanced", "speed", "quality", "efficiency")--weights: Custom weights as JSON string, e.g., '{"quality": 0.7, "speed": 0.3}'
Example Scripts
example_optimization.py: Full example with all optimization modesexample_quick_optimization.py: Simplified example for quick testinggemini_optimization.py: Example using Gemini 2.0 Flash via OpenRouterllm_multi_benchmark.py: Example with multi-benchmark optimization and custom LLM settings
Utility Scripts
-
update_llm_config.py: Update LLM configuration in the databasepython update_llm_config.py --model "google/gemini-2.0-flash" --provider "openai_endpoint" --endpoint "https://openrouter.ai/api/v1" --api-key "your-api-key" -
run_gemini_benchmark.py: Run benchmarks with Gemini 2.0 Flash via OpenRouterpython run_gemini_benchmark.py --api-key "your-api-key" --examples 10
Important: Always update the LLM configuration in the database before running benchmarks or optimization to ensure consistent behavior. The utility scripts above help you do this.
How Optimization Works
The optimization process:
- Defines a parameter space to explore (iterations, questions per iteration, search strategy, etc.)
- Runs multiple trials with different parameter combinations
- Evaluates each combination using benchmarks
- Uses Optuna to efficiently search for the best parameters
- Returns the optimal parameters and stores detailed results
Example Parameter Space
Optimization explores parameters such as:
iterations: Number of search iterationsquestions_per_iteration: Number of questions to generate per iterationsearch_strategy: Search strategy to use ("standard", "rapid", "iterdrag", etc.)max_results: Maximum number of search results to consider- Other system-specific parameters
Using Custom LLM Models
The optimization tools support different LLM providers and models:
Via OpenRouter
To use models like Gemini or other models via OpenRouter:
python run_optimization.py "Research query" --model "google/gemini-2.0-flash-001" --provider "openai_endpoint" --endpoint-url "https://openrouter.ai/api/v1" --api-key "your-openrouter-api-key"
Or use the dedicated example:
python gemini_optimization.py --api-key "your-openrouter-api-key"
Direct Provider Access
To use models directly from providers like Anthropic or OpenAI:
python run_optimization.py "Research query" --model "claude-3-sonnet-20240229" --provider "anthropic" --api-key "your-anthropic-api-key"
Or for OpenAI:
python run_optimization.py "Research query" --model "gpt-4-turbo" --provider "openai" --api-key "your-openai-api-key"
Using Optimization Results
After running optimization, you can use the resulting parameters by updating your configuration:
from local_deep_research.api import quick_summary
results = quick_summary(
query="What are the latest developments in fusion energy?",
iterations=best_params["iterations"],
questions_per_iteration=best_params["questions_per_iteration"],
search_strategy=best_params["search_strategy"],
# Other optimized parameters
# You can also use custom LLM configuration:
model_name="your-model",
provider="your-provider"
)