Files
local-deep-research/examples/optimization/example_optimization.py
LearningCircuit 0c6635ecc2 feat: Add pre-commit hook to enforce pathlib usage (issue #640) (#656)
* feat: Add pre-commit hook to enforce pathlib usage (issue #640)

- Created check-pathlib-usage.py pre-commit hook using AST parsing
- Detects os.path usage and suggests pathlib alternatives
- Fixed os.path.normpath usage in auth/routes.py to use PurePosixPath
- Added hook configuration to .pre-commit-config.yaml

The hook provides helpful suggestions for replacing os.path calls with
their pathlib equivalents for better cross-platform compatibility.

Co-Authored-By: djpetti <djpetti@users.noreply.github.com>

* feat: Add missing pathlib pre-commit hook script

Co-Authored-By: djpetti <djpetti@users.noreply.github.com>

* refactor: Migrate core src modules from os.path to pathlib

- Fixed web/app_factory.py, config/llm_config.py, metrics/token_counter.py
- Fixed utilities/es_utils.py, web/routes/benchmark_routes.py
- Fixed web/routes/settings_routes.py, web_search_engines/engines/search_engine_local.py
- Replaced os.path.join() with Path() / syntax
- Replaced os.path.exists() with Path().exists()
- Replaced os.path.basename() with Path().name
- Replaced os.path.dirname() with Path().parent

Part of the migration to modern pathlib API for better cross-platform
compatibility and cleaner code.

Co-Authored-By: djpetti <djpetti@users.noreply.github.com>

* refactor: Migrate from os.path to pathlib in src and tests (issue #640)

Replaced os.path usage with pathlib.Path throughout:
- src/local_deep_research/benchmarks: All os.path.join, exists, dirname, basename, abspath replaced
- tests directory: Complete migration of all test files
- Improved cross-platform compatibility and code readability
- Kept os.path.expandvars in env_settings.py (no pathlib equivalent)

Part of pre-commit hook enforcement for pathlib usage.
Remaining work: examples/ and scripts/ directories.

Co-Authored-By: djpetti

* fix: Complete migration from os.path to pathlib.Path (issue #640)

Completed manual migration of all os.path usage to pathlib.Path across:
- scripts/ directory (3 files)
- examples/ directory (25 files total)
  - examples/benchmarks/ (8 files)
  - examples/optimization/ (16 files)
  - examples/show_env_vars.py
- src/local_deep_research/settings/env_settings.py

Changes made:
- Replaced os.path.join() with Path() / syntax
- Replaced os.path.exists() with Path().exists()
- Replaced os.path.dirname() with Path().parent
- Replaced os.path.basename() with Path().name or Path().stem
- Replaced os.path.abspath() with Path().resolve()
- Replaced os.makedirs() with Path().mkdir(parents=True, exist_ok=True)
- Added pathlib import where needed

Note: Kept os.path.expandvars in env_settings.py as there is no pathlib
equivalent. Added comment explaining this limitation.

This completes the pathlib migration for issue #640.

Co-Authored-By: djpetti

* fix: Allow os.path.expandvars in pathlib pre-commit hook

Updated the check-pathlib-usage.py pre-commit hook to skip checking
os.path.expandvars since it has no pathlib equivalent.

Changes:
- Added exception for expandvars in both visit_Attribute and visit_Call methods
- Added comment in equivalents dictionary noting expandvars is allowed
- This allows env_settings.py to use os.path.expandvars without failing checks

This resolves the pre-commit CI failure while maintaining the pathlib
enforcement for all other os.path methods.

Co-Authored-By: djpetti

---------

Co-authored-by: djpetti
2025-08-17 22:52:35 +02:00

94 lines
2.5 KiB
Python

# example_optimization.py - Quick Demo Version
"""
Full parameter optimization example for Local Deep Research.
This script demonstrates the full parameter optimization functionality.
Usage:
# Install dependencies with PDM
cd /path/to/local-deep-research
pdm install
# Run the script with PDM
pdm run python examples/optimization/example_optimization.py
"""
import json
from datetime import datetime, UTC
from pathlib import Path
# Import the optimization functionality
from local_deep_research.benchmarks.optimization import (
optimize_parameters,
)
# Loguru automatically handles logging configuration
def main():
# Create timestamp for unique output directory
timestamp = datetime.now(UTC).strftime("%Y%m%d_%H%M%S")
output_dir = str(
Path("examples")
/ "optimization"
/ "results"
/ f"optimization_results_{timestamp}"
)
Path(output_dir).mkdir(parents=True, exist_ok=True)
print(
f"Starting quick optimization demo - results will be saved to {output_dir}"
)
# Demo with just a single simple optimization
print("\n=== Running quick demo optimization ===")
# Create a very simple parameter set to test
param_space = {
"iterations": {
"type": "int",
"low": 1,
"high": 2,
"step": 1,
},
"questions_per_iteration": {
"type": "int",
"low": 1,
"high": 2,
"step": 1,
},
"search_strategy": {
"type": "categorical",
"choices": ["rapid"], # Just use the fastest strategy
},
}
balanced_params, balanced_score = optimize_parameters(
query="SimpleQA quick demo", # Task descriptor
search_tool="searxng", # Using SearXNG
n_trials=2, # Just 2 trials for quick demo
output_dir=str(Path(output_dir) / "demo"),
param_space=param_space, # Limited parameter space
metric_weights={"quality": 0.5, "speed": 0.5},
)
print(f"Best parameters: {balanced_params}")
print(f"Best score: {balanced_score:.4f}")
# Save demo results to a summary file
summary = {
"timestamp": timestamp,
"demo": {"parameters": balanced_params, "score": balanced_score},
}
with open(Path(output_dir) / "optimization_summary.json", "w") as f:
json.dump(summary, f, indent=2)
print(f"\nDemo complete! Results saved to {output_dir}")
print(f"Recommended parameters: {balanced_params}")
if __name__ == "__main__":
main()