geckon01/local-deep-research
1
0
Fork 0
mirror of https://github.com/LearningCircuit/local-deep-research.git synced 2026-06-15 19:46:56 +03:00
Code Issues Packages Projects Releases Wiki Activity

refactor: cleanup remaining verified dead code across 5 areas (#3263)

Browse Source 
* refactor: cleanup remaining verified dead code across 5 areas

Dead templates, functions, storage ABCs, eslint duplicate, dev scripts.
All verified by 40 agents (20 scanning + 20 verification).

* revert: keep 3 dev scripts that have active references

- regenerate_golden_master.py: called by pre-commit hook
  .pre-commit-hooks/check-golden-master-settings.py
- restart_server.sh: documented in API testing guide, examples,
  and multiple README files
- run_tests.py: referenced in CONTRIBUTING.md testing section

Added inline comments noting the references so future cleanup
attempts don't remove them without updating dependents.

* revert: keep restart_server_debug.sh dev script

* revert: keep debug_pytest.py and stop_server.sh dev scripts

Small utility scripts that cost nothing to keep and are useful
for developers debugging CI failures and managing the dev server.

* docs: add do-not-remove comments to all dev scripts

Each script now documents why it must be kept:
- regenerate_golden_master.py: pre-commit hook dependency
- restart_server.sh: documented in API guides and examples
- restart_server_debug.sh: companion to restart_server.sh
- run_tests.py: referenced in CONTRIBUTING.md
- debug_pytest.py: developer utility for CI failure reproduction
- stop_server.sh: companion to restart_server.sh
This commit is contained in:
LearningCircuit
2026-03-28 16:03:21 +01:00
committed by GitHub
parent 18efd89eca
commit b28c80466c
13 changed files with 19 additions and 565 deletions
Download Patch File Download Diff File Expand all files Collapse all files

100
eslint.config.mjs
View File

@@ -1,100 +0,0 @@
export default [
{
files: ["**/*.js", "**/*.mjs"],
languageOptions: {
ecmaVersion: 2022,
sourceType: "module",
globals: {
// Browser globals
window: "readonly",
document: "readonly",
console: "readonly",
fetch: "readonly",
setTimeout: "readonly",
setInterval: "readonly",
clearTimeout: "readonly",
clearInterval: "readonly",
localStorage: "readonly",
sessionStorage: "readonly",
navigator: "readonly",
location: "readonly",
history: "readonly",
alert: "readonly",
confirm: "readonly",
FormData: "readonly",
URLSearchParams: "readonly",
URL: "readonly",
Blob: "readonly",
File: "readonly",
FileReader: "readonly",
Event: "readonly",
CustomEvent: "readonly",
HTMLElement: "readonly",
Node: "readonly",
NodeList: "readonly",
Element: "readonly",
MutationObserver: "readonly",
ResizeObserver: "readonly",
IntersectionObserver: "readonly",
WebSocket: "readonly",
AbortController: "readonly",
Request: "readonly",
Response: "readonly",
Headers: "readonly",
crypto: "readonly",
performance: "readonly",
requestAnimationFrame: "readonly",
cancelAnimationFrame: "readonly",
getComputedStyle: "readonly",
matchMedia: "readonly",
// ES2021+ globals
Promise: "readonly",
Map: "readonly",
Set: "readonly",
WeakMap: "readonly",
WeakSet: "readonly",
Symbol: "readonly",
Proxy: "readonly",
Reflect: "readonly",
BigInt: "readonly",
globalThis: "readonly",
// Common libraries used in project
marked: "readonly",
DOMPurify: "readonly",
io: "readonly",
Prism: "readonly",
ClipboardJS: "readonly",
jspdf: "readonly",
html2canvas: "readonly",
},
},
rules: {
// Critical errors that should block commits
"no-const-assign": "error",
"no-dupe-args": "error",
"no-dupe-keys": "error",
"no-duplicate-case": "error",
"no-func-assign": "error",
"valid-typeof": "error",
// Warnings for less critical issues
"no-empty": "warn",
"no-extra-semi": "warn",
"no-unreachable": "warn",
// Disabled for existing codebase compatibility
"no-unused-vars": "off",
"no-undef": "off",
"no-redeclare": "off",
},
},
{
ignores: [
"node_modules/**",
"dist/**",
"build/**",
"**/*.min.js",
"**/static/dist/**",
// Temporarily exclude until duplicate function declarations are refactored
"**/pages/news.js",
],
},
];

5
scripts/dev/debug_pytest.py Executable file → Normal file
View File

@@ -1,5 +1,8 @@
#!/usr/bin/env python3
"""Debug script to run pytest with CI settings and see what's failing."""
"""Debug script to run pytest with CI settings and see what's failing.
Do not remove — developer utility for reproducing CI failures locally.
"""
import os
import subprocess

3
scripts/dev/regenerate_golden_master.py
View File

@@ -3,6 +3,9 @@
Usage:
python scripts/dev/regenerate_golden_master.py
NOTE: This script is called by the pre-commit hook
.pre-commit-hooks/check-golden-master-settings.py — do not delete.
"""
import json

6
scripts/dev/restart_server.sh
View File

@@ -1,5 +1,11 @@
#!/bin/bash
# Script to restart the LDR server
#
# Referenced in:
# - tests/api_tests_with_login/README.md
# - examples/api_usage/http/README.md
# - examples/api_usage/http/advanced/simple_http_example.py
# Do not delete without updating those references.
echo "Stopping existing LDR server..."
pkill -f "python -m local_deep_research.web.app" 2>/dev/null || echo "No existing server found"

1
scripts/dev/restart_server_debug.sh
View File

@@ -1,5 +1,6 @@
#!/bin/bash
# Script to restart the LDR server with DEBUG logging enabled.
# Do not remove — companion to restart_server.sh for debug workflows.
#
# WARNING: Debug logs may contain sensitive data (queries, answers, API responses).
# This is for local development and feature testing only.

2
scripts/dev/run_tests.py
View File

@@ -3,6 +3,8 @@
Run tests for Local Deep Research.
This script runs pytest with appropriate configuration for the project.
Referenced in CONTRIBUTING.md — do not delete without updating that file.
"""
import os

4
scripts/dev/stop_server.sh
View File

@@ -1,5 +1,7 @@
#!/bin/bash
# Script to stop the LDR server
# Script to stop the LDR server.
# Do not remove — companion to restart_server.sh; handles graceful
# and forced shutdown of the dev server.
echo "Stopping LDR server..."

65
src/local_deep_research/news/core/storage.py
View File

@@ -202,68 +202,3 @@ class PreferenceStorage(BaseStorage):
) -> bool:
"""Update the user's preference embedding"""
pass
class SearchHistoryStorage(BaseStorage):
"""Interface for search history storage (if tracking enabled)"""
@abstractmethod
def record_search(
self, user_id: str, query: str, search_data: Dict[str, Any]
) -> str:
"""Record a search query"""
pass
@abstractmethod
def get_recent_searches(
self, user_id: str, hours: int = 48, limit: int = 50
) -> List[Dict[str, Any]]:
"""Get recent searches for a user"""
pass
@abstractmethod
def link_to_subscription(
self, search_id: str, subscription_id: str
) -> bool:
"""Link a search to a subscription created from it"""
pass
@abstractmethod
def get_popular_searches(
self, hours: int = 24, limit: int = 20
) -> List[Dict[str, Any]]:
"""Get popular searches across all users"""
pass
class NewsItemStorage(BaseStorage):
"""Interface for news item storage (the raw news data)"""
@abstractmethod
def get_recent(
self, hours: int = 24, limit: int = 50
) -> List[Dict[str, Any]]:
"""Get recent news items"""
pass
@abstractmethod
def store_batch(self, news_items: List[Dict[str, Any]]) -> List[str]:
"""Store multiple news items at once"""
pass
@abstractmethod
def update_votes(self, news_id: str, vote_type: str) -> bool:
"""Update vote counts"""
pass
@abstractmethod
def get_by_category(
self, category: str, limit: int = 50
) -> List[Dict[str, Any]]:
"""Get news items by category"""
pass
@abstractmethod
def cleanup_old_items(self, days: int = 7) -> int:
"""Remove old news items, return count deleted"""
pass

5
src/local_deep_research/research_library/services/download_service.py
View File

@@ -1524,11 +1524,6 @@ class DownloadService:
logger.exception("bioRxiv download failed")
return None
def _download_medrxiv(self, url: str) -> Optional[bytes]:
"""Download from medRxiv."""
# Same as bioRxiv
return self._download_biorxiv(url)
def _save_text_with_db(
self,
resource: ResearchResource,

20
src/local_deep_research/research_library/services/library_rag_service.py
View File

@@ -1490,23 +1490,3 @@ class LibraryRAGService:
"status": "error",
"error": f"Operation failed: {type(e).__name__}",
}
def search_library(
self, query: str, limit: int = 10, score_threshold: float = 0.0
) -> List[Dict[str, Any]]:
"""
Search library documents using semantic search.
Args:
query: Search query
limit: Maximum number of results
score_threshold: Minimum similarity score
Returns:
List of results with content, metadata, and similarity scores
"""
# This will be implemented when we integrate with the search system
# For now, raise NotImplementedError
raise NotImplementedError(
"Library search will be implemented in the search integration phase"
)

19
src/local_deep_research/research_library/utils/__init__.py
View File

@@ -235,25 +235,6 @@ def open_file_location(file_path: str) -> bool:
return False
def get_relative_library_path(absolute_path: str, username: str) -> str:
"""
Get the relative path from the library root.
Args:
absolute_path: The absolute file path
username: The username
Returns:
The relative path from the library root
"""
try:
library_root = get_library_storage_path(username)
return str(Path(absolute_path).relative_to(library_root))
except ValueError:
# Path is not relative to library root
return Path(absolute_path).name
def get_absolute_library_path(
relative_path: str, username: str
) -> Optional[Path]:

198
src/local_deep_research/web/templates/benchmark/dashboard.html
View File

@@ -1,198 +0,0 @@
{% extends "base.html" %}
{% block title %}Benchmarks{% endblock %}
{% block content %}
<div class="container">
<h1>LDR Benchmarks</h1>
<div class="card mb-4">
<div class="card-header">
<h2>Run Benchmark</h2>
</div>
<div class="card-body">
<form id="benchmarkForm">
<div class="form-group">
<label for="benchmarkType">Benchmark</label>
<select class="form-control" id="benchmarkType" required>
{% for benchmark in benchmarks %}
<option value="{{ benchmark.id }}">{{ benchmark.name }} - {{ benchmark.description }}</option>
{% endfor %}
</select>
</div>
<div class="form-group">
<label for="numExamples">Number of Examples</label>
<input type="number" class="form-control" id="numExamples" value="10" min="1" max="1000">
<small class="form-text text-muted">Higher numbers give more accurate results but take longer</small>
</div>
<div class="form-group">
<label for="searchIterations">Search Iterations</label>
<input type="number" class="form-control" id="searchIterations" value="3" min="1" max="10">
</div>
<div class="form-group">
<label for="questionsPerIteration">Questions Per Iteration</label>
<input type="number" class="form-control" id="questionsPerIteration" value="3" min="1" max="10">
</div>
<div class="form-group">
<label for="searchTool">Search Tool</label>
<select class="form-control" id="searchTool">
<option value="searxng">SearXNG</option>
<option value="wikipedia">Wikipedia</option>
<option value="auto">Auto (Multiple Engines)</option>
</select>
</div>
<button type="submit" class="btn btn-primary">Start Benchmark</button>
</form>
</div>
</div>
<div class="card mb-4" id="benchmarkStatus" style="display: none;">
<div class="card-header">
<h2>Benchmark Status</h2>
</div>
<div class="card-body">
<div class="ldr-progress mb-3" id="benchmarkProgressContainer" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="0" aria-label="Benchmark progress">
<div class="ldr-progress-bar" id="benchmarkProgress" style="width: 0%"></div>
</div>
<p id="statusMessage" aria-live="polite">Initializing benchmark...</p>
<button class="btn btn-secondary" id="viewResults" style="display: none;">View Results</button>
</div>
</div>
<div class="card" id="benchmarkResults" style="display: none;">
<div class="card-header">
<h2>Benchmark Results</h2>
</div>
<div class="card-body" id="resultsContent">
</div>
</div>
</div>
<script>
let currentJobId = null;
let statusInterval = null;
document.getElementById('benchmarkForm').addEventListener('submit', function(e) {
e.preventDefault();
const benchmarkType = document.getElementById('benchmarkType').value;
const numExamples = document.getElementById('numExamples').value;
const searchIterations = document.getElementById('searchIterations').value;
const questionsPerIteration = document.getElementById('questionsPerIteration').value;
const searchTool = document.getElementById('searchTool').value;
// Show status card
document.getElementById('benchmarkStatus').style.display = 'block';
document.getElementById('benchmarkResults').style.display = 'none';
document.getElementById('viewResults').style.display = 'none';
// Reset progress bar state
const progressBar = document.getElementById('benchmarkProgress');
const progressContainer = document.getElementById('benchmarkProgressContainer');
progressBar.style.width = '0%';
progressContainer.setAttribute('aria-valuenow', 0);
progressBar.classList.remove('bg-danger');
// Start benchmark
fetch('/benchmark/run', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
benchmark_type: benchmarkType,
num_examples: parseInt(numExamples),
search_iterations: parseInt(searchIterations),
questions_per_iteration: parseInt(questionsPerIteration),
search_tool: searchTool
})
})
.then(response => response.json())
.then(data => {
if (data.job_id) {
currentJobId = data.job_id;
document.getElementById('statusMessage').textContent = data.message;
// Start polling for status updates
statusInterval = setInterval(checkStatus, 2000);
} else {
document.getElementById('statusMessage').textContent = 'Error: ' + data.error;
}
})
.catch(error => {
document.getElementById('statusMessage').textContent = 'Error: ' + error;
});
});
function checkStatus() {
if (!currentJobId) return;
fetch('/benchmark/status/' + currentJobId)
.then(response => response.json())
.then(data => {
if (data.status === 'running') {
const runtime = data.runtime || 0;
document.getElementById('statusMessage').textContent = `Running benchmark... (${Math.round(runtime)}s elapsed)`;
const progressBar = document.getElementById('benchmarkProgress');
progressBar.style.width = '50%';
document.getElementById('benchmarkProgressContainer').setAttribute('aria-valuenow', 50);
} else if (data.status === 'completed') {
clearInterval(statusInterval);
document.getElementById('statusMessage').textContent = 'Benchmark completed successfully!';
const progressBar = document.getElementById('benchmarkProgress');
progressBar.style.width = '100%';
document.getElementById('benchmarkProgressContainer').setAttribute('aria-valuenow', 100);
document.getElementById('viewResults').style.display = 'inline-block';
document.getElementById('viewResults').onclick = function() {
showResults(currentJobId);
};
} else if (data.status === 'error') {
clearInterval(statusInterval);
document.getElementById('statusMessage').textContent = 'Error: ' + data.error;
const progressBar = document.getElementById('benchmarkProgress');
progressBar.style.width = '100%';
document.getElementById('benchmarkProgressContainer').setAttribute('aria-valuenow', 100);
progressBar.classList.add('bg-danger');
}
})
.catch(error => {
document.getElementById('statusMessage').textContent = 'Error checking status: ' + error;
});
}
function showResults(jobId) {
fetch('/benchmark/results/' + jobId)
.then(response => response.json())
.then(data => {
document.getElementById('benchmarkResults').style.display = 'block';
let html = '';
if (data.metrics) {
html += `<h3>Summary</h3>`;
html += `<p><strong>Accuracy:</strong> ${(data.metrics.accuracy * 100).toFixed(1)}%</p>`;
html += `<p><strong>Examples:</strong> ${data.metrics.total_examples}</p>`;
html += `<p><strong>Correct:</strong> ${data.metrics.correct}</p>`;
if (data.metrics.average_processing_time) {
html += `<p><strong>Average Processing Time:</strong> ${data.metrics.average_processing_time.toFixed(2)}s</p>`;
}
html += `<p><a href="${escapeHtml(data.report_path)}" target="_blank" class="btn btn-info">View Full Report</a></p>`;
} else {
html += `<p>No metrics available. Check the results file for details.</p>`;
html += `<p><a href="${escapeHtml(data.results_path)}" target="_blank" class="btn btn-info">View Results File</a></p>`;
}
document.getElementById('resultsContent').innerHTML = html;
})
.catch(error => {
document.getElementById('resultsContent').innerHTML = `<p>Error loading results: ${escapeHtml(error)}</p>`;
});
}
</script>
{% endblock %}

156
src/local_deep_research/web/templates/pages/news_cleaned.html
View File

@@ -1,156 +0,0 @@
{% extends "base.html" %}
{% block title %}LDR News - AI-Powered Research Feed{% endblock %}
{% block extra_head %}
<link rel="stylesheet" href="/static/css/news.css">
{% endblock %}
{% block content %}
<!-- Alert Container -->
<div id="news-alert" class="ldr-settings-alert-container" role="alert" style="display: none; position: fixed; top: 20px; right: 20px; z-index: 1000;"></div>
<div class="ldr-news-page-wrapper">
<!-- Main Content Container with Right Sidebar -->
<div class="ldr-news-container">
<!-- News Feed Section -->
<div class="ldr-news-feed">
<!-- Header with integrated search -->
<div class="ldr-feed-header-section">
<div class="ldr-feed-header">
<h1><i class="bi bi-newspaper"></i> Research News Feed</h1>
<div class="ldr-news-search-box">
<input type="text" id="news-search" class="ldr-form-control" placeholder="Search news topics...">
<button id="search-btn" class="btn btn-primary">
<i class="bi bi-search"></i>
</button>
</div>
</div>
<!-- Quick actions row -->
<div class="ldr-quick-actions-row">
<button class="btn btn-warning" id="refresh-feed-btn">
<i class="bi bi-arrow-clockwise"></i> Refresh Feed
</button>
<button class="btn btn-outline-secondary" data-bs-toggle="modal" data-bs-target="#subscriptionsModal">
<i class="bi bi-bell"></i> Manage Subscriptions
</button>
<div class="form-check form-switch d-inline-block ms-3">
<input class="form-check-input" type="checkbox" id="auto-refresh" checked>
<label class="form-check-label" for="auto-refresh">
<i class="bi bi-arrow-clockwise"></i> Auto-refresh
</label>
</div>
</div>
</div>
<!-- Filters Bar -->
<div class="ldr-filters-bar">
<div class="ldr-filter-section">
<div class="ldr-subscriptions-horizontal">
<select id="subscription-filter" class="form-select">
<option value="">All Topics</option>
<option value="ai">AI & Technology</option>
<option value="science">Science</option>
<option value="business">Business</option>
<option value="health">Health</option>
<option value="climate">Climate</option>
</select>
</div>
<div class="ldr-time-filter-group">
<button class="ldr-filter-btn active" data-time="today">Today</button>
<button class="ldr-filter-btn" data-time="week">This Week</button>
<button class="ldr-filter-btn" data-time="month">This Month</button>
</div>
<div class="ldr-impact-filter-group">
<label>Impact:</label>
<input type="range" class="ldr-impact-slider" min="0" max="10" value="0" id="impact-filter">
<span class="ldr-impact-value">0+</span>
</div>
</div>
</div>
<!-- News Cards Container -->
<div class="ldr-news-cards-container" id="news-container">
<!-- Loading skeleton -->
<div class="ldr-loading-skeleton">
<div class="ldr-skeleton-card"></div>
<div class="ldr-skeleton-card"></div>
<div class="ldr-skeleton-card"></div>
</div>
</div>
<!-- Load More -->
<div class="ldr-load-more-section">
<button class="btn btn-outline-primary" id="load-more-btn">
<i class="bi bi-plus-circle"></i> Load More Stories
</button>
</div>
</div>
<!-- Right Sidebar - Recommendations -->
<div class="ldr-recommendations">
<h3><i class="bi bi-stars"></i> Recommended Topics</h3>
<div class="ldr-recommendation-list" id="recommendations-list">
<div class="ldr-recommendation-item">
<i class="bi bi-lightning-charge"></i>
<span>Quantum Computing Breakthroughs</span>
<button class="btn btn-sm btn-outline-primary">Follow</button>
</div>
<div class="ldr-recommendation-item">
<i class="bi bi-cpu"></i>
<span>AI Safety Research</span>
<button class="btn btn-sm btn-outline-primary">Follow</button>
</div>
<div class="ldr-recommendation-item">
<i class="bi bi-graph-up"></i>
<span>Tech Market Analysis</span>
<button class="btn btn-sm btn-outline-primary">Follow</button>
</div>
</div>
<h3 class="mt-4"><i class="bi bi-clock-history"></i> Recent Searches</h3>
<div class="ldr-recent-searches" id="recent-searches">
<!-- Will be populated dynamically -->
</div>
</div>
</div>
</div>
<!-- Subscriptions Modal -->
<div class="modal fade" id="subscriptionsModal" tabindex="-1">
<div class="modal-dialog modal-lg">
<div class="modal-content">
<div class="modal-header">
<h5 class="modal-title"><i class="bi bi-bell"></i> Manage Subscriptions</h5>
<button type="button" class="btn-close" data-bs-dismiss="modal"></button>
</div>
<div class="modal-body">
<div class="ldr-subscription-section">
<h6>Your Subscriptions</h6>
<div id="current-subscriptions" class="ldr-subscriptions-list">
<!-- Will be populated dynamically -->
</div>
<h6 class="mt-4">Add New Subscription</h6>
<form id="add-subscription-form">
<div class="input-group">
<input type="text" class="ldr-form-control" placeholder="Enter topic or search query" id="new-subscription-input">
<button class="btn btn-primary" type="submit">
<i class="bi bi-plus"></i> Subscribe
</button>
</div>
</form>
</div>
</div>
</div>
</div>
</div>
<!-- Add news page JavaScript -->
<script defer src="/static/js/pages/news.js"></script>
<!-- Add Bootstrap JavaScript -->
{% endblock %}