fix(tests): two release-pipeline failures — UA assertions + WebKit budget (#4232)

Two test failures have been blocking every Create Release run on main
since 1f0b0a4a9 was the last release that actually exercised the test
gates. Both have the same shape: a test asserts a constant that the
codebase already changed.

1. Pytest UA assertions (search_engines, 4 files)
   #4130 / #4179 centralized every academic search engine on the
   canonical ``USER_AGENT`` constant in ``src/local_deep_research/
   constants.py``. The header value changed from ``LocalDeepResearch``
   to ``Local-Deep-Research/{__version__} (Academic Research Tool;
   ...)``, but the four engine-specific tests for gutenberg, openlibrary,
   pubchem, and zenodo still asserted the pre-fix substring and so
   failed in every release run since (FAILED ... tests/search_engines/
   test_search_engine_*.py::Test*Init::test_user_agent_header_set).
   Update them to assert the new canonical substring
   ``Local-Deep-Research``. The same value is what
   tests/web_search_engines/test_engine_user_agents.py enforces at the
   source level — no other test referenced the old literal.

2. Mobile Safari "Content Above Mobile Nav" for ``/`` (Research Home)
   The test has been the sole failure of the test-gate / mobile-safari
   job in every recent release run (runs 26331108360, 26331198212, etc).
   PR #4076 dropped the WebKit-skip on this assertion and PR #4215
   capped one defensive ``waitForLoadState`` at 3s, but the test still
   times out at 30s and dies inside the very first ``page.evaluate``
   of ``findElementsBehindMobileNav`` with the misleading
   "Target page, context or browser has been closed" message.

   Root cause: two stacked defensive waits ate the remaining budget on
   slow WebKit CI runners on ``/``:
   - A ``waitForFunction`` polling for the help-panel collapse
     condition. The preceding ``page.evaluate`` already sets
     ``style.display = 'none'`` synchronously on every panel, so the
     follow-up poll is redundant — but on WebKit it occasionally
     consumed its full 5s timeout.
   - A ``waitForFunction`` polling for
     ``scrollY >= scrollHeight - 1`` after scrolling to the bottom.
     ``window.scrollTo`` is synchronous inside ``page.evaluate`` and
     commits before the evaluate resolves, so this poll is also
     redundant. Worse, on ``/`` lazy-loaded content keeps
     ``document.body.scrollHeight`` growing, so the condition could
     never become true and the poll always ran the full 5s timeout.

   Drop both polls and raise the per-test budget to 60s with
   ``test.describe.configure({ timeout: 60000 })`` so the test reflects
   real WebKit timing under CI load rather than misattributing slowness
   to a closed page. The 60s ceiling is generous: locally the assertion
   completes in well under 10s; the extra headroom only matters when
   the runner is under load.
This commit is contained in:
LearningCircuit
2026-05-24 08:58:09 +02:00
committed by GitHub
parent aba32cb627
commit 4b5f8f03db
5 changed files with 20 additions and 15 deletions

View File

@@ -61,7 +61,7 @@ class TestGutenbergSearchEngineInit:
engine = GutenbergSearchEngine()
assert "User-Agent" in engine.headers
assert "LocalDeepResearch" in engine.headers["User-Agent"]
assert "Local-Deep-Research" in engine.headers["User-Agent"]
class TestGutenbergQueryBuilding:

View File

@@ -62,7 +62,7 @@ class TestOpenLibrarySearchEngineInit:
engine = OpenLibrarySearchEngine()
assert "User-Agent" in engine.headers
assert "LocalDeepResearch" in engine.headers["User-Agent"]
assert "Local-Deep-Research" in engine.headers["User-Agent"]
class TestOpenLibraryQueryBuilding:

View File

@@ -58,7 +58,7 @@ class TestPubChemSearchEngineInit:
engine = PubChemSearchEngine()
assert "User-Agent" in engine.headers
assert "LocalDeepResearch" in engine.headers["User-Agent"]
assert "Local-Deep-Research" in engine.headers["User-Agent"]
class TestPubChemSearchExecution:

View File

@@ -64,7 +64,7 @@ class TestZenodoSearchEngineInit:
engine = ZenodoSearchEngine()
assert "User-Agent" in engine.headers
assert "LocalDeepResearch" in engine.headers["User-Agent"]
assert "Local-Deep-Research" in engine.headers["User-Agent"]
class TestZenodoQueryBuilding:

View File

@@ -336,6 +336,14 @@ test.describe('All Pages - Touch Targets', () => {
// ============================================
test.describe('All Pages - Content Above Mobile Nav', () => {
// WebKit on CI is markedly slower than Chromium/Firefox at firing `load`
// and at running cross-process page.evaluate against `/`. The defensive
// waits below + the chunked DOM walk legitimately need more than the
// default 30s budget when the runner is under load. Raise the per-test
// budget so the test reflects real WebKit timing rather than misleadingly
// failing with "Target page... has been closed" on the first evaluate.
test.describe.configure({ timeout: 60000 });
for (const pageInfo of PAGES) {
test(`${pageInfo.name} content not hidden behind mobile nav`, async ({ page, isMobile }) => {
if (!isMobile) {
@@ -361,7 +369,9 @@ test.describe('All Pages - Content Above Mobile Nav', () => {
await page.waitForSelector('#benchmark-form input, #benchmark-form select, #benchmark-form textarea', { timeout: 10000 }).catch(() => {});
}
// Collapse all help panels before checking - they can expand from localStorage state
// Collapse all help panels before checking - they can expand from localStorage state.
// The evaluate runs synchronously in-page, so panels are already hidden when it returns
// — no need for a follow-up waitForFunction that previously ate up to 5s of budget.
await page.evaluate(() => {
if (window.HelpService && typeof window.HelpService.collapseAll === 'function') {
window.HelpService.collapseAll();
@@ -371,11 +381,6 @@ test.describe('All Pages - Content Above Mobile Nav', () => {
el.style.display = 'none';
});
});
// Wait for help panels to finish collapsing
await page.waitForFunction(() => {
const panels = document.querySelectorAll('.ldr-help-panel-content');
return panels.length === 0 || Array.from(panels).every(el => el.style.display === 'none' || el.offsetHeight === 0);
}, { timeout: 5000 }).catch(() => {});
// Ensure page is fully loaded before heavy DOM operations (WebKit stability).
// Cap with an explicit short timeout: without it, the default navigationTimeout
@@ -383,12 +388,12 @@ test.describe('All Pages - Content Above Mobile Nav', () => {
// `load`, leaving zero time for the DOM walk below.
await page.waitForLoadState('load', { timeout: 3000 }).catch(() => {});
// Scroll to bottom to check last content
// Scroll to bottom to check last content. scrollTo() is synchronous inside
// page.evaluate(): the browser commits the scroll before the evaluate resolves,
// so an additional `waitForFunction(scrollY >= scrollHeight)` poll is redundant.
// The old poll could also never resolve on `/` when lazy-loaded content kept
// growing `document.body.scrollHeight`, eating its full 5s timeout on every run.
await page.evaluate(() => window.scrollTo(0, document.body.scrollHeight));
// Wait for scroll to settle at the bottom
await page.waitForFunction(() => {
return (window.innerHeight + window.scrollY) >= document.body.scrollHeight - 1;
}, { timeout: 5000 }).catch(() => {});
const hiddenElements = await findElementsBehindMobileNav(page, MOBILE_NAV_SELECTOR);