* ci(nuclei): authenticate scan + seed URL list from Flask url_map
Previously the Nuclei DAST job ran against an unauthenticated single
target (`http://localhost:5000`) with no URL list. Because Nuclei is
template-driven (not a crawler) and the LDR app is auth-gated, the
scanner only ever saw `/auth/login`, the index, and a couple of
unauthenticated endpoints. The 2-minute scan over 10k templates produced
only 5 info-level findings, all of which were intentional design
choices (CSP `unsafe-inline`, SameSite=Lax, OPTIONS verb, form
detection) — i.e. the gate was effectively a green-checkmark.
Now the workflow:
1. Pre-creates the standard CI `test_admin` user via the existing
`init_test_database.py` helper (avoids slow registration + rate
limits).
2. Logs in via the real /auth/login flow with CSRF token, captures
the Flask session cookie, and verifies via /auth/check.
3. Dumps the Flask url_map (excluding parameterized routes, static,
and POST-only endpoints) into urls.txt so Nuclei probes every
blueprint route, not just `/`.
4. Runs Nuclei with `-list urls.txt` and the authenticated session
cookie via `-H "Cookie: session=..."`.
5. Filters to severity >= low to drop the four info-level findings
that are intentional design choices.
The session cookie is masked in logs via `::add-mask::` so it doesn't
leak into the run output. Test credentials match the convention used by
the playwright-webkit-tests and puppeteer-e2e-tests workflows.
Adds scripts/ci/dump_url_map.py as a small helper that imports
`create_app()` and iterates `app.url_map.iter_rules()` — reusable from
other DAST workflows (e.g. ZAP API scan) that benefit from URL seeding.
* ci(nuclei): address findings from review pass
Three differentiated review agents flagged five actionable items on the
authenticated-Nuclei PR. This commit addresses all five:
* dump_url_map.py: stop skipping parameterized routes. Substitute a
Flask-converter-appropriate placeholder (int/float→1, uuid→all-zeros,
default→"nuclei") so Nuclei still probes path-traversal / parameter-
injection / SQLi templates against routes like /research/<research_id>
and /api/research/<research_id>/status. Without this, the bulk of the
authenticated app surface (history, research, API blueprints) was
silently excluded — which defeats the PR's purpose.
* nuclei.yml -etags intrusive,dos,fuzz: now that Nuclei holds a real
session, default templates could mutate state or DoS the runner. This
is the standard exclusion set for authenticated DAST.
* nuclei.yml: replace `cat cookies.txt` in the missing-cookie error
branch with a column-filtered `awk` that omits the value column. The
cookie is masked via `::add-mask::` after this point, so the previous
branch could leak the session token in CI logs if the extraction
regex ever broke.
* nuclei.yml: add `sleep 2` between auth/check and the Nuclei step so
the post-login background thread (settings migration + library init,
see web/auth/routes.py:_perform_post_login_tasks) finishes before
probes start and 500 on settings-dependent routes.
* nuclei.yml: drop `# pragma: allowlist secret` on TEST_PASSWORD. The
repo uses gitleaks (.gitleaks.toml already allowlists `testpass123`),
not detect-secrets — the pragma was dead weight.
Out of scope for this PR (recorded but not changed):
- 3-way credential drift (init_test_database.py / nuclei.yml /
auth_helper.js all hardcode test_admin/testpass123)
- Nuclei binary version `latest` auto-updating (matches existing CI)
- create_app() side effects in dump_url_map.py (currently benign)
1. Move inline DB init script to external file (scripts/ci/init_test_database.py)
for better maintainability per djpetti's suggestion.
2. Fail fast in CI when pre-created user login fails instead of falling back
to slow registration. This makes debugging easier - if the CI user doesn't
work, something is wrong with workflow setup and should be fixed there.
Per djpetti's suggestion about developer experience.
The external script is now shared between critical-ui-tests.yml and
extended-ui-tests.yml, reducing duplication.
Co-authored-by: Daniel Petti <djpetti@gmail.com>
