ulysses/services/search
Kenny Van de Maele fafaf089c5
refactor(search): centralize the web-scraping User-Agent into one constant (#4325)
The outbound UA for web_fetch / web_search was inlined in four places with
two different values and nothing keeping them current: content.py pinned a
mid-2021 Chrome 91 build, and providers.py sent a bare Mozilla/5.0 in three
spots. Some sites serve a degraded or blocked page to a UA that old.

Add WEB_FETCH_USER_AGENT to src/constants.py (env-overridable, matching the
existing Copilot/Kimi UA-constant pattern) and import it in content.py and
providers.py. Default to a current, common desktop UA so pages return their
normal HTML: the market-leading desktop OS (Windows; NT 10.0 covers Windows
10 and 11) and browser (Chrome) on a current stable build. The version is now
bumped in one place.

Service-specific self-identifying agents (Copilot, Kimi, webhooks, cookbook)
are intentionally left separate. Adds a regression pinning the constant shape,
the env override, and a guard against a new inline Mozilla literal in the
search sources.

Closes #4324
2026-06-16 01:33:47 +00:00
..
__init__.py Odysseus v1.0 2026-05-31 23:58:26 +09:00
analytics.py fix: search analytics FileHandler crashes on startup writing to read-only image layer (#2366) 2026-06-07 19:26:22 +02:00
cache.py fix(search): write cache under DATA_DIR, guard mkdir against read-only path (#3334) 2026-06-07 22:37:12 +01:00
content.py refactor(search): centralize the web-scraping User-Agent into one constant (#4325) 2026-06-16 01:33:47 +00:00
core.py fix: web search content blocks numbered by fetch completion order break citations (#1672) 2026-06-03 14:22:55 +09:00
providers.py refactor(search): centralize the web-scraping User-Agent into one constant (#4325) 2026-06-16 01:33:47 +00:00
query.py fix: _extract_entities crashes on a non-string query (#1724) 2026-06-03 13:30:28 +09:00
ranking.py Word-boundary match for snippet and subject-term ranking (#1473 follow-up) (#2556) 2026-06-05 08:04:31 +01:00
service.py fix: SearchService.search() calls comprehensive_web_search incorrectly (broken public API) (#1720) 2026-06-03 13:33:56 +09:00