mirror of
https://github.com/Mintplex-Labs/anything-llm.git
synced 2026-06-16 07:31:24 +03:00
4e324105bb
* add model router backend: schema, models, API, evaluator, provider, and cooldown * add model router frontend: settings pages, workspace integration, rule builder with DnD, and system LLM support * add routing indicator, new rule types, and UX improvements for model router * add provider credential validation to model router and fix icon alignment * add llm classifcation rules for model router * lint * add vision routing, agent support, and configurable cooldown to model router * add english translations for model router * move isPathMatch to path utils * fix model router sticking to first model in agent mode * make model router notifications ephemeral and add CSV rule support * improve model route notification ux and fix agent image routing * split RuleForm into subcomponents * extract RuleRow subcomponent and move ModalWrapper to NewRouterModal * remove /admin prefix from model router endpoints * use single query for model router list endpoint * clean up model router error handling and http methods * refactor model router rule update method and fix stale error field names * add model router env vars to .env.example * update fallback model description to mention llm rule classification * fix model router rule row alignment and badge readability * Model Router: Multi-condition rules, updated designs, and router edit support (#5478) * Implement draft designs * add matches comparator for prompt content * implement multi conditional to a calculated rule * fix between comparator * restyle model router modals to match light mode designs --------- Co-authored-by: shatfield4 <seanhatfield5@gmail.com> * replace model route spinner with webm animation * route llm classifier through aibitat tool call * fix: rule form state not resetting when making new rules * rename router form to rules page + fix and/plus button styles * add telemetry for model router creation * fix x button alignment on rule condition rows * simple UI changes * fix router * Refactor model routing into singleton with sticky routes and LLM caching Consolidate router/index.js, router/deterministic.js, and router/cooldown.js into a single ModelRouterService singleton class that manages all routing state. Key changes: - Sticky model routing: once a rule matches, the model stays active for follow-up messages that don't match any rule (e.g., "tell me more"), preventing unwanted bouncing back to the fallback model. - LLM classification cache: expensive LLM classifier calls are cached for the full sticky window (default 5 min), running at most once per window. - Calculated rules always re-evaluate (they're instant), so topic shifts that match a different rule still route correctly. - Route notifications only fire when the model actually changes, not on every message. - Router config is cached in-memory and invalidated on CRUD operations. - Detailed logging for cache hits/misses/expiry and rule evaluation. * fix model router notification text to be i18n * prevent history overwrite on regen * fix in-processing post-routed chat spacing in UI * reset router on thread reset * update translation key * refactor route name label on chat history * use system LLM on new model router * frontend modular cleanup * fix UI/UX on how rules are shown * rule builder translations * translations * endpoint cleanup * refactor rules * fix router translation for no rule remove from default plugins port support for model router to ephemeral * simplify stream handler * do not route notify on first message (non-agent chat) * fix cache keys * ttl reset on followups * Differentiate between short TTL and long TTL when we actually have run an LLM eval route * port embed support for router option * frontend fixes * dynamic key * 5315 i18n (#5666) * translations * norm * readme entry * update comments --------- Co-authored-by: Marcello Fitton <106866560+angelplusultra@users.noreply.github.com> Co-authored-by: Timothy Carambat <rambat1010@gmail.com>