From dffd27f883a747f1a3eaa7970deae4f0a6914b28 Mon Sep 17 00:00:00 2001 From: Ivan-Apro Date: Wed, 10 Jun 2026 19:22:17 +0000 Subject: [PATCH] =?UTF-8?q?=F0=9F=8E=AB=20fix:=20Forward=20User=20Auth=20H?= =?UTF-8?q?eaders=20on=20Model=20Fetch=20(#13616)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * 🔐 fix: Resolve template vars and respect custom Authorization on model fetch The custom-endpoint model fetch path in `fetchModels` had two bugs that silently broke per-user authentication on `GET /v1/models`: 1. Template variables in the configured `headers:` block were not substituted on the OpenAI-compatible branch. Only the Ollama branch ran `resolveHeaders`, so placeholders like `{{LIBRECHAT_OPENID_ID_TOKEN}}` were forwarded as literal strings on every other endpoint. 2. After spreading the (unresolved) headers into the request, the code unconditionally executed `options.headers.Authorization = \`Bearer ${apiKey}\`` and clobbered any `Authorization` the operator had set in `headers:`. Combined, these meant a config like ```yaml endpoints: custom: - name: "MyProxy" apiKey: "${MY_API_KEY}" headers: authorization: "Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}" ``` sent `Authorization: Bearer ${MY_API_KEY}` on `/v1/models` instead of the user's resolved JWT — even with `OPENID_REUSE_TOKENS=true` set. Auth-aware proxies (e.g. LiteLLM with team-based JWT auth) therefore could not return a per-user filtered model list. This change runs `headers` through `resolveHeaders` (mirroring the Ollama branch) and only falls back to the apiKey-based default when the resolved headers do not already supply an `Authorization` (case-insensitive). All other endpoints behave unchanged: when no `Authorization` is configured, the existing `Bearer ${apiKey}` default still applies. Tests added: - Template variables in custom headers are resolved on the OpenAI path. - A config-supplied `Authorization` overrides the apiKey default. - The override check is case-insensitive (`authorization` works too). Co-Authored-By: Claude Opus 4.7 (1M context) * 🔐 fix: Address review — import order, P1 token leak guard, P2 token-config path - Fix sort-imports drift in `models.ts` and `custom/initialize.ts`. - P1: in `loadConfigModels` (`config/models.ts`), do not forward `endpointHeaders` to `fetchModels` when `baseURLIsUserProvided`. Configured templates such as `Authorization: Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}` would otherwise resolve and be sent to a destination the user controls — leaking the user's identity token. Header overrides remain in place when only the apiKey is user-provided (admin-trusted base URL). - P2: in `initializeCustom` (`custom/initialize.ts`), the token-config fetch path now forwards `headers` and `userObject` to `fetchModels` (mirroring the auth-aware behaviour), with the same `userProvidesURL` guard. Additionally, when `endpointConfig.headers` is set the model cache is skipped to avoid a per-user filtered response leaking across users; token-config caching was already user-keyed when key/URL are user-provided. Tests added: - `config/models.spec.ts` (new): verifies the P1 guard — headers are dropped when the base URL is user-provided, and forwarded when only the apiKey is user-provided. - `custom/initialize.spec.ts`: three cases for the P2 path covering header forwarding to admin-trusted base URLs, header drop on user-provided base URLs, and absence of `skipCache` when no headers are configured. Co-Authored-By: Claude Opus 4.7 (1M context) * 🔐 fix: Scope model + token-config caches when user-bound headers are forwarded Two follow-up fixes from the second review pass: P1.1 (`fetchModels` / `models.ts`): the MODEL_QUERIES cache is keyed by baseURL+apiKey only. When callers forward headers containing template variables that resolve against the current user (e.g. `Authorization: Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}`), one user's filtered list could be served to the next request that happens to share the same baseURL+apiKey. `shouldCache` now skips the cache whenever both `headers` and `userObject` are supplied — that's the unambiguous signal the response is being resolved against a specific user identity. Existing callers that pass neither (fetchOpenAIModels, fetchAnthropicModels) keep their cache. P1.2 (`initializeCustom` / `custom/initialize.ts`): the surrounding tokenConfigCache uses `tokenKey === endpoint` when key+URL are admin-configured. With user-bound headers forwarded, the first user's token config could be cached for the shared endpoint and served to other users until TTL. `tokenKey` is now also user-scoped when `endpointConfig.headers` will be forwarded (i.e. base URL is admin-trusted, so the security guard leaves headers in place). Also removed the explicit `skipCache: !!endpointConfig.headers` from the fetchModels call in initializeCustom — the new fetchModels-level rule covers it uniformly across both call sites. Tests added: - models.spec.ts: cache skipped on `headers + userObject`; cache used when only one of them is supplied (existing callers unaffected). - initialize.spec.ts: `tokenKey` is `${endpoint}:${userId}` when headers will be forwarded, and `endpoint` (unscoped) when no headers are configured. Co-Authored-By: Claude Opus 4.7 (1M context) * 🔐 fix: Include header fingerprint in in-request model fetch coalescing key `loadConfigModels` coalesces concurrent fetches for endpoints that share the same admin-trusted `${BASE_URL}__${API_KEY}` via `fetchPromisesMap`. With per-endpoint `headers:` overrides — including templates that resolve against the current user — that key is too coarse: two custom endpoints sharing a proxy URL/key but configuring different headers (e.g. distinct `X-Tenant` values, or different static `Authorization` strings) would share a single fetch promise, and the first endpoint's filtered response would be returned for the second endpoint within the same request. Fix: include a stable SHA-256 fingerprint of the configured headers in the coalescing key. Endpoints that genuinely share `baseURL + apiKey + headers` still share one fetch (preserves the existing optimisation); endpoints that differ in headers each get their own fetch. Test added in `config/models.spec.ts`: - Two endpoints sharing baseURL+apiKey but with different headers result in two `fetchModels` calls, each carrying the right headers. - Two endpoints sharing baseURL+apiKey AND identical headers still coalesce into a single `fetchModels` call. Co-Authored-By: Claude Opus 4.7 (1M context) --------- Co-authored-by: Claude Opus 4.7 (1M context) --- .../api/src/endpoints/config/models.spec.ts | 197 ++++++++++++++++++ packages/api/src/endpoints/config/models.ts | 32 ++- .../src/endpoints/custom/initialize.spec.ts | 142 +++++++++++++ .../api/src/endpoints/custom/initialize.ts | 32 ++- packages/api/src/endpoints/models.spec.ts | 123 +++++++++++ packages/api/src/endpoints/models.ts | 32 ++- 6 files changed, 550 insertions(+), 8 deletions(-) create mode 100644 packages/api/src/endpoints/config/models.spec.ts diff --git a/packages/api/src/endpoints/config/models.spec.ts b/packages/api/src/endpoints/config/models.spec.ts new file mode 100644 index 0000000000..b52074f0ac --- /dev/null +++ b/packages/api/src/endpoints/config/models.spec.ts @@ -0,0 +1,197 @@ +import { AuthType, EModelEndpoint } from 'librechat-data-provider'; +import type { ServerRequest } from '~/types'; +import { createLoadConfigModels } from './models'; + +jest.mock('~/utils', () => { + const original = jest.requireActual('~/utils'); + return { + ...original, + // Inline literal — jest.mock() factory may not reference imports. + isUserProvided: (val: string) => val === 'user_provided', + }; +}); + +describe('createLoadConfigModels – user-provided baseURL header guard', () => { + const fetchModels = jest.fn().mockResolvedValue([]); + + const buildAppConfig = (endpointOverrides: Record) => ({ + endpoints: { + [EModelEndpoint.custom]: [ + { + name: 'TestProxy', + baseURL: AuthType.USER_PROVIDED, + apiKey: AuthType.USER_PROVIDED, + models: { fetch: true }, + ...endpointOverrides, + }, + ], + }, + }); + + beforeEach(() => { + fetchModels.mockReset().mockResolvedValue([]); + }); + + it('does NOT forward configured headers when baseURL is user-provided', async () => { + const headers = { + Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}', + 'X-User-Email': '{{LIBRECHAT_USER_EMAIL}}', + }; + + const loadConfigModels = createLoadConfigModels({ + getAppConfig: jest.fn().mockResolvedValue(buildAppConfig({ headers })), + getUserKeyValues: jest.fn().mockResolvedValue({ + apiKey: 'sk-user-key', + baseURL: 'https://user-controlled.example.com/v1', + }), + fetchModels, + }); + + const req = { + user: { id: 'user-1', email: 'user@example.com' }, + config: undefined, + } as unknown as ServerRequest; + + await loadConfigModels(req); + + expect(fetchModels).toHaveBeenCalledTimes(1); + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + name: 'TestProxy', + baseURL: 'https://user-controlled.example.com/v1', + headers: undefined, + }), + ); + }); + + it('DOES forward configured headers when baseURL is admin-trusted (only apiKey is user-provided)', async () => { + const headers = { + Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}', + }; + + const loadConfigModels = createLoadConfigModels({ + getAppConfig: jest.fn().mockResolvedValue({ + endpoints: { + [EModelEndpoint.custom]: [ + { + name: 'TrustedProxy', + baseURL: 'https://admin-trusted.example.com/v1', + apiKey: AuthType.USER_PROVIDED, + models: { fetch: true }, + headers, + }, + ], + }, + }), + getUserKeyValues: jest.fn().mockResolvedValue({ + apiKey: 'sk-user-key', + baseURL: undefined, + }), + fetchModels, + }); + + const req = { + user: { id: 'user-1' }, + config: undefined, + } as unknown as ServerRequest; + + await loadConfigModels(req); + + expect(fetchModels).toHaveBeenCalledTimes(1); + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + name: 'TrustedProxy', + baseURL: 'https://admin-trusted.example.com/v1', + headers, + }), + ); + }); +}); + +describe('createLoadConfigModels – in-request fetch coalescing', () => { + const fetchModels = jest.fn().mockResolvedValue([]); + + beforeEach(() => { + fetchModels.mockReset().mockResolvedValue([]); + }); + + it('does NOT coalesce two endpoints with the same baseURL+apiKey but different headers', async () => { + const loadConfigModels = createLoadConfigModels({ + getAppConfig: jest.fn().mockResolvedValue({ + endpoints: { + [EModelEndpoint.custom]: [ + { + name: 'TenantA', + baseURL: 'https://shared-proxy.example.com/v1', + apiKey: 'sk-shared', + models: { fetch: true }, + headers: { 'X-Tenant': 'a' }, + }, + { + name: 'TenantB', + baseURL: 'https://shared-proxy.example.com/v1', + apiKey: 'sk-shared', + models: { fetch: true }, + headers: { 'X-Tenant': 'b' }, + }, + ], + }, + }), + getUserKeyValues: jest.fn(), + fetchModels, + }); + + const req = { + user: { id: 'user-1' }, + config: undefined, + } as unknown as ServerRequest; + + await loadConfigModels(req); + + expect(fetchModels).toHaveBeenCalledTimes(2); + const headersByName = new Map | undefined>(); + for (const call of fetchModels.mock.calls) { + headersByName.set(call[0].name, call[0].headers); + } + expect(headersByName.get('TenantA')).toEqual({ 'X-Tenant': 'a' }); + expect(headersByName.get('TenantB')).toEqual({ 'X-Tenant': 'b' }); + }); + + it('still coalesces two endpoints that share baseURL+apiKey AND identical headers', async () => { + const sharedHeaders = { Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}' }; + const loadConfigModels = createLoadConfigModels({ + getAppConfig: jest.fn().mockResolvedValue({ + endpoints: { + [EModelEndpoint.custom]: [ + { + name: 'AliasOne', + baseURL: 'https://shared-proxy.example.com/v1', + apiKey: 'sk-shared', + models: { fetch: true }, + headers: sharedHeaders, + }, + { + name: 'AliasTwo', + baseURL: 'https://shared-proxy.example.com/v1', + apiKey: 'sk-shared', + models: { fetch: true }, + headers: sharedHeaders, + }, + ], + }, + }), + getUserKeyValues: jest.fn(), + fetchModels, + }); + + const req = { + user: { id: 'user-1' }, + config: undefined, + } as unknown as ServerRequest; + + await loadConfigModels(req); + + // Same baseURL + apiKey + headers → one fetch shared across both endpoints. + expect(fetchModels).toHaveBeenCalledTimes(1); + }); +}); diff --git a/packages/api/src/endpoints/config/models.ts b/packages/api/src/endpoints/config/models.ts index 22c5207b1d..6bd47d10ca 100644 --- a/packages/api/src/endpoints/config/models.ts +++ b/packages/api/src/endpoints/config/models.ts @@ -1,3 +1,4 @@ +import crypto from 'crypto'; import { logger } from '@librechat/data-schemas'; import { ErrorTypes, @@ -12,6 +13,23 @@ import type { FetchModelsParams } from '~/endpoints/models'; import { fetchModels as defaultFetchModels } from '~/endpoints/models'; import { isUserProvided } from '~/utils'; +/** + * Stable fingerprint of a headers object, used to disambiguate the + * in-request fetch-coalescing key. Two endpoints that share the same + * baseURL+apiKey but configure different headers must NOT share a fetch + * promise, otherwise the first endpoint's filtered /models response would + * be reused for the other in the same request. + */ +function headersFingerprint(headers: Record | undefined): string { + if (!headers || Object.keys(headers).length === 0) { + return ''; + } + const ordered = Object.keys(headers) + .sort() + .map((k) => [k, headers[k]]); + return crypto.createHash('sha256').update(JSON.stringify(ordered)).digest('hex').slice(0, 16); +} + interface ResolvedEndpoint { name: string; endpoint: TEndpoint; @@ -149,7 +167,11 @@ export function createLoadConfigModels(deps: LoadConfigModelsDeps) { baseURLIsUserProvided, } of resolved) { const { models, headers: endpointHeaders } = endpoint; - const uniqueKey = `${BASE_URL}__${API_KEY}`; + // Include a fingerprint of the configured headers so two admin-trusted + // endpoints that happen to share the same baseURL+apiKey but configure + // different (potentially user-bound) headers don't reuse each other's + // fetched model list within the same request. + const uniqueKey = `${BASE_URL}__${API_KEY}__${headersFingerprint(endpointHeaders)}`; if (models?.fetch && !apiKeyIsUserProvided && !baseURLIsUserProvided) { fetchPromisesMap[uniqueKey] = @@ -184,7 +206,13 @@ export function createLoadConfigModels(deps: LoadConfigModelsDeps) { baseURL: resolvedBaseURL, user: req.user?.id, userObject: req.user, - headers: endpointHeaders, + // Do not forward header overrides when the base URL is + // user-supplied: configured templates such as + // {{LIBRECHAT_OPENID_ID_TOKEN}} would otherwise resolve and be + // sent to a destination the user controls, leaking the user's + // identity token. Header overrides are only safe for endpoints + // whose base URL is admin-trusted. + headers: baseURLIsUserProvided ? undefined : endpointHeaders, direct: endpoint.directEndpoint, userIdQuery: models.userIdQuery, skipCache: true, diff --git a/packages/api/src/endpoints/custom/initialize.spec.ts b/packages/api/src/endpoints/custom/initialize.spec.ts index 3c384bf036..cafb06ec19 100644 --- a/packages/api/src/endpoints/custom/initialize.spec.ts +++ b/packages/api/src/endpoints/custom/initialize.spec.ts @@ -210,3 +210,145 @@ describe('initializeCustom – SSRF guard wiring', () => { expect(mockGetOpenAIConfig).not.toHaveBeenCalled(); }); }); + +describe('initializeCustom – token-config fetch header forwarding', () => { + const { fetchModels } = jest.requireMock('~/endpoints/models'); + + function createTokenConfigParams(overrides: { + apiKey?: string; + baseURL?: string; + userBaseURL?: string; + headers?: Record; + }): BaseInitializeParams { + const { apiKey = 'sk-test-key', baseURL = 'https://openrouter.ai/api/v1' } = overrides; + + mockGetCustomEndpointConfig.mockReturnValue({ + apiKey, + baseURL, + models: { fetch: true }, + headers: overrides.headers, + }); + + const db = { + getUserKeyValues: jest.fn().mockResolvedValue({ + apiKey: 'sk-user-key', + baseURL: overrides.userBaseURL ?? 'https://user-api.example.com/v1', + }), + } as unknown as BaseInitializeParams['db']; + + return { + req: { + user: { id: 'user-1', email: 'user@example.com' }, + body: { key: '2099-01-01' }, + config: {}, + } as unknown as BaseInitializeParams['req'], + // openrouter is in FetchTokenConfig, so the fetchModels call is reached + endpoint: 'openrouter', + model_parameters: { model: 'gpt-4' }, + db, + }; + } + + beforeEach(() => { + jest.clearAllMocks(); + fetchModels.mockReset().mockResolvedValue([]); + }); + + it('forwards configured headers and user object to fetchModels for admin-trusted base URL', async () => { + const headers = { + Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}', + 'X-User-Email': '{{LIBRECHAT_USER_EMAIL}}', + }; + const params = createTokenConfigParams({ + apiKey: 'sk-test-key', + baseURL: 'https://openrouter.ai/api/v1', + headers, + }); + + await initializeCustom(params); + + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + name: 'openrouter', + headers, + userObject: params.req.user, + }), + ); + }); + + it('drops headers when base URL is user-provided (token leak guard)', async () => { + const headers = { + Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}', + }; + const params = createTokenConfigParams({ + apiKey: 'sk-test-key', + baseURL: AuthType.USER_PROVIDED, + userBaseURL: 'https://user-controlled.example.com/v1', + headers, + }); + + await initializeCustom(params); + + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + name: 'openrouter', + headers: undefined, + userObject: params.req.user, + }), + ); + }); + + it('uses the unscoped endpoint tokenKey when no user-bound headers are configured', async () => { + const params = createTokenConfigParams({ + apiKey: 'sk-test-key', + baseURL: 'https://openrouter.ai/api/v1', + }); + + await initializeCustom(params); + + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + name: 'openrouter', + tokenKey: 'openrouter', + headers: undefined, + }), + ); + }); + + it('user-scopes the tokenKey when headers will be forwarded (admin-trusted base URL)', async () => { + const params = createTokenConfigParams({ + apiKey: 'sk-test-key', + baseURL: 'https://openrouter.ai/api/v1', + headers: { Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}' }, + }); + + await initializeCustom(params); + + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + tokenKey: 'openrouter:user-1', + }), + ); + }); + + it('does NOT user-scope the tokenKey when headers are dropped (user-provided base URL)', async () => { + const params = createTokenConfigParams({ + apiKey: 'sk-test-key', + baseURL: AuthType.USER_PROVIDED, + userBaseURL: 'https://user-controlled.example.com/v1', + headers: { Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}' }, + }); + + await initializeCustom(params); + + // baseURL is user-provided so tokenKey is already user-scoped via the + // existing rule, not via the new headers signal. Either way the value + // should be the user-scoped key. + expect(fetchModels).toHaveBeenCalledWith( + expect.objectContaining({ + tokenKey: 'openrouter:user-1', + headers: undefined, + }), + ); + }); +}); diff --git a/packages/api/src/endpoints/custom/initialize.ts b/packages/api/src/endpoints/custom/initialize.ts index cc5ab14a9f..19d4e2c35e 100644 --- a/packages/api/src/endpoints/custom/initialize.ts +++ b/packages/api/src/endpoints/custom/initialize.ts @@ -7,8 +7,8 @@ import { import type { TEndpoint } from 'librechat-data-provider'; import type { AppConfig } from '@librechat/data-schemas'; import type { BaseInitializeParams, InitializeResultBase, EndpointTokenConfig } from '~/types'; -import { getOpenAIConfig } from '~/endpoints/openai/config'; import { isUserProvided, checkUserKeyExpiry } from '~/utils'; +import { getOpenAIConfig } from '~/endpoints/openai/config'; import { getCustomEndpointConfig } from '~/app/config'; import { fetchModels } from '~/endpoints/models'; import { validateEndpointURL } from '~/auth'; @@ -138,8 +138,17 @@ export async function initializeCustom({ const cache = tokenConfigCache(); /** tokenConfig is an optional extended property on custom endpoints */ const hasTokenConfig = (endpointConfig as Record).tokenConfig != null; + // When `endpointConfig.headers` will be forwarded to the model fetch (i.e. + // base URL is admin-trusted, so the security guard below leaves them in + // place), header templates may resolve against the current user — making + // the response, and therefore the derived token config, user-specific. + // User-scope the token-config cache key in that case so a cached entry + // for one user can't be served to another. + const willForwardUserScopedHeaders = !!endpointConfig?.headers && !userProvidesURL; const tokenKey = - !hasTokenConfig && (userProvidesKey || userProvidesURL) ? `${endpoint}:${userId}` : endpoint; + !hasTokenConfig && (userProvidesKey || userProvidesURL || willForwardUserScopedHeaders) + ? `${endpoint}:${userId}` + : endpoint; const cachedConfig = !hasTokenConfig && @@ -154,7 +163,24 @@ export async function initializeCustom({ endpointConfig.models?.fetch && !endpointTokenConfig ) { - await fetchModels({ apiKey, baseURL, name: endpoint, user: userId, tokenKey }); + await fetchModels({ + apiKey, + baseURL, + name: endpoint, + user: userId, + tokenKey, + userObject: req.user, + // Mirror the security guard in `loadConfigModels`: never forward + // header overrides when the base URL is user-supplied — configured + // templates like {{LIBRECHAT_OPENID_ID_TOKEN}} would otherwise resolve + // and leak the user's identity token to a destination the user controls. + headers: userProvidesURL ? undefined : endpointConfig.headers, + // Note: when both `headers` and `userObject` are supplied below, the + // MODEL_QUERIES cache inside `fetchModels` is automatically skipped, + // which prevents a per-user filtered model list from leaking across + // users. The token-config cache key (`tokenKey`) is also user-scoped + // above when these headers will be forwarded. + }); endpointTokenConfig = (await cache.get(tokenKey)) as EndpointTokenConfig | undefined; } diff --git a/packages/api/src/endpoints/models.spec.ts b/packages/api/src/endpoints/models.spec.ts index a2101053d0..4f2c4efe78 100644 --- a/packages/api/src/endpoints/models.spec.ts +++ b/packages/api/src/endpoints/models.spec.ts @@ -149,6 +149,92 @@ describe('fetchModels', () => { ); }); + it('should resolve template variables in custom headers on the OpenAI-compatible path', async () => { + const customHeaders = { + Authorization: 'Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}', + 'X-User-Email': '{{LIBRECHAT_USER_EMAIL}}', + }; + const userObject = { id: 'user123', email: 'user@example.com' }; + + (resolveHeaders as jest.Mock).mockReturnValueOnce({ + Authorization: 'Bearer resolved-jwt', + 'X-User-Email': 'user@example.com', + }); + + await fetchModels({ + user: 'user123', + apiKey: 'testApiKey', + baseURL: 'https://api.test.com', + name: 'TestAPI', + headers: customHeaders, + userObject, + }); + + expect(resolveHeaders).toHaveBeenCalledWith({ + headers: customHeaders, + user: userObject, + }); + expect(mockedAxios.get).toHaveBeenCalledWith( + expect.stringContaining('https://api.test.com/models'), + expect.objectContaining({ + headers: expect.objectContaining({ + Authorization: 'Bearer resolved-jwt', + 'X-User-Email': 'user@example.com', + }), + }), + ); + }); + + it('should preserve a config-supplied Authorization header instead of overwriting with the apiKey default', async () => { + const customHeaders = { + Authorization: 'Bearer user-jwt-token', + }; + + await fetchModels({ + user: 'user123', + apiKey: 'testApiKey', + baseURL: 'https://api.test.com', + name: 'TestAPI', + headers: customHeaders, + }); + + expect(mockedAxios.get).toHaveBeenCalledWith( + expect.stringContaining('https://api.test.com/models'), + expect.objectContaining({ + headers: expect.objectContaining({ + Authorization: 'Bearer user-jwt-token', + }), + }), + ); + expect(mockedAxios.get).not.toHaveBeenCalledWith( + expect.any(String), + expect.objectContaining({ + headers: expect.objectContaining({ + Authorization: 'Bearer testApiKey', + }), + }), + ); + }); + + it('should treat Authorization header case-insensitively when skipping the apiKey default', async () => { + const customHeaders = { + authorization: 'Bearer lower-case-user-jwt', + }; + + await fetchModels({ + user: 'user123', + apiKey: 'testApiKey', + baseURL: 'https://api.test.com', + name: 'TestAPI', + headers: customHeaders, + }); + + const lastCall = mockedAxios.get.mock.calls[mockedAxios.get.mock.calls.length - 1]; + const sentHeaders = lastCall[1]?.headers ?? {}; + expect(sentHeaders.authorization).toBe('Bearer lower-case-user-jwt'); + expect(sentHeaders.Authorization).toBeUndefined(); + }); + afterEach(() => { jest.clearAllMocks(); }); @@ -798,4 +884,41 @@ describe('fetchModels caching behavior', () => { expect.any(Number), ); }); + + it('skips MODEL_QUERIES cache when both headers and userObject are supplied (user-scoped response)', async () => { + await fetchModels({ + apiKey: 'key', + baseURL: 'https://api.test.com', + name: 'TestAPI', + headers: { Authorization: 'Bearer some-user-token' }, + userObject: { id: 'user-1' }, + }); + + expect(mockCacheGet).not.toHaveBeenCalled(); + expect(mockCacheSet).not.toHaveBeenCalled(); + }); + + it('still uses cache when headers are supplied without a userObject (no per-user resolution)', async () => { + await fetchModels({ + apiKey: 'key', + baseURL: 'https://api.test.com', + name: 'TestAPI', + headers: { 'X-Static-Header': 'static-value' }, + }); + + expect(mockCacheGet).toHaveBeenCalled(); + expect(mockCacheSet).toHaveBeenCalled(); + }); + + it('still uses cache when userObject is supplied without headers', async () => { + await fetchModels({ + apiKey: 'key', + baseURL: 'https://api.test.com', + name: 'TestAPI', + userObject: { id: 'user-1' }, + }); + + expect(mockCacheGet).toHaveBeenCalled(); + expect(mockCacheSet).toHaveBeenCalled(); + }); }); diff --git a/packages/api/src/endpoints/models.ts b/packages/api/src/endpoints/models.ts index 7246c83e5c..197301df4d 100644 --- a/packages/api/src/endpoints/models.ts +++ b/packages/api/src/endpoints/models.ts @@ -126,7 +126,16 @@ export async function fetchModels({ return models; } - const shouldCache = !skipCache && !(userIdQuery && user); + // The MODEL_QUERIES cache is keyed by baseURL+apiKey only. That's safe + // when the response is identical for every caller, but fails when callers + // forward header templates that resolve to a user-bound value (e.g. + // `Authorization: Bearer {{LIBRECHAT_OPENID_ID_TOKEN}}`): one user's + // filtered list could otherwise be served to the next request that + // shares the same baseURL+apiKey. Skip the cache whenever both `headers` + // and `userObject` are supplied, since that's the signal the caller is + // resolving headers against a specific user's identity. + const hasUserScopedHeaders = !!headers && Object.keys(headers).length > 0 && !!userObject; + const shouldCache = !skipCache && !(userIdQuery && user) && !hasUserScopedHeaders; const cacheKey = shouldCache ? modelsCacheKey(baseURL ?? '', apiKey) : ''; const modelsCache = shouldCache ? standardCache(CacheKeys.MODEL_QUERIES) : null; if (modelsCache && cacheKey) { @@ -156,13 +165,21 @@ export async function fetchModels({ } try { + // Resolve template variables (e.g. {{LIBRECHAT_OPENID_ID_TOKEN}}) in the + // configured headers, mirroring fetchOllamaModels above. Without this, + // placeholder strings are forwarded literally on the model-fetch path. + const resolvedHeaders = resolveHeaders({ + headers: headers ?? undefined, + user: userObject, + }); + const options: { headers: Record; timeout: number; httpsAgent?: HttpsProxyAgent; } = { headers: { - ...(headers ?? {}), + ...resolvedHeaders, }, timeout: 5000, }; @@ -173,7 +190,16 @@ export async function fetchModels({ 'anthropic-version': process.env.ANTHROPIC_VERSION || '2023-06-01', }; } else { - options.headers.Authorization = `Bearer ${apiKey}`; + // Only fall back to the apiKey-based Bearer when the configured + // headers did not already supply an Authorization. This lets + // auth-aware proxies (e.g. LiteLLM with JWT auth) receive the user's + // token on /v1/models so they can return a per-user filtered list. + const hasAuthHeader = Object.keys(options.headers).some( + (k) => k.toLowerCase() === 'authorization', + ); + if (!hasAuthHeader) { + options.headers.Authorization = `Bearer ${apiKey}`; + } } if (process.env.PROXY) {