Groups and channels

What is a Group §

A Group corresponds to an upstream service and contains a pool of credentials. For each request, the gateway selects an available credential; if one is rate-limited or invalid, it automatically uses another without intervention.

See Core concepts for the roles of Groups and AccessKeys. In short, Groups connect to upstream services; AccessKeys authorize applications.

35 built-in channels §

When creating a Group, select one from these. Common channels are displayed directly as buttons, while other channels are collected under "Other Channels." Credentials have different forms depending on the category:

Each channel declares the protocols and capabilities it supports. The gateway converts between supported capabilities, but it is not a universal translator; see Protocols and conversion boundaries for the boundaries.

View the per-channel support matrix →

Create a Group §

Open Groups → New Group. You only need to provide three things: the channel, credentials, and models.

View full size

Create one Group per upstream service. Put multiple keys for the same provider in that Group instead of splitting them up.

Channel parameters §

Existing API-key groups can switch to another API-key channel in advanced configuration without reimporting credentials. Subscription groups cannot switch this way. Confirm that the target channel accepts the existing credentials, and check the preserved custom address and model configuration before switching.

After selecting a channel, the form shows parameters specific to this channel. Common ones:

• Endpoint URL — Official channels provide a default. Change it only for a relay or self-hosted upstream.
• Gateway base URL — For GPT-Load, New API, CLIProxyAPI, and Sub2API, enter the base URL or deployment prefix without /v1, /v1beta, or query parameters.
• Region — Cloud platforms such as AWS Bedrock and Google Vertex AI require a region.
• Deployment name — The model deployment identifier used by Azure OpenAI.

Incorrect values usually cause every request to fail. Use Route Check in Monitoring and troubleshooting to see which Groups became candidates and how many credentials are currently available.

Manage the credential pool §

The credential pool is the core of a Group. Place several keys, and the gateway will rotate among them, so a problem with one key does not affect the whole pool.

View full size

Top statistics by status. Available participants rotate, those in cooldown are temporarily skipped, and blacklisted ones are automatically removed.

Credentials can have optional names to help identify them in lists, logs, and route inspection; clearing a name restores the original label. Names are display-only and do not change secrets, account identity, scheduling, or health state.

Subscription accounts are masked by default and can be explicitly revealed in credential details. Masking only reduces on-screen exposure; it does not remove account information or mean that account fields in the management API are redacted.

Credentials can perform these operations:

• Batch Import — open the import panel on the Group’s Credentials page and paste multiple keys, one per line.
• Enable / Disable — A disabled credential stays in the pool but is excluded from rotation.
• Single weight — Send more or less traffic to a specific key.
• Test connection — sends one minimal real upstream request with this credential so you can see on the spot whether it still works
• View real value — Credentials are stored encrypted; reveal the value explicitly only when needed.
• Set proxy individually — Use this when one key needs a different egress route; see Proxy and request headers.

Select multiple credentials in an API Key group and choose Batch test, then select a shared protocol and model. Up to 10 credentials are tested concurrently. You can stop the batch or retry failed and unfinished items; leaving the group cancels the batch. Tests send real requests and may incur charges, without automatically changing enabled or blacklist states.

Group and individual credential proxy settings select enabled proxies maintained on the Proxies page, or use inherit or direct mode. A proxy can be referenced in multiple places: changing its address affects those references, while disabling or deleting it falls back to the parent configuration and may ultimately use a direct connection.

API-key groups can also select All credentials → Import from files on the credentials page. Choose multiple UTF-8 TXT, JSON, or JSONL files, review the credential count, and submit. Each import is limited to 5,000 credentials and 32 MiB; file contents must still match the selected channel’s credential format.

View full size

Duplicate keys are detected and skipped automatically, so you can safely paste the entire block.

Exposed models §

A Group must declare the models it exposes. You can discover them automatically from upstream or add them manually.

View full size

Only models listed here can be requested through this Group.

Configure model aliases in the same table. See Model management for pricing and cost estimates.

Runtime policy §

Set priority and weight in the group basics. Other runtime parameters include timeouts, blacklisting thresholds, session affinity, outbound proxy, and parameter override rules.

Priority defaults to 0 and accepts negative values. Higher values take precedence. For example, set primary groups to 100 and backups to 0; within a tier, the routing strategy and weights distribute traffic. See After failure for retry fallback rules.

View full size

For settings that support inheritance, enable an override or restore the system default. Changes take effect after saving.

See Runtime settings for what each parameter controls and when to change it.

Parameter override rules match the client protocol and model, using exact model names or a trailing * for prefix matching. Matching rules run in order: each removes the specified JSON paths before merging settings, and later rules override earlier values. Objects merge recursively; assigning an array value replaces the entire array, and null is an ordinary value.

For objects inside arrays, use * in set to update fields in each item, or * in remove paths to delete fields. Only existing items are traversed; other fields and array order are preserved. Array elements cannot be deleted, and numeric indexes are not supported. For example, remove reasoning_content from each message:

[{"remove":["/messages/*/reasoning_content"]}]

Parameter overrides apply to generation, Embeddings, and Rerank, not image editing, token counting, Responses resource operations, or probes. They cannot change root model, stream, or store, or inject or rewrite previous_response_id.

Custom Channel §

If a service is not built in but exposes an OpenAI-compatible API, choose OpenAI Compatible and enter its endpoint URL and key. Use this option for relay services and self-hosted inference services.

When to create separate Groups §

Create separate Groups when:

• Different upstream services — mandatory: each Group can use only one channel.
• Same provider, but two batches of keys must use different strategies (e.g., one batch primary, one batch fallback)
• Same provider, but different applications may have different model access

Do not split a Group when: multiple keys for the same provider. Put them in one pool so the gateway can rotate between them and route around failures; splitting them removes that shared fallback.