How scheduling works
This page explains the internal mechanisms. You can use it normally without understanding, but it is very useful when troubleshooting 'why this credential was used.'
The complete path of a single request §
After the request comes in, the gateway performs these steps sequentially:
- Authentication — Verify that the AccessKey is valid and enabled.
- Protocol Check — Verify that the AccessKey allows the current protocol.
- Select Group — Find authorized Groups that expose the requested model.
- Select Credential — Choose an available credential from the Group's pool.
- Forward — Convert protocols when needed, then send the request upstream.
- Retry if failed — Switch credentials until the request succeeds or retries are exhausted.
If any stage cannot select a target, the request fails with a reason code. These codes appear in Route Check and Request Logs; see the final section.
Select the Group first §
Candidate Groups must meet all of the following:
- Within the authorization scope of this AccessKey
- Enabled
- The requested model is exposed
- Effective weight is greater than 0
When multiple Groups qualify, choose by weight. This is how multiple sources for the same model provide automatic failover: another Group can serve the model when one Group has no working credentials.
Select a credential §
After selecting a Group, filter in its credential pool:
- Status is available (not disabled, cooling down, or blacklisted)
- Subscription accounts also need to have normal authorization status
- Effective weight is greater than 0
Choose randomly from the remaining candidates by weight. This is not simple polling: round-robin can repeatedly select a failing credential, while weighted selection plus cooldown is more resilient.
Weight §
Weight determines relative traffic share. Both Groups and credentials support automatic and manual modes:
- Group automatic — uses the default weight
- Credential automatic — calculated dynamically from recent successes and failures
- Manual — ranges from 1 to 100; higher values receive proportionally more traffic
The management UI and management API do not accept a manual weight of 0. To stop traffic temporarily, disable the Group or credential; its configuration and historical statistics remain available.
Session affinity §
When enabled, the gateway derives a soft-affinity key from the AccessKey, client protocol, and the request instructions or the prefix of the first user input. Later requests with the same stable prefix prefer the credential that previously succeeded.
The affinity mechanism does not read previous_response_id, conversation, or any other upstream resource ID, and it does not guarantee that a stateful resource returns to its original credential. For reliable stateful-resource access, keep only one credential in the Group or confirm that the upstream permits resource sharing across credentials.
Affinity records have a TTL and capacity limit (10,000 by default) and evict the oldest entries. Affinity is not absolute: if the remembered credential is cooling down or blacklisted, the gateway selects another available credential.
See Runtime settings for configuration.
After failure §
When a request fails, the gateway retries with a different credential until it succeeds or reaches the retry limit.
The key point is "what counts as a failure":
- Retry — Upstream rate limits, server errors, and network timeouts are issues that may be resolved by changing credentials.
- Does not retry — Request errors such as invalid parameters or a missing model fail on every credential, so retrying only wastes time.
Once streaming output starts, the request will not be retried: the client already has partial content, so replaying on another credential would corrupt the response. The gateway switches safely only before the first data block arrives; later errors are returned as-is.
Cooldown and blacklist §
The two-level protection mechanism prevents bad credentials from repeatedly slowing down requests:
- Cooldown — A credential is temporarily skipped after an error, then restored automatically when cooldown expires. This is normal after upstream rate limiting.
- Blacklist — Automatically removed after consecutive failures exceed the threshold. It requires manual verification and does not recover automatically.
Cooldown is temporary avoidance, assuming the problem is temporary. Blacklisting means the credential is considered broken, such as when a key is revoked or an account is unpaid.
A successful attempt resets the consecutive failure count. Transient failures therefore do not accumulate toward blacklisting.
Configure thresholds in Runtime settings and inspect current status on the Health tab in Monitoring and troubleshooting.
Reason codes for routing failures §
Route checking and request logs will give specific reason codes. Reference table:
| Reason code | Meaning | How to handle |
|---|---|---|
| access_key_disabled | AccessKey has been deactivated | Enable it on the AccessKeys page |
| access_key_expired | AccessKey has expired | Create a new key or extend validity |
| protocol_filtered | This key has not selected this protocol | Add a check for the corresponding protocol in the key |
| model_filtered | Requested model is not within the allowed range | Check model limitations for the key |
| model_required_by_filter | The key restricts models, but the request carried no model name | Name the model explicitly in the request, or drop the key's model restriction |
| operation_unsupported | The channel does not support this operation under that protocol | Switch to a Group that supports the capability; see Protocols and conversion limits |
| native_route_required | The request requires a native route, but this Group can only serve it through conversion | Use a Group whose protocol matches the client |
| no_route_target | No routable target was found | Confirm the key has authorized at least one Group |
| group_disabled | Group is deactivated | Enable this Group |
| group_filtered | Group is not within the authorization scope of this key | Add this Group in the key |
| no_available_group | No Group can provide this model | Confirm the model is open in some Group |
| no_credentials | There is not a single credential in the Group. | Add credentials to the Group |
| group_weight_zero | A legacy configuration has a Group weight of 0 | Switch to automatic weight or a manual weight from 1 to 100 |
| credential_disabled | Credential has been deactivated | Enable it, or depend on other credentials |
| credential_auth_unavailable | Subscription account authorization has expired | Reauthorize, see subscription account page |
| credential_blacklisted | Credential has been blacklisted | Restore it after confirming the credential itself is valid |
| credential_cooldown | Credential is in cooldown | Wait for automatic recovery, or add more credentials to share the load |
| credential_weight_zero | A credential weight of 0 left over from an old configuration | Switch to automatic weight or a manual weight from 1 to 100 |
| credential_not_allowed | This request already excluded this credential | Expected behavior; a retry will not hit the one that just failed |
| no_available_credential | All credentials are unavailable | Check the health page; most likely it's a collective rate limit or key expiration. |
After receiving a reason code, adjust the conditions in Route checking and retry the check to verify the change without sending a real request.