Key takeaways

  • AWS published the Bedrock deployment walkthrough on November 13, 2026. The available artifact is sample code for evaluation, not a new managed Bedrock routing service.
  • Stage routing selects between two models using tool-result history; Random routing supports weighted comparisons. Neither strategy calls a separate judge model in this integration.
  • The example uses global.anthropic.claude-opus-5-5 and nvidia.nemotron-super-3-120b through Bedrock Converse. AWS says the pair was tested in us-east-1.
  • Routing must beat single-model baselines on solved tasks and total cost, not just token price. No independently verified saving is established by this release.
01

What AWS published

On November 13, 2026, AWS published a walkthrough and sample repository for routing coding-agent requests between Amazon Bedrock models with NVIDIA NeMo Switchyard. The sample deploys an OpenAI-compatible LiteLLM gateway on ECS with Fargate in the reader’s own AWS account. It is available as code now, rather than a newly managed Bedrock service or a new foundation model.

The default model group is switchyard. Its candidates are Claude Opus 5.5, addressed through global.anthropic.claude-opus-5-5, and NVIDIA Nemotron 3 Super 120B, addressed through nvidia.nemotron-super-3-120b. The sample calls both through Bedrock Converse or ConverseStream. AWS says this pair has been tested in US East (N. Virginia); access, tool support and availability elsewhere need separate checks.

AWS explicitly labels the code non-production and the LiteLLM integration experimental. The walkthrough specifies LiteLLM v1.102.0 and switchyard-server 0.3.0, describing the latter as a demo and evaluation component. NVIDIA’s pinned guide adds a packaging constraint: the published nemo-switchyard release does not contain the decision-only Python API used by this example, so the integration must run from a repository checkout rather than an assumed standalone package installation.

02

How the router chooses a model

Stage routing needs exactly two candidates: a capable model first, and an efficient model second. It inspects recent tool activity and results in the conversation history. The default efficient_first configuration sends an initial request without tool results to Nemotron; failure signals can cause later requests to use the capable model. The sample examines the three most recent tool results with a confidence threshold of 0.5.

Random routing instead chooses from a pool with optional weights, which makes it useful as a traffic-split comparator. Neither shipped strategy makes an intermediate judge-model call. That statement should not be expanded to every Switchyard algorithm: classifier-backed strategies requiring intermediate model calls are unsupported by this integration.

Our analysis: history-based escalation is a different trade-off from predicting task difficulty before the first turn. AWS itself warns that a hard task can start on the efficient model, accumulate failed attempts and then replay that history during a more expensive recovery. A router can therefore lower some request costs while making a difficult task more expensive overall.

03

Compatibility and location are part of the decision

Clients send Chat Completions requests to /v1/chat/completions with model set to switchyard. The response body retains that public group name; the x-litellm-model-name response header identifies the concrete model selected. Recording only the body’s model field would lose the information needed to attribute each turn’s cost.

The default load-balancer configuration does not forward OpenAI Responses or Anthropic Messages routes. AWS says clients using those APIs receive a 404 until the deployment is changed and tested. The adapter supports text messages, function tool calls and text tool results; NVIDIA documents rejection of media, audio and other unsupported shapes. OpenAI-compatible is not a promise that every coding-agent client works unchanged.

The global. prefix on the Opus profile also matters: AWS says processing may occur in supported commercial Regions worldwide. Keeping gateway-to-Bedrock traffic on the AWS network through a VPC endpoint is not the same as guaranteeing in-Region model processing. Readers with location requirements need an appropriate geographic profile or in-Region identifier and must revalidate access and behavior.

04

Measure completed work, not cheap requests

The sample includes routed and single-model baseline groups plus small coding and conversation runners. Its central metric is estimated total run cost divided by solved runs, including spending on failures. AWS cautions that a few runs on illustrative tasks are insufficient to conclude that routing improves cost or quality. This announcement does not establish a quantified saving, and AccessAllGPT did not reproduce one.

The coding runner uses the selected-model header, returned token usage and a local PRICES table. Cached-input turns, unknown model prices or missing accounting fields remain unpriced rather than being silently counted as free. Estimates are not a bill: gateway infrastructure, current model rates, caching behavior and failed attempts still matter. We do not quote a current API rate here because we did not independently reconcile the candidate models’ billing schedules.

A useful evaluation starts with the same tasks, turn budgets and output caps for the routed group and both baselines. Compare solve rates before costs; inspect hard tasks separately, repeat trials and retain transcripts. If a router solves fewer tasks, a lower average request cost is not enough to justify it.

05

What readers should do next

Teams already buying Bedrock coding inference can inspect the public sample as an evaluation starting point. First confirm that their client uses supported request paths and sends the required history. Then compare the routed group with each single-model group on representative, non-sensitive tasks. Keep the simpler baseline if routing does not improve the trade-off that matters to the team.

Do not deploy the sample as an unattended production gateway or run its model-written code on a workstation with valuable files and persistent credentials. AWS warns that the benchmark subprocess is not an operating-system sandbox even though its environment omits credentials. Use a disposable environment, protect saved prompts and code, and remove cloud resources after evaluation to avoid ongoing charges. These are proposed next steps, not actions we performed for this briefing.

06

A routing trial worth running

Use this small comparison before deciding whether another gateway layer earns its place.

Entries stay in this browser tab and are not submitted to AccessAllGPT. Blank responses are copied as [Unresolved].

Check Chat Completions support, full history, tool-result shapes and both model entitlements before starting.

Run the same representative tasks against the routed group and each model alone, with matched turn and output budgets.

Keep selected-model headers, usage and failed-run costs; mark missing or cached pricing as unresolved, not zero.

Require acceptable solve rate, latency and total cost, including gateway overhead. Review processing location and experimental dependencies separately.

Primary sources

  1. Route coding agent requests to the right model on Amazon Bedrock with NVIDIA NeMo SwitchyardAWS · Reviewed: November 13 date; solution overview; prerequisites; model profiles; Stage routing; gateway tests; cost per solved task; interpretation; cleanup · Retrieved · Supports: Publication date, architecture, configured model IDs, reported us-east-1 testing, API compatibility, worldwide inference-profile caveat, evaluation accounting and non-production warning.
  2. AWS Bedrock Switchyard sample repositoryAWS Samples · Reviewed: README sample-code warning; why a router; architecture; request flow; supported strategies; models · Retrieved · Supports: Public implementation availability, experimental scope, two routing profiles, baseline groups and exact configured Bedrock model identifiers.
  3. Switchyard routing plugin for LiteLLM — pinned integration guideNVIDIA NeMo · Reviewed: Experimental integration; checkout-only dependency; request flow; supported algorithm semantics; message adapter; quick-start response headers · Retrieved · Supports: Checkout-only packaging requirement, LiteLLM 1.102.0 pin, lack of intermediate model-call support, supported and rejected message types and selected-model header semantics.

Limitations

Source-backed briefing, not hands-on testing. AccessAllGPT did not deploy the gateway, call either model, run the sample benchmark or verify savings, latency, account access or worldwide model availability. Configuration details reflect the reviewed AWS sample and pinned NVIDIA guide, not guarantees about all Switchyard integrations. No current numerical API price or production-readiness certification is claimed.

Disclosures

AccessAllGPT received no payment, credits or private access for this briefing. AWS and NVIDIA did not review or endorse it. AccessAllGPT Research is operated by NeuralArc, is independent and is not affiliated with AWS, NVIDIA or Anthropic. Publication-wide relationships are listed on the disclosures page.

Further AccessAllGPT guidance

  1. Benchmark design for agent systems
  2. AI agent news and analysis
  3. What AI models cost
  4. Publication methodology