AWS Publishes SageMaker Multi-Turn RL Workflow With GPT-OSS-20B Cost and Evaluation Results

AWS published a concrete multi-turn RL path on October 23

On October 23, 2026, AWS published a hands-on workflow for Multi-Turn Reinforcement Learning in Amazon SageMaker AI. The service trains a policy model from the outcome of an episode rather than requiring a correct response or trajectory for every turn. SageMaker sends a task to an agent hosted on Amazon Bedrock AgentCore—or customer infrastructure reached through a Lambda forwarder—and receives a reward after the agent has acted in its environment.

This is a narrower and more managed contract than SageMaker training session APIs, which let developers orchestrate low-level sampling and gradient operations. Here, AWS owns the MTRL training job and LoRA update loop; the customer owns the agent, tools, environment, task data and reward. The published Wordle sample is available now in an Apache-2.0 aws-samples repository, but account access, quotas, IAM, same-Region storage and supported-model availability still determine whether a team can execute it.

Four model-and-Region combinations define the current access boundary

AWS lists four supported base models at the time of the guide. Amazon Nova 2 Lite and GPT-OSS-20B are available for MTRL in US East (N. Virginia) and US West (Oregon). Gemma 4 31B instruction-tuned and Qwen 3.6 27B are listed in US West (Oregon). AWS says each is trained with a LoRA adapter; this is not bring-your-own-model or unrestricted full-parameter training.

The customer must deploy an agent runtime, create a training-job role that trusts job.sagemaker.amazonaws.com, attach the documented permissions, put datasets in an S3 bucket in the same Region, and provide an MLflow app. The agent calls the temporary policy endpoint through the SageMaker Job Runtime. Redeploying the agent while training restarts the runtime and can fail the job, so agent code and configuration need to be frozen before paid rollouts begin.

The AWS run exposes token volume and separate runtime cost

AWS’s 100-step GPT-OSS-20B job ran for 12 hours 40 minutes and generated 105,486 rollouts. It reports 491,616,806 prefill tokens at $0.12 per million, 159,737,098 sample tokens at $0.30 per million and 643,308,554 train tokens at $0.36 per million. The published line items are $58.99, $47.92 and $231.59, which sum to $338.50 for the training job.

AgentCore is billed separately. AWS estimates about $79 from the runtime’s CloudWatch usage at us-east-1 list prices, bringing the two stated components to about $417.50. That is not a universal quote: it excludes surrounding S3, MLflow, logs, evaluation, model deployment and engineering costs, and rates can change. Multi-turn prompts repeatedly include history, while reasoning and retries add sample tokens, so trajectory length and failure rate are budget variables rather than incidental details.

The 61% to 84% result is sensitive to serving settings

AWS reports that the trained model solved 84% of 128 held-out Wordle words at medium reasoning effort and temperature 1.0, compared with 61% for the base model. At low effort, it reports 69% for the trained model versus 15% for the base. The adapter reportedly improved results by 14 to 54 percentage points across the four tested combinations of low or medium effort and temperature 0 or 1.0.

The training job’s own deterministic validation curve ended at 37% and peaked at 48%, because it used temperature 0. AWS chose to headline the serving-style temperature-1.0 result and averaged two trained runs—80% and 88%—for the medium-effort point. That choice is disclosed and useful, but it also shows why a single training curve is not a deployment metric. AccessAllGPT did not obtain per-word outcomes, confidence intervals or independent reruns.

Reward design can silently teach the wrong behavior

AWS says its first reward summed per-guess shaping across the episode. That made a lost game earn a positive reward and allowed a third-turn solve to outscore a first-turn solve. The revised function gives the terminal outcome priority: a solve pays 1.05 to 1.55 depending on speed, a six-guess loss pays -0.5, and no valid guess pays -1.5. A clipped shaping term is scaled so it cannot move an episode into another outcome tier.

That failure is more decision-useful than the final score. Before training, enumerate reward ordering over successful, failed, abstaining, malformed and adversarial trajectories; test those cases directly; then inspect real traces for shortcuts. AWS also reports authentication failures under concurrency, plain-text tool-call drift at an excessive learning rate and a plateau after early gains. A production trial needs explicit handling for every one of those modes.

What agent-training teams should do next

Trial MTRL only when the task has an objective episode outcome, repeated safe simulations, a supported model and Region, and enough expected volume to justify post-training. Start with the repository’s smoke-test pattern, but do not treat its expected failure as a production health check. Freeze the agent image and reward, record the repository commit, establish no-training baselines at actual serving settings and put ceilings on prefill, sample, train and runtime spend.

Adopt only when repeat runs improve held-out accepted outcomes without increasing severe errors or reward exploitation, and when the gain survives the intended model settings, tools and traffic. Constrain the trained adapter to the evaluated workflow. Wait if the model, Region, rate or quota is unsuitable. Reject the approach when the environment cannot be safely simulated, the reward cannot distinguish success from shortcuts, or a simpler prompt, tool or supervised example change reaches the same outcome.

Disclosures

AccessAllGPT received no AWS account, credits, model access, dataset, repository contribution, private traces, MLflow export, briefing, review, payment or compensation for this article. AWS, OpenAI, Google, Qwen and sample dependencies did not sponsor, review or endorse it. AccessAllGPT did not test, score, rank or train the service or models. AccessAllGPT Research is operated by NeuralArc, is independent, and is not affiliated with Amazon Web Services or organizations cited. Publication-wide relationships are listed on the disclosures page.