AWS Adds SageMaker Inference Benchmarking to Coding Agents

AWS launched the aws-ai-ml skill on October 5

Amazon Web Services announced the aws-ai-ml agent skill on October 5, 2026 through the Agent Toolkit for AWS. AWS names Kiro, Claude Code and Codex, and says the skill can work with other compatible coding agents. The launch adds a conversational code-generation route for SageMaker inference optimization; it does not introduce a new foundation model, model ID or inference endpoint.

The underlying optimized generative AI inference recommendation service predates this release. AWS announced that capability on April 22 and described it as a way to benchmark candidate deployment configurations using NVIDIA AIPerf. The new event is the agent skill that turns natural-language intent into SageMaker Python SDK v3 code for that workflow.

The skill covers three decision tasks

For an existing SageMaker endpoint, the agent can generate a notebook that uses Workload.synthetic() and start_benchmark() to drive a load test. AWS says the report includes requests and output tokens per second, p50 and p99 latency, time to first token, inter-token latency and supported concurrency. Those are measurements of the selected endpoint and workload—not model-quality scores.

For a new deployment choice, the skill can generate code to test candidate instances and configurations for a model in S3, SageMaker JumpStart or the Hugging Face Hub. It can also compare two named benchmark jobs and calculate metric deltas. The launch post says deploying a model is outside this optimization workflow’s scope; generated configuration or recommendations should not be mistaken for a completed production deployment.

Setup requires AWS CLI 2.35+, uv and usable credentials

AWS’s local path starts with aws configure agent-toolkit and requires AWS CLI 2.35 or later plus uv. The launch post then shows npx skills add aws/agent-toolkit-for-aws/skills/aws-ai-ml. AWS also offers a preconfigured private SageMaker Studio JupyterLab space; its first boot is described as taking 5–10 minutes.

The public repository currently stores version 4 of the source at skills/core-skills/aws-ai-ml, while the launch post’s npx argument uses a shorter skills/aws-ai-ml selector. AccessAllGPT did not run the installer, so teams should verify what artifact and revision the command resolves before approving it. Pin the source commit or plugin version where reproducibility matters.

The agent acts with your IAM authority

AWS says credentials need permissions for SageMaker operations such as creating endpoints and running benchmark or recommendation jobs, and that generated code runs under those credentials. The Agent Toolkit product page separately describes IAM controls, CloudWatch visibility and a sandboxed MCP script environment. Those platform features do not make a broad local AWS credential least-privileged by default.

Use a dedicated role with only the required SageMaker, S3, logging and supporting actions; constrain account and Region; and deny unrelated mutation. Keep confirmation in a deterministic wrapper or reviewer workflow rather than relying only on an agent instruction to ask before load-testing. A benchmark intentionally sends traffic to a live endpoint and can affect capacity, latency and cost.

Toolkit access is free; the resources are not

AWS says the Agent Toolkit has no additional charge and customers pay for resources the agent provisions or interacts with. The October launch post does not provide endpoint, benchmark-job, recommendation-job, Studio, storage or data-transfer prices. “No additional charge” therefore does not mean a benchmark run is free.

Budget from the concrete plan: instance types and counts, endpoint uptime, load-test duration, recommendation candidates, Studio runtime, S3 artifacts, logs and retries. Apply spend alerts and resource tags before execution. The launch cleanup list explicitly calls for deleting created endpoints, stopping or deleting JupyterLab spaces and removing benchmark and recommendation objects from the default SageMaker bucket.

AWS’s example is not a model ranking

AWS compares Qwen3-8B on a four-GPU ml.g5.12xlarge with Qwen3-1.7B on a one-GPU ml.g6.4xlarge under a 512-input, 256-output-token workload at concurrency four. AWS reports roughly 44–47% better throughput and end-to-end latency for the larger setup, while the smaller model has better time to first token.

The post itself attributes much of the delta to approximately four times the compute. This is a product walkthrough, not a controlled model comparison: model size, hardware and configuration all change. Reuse the workflow, not the conclusion. Hold task, traffic, output constraints and acceptance criteria constant when comparing candidate systems, and report quality and complete cost beside speed.

What builders should do next

Adopt the skill for a bounded pilot when SageMaker is already the target and a reviewer will inspect every generated operation. Constrain it to recommendation and read-only comparison when endpoint mutation or live load is unnecessary. Wait if the resolved source revision, required IAM policy, regional service availability or expected cost is unknown. Reject autonomous execution with broad production credentials or an unbounded load test.

Use one disposable endpoint and one representative workload. Record the skill revision, agent and plugin version, generated notebook hash, IAM policy, Region, model artifact, container, instance, concurrency, token shape, duration, raw percentiles, throughput, failures and cost. Require a human or external policy gate before create, benchmark and delete actions, then verify cleanup from AWS—not from the agent’s completion message.

Disclosures

AccessAllGPT received no AWS account, credits, software access, briefing, demo, benchmark results, review or compensation for this article. AWS did not sponsor, review or endorse it. AccessAllGPT Research is operated by NeuralArc, is independent, and is not affiliated with Amazon Web Services, Anthropic, OpenAI, Kiro, NVIDIA, Qwen or Hugging Face. Publication-wide relationships are listed on the disclosures page.