# AccessAllGPT Research

> Independent, evidence-led technical research for founders, CTOs, AI/ML leads, engineering managers, senior builders and technical buyers deciding what AI to build, buy, deploy or avoid.

AccessAllGPT Research is operated by NeuralArc. It is independent and is not affiliated with OpenAI. This file is a machine-readable discovery aid, not a substitute for the dated evidence, limitations and disclosures on each article page. Vendor documentation establishes documented vendor claims, not independent performance. AccessAllGPT does not claim original tests unless the linked article explicitly describes a reproducible method and preserved outputs.

## Start here

- [Research index](https://www.accessallgpt.com/research): all decision guides, searchable by topic and evidence status
- [Evidence index](https://www.accessallgpt.com/evidence): source-level review scope, retrieval dates and bounded claim notes
- [Methodology](https://www.accessallgpt.com/methodology): evidence hierarchy, review method and correction policy
- [Disclosures](https://www.accessallgpt.com/disclosures): ownership, independence and commercial disclosures
- [RSS feed](https://www.accessallgpt.com/feed.xml): publication updates
- [Sitemap](https://www.accessallgpt.com/sitemap.xml): canonical public routes

## Decision desks

- [Models](https://www.accessallgpt.com/models): Capability, cost, deployment and model-selection decisions.
- [Agents](https://www.accessallgpt.com/agents): Coding agents, tool use, orchestration, control and reliability.
- [Automation](https://www.accessallgpt.com/automation): Operational workflows that survive contact with production.
- [Papers to production](https://www.accessallgpt.com/papers): What research changes, what it proves and what teams can use now.
- [Benchmarks](https://www.accessallgpt.com/benchmarks): Evaluation design, reproducibility and evidence beyond leaderboards.
- [Buyer guides](https://www.accessallgpt.com/buyer-guides): Evidence-led frameworks for expensive AI tooling decisions.

## Published decision guides

### [MCP Output Validation Depends on Tool Discovery](https://www.accessallgpt.com/research/mcp-output-schema-discovery-cache-validation-boundary)
- Decision desk: Agents
- Evidence status: Reproducible agent-protocol investigation
- Published: 2026-08-31; last material update: 2026-08-31
- Summary: In an official MCP TypeScript SDK 1.30.0 probe, Client.callTool accepted a string where a tool promised an integer until listTools populated the output-validator cache. Discovery order is therefore part of the client’s validation boundary.
- Method: AccessAllGPT reviewed the current MCP 2026-07-28 tools specification and CallToolResult schema, the release record and pinned client/server implementation for @modelcontextprotocol/sdk 1.30.0, Anthropic’s original MCP announcement and OWASP’s independent prompt-injection defensive guidance. I wrote and ran scripts/audit-mcp-output-schema-cache.mjs on Node v24.10.0 and Darwin arm64 with the official SDK 1.30.0, Zod 3.25.76 and linked in-memory transports. A deliberately nonconforming low-level synthetic server advertised an integer score from 0 to 100 but returned {score:"ninety-nine"}. Ten assertions compared a direct call before discovery with the same call after listTools. The pinned client source was bound to SHA-256 db6cc643f5cc095b2fd0b70bf691d1775f07a317e97555e3928559e815f5e32a. No model, production client, remote server, credential, network target or user data was used.
- Limitations: This is a bounded local reproduction on Node v24.10.0, Darwin arm64, @modelcontextprotocol/sdk 1.30.0 and Zod 3.25.76 using one deliberately nonconforming low-level in-memory server, one tool, two calls and one integer schema. It did not test a model, production MCP client, high-level server mismatch end to end, remote or stdio transport, authentication, concurrency, reconnect, list-change races, task streaming, another SDK, hostile schema complexity, content injection, malware, external effects or real data. It demonstrates that the tested Client.callTool path lacked tool-specific validation before listTools populated its cache and rejected the same invalid result afterward; it does not measure prevalence, prove exploitability, allege a protocol defect or replace a security review. OWASP guidance is adjacent defensive evidence, not validation of this probe.
- Primary sources:
  - [Tools (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/server/tools) — Model Context Protocol Specification. Retrieved: 2026-08-31. Reviewed: Tool definition, output schema, structured content, calling tools, error handling and security considerations. Supports: The current protocol defines outputSchema as an optional contract for structuredContent and says a conforming structured result must match that schema; it also separates tool execution errors from protocol errors..
  - [Schema reference: CallToolResult (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/schema#calltoolresult) — Model Context Protocol Specification. Retrieved: 2026-08-31. Reviewed: CallToolResult content, structuredContent, isError and result metadata fields. Supports: The schema requires structuredContent to be a JSON object when present, but the tool-specific output contract comes from the separately discovered Tool.outputSchema..
  - [@modelcontextprotocol/sdk 1.30.0 client implementation](https://unpkg.com/@modelcontextprotocol/sdk@1.30.0/dist/esm/client/index.js) — Model Context Protocol TypeScript SDK. Retrieved: 2026-08-31. Reviewed: Client.callTool, cacheToolMetadata, getToolOutputValidator and listTools implementation. Supports: The pinned client validates a tool result only when a validator is present in its metadata cache; listTools clears and repopulates that cache from discovered output schemas..
  - [@modelcontextprotocol/sdk 1.30.0 server implementation](https://unpkg.com/@modelcontextprotocol/sdk@1.30.0/dist/esm/server/mcp.js) — Model Context Protocol TypeScript SDK. Retrieved: 2026-08-31. Reviewed: High-level McpServer tool registration, input validation, output validation and callback dispatch. Supports: The high-level McpServer validates registered tool structuredContent against its configured output schema before returning a successful result; this differs from the deliberately nonconforming low-level fixture used in the client probe..
  - [TypeScript SDK release 1.30.0](https://github.com/modelcontextprotocol/typescript-sdk/releases/tag/1.30.0) — Model Context Protocol on GitHub. Retrieved: 2026-08-31. Reviewed: Release identity, tag, publication date and package change record. Supports: The official release record binds the tested package to SDK version 1.30.0, released July 27, 2026..
  - [LLM Prompt Injection Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html) — OWASP Cheat Sheet Series. Retrieved: 2026-08-31. Reviewed: Agent-specific attacks, thought and observation injection, least privilege, tool validation and trust-boundary guidance. Supports: Independent defensive guidance treats tool output as untrusted content and recommends validating tool calls against user permissions and session context; it does not test this SDK cache behavior..
  - [Introducing the Model Context Protocol](https://www.anthropic.com/news/model-context-protocol) — Anthropic. Retrieved: 2026-08-31. Reviewed: November 25, 2024 announcement, architecture, SDKs and ecosystem purpose. Supports: Anthropic introduced MCP as an open standard for connecting AI assistants to data sources and tools, establishing why discovered tool contracts enter an agent client’s trust boundary..

### [MCP’s readOnlyHint Is Not a Permission Boundary](https://www.accessallgpt.com/research/mcp-tool-annotations-permission-boundary)
- Decision desk: Agents
- Evidence status: Reproducible agent-protocol investigation
- Published: 2026-08-31; last material update: 2026-08-31
- Summary: In an official MCP TypeScript SDK 1.30.0 probe, a tool advertised readOnlyHint: true and idempotentHint: true, then wrote a file on both calls. The SDK faithfully transported the hints and faithfully dispatched the mutations.
- Method: AccessAllGPT reviewed the current MCP 2026-07-28 tools specification and ToolAnnotations schema, the generated types and release record for @modelcontextprotocol/sdk 1.30.0, Anthropic’s original MCP announcement and Invariant Labs’ independent tool-poisoning report. I wrote and ran scripts/audit-mcp-tool-annotations.mjs on Node v24.10.0 and Darwin arm64 using the official SDK 1.30.0, Zod 3.25.76 and its linked in-memory client/server transport. The synthetic tool declared readOnlyHint=true, destructiveHint=false, idempotentHint=true and openWorldHint=false, then wrote one temporary marker on each of two calls. Eleven assertions verified discovery metadata, pre-call state, both responses, both writes and call count. The SDK specification type source was bound to SHA-256 f612706dcbe34de49a77f2458ba4e0ef9e767efffc0f38024e3d84a8f1f66414. No model, production client, external tool, credential, network target or user data was used.
- Limitations: This is a bounded local reproduction on Node v24.10.0, Darwin arm64, @modelcontextprotocol/sdk 1.30.0 and Zod 3.25.76 using one synthetic in-memory server, one zero-argument tool, one temporary file and two calls. It did not test a language model, agent planner, production MCP client, remote or stdio transport, authentication, registry, signature, approval interface, network failure, concurrency, database, external API, Windows, Linux, malware or real data. It proves that the tested SDK path transports behavioral annotations and dispatches callbacks without enforcing read-only or idempotent effects, which is consistent with the specification; it does not prove a protocol defect, vendor vulnerability, client prevalence or exploitability. The independent Invariant report concerns description-based tool poisoning and does not validate this probe.
- Primary sources:
  - [Tools (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/server/tools) — Model Context Protocol Specification. Retrieved: 2026-08-31. Reviewed: Tool definition, annotations, user interaction model, calling tools and security considerations. Supports: The current specification defines annotations as optional descriptions of tool behavior and normatively requires clients to consider them untrusted unless they come from trusted servers..
  - [Schema reference: ToolAnnotations (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/schema#toolannotations) — Model Context Protocol Specification. Retrieved: 2026-08-31. Reviewed: ToolAnnotations fields, defaults and semantic qualifications for read-only, destructive, idempotent and open-world hints. Supports: The schema defines readOnlyHint, destructiveHint, idempotentHint and openWorldHint as behavioral hints, including defaults and the conditions under which some fields are meaningful..
  - [@modelcontextprotocol/sdk 1.30.0 generated specification types](https://unpkg.com/@modelcontextprotocol/sdk@1.30.0/dist/esm/spec.types.d.ts) — Model Context Protocol TypeScript SDK. Retrieved: 2026-08-31. Reviewed: ToolAnnotations interface, field comments, trust warning and Tool definition. Supports: The pinned SDK type declarations say every ToolAnnotations property is a hint, may not faithfully describe behavior, and must not drive tool-use decisions when received from an untrusted server..
  - [TypeScript SDK release 1.30.0](https://github.com/modelcontextprotocol/typescript-sdk/releases/tag/1.30.0) — Model Context Protocol on GitHub. Retrieved: 2026-08-31. Reviewed: Release identity, tag, commit, publication date and changed-package record. Supports: The official release page binds the tested SDK package to version 1.30.0 and tag commit 2d889f2, released July 27, 2026..
  - [MCP Security Notification: Tool Poisoning Attacks](https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks) — Invariant Labs. Retrieved: 2026-08-31. Reviewed: Threat definition, malicious tool-description example, reported client experiments, limitations and proposed mitigations. Supports: Independent security researchers reported that malicious instructions in MCP tool descriptions could steer tested agents toward unauthorized reads and actions; this is adjacent threat evidence, not validation of this annotation probe..
  - [Introducing the Model Context Protocol](https://www.anthropic.com/news/model-context-protocol) — Anthropic. Retrieved: 2026-08-31. Reviewed: November 25, 2024 announcement, protocol purpose, architecture, SDKs and early ecosystem description. Supports: Anthropic introduced MCP on November 25, 2024 as an open standard connecting AI assistants to data sources and tools, establishing the chronology and intended AI integration boundary..

### [npm ci --ignore-scripts Still Installs Runnable Package Binaries](https://www.accessallgpt.com/research/npm-ci-ignore-scripts-bin-links-execution-boundary)
- Decision desk: Agents
- Evidence status: Reproducible package-manager investigation
- Published: 2026-08-31; last material update: 2026-08-31
- Summary: In npm 11.6.0, a synthetic package installed by npm ci --ignore-scripts did not run its install hook, but npm still created node_modules/.bin/aag-ci-probe. Invoking that link executed the package. --bin-links=false removed the shim, not the underlying executable file.
- Method: AccessAllGPT reviewed npm CLI v11 ci and package.json bin documentation, npm 11.6.0 ci source, bundled @npmcli/arborist 9.1.4 source and the independent OpenSSF Package Analysis design. I wrote and ran scripts/audit-npm-ci-ignore-scripts-bin-links.mjs on Node v24.10.0, npm 11.6.0 and Darwin arm64. The audit bound installed source to SHA-256 digests 76cacc0ed9bbee8a7c1e5d5656201403a4dd688d31ee141b531c9e85a7e75e9e and 0915a24508d04986869407f80ef6f793a85abd1a4785e27b4029efcb7f4921dd, packed one synthetic local dependency, created a frozen lockfile, compared three fresh npm ci flows, explicitly invoked only the synthetic binary, and passed 20 assertions over hook markers, package files, symlink targets and invocation output. No registry package executable, external service, exploit, network access or publication was used.
- Limitations: This is a bounded local reproduction on Node v24.10.0, npm 11.6.0, @npmcli/arborist 9.1.4 and Darwin arm64. It used one synthetic local tarball, one install hook, one JavaScript bin and three frozen-lockfile projects. It did not test Windows shims, Linux, containers, workspaces, global mode, npm exec/npx, PATH attacks, collisions, native binaries, generated bins, registries, caches, credentials, egress or malicious packages. The direct source review covered npm ci and Arborist’s builder, not every reify mixin or wrapper. The OpenSSF source supplies independent operational context and does not validate the experiment. --ignore-scripts and --bin-links=false are configuration controls, not a sandbox, malware assessment or proof that installed dependency code cannot execute.
- Primary sources:
  - [npm ci documentation, CLI v11](https://docs.npmjs.com/cli/v11/commands/npm-ci) — npm Docs. Retrieved: 2026-08-31. Reviewed: Description, clean-install guarantees, lockfile requirements, ignore-scripts and bin-links configuration. Supports: npm documents ci as a lockfile-frozen clean install and lists ignore-scripts and bin-links as separate configuration controls; ignore-scripts suppresses package.json scripts while bin-links governs executable links..
  - [package.json bin field documentation, CLI v11](https://docs.npmjs.com/cli/v11/configuring-npm/package-json#bin) — npm Docs. Retrieved: 2026-08-31. Reviewed: bin field mapping, executable files, local installation links and shebang requirement. Supports: npm documents that local installs create links in node_modules/.bin for package bin entries so commands can be run by npm scripts and other callers..
  - [npm 11.6.0 ci command implementation](https://github.com/npm/cli/blob/v11.6.0/lib/commands/ci.js) — npm CLI source. Retrieved: 2026-08-31. Reviewed: Accepted configuration, lockfile validation, node_modules removal, Arborist reify dispatch and root lifecycle guard. Supports: The pinned command accepts ignore-scripts and bin-links independently, passes flat options to Arborist reify, and uses ignore-scripts to guard root lifecycle events after dependency reification..
  - [Arborist 9.1.4 rebuild implementation bundled with npm 11.6.0](https://unpkg.com/@npmcli/arborist@9.1.4/lib/arborist/rebuild.js) — npm CLI source via unpkg. Retrieved: 2026-08-31. Reviewed: Early return, build queues, lifecycle guards, binLinks branch, package metadata inspection and executable linking. Supports: The implementation queues bins and lifecycle events together but guards them separately: ignoreScripts gates lifecycle execution while binLinks gates #linkAllBins..
  - [OpenSSF Package Analysis README at commit c5c4500](https://github.com/ossf/package-analysis/blob/c5c45008da694036d701ba76fe9567fc8a5b9675/README.md) — Open Source Security Foundation. Retrieved: 2026-08-31. Reviewed: Project purpose, behavioral signals, architecture, sandboxing and captured process and network data. Supports: Independent ecosystem infrastructure treats commands, file access and network destinations as behaviors worth observing and dynamically analyzes packages in isolated gVisor containers; it does not test this npm fixture or establish maliciousness..

### [npm --ignore-scripts Is Not a Lasting Quarantine](https://www.accessallgpt.com/research/npm-rebuild-after-ignore-scripts-execution-boundary)
- Decision desk: Agents
- Evidence status: Reproducible package-manager investigation
- Published: 2026-08-31; last material update: 2026-08-31
- Summary: In npm 11.6.0, I installed a synthetic tarball with --ignore-scripts and observed no hooks. A later ordinary npm rebuild ran its preinstall, install and postinstall scripts. The install flag did not persist as package approval state; script policy must cover every later build command.
- Method: AccessAllGPT reviewed npm CLI v11 rebuild and lifecycle documentation, npm 11.6.0 rebuild source, bundled @npmcli/arborist 9.1.4 source and an independent historical supply-chain paper. I wrote and ran scripts/audit-npm-rebuild-after-ignore-scripts.mjs on Node v24.10.0, npm 11.6.0 and Darwin arm64. The audit bound installed source files to SHA-256 digests 8a4616e70380702f7fd5c76fc9e9fd7f3e10683bdd12fd643a83c65a8d6fbcad and 0915a24508d04986869407f80ef6f793a85abd1a4785e27b4029efcb7f4921dd, packed only a synthetic local dependency, compared three fresh install/rebuild flows, and passed 14 assertions over hook traces, lockfile metadata and source control flow. No registry dependency script, external service, exploit, network exfiltration or publication was used.
- Limitations: This is a bounded local reproduction on Node v24.10.0, npm 11.6.0, @npmcli/arborist 9.1.4 and Darwin arm64. It used one synthetic local tarball with preinstall, install and postinstall markers. It did not test npm ci, install-scripts, approve-scripts, trust, workspaces, links, prepare, node-gyp defaults, native compilation, bundled or optional dependencies, global mode, bins, Linux, Windows, containers, caches, registries, credentials, network access or malicious packages. The source review covered the direct rebuild command and bundled Arborist path, not every wrapper or delegated implementation. The independent paper is historical context and does not validate this behavior. --ignore-scripts is a lifecycle control, not a sandbox, malware assessment or proof of zero filesystem changes.
- Primary sources:
  - [npm rebuild documentation, CLI v11](https://docs.npmjs.com/cli/v11/commands/npm-rebuild) — npm Docs. Retrieved: 2026-08-31. Reviewed: Description, synopsis, lifecycle behavior, package selection and ignore-scripts configuration. Supports: npm documents rebuild as running matching lifecycle scripts for installed packages and exposes ignore-scripts as a command configuration, not as persisted package approval state..
  - [npm scripts documentation, CLI v11](https://docs.npmjs.com/cli/v11/using-npm/scripts#npm-rebuild) — npm Docs. Retrieved: 2026-08-31. Reviewed: Life cycle scripts, dependencies, working directory and npm rebuild operation order. Supports: npm documents preinstall, install and postinstall as the lifecycle order for npm rebuild and explains that scripts run from the package root..
  - [npm 11.6.0 rebuild command implementation](https://github.com/npm/cli/blob/v11.6.0/lib/commands/rebuild.js) — npm CLI source. Retrieved: 2026-08-31. Reviewed: Accepted configuration, Arborist option construction, package-spec matching and rebuild dispatch. Supports: The pinned command accepts ignore-scripts, passes current flat options into Arborist and invokes rebuild over the selected installed nodes; it does not read a previous install command or persisted approval record..
  - [Arborist 9.1.4 rebuild implementation bundled with npm 11.6.0](https://unpkg.com/@npmcli/arborist@9.1.4/lib/arborist/rebuild.js) — npm CLI source via unpkg. Retrieved: 2026-08-31. Reviewed: Default-node loading, build queues, ignoreScripts guards, package metadata refresh and lifecycle execution. Supports: The implementation discovers scripts from installed package metadata, queues preinstall/install/postinstall, and suppresses those queues only when the current rebuild options set ignoreScripts..
  - [Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks](https://arxiv.org/abs/2005.09535) — DIMVA preprint on arXiv. Retrieved: 2026-08-31. Reviewed: Abstract, historical malicious-package dataset, attack-tree scope and execution-stage framing. Supports: Independent researchers documented historical malicious packages and code-execution opportunities across package ecosystems; this is risk context, not validation of npm 11.6.0 or the synthetic experiment..

### [npm pack --dry-run Still Executes Package Scripts](https://www.accessallgpt.com/research/npm-pack-dry-run-lifecycle-script-execution-boundary)
- Decision desk: Agents
- Evidence status: Reproducible package-manager investigation
- Published: 2026-08-30; last material update: 2026-08-30
- Summary: In npm 11.6.0, a synthetic package’s prepack, prepare and postpack hooks all ran under npm pack --dry-run. The tarball file was not written, but the scripts changed the working tree. Use --ignore-scripts and an isolated builder when inspection must not grant package code authority.
- Method: AccessAllGPT reviewed npm CLI v11 pack and scripts documentation, npm 11.6.0 pack source, bundled libnpmpack 9.0.7 source and an independent historical supply-chain paper. I wrote and ran scripts/audit-npm-pack-dry-run.mjs on Node v24.10.0, npm 11.6.0, Darwin arm64. The audit bound installed source with SHA-256 digests 7580758d2fd46c6d3998f72ad623e93e91be3f8ce1893c9c914ec4b147874de5 and 665eeb6479829e1c21699e55476edf7b020da3aac12e73d8bcf2a69c0a33951a, created only a synthetic local package, ran normal, dry-run and ignore-scripts pack modes, and passed 13 assertions over hook order, filesystem writes and reported file lists. No registry package lifecycle script, publish operation, network exfiltration or exploit was attempted.
- Limitations: This is a bounded local reproduction on Node v24.10.0, npm 11.6.0, bundled libnpmpack 9.0.7 and Darwin arm64. It used one synthetic directory package and did not test npm publish, package registries, Git or tarball specs, dependencies, workspaces, monorepo wrappers, hook failures, concurrency, Linux, Windows, containers, caches, network access or credential exposure. Source-level review covered the adjacent npm pack and libnpmpack paths but not every delegated pacote implementation path. The independent paper is historical context and does not validate this npm behavior. ignore-scripts suppressed the three fixture hooks but is not a sandbox, malware control, package safety assessment or proof of zero side effects.
- Primary sources:
  - [npm pack documentation, CLI v11](https://docs.npmjs.com/cli/v11/commands/npm-pack) — npm Docs. Retrieved: 2026-08-30. Reviewed: Synopsis, dry-run, ignore-scripts and description sections for npm pack. Supports: npm documents dry-run as reporting what pack would do without making changes, and documents ignore-scripts as preventing package.json scripts from running..
  - [npm scripts documentation, CLI v11](https://docs.npmjs.com/cli/v11/using-npm/scripts) — npm Docs. Retrieved: 2026-08-30. Reviewed: Life cycle operation order plus prepare, prepack and postpack definitions. Supports: npm documents prepack and prepare before tarball generation and postpack after generation; prepare has run during npm pack since npm 4..
  - [npm 11.6.0 pack command implementation](https://github.com/npm/cli/blob/v11.6.0/lib/commands/pack.js) — npm CLI source. Retrieved: 2026-08-30. Reviewed: Pack command parameters, manifest resolution, libnpmpack invocation and output path. Supports: The command forwards flat options, including dry-run and ignore-scripts, into libnpmpack and then reports the generated tarball metadata..
  - [libnpmpack 9.0.7 pack implementation bundled with npm 11.6.0](https://unpkg.com/libnpmpack@9.0.7/lib/index.js) — npm CLI source. Retrieved: 2026-08-30. Reviewed: Directory-script guards, tarball creation, dry-run write guard and postpack execution order. Supports: The pinned implementation gates prepack and postpack on ignoreScripts, delegates tarball creation between them, and uses dryRun only to decide whether the completed tarball buffer is written to disk..
  - [Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks](https://arxiv.org/abs/2005.09535) — DIMVA preprint on arXiv. Retrieved: 2026-08-30. Reviewed: Abstract, historical package-registry attack corpus, supply-chain stages and stated scope. Supports: Independent researchers documented historical malicious packages and execution opportunities in package ecosystems; this supplies bounded risk context, not evidence about npm pack 11.6.0 or the synthetic fixture..

### [A pnpm Build Approval Can Outlive the Version You Reviewed](https://www.accessallgpt.com/research/pnpm-allow-builds-package-version-execution-scope)
- Decision desk: Agents
- Evidence status: Source-backed package-manager investigation
- Published: 2026-08-29; last material update: 2026-08-29
- Summary: pnpm 11.24.0 blocks unapproved dependency builds, but its normal approve-builds flow records a bare package name. The pinned implementation treats that name as approval for every registry version. Exact name@version rules are supported and give CI a narrower execution boundary.
- Method: AccessAllGPT reviewed pnpm 10.0.0, 10.1.0, 11.23.0 and 11.24.0 release records; the current approve-builds documentation; pnpm 11.24.0 source at commit cef4816dfbc9aa7ffbe67fa727c1eb9be5d5e1e7; and an independent historical supply-chain paper. The registry reported pnpm 11.24.0 with integrity sha512-vSfjRel23LC+C3oSKCF7BJqBfiGx81XJDb59xGZxiVqLwebQbCRVRQXqk+oLRfSJon7Bv7yN5qlln8oPFvoAAA==. I wrote and ran scripts/audit-pnpm-allow-builds-policy.mjs on Node v24.10.0, Darwin arm64. It fetched two commit-pinned source files, verified SHA-256 digests b0fcc6a8364c02f384115869030b51dc4643c2a7649eb2b869a05b760589079d and b4a931881f1b5c28689c47dad109e4c4e84e788c8eb63802b63cf58f196bb0bf, asserted the relevant matching branches, and exercised a six-row policy matrix with 12 passing assertions. The matrix is a source-level reproduction of the matching algorithm, not an end-to-end package installation or security exploit.
- Limitations: This is a source-level policy investigation, not an end-to-end pnpm installer test, package audit, exploit, malware analysis, sandbox evaluation or cross-platform survey. The unattended command guard blocked the proposed real-package lifecycle-script run, so no third-party install script was executed and no installer output is claimed. The six-row matrix mirrors only the inspected matching branches and cannot prove call-site behavior for aliases, peers, patches, custom registries, Git, tarball, local, workspace or globally installed packages. Source was pinned to pnpm 11.24.0 commit cef4816dfbc9aa7ffbe67fa727c1eb9be5d5e1e7; later versions may differ. The independent paper is historical context and does not assess pnpm 11. Exact-version policy reduces approval drift but does not establish package safety, publisher trust, artifact provenance or runtime containment.
- Primary sources:
  - [pnpm v10.0.0 release notes](https://github.com/pnpm/pnpm/releases/tag/v10.0.0) — pnpm project. Retrieved: 2026-08-29. Reviewed: Major change that stopped dependency lifecycle scripts by default, onlyBuiltDependencies migration example and security motivation. Supports: pnpm 10.0.0 was released on 2025-01-07 and changed dependency lifecycle scripts to blocked by default, with named-package exceptions through onlyBuiltDependencies at that time..
  - [pnpm v10.1.0 release notes](https://github.com/pnpm/pnpm/releases/tag/v10.1.0) — pnpm project. Retrieved: 2026-08-29. Reviewed: Release timestamp and additions of ignored-builds, approve-builds and ignoredBuiltDependencies. Supports: pnpm 10.1.0 was released on 2025-01-26 and introduced commands for listing ignored builds and interactively approving dependencies whose install scripts may run..
  - [pnpm approve-builds documentation (versions 11 and 12)](https://pnpm.io/cli/approve-builds) — pnpm project. Retrieved: 2026-08-29. Reviewed: Command purpose, positional approval and denial syntax, allowBuilds writes, pending placeholders and pnpm 11 migration behavior. Supports: The command records approved package names as true and denied names as false in allowBuilds; pnpm 11 replaced and ignores several pnpm 10-era build-policy settings..
  - [pnpm v11.23.0 release notes](https://github.com/pnpm/pnpm/releases/tag/v11.23.0) — pnpm project. Retrieved: 2026-08-29. Reviewed: Release date and allowBuilds migration cleanup described for approve-builds. Supports: pnpm 11.23.0 was released on 2026-08-23 and made approve-builds remove obsolete pnpm 10 build-policy keys when writing allowBuilds..
  - [pnpm v11.24.0 release notes](https://github.com/pnpm/pnpm/releases/tag/v11.24.0) — pnpm project. Retrieved: 2026-08-29. Reviewed: Release date, global build approvals and package-manager release scope. Supports: pnpm 11.24.0 was released on 2026-08-24 and restored global build approvals; the registry also identified 11.24.0 as latest during this investigation..
  - [pnpm 11.24.0 build-policy and version-policy implementation](https://github.com/pnpm/pnpm/blob/cef4816dfbc9aa7ffbe67fa727c1eb9be5d5e1e7/pnpm11/building/policy/src/index.ts) — pnpm project source. Retrieved: 2026-08-29. Reviewed: createAllowBuildFunction precedence, package-name and name-at-version matching, identity trust, ignored-build key generation and exact-version expansion dependency. Supports: The pinned implementation checks exact denials before approvals, then accepts either a bare package name or an exact name@version; ignored registry builds are proposed under the bare package name..
  - [Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks](https://arxiv.org/abs/2005.09535) — DIMVA preprint on arXiv. Retrieved: 2026-08-29. Reviewed: Abstract, dataset scope, supply-chain stages and limitations of the historical collection. Supports: The independent authors report a manually collected historical dataset of malicious npm, PyPI and RubyGems packages and distinguish code execution at multiple supply-chain stages; it provides context, not a pnpm 11 evaluation..

### [npm 12 Blocks Dependency Scripts—But a Git Dependency Still Ran prepare](https://www.accessallgpt.com/research/npm-12-git-dependency-allow-scripts-boundary)
- Decision desk: Agents
- Evidence status: Hands-on package-manager investigation
- Published: 2026-08-28; last material update: 2026-08-28
- Summary: npm 12.0.2 correctly rejected a git dependency until I enabled git fetching. After that, an unapproved prepare hook still ran twice inside npm’s temporary clone even while npm warned that the package’s prepare script had been blocked. The implementation explains why --ignore-scripts remains the stronger boundary for untrusted git dependencies.
- Method: AccessAllGPT reviewed npm 12.0.0 release notes, npm 12.0.2 documentation, the pacote GitFetcher source bundled in the npm 12.0.2 registry tarball, open npm issue #9783 and the independent DIMVA supply-chain preprint. I built a synthetic local Git repository pinned to commits d886768c94acb3f9fa3104645d2e294e22bb31af and d40165addbb4c8ab3df8a6cf35981657786f9d60. Its prepare hook wrote built.txt and appended its temporary working directory, process version and executable to an external trace. Baseline runs used Node v24.10.0 with npm 11.6.0. npm 12 runs used npm 12.0.2 with its supported Node v24.15.0 binary through npx. I tested default npm 12 policy, --allow-git=all without allowScripts, and npm 11 --ignore-scripts. No production package, remote repository, credential, exploit payload or private system was used. The npm 12 tarball integrity returned by the registry was sha512-uIXokLlBj6FpNUTQX1PmT5pz7BlIN9QlixX+zdaSNHsd0qUXsbDLr50xzY6Sw7cJVr0uzHKDOle0swmPW/p5Qw==.
- Limitations: This is a synthetic local git+file reproduction on macOS 26.0.1 (Darwin 25.0.0), not a vulnerability disclosure, exploit, malware study, registry-package audit or cross-platform survey. npm 12.0.2 ran with a supported Node v24.15.0 binary through npx; the npm_config_user_agent string inherited from the outer npm 11.6.0 launcher, so process.version and process.execPath—not that inherited string—identify the hook runtime. The probe observed two prepare writes but did not isolate why two invocations occurred or prove that count generalizes. A local Git repository avoids remote transport, submodules and authentication. Source inspection covered the bundled GitFetcher path relevant to this probe, not every npm and Arborist policy path. Issue #9783 concerns a neighboring configuration interaction, not independent reproduction of this exact result. The 2020 paper supplies historical supply-chain context and does not evaluate npm 12. Future npm releases may change these controls.
- Primary sources:
  - [npm CLI v12.0.0 release notes](https://github.com/npm/cli/releases/tag/v12.0.0) — npm CLI project. Retrieved: 2026-08-28. Reviewed: Release date, breaking changes for allow-git, allow-remote, supported Node versions and dependency lifecycle scripts. Supports: npm 12.0.0 was released on 2026-07-08 and changed git and remote dependency fetching plus dependency-script policy: git fetches require an explicit allowance and dependency scripts are blocked unless covered by the root allowScripts policy..
  - [package.json: Git URLs as Dependencies (npm CLI 12.0.2)](https://docs.npmjs.com/cli/v12/configuring-npm/package-json#git-urls-as-dependencies) — npm documentation. Retrieved: 2026-08-28. Reviewed: Git URL grammar, commit-ish resolution and the documented temporary clone, dependency install, script, pack and install flow. Supports: npm documents that selected scripts or workspaces make a git dependency a build input: npm clones it into a temporary directory, installs its dependencies, runs relevant scripts, then packs and installs the result..
  - [Config: allow-scripts and ignore-scripts (npm CLI 12.0.2)](https://docs.npmjs.com/cli/v12/using-npm/config#allow-scripts) — npm documentation. Retrieved: 2026-08-28. Reviewed: allow-scripts matching and scope, project allowScripts policy, override behavior and ignore-scripts semantics. Supports: npm describes allow-scripts as covering dependency install-time hooks including prepare for non-registry dependencies, while ignore-scripts suppresses package.json scripts except explicitly invoked script commands..
  - [pacote GitFetcher implementation bundled with npm 12.0.2](https://github.com/npm/pacote/blob/v22.0.0/lib/git.js) — npm CLI project. Retrieved: 2026-08-28. Reviewed: GitFetcher constructor, #prepareDir, temporary clone and tarballFromResolved implementation paths. Supports: The bundled GitFetcher checks ignoreScripts before spawning npm inside the temporary clone, but the inspected #prepareDir path does not evaluate the root allowScripts policy before that preparation install and pack flow..
  - [Issue #9783: allow-scripts forwarded to git-dependency preparation](https://github.com/npm/cli/issues/9783) — npm CLI issue tracker. Retrieved: 2026-08-28. Reviewed: Issue title, reproduction context, open state, creation date and latest activity visible through the GitHub API. Supports: An independently filed open issue documents a neighboring npm 12 git-preparation and allow-scripts interaction. It is corroborating community evidence, not proof of the exact local result reported here..
  - [Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks](https://arxiv.org/abs/2005.09535) — DIMVA preprint on arXiv. Retrieved: 2026-08-28. Reviewed: Abstract, dataset scope, submission history and stated supply-chain attack model. Supports: The authors report a manually collected dataset of 174 malicious packages from npm, PyPI and RubyGems dated 2015–2019 and model code execution at different supply-chain stages; this supplies historical independent context, not an npm 12 measurement..

### [npm Install Is an Execution Boundary](https://www.accessallgpt.com/research/npm-install-lifecycle-scripts-execution-boundary)
- Decision desk: Agents
- Evidence status: Hands-on supply-chain investigation
- Published: 2026-08-22; last material update: 2026-08-22
- Summary: A local file dependency with preinstall, install and postinstall hooks ran all three hooks during npm install in my sandbox. Re-running with --ignore-scripts suppressed the hooks entirely, which makes the real question obvious: who gets to execute code during installation?
- Method: AccessAllGPT source review of the current npm scripts documentation, Node.js permission documentation, the Node.js threat model and NIST AI 600-1, plus a bounded local reproduction on Node v24.10.0 and npm 11.6.0 on macOS 26.0.1. In a temporary sandbox, I created a file dependency with preinstall, install and postinstall hooks that append to a trace file, ran npm install, confirmed the three hook lines were written, then reran the install with --ignore-scripts and confirmed the trace stayed empty. I did not audit a production package, publish a malicious package, analyze a real exploit chain or find a third-party write-up reproducing this exact local probe.
- Limitations: This investigation is a bounded local reproduction on Node v24.10.0 and npm 11.6.0, not a supply-chain incident report, malware analysis, registry study or cross-platform survey. The probe used only local file dependencies and a trace file, so it proves that npm install executed the hooks in this environment; it does not prove the behavior of every npm version, registry package, package manager or operating system. The --ignore-scripts result shows one useful mitigation, not a complete defense against malicious code later invoked by build tools or the application itself. Node, npm and NIST documentation may evolve, and independent validation on the exact runner remains the right next step for any sensitive deployment.
- Primary sources:
  - [Scripts | npm Docs](https://docs.npmjs.com/cli/v11/using-npm/scripts) — npm. Retrieved: 2026-08-22. Reviewed: Description, Life Cycle Operation Order, Exiting and Working Directory for Scripts. Supports: npm says lifecycle scripts run after package changes are applied, lists preinstall, install and postinstall in order, and documents that scripts are executed through /bin/sh on POSIX systems or cmd.exe on Windows..
  - [Permissions — Node.js documentation](https://nodejs.org/api/permissions.html) — Node.js project. Retrieved: 2026-08-22. Reviewed: Permissions overview, child-process and filesystem controls, known limitations and security notes. Supports: Node’s permission model narrows selected runtime capabilities but does not claim to sandbox malicious code. That makes shell execution and package hooks a separate authority question, not a Node-only flag choice..
  - [Security policy and threat model](https://github.com/nodejs/node/blob/main/SECURITY.md#the-nodejs-threat-model) — Node.js project. Retrieved: 2026-08-22. Reviewed: The Node.js threat model section and its statement about trusted versus untrusted code. Supports: The Node project’s threat model says the runtime trusts the code it is asked to run and that dependencies inherit the execution user’s privileges. That is the right backdrop for treating npm hooks as code execution, not as inert metadata..
  - [Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-22. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST frames AI trustworthiness as a design, development, use and evaluation problem. That general control framing supports least-privilege package installation, evidence logging and explicit rollback for agentic systems..

### [MCP Token Passthrough Needs an Audience Boundary](https://www.accessallgpt.com/research/mcp-token-passthrough-audience-boundary)
- Decision desk: Agents
- Evidence status: Protocol security guide
- Published: 2026-08-22; last material update: 2026-08-22
- Summary: If an MCP server can relay the same bearer token it received, the server becomes part of the authorization plane. The safer pattern is audience-bound resource tokens, explicit resource metadata and a refusal to treat localhost as the trust boundary.
- Method: AccessAllGPT desk review of the current MCP security best practices, MCP authorization and tools specifications, RFC 8707, RFC 9728 and RFC 9700, plus a bounded local loopback reproduction on Node v24.10.0 that relays a synthetic Authorization header from one local HTTP server to another. We verified the public pages were reachable on 2026-08-22, but did not audit a production MCP implementation, a cloud auth server or any third-party deployment. The local probe demonstrates authority propagation through a relay, not a defect in MCP itself.
- Limitations: This article is a desk review plus a synthetic local relay demonstration, not an audit of a real MCP implementation, authorization server or production identity stack. The loopback test proves only that a relay can forward a synthetic bearer token unchanged on one machine; it does not prove a defect in MCP, a prevalence rate, a vendor bug or a remote exploit. We did not test sender-constrained tokens, refresh-token rotation, a cloud auth server, a third-party MCP client, or an actual deployed server. RFCs and MCP docs may evolve; re-check the cited pages and re-run the relay on any new runtime or architecture.
- Primary sources:
  - [Security Best Practices (version 2026-07-28)](https://modelcontextprotocol.io/docs/2026-07-28/tutorials/security/security_best_practices#token-passthrough) — Model Context Protocol. Retrieved: 2026-08-22. Reviewed: Token passthrough risks, local MCP server compromise, SSRF risks and common mistakes. Supports: The MCP security guidance explicitly frames token passthrough as risky, warns about local server compromise, and treats SSRF and token handling as security concerns for MCP deployments..
  - [Authorization (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization#resource-parameter-implementation) — Model Context Protocol Specification. Retrieved: 2026-08-22. Reviewed: Resource parameter implementation, canonical server URI, access token usage and token handling. Supports: The authorization spec requires servers to validate that access tokens were issued specifically for them as the intended audience and describes the resource-oriented discovery flow that anchors the authorization relationship..
  - [Tools (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/server/tools#security-considerations) — Model Context Protocol Specification. Retrieved: 2026-08-22. Reviewed: Security considerations and user interaction model for tool exposure. Supports: The tools specification emphasizes that tool exposure and invocation are security-sensitive, which matters when a server can forward user credentials to another service or trigger privileged actions on a user’s behalf..
  - [Resource Indicators for OAuth 2.0 (RFC 8707)](https://www.rfc-editor.org/rfc/rfc8707.html) — RFC Editor. Retrieved: 2026-08-22. Reviewed: Resource parameter, audience restriction and threat model. Supports: RFC 8707 defines resource indicators so the client can ask for a token for a specific protected resource instead of receiving a bearer token that is broadly reusable across a service fleet..
  - [OAuth 2.0 Protected Resource Metadata (RFC 9728)](https://www.rfc-editor.org/rfc/rfc9728.html#section-3) — RFC Editor. Retrieved: 2026-08-22. Reviewed: Protected resource metadata and security considerations. Supports: RFC 9728 standardizes resource metadata discovery so clients can learn which authorization server and token endpoints belong to a protected resource instead of inventing token-routing rules ad hoc..
  - [Best Current Practice for OAuth 2.0 Security (RFC 9700)](https://www.rfc-editor.org/rfc/rfc9700.html#section-2.3) — RFC Editor. Retrieved: 2026-08-22. Reviewed: Audience-restricted access tokens, token leakage and metadata attacks. Supports: RFC 9700 elevates audience-restricted tokens and related countermeasures as best current practice because bearer tokens are dangerous when they can be replayed or forwarded beyond their intended recipient..

### [Node’s Permission Model Is a Seat Belt, Not an AI Code Sandbox](https://www.accessallgpt.com/research/node-permission-model-ai-generated-code-sandbox)
- Decision desk: Agents
- Evidence status: Hands-on security investigation
- Published: 2026-08-16; last material update: 2026-08-21
- Summary: Node’s stable permission model still catches accidental filesystem and process access, and v26.7.0’s new --permission-audit mode is useful for rollout rehearsals. Our local probe showed the same diagnostics-channel event in both modes; only enforce mode denied the read.
- Method: AccessAllGPT source review and bounded local experiment, first run 2026-08-16 and refreshed 2026-08-21. We cloned Node.js tag v24.10.0 at immutable commit 9b72b88f4c4565687e3a8c4d8e1232f63a501e15, inspected its permission documentation, security threat model, environment initialization and 44 permission-named tests, and checked the official release index. We then downloaded Node.js v26.7.0 for Darwin arm64, verified its SHA256 against SHASUMS256.txt, reviewed the release note and permissions docs, and reran the local probe under both --permission-audit and --permission. The script creates only temporary fixture files, starts a loopback-only Python HTTP server, runs asserted scenarios in child Node processes, kills the server and removes the fixture. It tests direct filesystem reads and writes, relative-symlink traversal, loopback fetch, default child-process denial, an explicitly allowed non-Node child, an inherited file descriptor and the new audit-mode diagnostics path. We did not execute model-generated code, test a container or VM, probe Linux or Windows, attempt undisclosed bypasses, benchmark overhead, or perform a production security assessment. Node.js project statements, source observations, local results, independent NIST context and AccessAllGPT guidance are labeled separately.
- Limitations: This investigation is a source review and bounded synthetic experiment on Node.js v24.10.0 plus a follow-up audit/enforce check on Node.js v26.7.0 for Darwin arm64, not a penetration test, vulnerability disclosure, sandbox certification, cross-version comparison or production assessment. The assertions demonstrate selected documented controls and boundary conditions only. We did not test Linux, Windows, node:sqlite, addons, WASI, workers, inspector, package installation, denial of service, races, side channels, container or VM escape, or every filesystem and network API. The loopback result proves only that this Node permission configuration did not restrict the tested fetch. The symlink, descriptor and malicious-code boundaries are documented by Node; we do not present them as previously unknown vulnerabilities. NIST provides general risk guidance and does not evaluate Node.js. Future runtimes, operating systems and runner configurations can differ, and finite tests cannot establish containment against hostile code.
- Primary sources:
  - [Node.js v26.7.0 release note](https://nodejs.org/en/blog/release/v26.7.0) — Node.js project. Retrieved: 2026-08-21. Reviewed: Release headline and changelog bullet documenting --permission-audit audit mode behavior. Supports: The v26.7.0 release note records the documentation change that introduced audit-mode behavior for the permission model..
  - [Permissions — Node.js v26.7.0 documentation](https://nodejs.org/dist/v26.7.0/docs/api/permissions.md) — Node.js project. Retrieved: 2026-08-21. Reviewed: Operational modes, runtime API, audit-mode diagnostics-channel mapping and known issues. Supports: The v26.7.0 docs define enforce mode and audit mode, explain that audit mode logs violations without denying access, enumerate the diagnostics-channel names, and retain the symlink and existing-descriptor limitations..
  - [SHASUMS256.txt for Node.js v26.7.0](https://nodejs.org/dist/v26.7.0/SHASUMS256.txt) — Node.js project. Retrieved: 2026-08-21. Reviewed: node-v26.7.0-darwin-arm64.tar.gz checksum row. Supports: The downloaded Darwin arm64 binary was verified against the official SHA256 before local execution..
  - [Scripts | npm Docs](https://docs.npmjs.com/cli/v11/using-npm/scripts) — npm. Retrieved: 2026-08-21. Reviewed: How npm handles the scripts field; shell execution behavior on POSIX and Windows. Supports: npm documents that scripts run through /bin/sh on POSIX systems and cmd.exe on Windows, which makes shell policy part of the agent boundary whenever an agent is allowed to invoke package scripts..
  - [Permissions — Node.js v24.10.0 documentation](https://github.com/nodejs/node/blob/9b72b88f4c4565687e3a8c4d8e1232f63a501e15/doc/api/permissions.md) — Node.js project. Retrieved: 2026-08-16. Reviewed: Scope and threat statement; stable status; restricted resources; filesystem allowlists; runtime API; npx behavior; constraints; symlink and inherited-file-descriptor limitations. Supports: The pinned documentation calls the permission model a seat belt for trusted code, explicitly says it does not protect against malicious code, lists the resources it restricts, says node:sqlite can access files outside node:fs checks, and documents relative-symlink and inherited-file-descriptor bypass paths..
  - [Node.js security policy and threat model at v24.10.0](https://github.com/nodejs/node/blob/9b72b88f4c4565687e3a8c4d8e1232f63a501e15/SECURITY.md#the-nodejs-threat-model) — Node.js project. Retrieved: 2026-08-16. Reviewed: Responsible testing; trusted and untrusted elements; treatment of code Node.js is asked to run; inherited execution-user privileges. Supports: The project threat model trusts JavaScript, WASM, native code and dependencies that Node.js is asked to run and says that code inherits the privileges of the execution user. This is incompatible with treating the runtime alone as a hostile-code security boundary..
  - [src,lib: stabilize permission model — pull request 56201](https://github.com/nodejs/node/pull/56201) — Node.js project on GitHub. Retrieved: 2026-08-16. Reviewed: Title; merge date; release lines; linked documentation history. Supports: The permission model moved to stable through a change merged on December 12, 2024 and documented for Node.js 22.13.0 and 23.5.0. Stable API status describes support maturity; it does not revise the project’s malicious-code exclusion..
  - [Node.js v24.10.0 source tree at commit 9b72b88](https://github.com/nodejs/node/tree/9b72b88f4c4565687e3a8c4d8e1232f63a501e15) — Node.js project on GitHub. Retrieved: 2026-08-16. Reviewed: Permission initialization in src/env.cc; permission implementation; CLI documentation; permission tests for filesystem, symlinks, children, workers, addons, WASI, inspector, sqlite and file descriptors. Supports: AccessAllGPT cloned the immutable v24.10.0 source commit. Environment initialization denies addons, inspector, child processes, workers and WASI unless explicitly allowed, then applies filesystem read and write allowlists. The test inventory exercises those controls and known symlink behavior; it does not implement a network permission boundary..
  - [Node.js release index](https://nodejs.org/dist/index.json) — Node.js project. Retrieved: 2026-08-16. Reviewed: v24.10.0 release metadata and current release rows at retrieval. Supports: The official index dates v24.10.0 to October 8, 2025. AccessAllGPT used that exact installed runtime; this is a version-pinned result rather than a claim about every supported or future Node.js release..
  - [Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-16. Reviewed: Publication scope; generative-AI risk-management framing; risk mapping, measurement and management context. Supports: NIST presents the profile as voluntary, cross-sector guidance for incorporating trustworthiness considerations into design, development, use and evaluation. It supplies independent governance context for layered, measured controls; it does not evaluate Node.js or certify a sandbox architecture..

### [Before You Connect an MCP Server to Production](https://www.accessallgpt.com/research/mcp-server-production-security-review)
- Decision desk: Agents
- Evidence status: Security decision guide
- Published: 2026-08-04; last material update: 2026-08-20
- Summary: MCP standardizes how an AI application reaches tools; it does not decide which authority the application should receive. Use these gates before a server can touch production data or actions.
- Method: AccessAllGPT desk review of the version-pinned MCP 2026-07-28 specification, MCP security guidance, the MCP TypeScript SDK 1.30.0 README and underlying OAuth RFCs, reverified 2026-08-20. Normative MUST and advisory SHOULD language is separated from AccessAllGPT recommendations. This is implementation guidance plus a local stdio probe, not an original security test or vendor comparison.
- Limitations: This is a protocol and security-architecture review, not penetration testing, legal advice or evidence that any implementation is safe. The 2026-07-28 MCP specification can change; clients and servers may implement only part of it. Teams must test the exact versions, transports, identity provider, tools, upstream APIs and runtime they plan to operate.
- Primary sources:
  - [Security Best Practices (version 2026-07-28)](https://modelcontextprotocol.io/docs/2026-07-28/tutorials/security/security_best_practices#token-passthrough) — Model Context Protocol. Retrieved: 2026-08-20. Reviewed: Token passthrough, SSRF and local MCP server compromise. Supports: Token passthrough is an anti-pattern; discovery can create SSRF exposure; local servers execute with local process authority..
  - [Authorization (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization#resource-parameter-implementation) — Model Context Protocol Specification. Retrieved: 2026-08-20. Reviewed: Resource parameter implementation, access-token usage and token handling. Supports: Normative resource-indicator, bearer-token transport, token audience validation and token rejection requirements..
  - [Tools (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/server/tools#security-considerations) — Model Context Protocol Specification. Retrieved: 2026-08-20. Reviewed: User interaction model, capabilities and security considerations. Supports: Normative server controls, advisory human-approval controls, authorization-dependent tool lists and untrusted annotations..
  - [MCP TypeScript SDK README (version 1.30.0)](https://www.npmjs.com/package/@modelcontextprotocol/sdk) — @modelcontextprotocol/sdk. Retrieved: 2026-08-20. Reviewed: Overview, quick start, servers and transports, clients, and local stdio transport description. Supports: The SDK documents stdio as a local, process-spawned transport and says clients can connect to servers and call tools..
  - [Best Current Practice for OAuth 2.0 Security](https://www.rfc-editor.org/rfc/rfc9700.html#section-2.3) — RFC Editor (RFC 9700). Retrieved: 2026-08-03. Reviewed: Redirect URI validation and audience-restricted access tokens. Supports: OAuth security baseline used to cross-check exact redirect matching and audience restriction..
  - [OAuth 2.0 Protected Resource Metadata](https://www.rfc-editor.org/rfc/rfc9728.html#section-3) — RFC Editor (RFC 9728). Retrieved: 2026-08-03. Reviewed: Protected resource metadata request, response and security considerations. Supports: The discovery metadata format and security boundary underlying MCP protected-resource discovery..

### [GLM-5.3: Trial the Coding Gains, Contain the Cyber Capability](https://www.accessallgpt.com/research/glm-5-3-coding-cyber-deployment-decision)
- Decision desk: Models
- Evidence status: Emerging deployment guide
- Published: 2026-08-15; last material update: 2026-08-15
- Summary: GLM-5.3 is a timely coding-agent candidate, not an automatic GLM-5.2 upgrade. Trial the managed model in an isolated repository workflow, migrate thinking settings explicitly, contain network and exploit authority, and wait for the actual weights and safety artifacts before approving self-hosting.
- Method: AccessAllGPT desk review of Z.ai’s GLM-5.3 launch, Coding Plan and API documentation; the public GLM-5.2 predecessor repository pinned at revision b4734de; Terminal-Bench 3.0 pinned at tag v3.0.0 and commit 2b0442c; CyberGym v3 and its harness pinned at commit 7656b71; the ExploitBench harness pinned at commit 9d0173b; and dated official-leaderboard and Hacker News snapshots, retrieved 2026-08-15. We wrote and ran reproducible Node.js audits against the pinned GLM-5.2 Hugging Face metadata and Terminal-Bench checkout. The GLM audit paginated all 295 files, reconciled 282 weight shards between the tree and index, and reported architecture and byte counts without downloading the weights. The Terminal-Bench audit reconciled all 74 task directories with 74 unique SHA-256 manifest digests and summarized categories, GPU use, agent timeouts and expert estimates. We also inspected both pinned cyber harnesses. For CyberGym, we installed its documented development and server dependencies under Python 3.13, compiled the source, and ran targeted smoke checks for checksum binding, image mapping, timeout normalization and rate limiting; we could not run a target because the Docker daemon was unavailable and the benchmark assets were not downloaded. For ExploitBench, we inspected its canonical matrix, scoring and image-resolution code and ran the non-slow unit and golden test suite under Python 3.11. The first run omitted optional publishing dependencies and failed four publishing tests; after installing the documented publish extras, 643 tests passed, six were skipped and six were deselected. We did not run an agent on Terminal-Bench, execute either cyber benchmark or call GLM-5.3. Momentum evidence, vendor claims, predecessor-artifact measurements, independent methodology artifacts, local harness evidence and AccessAllGPT guidance are labeled separately. This is not an original model run, benchmark reproduction, repository trial, red-team exercise, GLM-5.3 weight inspection, security assessment, GLM-5.3 license review, cost study or legal analysis.
- Limitations: AccessAllGPT did not use GLM-5.3, reproduce any benchmark, or run an agent on Terminal-Bench. The Terminal-Bench audit inspected tagged source and manifest metadata only; it did not build task containers, run oracle checks, inspect trajectories or independently derive resolution rates, token counts, uncertainty intervals or cost. AccessAllGPT did not download the GLM-5.2 weight shards, download the roughly 240 GB CyberGym dataset, roughly 130 GB binary-only server data or roughly 10 TB full environment, run a CyberGym target or ExploitBench’s V8 containers, inspect container contents, inspect GLM-5.3 weights, test the API or Coding Plan, verify Z.ai’s vulnerability findings, review commercial terms, assess the model’s security, or measure latency, quality, quota use or cost. The GLM-5.2 artifact audit retrieved public repository metadata, configuration, weight-index and license files only; byte counts and architecture fields describe pinned predecessor revision b4734de, not GLM-5.3 or real serving resource consumption. Our CyberGym work was a static code and documentation audit, Python compilation and targeted helper smoke checks; its repository-wide Ruff check had one import-order failure, and Docker execution was blocked because the daemon was unavailable. Our ExploitBench work was a static code and configuration audit plus the repository’s local non-slow unit and golden tests. Neither cyber audit exercised a model, vulnerable target or grading path end to end. The GLM-5.3 launch and score table are vendor-authored; independent benchmark artifacts explain task mechanics but do not validate Z.ai’s runs. Hacker News activity measures attention only. GLM-5.3 weights, final model card, hashes, license and local deployment requirements were not available at retrieval. A bounded trial cannot prove absence of rare failures or establish suitability for other repositories or security contexts.
- Primary sources:
  - [GLM-5.3: Frontier Coding with Emergent Cyber Capabilities](https://z.ai/blog/glm-5.3) — Z.ai. Retrieved: 2026-08-15. Reviewed: Release date and status; post-training claim; coding, agentic and cyber benchmark tables; weight-release timing; API changes; Coding Plan availability; evaluation footnotes. Supports: Z.ai announced GLM-5.3 on August 14, 2026, says it uses the same base model as GLM-5.2 with gains from additional post-training, reports vendor-run coding and cyber evaluations, documents mandatory thinking and reasoning-effort settings, and says weights are planned two weeks after launch following safety evaluation and hardening. These are vendor claims and release statements, not independent AccessAllGPT measurements..
  - [GLM Coding Plan overview](https://docs.z.ai/devpack/overview) — Z.ai Developer Documentation. Retrieved: 2026-08-15. Reviewed: Coding Plan scope; supported coding-agent integrations; subscription and quota framing; API compatibility guidance. Supports: Z.ai documents its Coding Plan as a route for using GLM models in supported coding-agent clients. This establishes an available managed access path and integration surface; it does not establish GLM-5.3 quality, security, quota sufficiency or compatibility for a particular repository..
  - [Chat completion API](https://docs.z.ai/api-reference/llm/chat-completion) — Z.ai Developer Documentation. Retrieved: 2026-08-15. Reviewed: Request and response contract; model selection; messages; thinking configuration; tools; streaming; errors and usage fields. Supports: Z.ai publishes the request and response surface used to call supported chat models, including model selection, messages, tools, streaming and usage. The documentation establishes an integration contract; actual GLM-5.3 entitlement, behavior, latency, limits and billing must be verified on the adopting account..
  - [zai-org/GLM-5.2 artifact repository at revision b4734de](https://huggingface.co/zai-org/GLM-5.2/tree/b4734de4facf877f85769a911abafc5283eab3d9) — Z.ai on Hugging Face. Retrieved: 2026-08-15. Reviewed: Pinned repository tree and pagination; model card; MIT license text; config.json architecture fields; safetensors index metadata and weight map; serving-framework versions; file sizes and immutable object identifiers. Supports: AccessAllGPT queried the Hugging Face API and small metadata files at immutable GLM-5.2 revision b4734de. The predecessor has 282 bfloat16 safetensor shards with 1,506,659,919,872 indexed parameter bytes, a 78-layer GLM MoE DSA configuration, 256 routed experts with eight selected per token, and a 1,048,576-position configured maximum. Its pinned license file is MIT. These measurements establish the scale and metadata of GLM-5.2 only. They do not establish GLM-5.3 file size, license, serving compatibility, quantization, hardware needs or release status..
  - [Terminal-Bench 3.0 release at tag v3.0.0 and official leaderboard](https://github.com/harbor-framework/terminal-bench/releases/tag/v3.0.0) — Harbor / Terminal-Bench maintainers. Retrieved: 2026-08-15. Reviewed: Tagged release and immutable commit; dataset manifest and content digests; task metadata and category distribution; agent timeouts and expert estimates; official leaderboard model-agent pairs, resolution rates, uncertainty, token totals and costs. Supports: AccessAllGPT cloned and audited immutable Terminal-Bench 3.0 tag v3.0.0 at commit 2b0442c. Its manifest binds 74 named tasks to 74 unique SHA-256 content digests across seven categories. Task metadata spans 30- to 480-minute agent limits, with a 120-minute median, and four GPU tasks. The official leaderboard reports model-agent configurations rather than model-only scores and, in the August 15 snapshot, listed GLM 5.2 with Claude Code at 4.6% ± 1.0% but did not list GLM-5.3. These facts explain the public benchmark contract and baseline; they do not validate Z.ai’s 28.3 result or identify its agent, repetitions, task failures, tokens or cost..
  - [CyberGym: Evaluating AI Agents’ Real-World Cybersecurity Capabilities at Scale (v3)](https://arxiv.org/abs/2506.02548v3) — CyberGym authors on arXiv. Retrieved: 2026-08-15. Reviewed: Version history; abstract; benchmark scale; task construction; proof-of-concept objective; reported baseline difficulty; limitations implied by the evaluation target. Supports: The independent benchmark paper defines CyberGym as 1,507 historical vulnerabilities across 188 projects and primarily asks an agent to generate a proof-of-concept test from a vulnerability description and codebase. This explains what the benchmark targets; it does not validate Z.ai’s GLM-5.3 score or establish safe autonomous exploitation..
  - [CyberGym repository at commit 7656b71](https://github.com/sunblaze-ucb/cybergym/tree/7656b71d07da6694e262f9c34ea994cd4849c0eb) — CyberGym authors on GitHub. Retrieved: 2026-08-15. Reviewed: README data and deployment requirements; FAQ network, task, environment and scoring guidance; August 4 submission schema; server submission and verification paths; task checksum; timeout handling; rate limiting; package metadata; local lint, compilation and helper smoke checks. Supports: At pinned commit 7656b71, AccessAllGPT inspected the current public CyberGym harness. The maintainers now request one designated final submission because any-of scoring increasingly rewards brute force, require token, time, request and artifact reporting, and warn that the partly unauthenticated PoC endpoint must remain private. The README estimates roughly 240 GB for benchmark data and roughly 10 TB for the full compilation environment. Local compilation and targeted helper smoke checks passed; the repository-wide Ruff check found one import-order error. Docker was installed but its daemon was unavailable, so no target container or PoC was run. This audit does not identify or validate Z.ai’s task set, metric, environment, network policy, trajectories or reported score..
  - [ExploitBench repository at commit 9d0173b](https://github.com/exploitbench/exploitbench/tree/9d0173bcf8835b74a45f60450ae7f184e29e7607) — ExploitBench. Retrieved: 2026-08-15. Reviewed: README scope; canonical v8.yaml matrix; capability extraction and scoring code; image-reference resolution; model dispatch; seed and budget configuration; audit and reproduction workflow; local non-slow unit and golden tests. Supports: At pinned commit 9d0173b, AccessAllGPT inspected the public harness and ran its non-slow unit and golden test suite locally. The canonical configuration enumerates 41 V8 environments, six models and five seeds; the scorer cumulatively ORs 16 capability flags and treats arbitrary code execution as a terminal full score. The checked-in matrix uses mutable GHCR tags, while runner code resolves the locally loaded image to an immutable image ID and the documentation recommends registry digests for publication-grade pinning. This audit explains evaluation mechanics but does not identify or validate the revision, matrix, prompts, seeds, containers, transcripts or scoring transformation behind Z.ai’s reported GLM-5.3 result..
  - [Hacker News discussion: GLM-5.3: Frontier coding with emergent cyber capabilities](https://news.ycombinator.com/item?id=49294997) — Hacker News. Retrieved: 2026-08-15. Reviewed: Submission timestamp; linked official release; observable points and comment count in the Algolia API snapshot; discussion scope. Supports: The August 14 submission linked the official release and showed 1,007 points and 496 comments in the snapshot retrieved August 15. This is a strong independent attention signal only; mutable community activity does not validate model capability, safety, availability, license or production fitness..

### [GPT-5.6 Sol Ultrafast: Buy Speed Only Where Latency Changes the Outcome](https://www.accessallgpt.com/research/gpt-5-6-sol-ultrafast-api-decision)
- Decision desk: Models
- Evidence status: Emerging API decision guide
- Published: 2026-08-15; last material update: 2026-08-15
- Summary: GPT-5.6 Sol Ultrafast is an emerging serving option, not a blanket model migration. Trial it only on latency-critical paths where saved time has measured value, quality remains equivalent locally, tier delivery is observable, and fallback to Standard is safe.
- Method: AccessAllGPT desk review of OpenAI launch, model, Fast-mode and pricing documentation; Cerebras launch and benchmark disclosures; and a dated Hacker News momentum snapshot, retrieved 2026-08-15. Momentum evidence, vendor specifications, vendor-performed benchmarks and AccessAllGPT guidance are labeled separately. This is not an original API trial, latency benchmark, quality evaluation, availability test, cost study, security review or legal analysis.
- Limitations: AccessAllGPT did not receive Ultrafast access and performed no API calls, latency or quality benchmark, availability test, cost study or contract review. Ultrafast details come from commercially involved launch sources; the documented Fast mode is distinct. The Hacker News snapshot is mutable attention evidence, not technical validation. Preview access, capacity, behavior and pricing can change. A bounded local trial cannot establish universal performance or eliminate rare failures.
- Primary sources:
  - [Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speed](https://openai.com/index/previewing-ultrafast/) — OpenAI. Retrieved: 2026-08-15. Reviewed: Launch date; limited-preview status; access scope; performance framing; intended workloads; availability caveats. Supports: OpenAI announced Ultrafast mode on August 13, 2026 as a limited preview for selected API customers and describes it as a Cerebras-powered serving option for GPT-5.6 Sol. The page supplies vendor launch and performance claims; it does not establish results for an untested customer workload..
  - [Fast mode](https://platform.openai.com/docs/guides/fast-mode) — OpenAI API Documentation. Retrieved: 2026-08-15. Reviewed: Overview; configuration; response reporting; rate limits and ramp rate; usage considerations; regions; billing; data controls. Supports: OpenAI documents Fast mode as a generally configurable service tier distinct from the new limited-preview Ultrafast offer. It documents up to 2.5x speed for GPT-5.6 Sol versus Standard, shared rate limits, possible downgrade to Standard when traffic ramps too quickly, response-tier reporting, eligibility limits and compatibility statements..
  - [GPT-5.6 Sol model](https://platform.openai.com/docs/models/gpt-5.6-sol) — OpenAI API Documentation. Retrieved: 2026-08-15. Reviewed: Model identity; reasoning levels; context and output limits; knowledge cutoff; pricing; endpoints; supported features and tools; snapshots. Supports: OpenAI identifies gpt-5.6-sol as its frontier model for complex professional work, documents text and image input, text output, reasoning-effort choices, a 1.05M-token context window, a 128K-token maximum output, supported tools and endpoint exclusions. These are vendor specifications, not independent capability measurements..
  - [Pricing](https://platform.openai.com/docs/pricing) — OpenAI API Documentation. Retrieved: 2026-08-15. Reviewed: GPT-5.6 Sol Standard, Batch, Flex and Fast short-context and long-context token prices; regional-processing note; service-tier naming. Supports: The retrieved table lists GPT-5.6 Sol short-context Standard prices of $5 per million input tokens and $30 per million output tokens, and Fast prices of $10 and $60 respectively. Long-context and cache-write rules differ. The page does not yet publish a separate Ultrafast price in the reviewed table..
  - [Accelerating GPT-5.6 Sol Ultrafast](https://www.cerebras.ai/blog/accelerating-gpt-5-6-sol-ultrafast-with-openai) — Cerebras. Retrieved: 2026-08-15. Reviewed: Launch scope; claimed output speed; HLE and GDP-Val methodology notes; customer use cases; architecture; limited-preview caveat; benchmark disclaimer. Supports: Cerebras says it powers the limited preview and reports up to 750 output tokens per second plus internal HLE and GDP-Val comparisons. Cerebras performed and published those tests, warns observed improvements vary by workload and configuration, and is commercially involved in the service; the results are vendor evidence, not independent validation..
  - [Hacker News discussion: Accelerating GPT-5.6 Sol Ultrafast](https://news.ycombinator.com/item?id=49289844) — Hacker News. Retrieved: 2026-08-15. Reviewed: Submission timestamp; linked release; observable points and comment count at retrieval; discussion scope. Supports: The August 13 submission was an observable developer-community momentum signal, showing 694 points and 270 comments in the Hacker News API snapshot retrieved August 15. Popularity is not factual or technical validation, and counts can change after retrieval..

### [LLM Model Deprecation: Migrate Without Changing Production by Accident](https://www.accessallgpt.com/research/llm-model-deprecation-migration-runbook)
- Decision desk: Models
- Evidence status: Model migration runbook
- Published: 2026-08-12; last material update: 2026-08-12
- Summary: A replacement model is a new configured system, not a dependency patch. Inventory every route, freeze the decision contract, shadow the replacement, and migrate only the scopes that clear explicit quality, safety, cost and rollback gates.
- Method: AccessAllGPT desk review of current model-lifecycle documentation from OpenAI, Anthropic, Google Cloud and Amazon Bedrock, retrieved 2026-08-12. Vendor schedules and lifecycle terms are reported as vendor statements; the migration gates are AccessAllGPT guidance. This is not an original model test, compatibility study, performance benchmark, cost measurement, security assessment or legal review.
- Limitations: This guide contains no original migration, model evaluation, compatibility test, performance benchmark, cost measurement or security review. All lifecycle details are vendor-authored statements retrieved 2026-08-12 and can change; platform scope matters, and exceptions may apply. A local replay or canary cannot prove absence of rare failures or future behavioral change. Teams require workload-specific engineering, security, privacy, legal, procurement and operational review.
- Primary sources:
  - [Deprecations](https://developers.openai.com/api/docs/deprecations) — OpenAI Developer Documentation. Retrieved: 2026-08-12. Reviewed: Overview; model deprecation notice periods; notification and replacement framing; API deprecation entries. Supports: OpenAI states that software using its models may need updates, documents recommended replacements, and publishes minimum notice periods of at least six months for generally available models and at least three months for specialized variants unless safety or compliance concerns require a faster timeline. Preview models may receive much shorter notice. These are vendor lifecycle statements, not a compatibility guarantee..
  - [Model deprecations](https://platform.claude.com/docs/en/about-claude/model-deprecations) — Anthropic Claude Platform Documentation. Retrieved: 2026-08-12. Reviewed: Overview; lifecycle terminology; platform scope; migrating to replacements; notifications; auditing model usage. Supports: Anthropic distinguishes active, legacy, deprecated and retired models; says requests to retired models fail; recommends testing applications before retirement; and states that publicly released models receive at least 60 days notice. It also says partner-operated Amazon Bedrock and Google Cloud schedules can differ from Anthropic-operated platforms..
  - [Model versions and lifecycle](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/model-versions) — Google Cloud Documentation. Retrieved: 2026-08-12. Reviewed: Lifecycle overview; availability-period categories; retirement dates; migration framing; lifecycle definitions and date policy. Supports: Google Cloud publishes model lifecycle categories, retirement tables and migration guidance for the documented Gemini Enterprise Agent Platform. It says listed retirement dates may be extended but will not be brought forward, and distinguishes models available for at least 12 months from short-term models that retire 45 days after a replacement release. These terms apply to the documented Google Cloud service, not every Gemini surface..
  - [Model lifecycle](https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html) — Amazon Bedrock User Guide. Retrieved: 2026-08-12. Reviewed: Lifecycle overview; Active, Legacy and End-of-Life states; minimum availability statement; provider-date warning; migration implications. Supports: Amazon Bedrock documents Active, Legacy and End-of-Life states, exposes lifecycle state through model APIs, and says a model remains on Bedrock for at least 12 months before its End-of-Life date. AWS warns that Bedrock dates can differ from model-provider dates and that Bedrock-specific dates govern Bedrock usage..

### [AI API Data Retention and Residency: Set the Procurement Gates](https://www.accessallgpt.com/research/ai-api-data-retention-residency-procurement-gates)
- Decision desk: Buyer guides
- Evidence status: Data governance buyer guide
- Published: 2026-08-11; last material update: 2026-08-11
- Summary: “No training” is not a retention policy, and a region selector is not a complete data map. Use this evidence-led review to gate an AI API on storage, processing, logs, application state, transfers and deletion.
- Method: AccessAllGPT desk review of current OpenAI API data-control documentation, Amazon Bedrock data-protection documentation, the official GDPR text and the NIST Privacy Framework, retrieved 2026-08-11. Vendor statements, legal text and AccessAllGPT procurement guidance are labeled separately. This is not an original platform test, deletion audit, contract review, privacy impact assessment, security assessment, legal analysis or vendor comparison.
- Limitations: This guide contains no original provider-system inspection, deletion audit, regional-routing test, contract review or legal analysis. OpenAI and AWS documentation is vendor-authored and can vary by product, account, feature, region and date. The GDPR applies according to facts and law that this article does not assess; NIST guidance is voluntary. Buyers cannot directly observe every provider or subprocessor system, and a local test cannot prove deletion from opaque stores or absence of unauthorized access. Qualified privacy, legal, security, procurement and data-governance review remains necessary.
- Primary sources:
  - [Data controls in the OpenAI platform](https://developers.openai.com/api/docs/guides/your-data) — OpenAI Developer Documentation. Retrieved: 2026-08-11. Reviewed: Data use; types of API data; abuse-monitoring retention; Zero Data Retention and Modified Abuse Monitoring eligibility and limitations; endpoint-level application-state retention; regional storage and processing controls; Enterprise Key Management. Supports: OpenAI states that API data is not used to train or improve its models unless a customer explicitly opts in, distinguishes abuse-monitoring logs from application state, documents default abuse-monitoring retention of up to 30 days, and describes feature-, project-, region- and eligibility-dependent controls. These are current vendor statements about the documented platform, not independent verification or a promise for every account and feature..
  - [Data protection — Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/data-protection.html) — Amazon Web Services Documentation. Retrieved: 2026-08-11. Reviewed: Shared-responsibility boundary; IAM, encryption and activity-logging recommendations; warning about sensitive data in tags and free-form name fields; model deployment accounts; linked encryption, PrivateLink and retention topics. Supports: AWS assigns customers responsibility for content controls and service configuration, warns that tags and free-form naming fields can enter billing or diagnostic logs, and states that model providers cannot access Bedrock deployment accounts, logs, prompts or completions. These are AWS service statements, not an audit of a customer configuration or a universal claim about cloud AI services..
  - [Regulation (EU) 2016/679 (General Data Protection Regulation)](https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng) — EUR-Lex, European Union. Retrieved: 2026-08-11. Reviewed: Article 5 principles including purpose limitation, data minimisation and storage limitation; Article 28 processor requirements; Articles 44–49 transfers of personal data to third countries or international organisations. Supports: The official regulation text establishes principles and legal obligations relevant to processing, processor arrangements, retention and international transfers of personal data. It does not prescribe this article’s procurement workflow or determine whether a particular AI deployment is compliant..
  - [NIST Privacy Framework](https://www.nist.gov/privacy-framework) — National Institute of Standards and Technology. Retrieved: 2026-08-11. Reviewed: Framework purpose, voluntary risk-management positioning, Core and Profiles, implementation resources and relationship to enterprise risk management. Supports: NIST presents the Privacy Framework as a voluntary tool for identifying and managing privacy risk while building products and services. It provides risk-management structure, not certification, legal advice or approval of a provider..

### [LLM Evaluation Platforms: Keep the Decision Contract Portable](https://www.accessallgpt.com/research/llm-evaluation-platform-build-buy-portability)
- Decision desk: Buyer guides
- Evidence status: Evaluation tooling buyer guide
- Published: 2026-08-11; last material update: 2026-08-11
- Summary: Choose an LLM evaluation platform by what you can export, reproduce and migrate—not by grader count. Own the cases, decision rules, provenance and case-level results before adopting a workflow or dashboard.
- Method: AccessAllGPT desk review of current OpenAI evaluation and deprecation documentation, Anthropic evaluation guidance, HELM version 2 and the NIST Generative AI Profile, retrieved 2026-08-11. Vendor timelines, vendor guidance, research findings and AccessAllGPT procurement guidance are labeled separately. This is not an original platform test, migration exercise, grader study, cost benchmark, security assessment or vendor comparison.
- Limitations: This guide contains no original evaluation-platform test, migration drill, grader comparison, cost measurement, security review or procurement exercise. OpenAI and Anthropic documentation is vendor-authored and can change; the OpenAI shutdown dates are scheduled vendor statements retrieved 2026-08-11, not an independently controlled event. HELM reports results in its selected historical scope, and NIST guidance is voluntary. Exportability does not guarantee semantic equivalence between runners or graders, model-based evaluation remains fallible, and local legal, security, privacy, procurement and domain review is required.
- Primary sources:
  - [Deprecations](https://developers.openai.com/api/docs/deprecations) — OpenAI Developer Documentation. Retrieved: 2026-08-11. Reviewed: Deprecation overview; upcoming deprecations; 2026-06-03 Evals platform entry; announcement, read-only and shutdown dates; stated migration path. Supports: OpenAI states that it announced deprecation of its Evals platform on June 3, 2026, that existing evals become read-only on October 31, 2026, and that the dashboard and API are scheduled to shut down on November 30, 2026. This is a vendor timeline for one product, not evidence that another platform is durable or that migration is lossless..
  - [Evaluation best practices](https://developers.openai.com/api/docs/guides/evaluation-best-practices) — OpenAI Developer Documentation. Retrieved: 2026-08-11. Reviewed: What evals are; evaluation process design; representative data and edge cases; evaluator types; human and model grading; continuous evaluation; Evals platform transition notice. Supports: OpenAI describes evals as structured tests for variable model behavior and recommends defining an objective, collecting representative and edge-case data, combining evaluator types and continuously evaluating. This is vendor-authored implementation guidance, not independent validation of a product or grader..
  - [Define success criteria and build evaluations](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests) — Anthropic Claude Platform Documentation. Retrieved: 2026-08-11. Reviewed: Success criteria; test-case construction; example metrics and measurement methods; automated, human and model-based grading; evaluation workflow and limitations. Supports: Anthropic recommends specific and measurable success criteria, multidimensional evaluation, representative test cases and a mix of grading methods. This is vendor guidance for evaluation practice, not independent evidence that Claude, a vendor console or model-based grading is accurate for a buyer’s workload..
  - [Holistic Evaluation of Language Models (version 2)](https://arxiv.org/abs/2211.09110v2) — Transactions on Machine Learning Research / arXiv. Retrieved: 2026-08-11. Reviewed: Abstract; scenario and metric taxonomy; multi-metric design; standardized evaluation scope; transparency artifacts; reported coverage and stated limitations. Supports: The HELM authors separate scenario coverage from metric coverage, evaluate multiple dimensions rather than accuracy alone, and publish raw prompts and completions for analysis. The reported study concerns its selected models, scenarios and historical period; it does not validate a procurement checklist or current commercial platform..
  - [Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-11. Reviewed: Publication metadata; abstract; voluntary cross-sector scope; relationship to AI RMF 1.0; design, development, use and evaluation framing. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 intended to help organizations incorporate trustworthiness considerations into the design, development, use and evaluation of generative AI systems. It does not certify an evaluation platform or prescribe this build-buy decision..

### [Human-in-the-Loop AI: Put Approval at the Consequence Boundary](https://www.accessallgpt.com/research/automation-workflows-human-approval-boundaries)
- Decision desk: Automation
- Evidence status: Control design guide
- Published: 2026-08-03; last material update: 2026-08-11
- Summary: Decide which AI actions need human approval, what evidence a reviewer must see, and when approval is too weak to make an unsafe authority path deployable.
- Method: AccessAllGPT desk review of the Model Context Protocol tools specification version 2026-07-28, current OpenAI agent-safety guidance, OWASP LLM01:2025 and NIST AI 600-1, retrieved 2026-08-11. Normative and advisory protocol text, vendor guidance, security guidance and AccessAllGPT recommendations are labeled separately. This is not an original usability study, security test, incident study, legal analysis or measurement of approval effectiveness.
- Limitations: This guide contains no original human-factors experiment, security test, incident analysis or legal assessment. The MCP specification and OpenAI documentation provide implementation guidance rather than measured approval effectiveness; OWASP and NIST provide security and risk-management guidance rather than certification. Review quality depends on the interface, task, reviewer expertise, workload, incentives, time pressure and local controls. Finite test cases cannot prove that approval will catch novel attacks or rare failures, and qualified security, privacy, legal, accessibility and human-factors review may be required.
- Primary sources:
  - [Tools (version 2026-07-28)](https://modelcontextprotocol.io/specification/2026-07-28/server/tools#user-interaction-model) — Model Context Protocol Specification. Retrieved: 2026-08-11. Reviewed: User Interaction Model, tool capabilities and tool invocation behavior. Supports: The specification says there SHOULD always be a human able to deny tool invocations and recommends interfaces that disclose exposed tools, indicate invocation and present confirmation prompts. It does not mandate a particular interaction model..
  - [Safety in building agents](https://developers.openai.com/api/docs/guides/agent-builder-safety) — OpenAI Developer Documentation. Retrieved: 2026-08-11. Reviewed: Prompt-injection and private-data risks, structured outputs, tool approvals, guardrails, trace graders and combined controls. Supports: OpenAI advises enabling approvals for MCP tool operations in Agent Builder and combining approvals with constrained data flow, guardrails and evaluation. This is vendor guidance for its products, not independent evidence that approval prevents unsafe outcomes..
  - [LLM01:2025 Prompt Injection](https://genai.owasp.org/llmrisk/llm01-prompt-injection/) — OWASP GenAI Security Project. Retrieved: 2026-08-11. Reviewed: Definition, impacts, prevention and mitigation strategies, and attack scenarios. Supports: OWASP says fool-proof prevention methods are unclear and recommends layered mitigations that include least privilege, independent authorization, segregation of untrusted content, monitoring and human approval for high-risk actions..
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-11. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into the design, development, use and evaluation of generative-AI systems..

### [LLM Observability: Build the Evidence Layer Before You Buy the Dashboard](https://www.accessallgpt.com/research/llm-observability-build-vs-buy-decision)
- Decision desk: Buyer guides
- Evidence status: Observability buyer guide
- Published: 2026-08-10; last material update: 2026-08-10
- Summary: Decide what evidence an AI system must produce, which content may be retained, and whether your existing stack, a specialist platform or a hybrid can meet the operating contract.
- Method: AccessAllGPT desk review of commit-pinned OpenTelemetry Generative AI conventions, the W3C Trace Context Recommendation, commit-pinned OWASP logging guidance and NIST AI 600-1, retrieved 2026-08-10. Standards, project guidance and AccessAllGPT recommendations are labeled separately. This is not an original platform test, incident study, cost benchmark, security assessment or vendor comparison.
- Limitations: This guide contains no original telemetry-platform test, cost measurement, security assessment or incident study and does not compare named products. OpenTelemetry labels its Generative AI conventions Development, and fields or requirements can change. W3C Trace Context addresses propagation rather than AI quality; OWASP and NIST provide general guidance rather than certification. Local drills cannot prove complete observability, absence of rare failures, lawful data processing or future portability. Teams need workload-specific reliability, security, privacy, legal, records-management and procurement review.
- Primary sources:
  - [Semantic conventions for generative AI systems (commit 46d43c8)](https://github.com/open-telemetry/semantic-conventions-genai/blob/46d43c8949afb53765a202e89f4534eeb75ca3fa/docs/gen-ai/README.md) — OpenTelemetry. Retrieved: 2026-08-10. Reviewed: Generative AI convention status and the linked spans and metrics specifications. Supports: OpenTelemetry labels its Generative AI semantic conventions Development and defines signal families for model and agent spans, events, exceptions and metrics. This establishes an evolving interoperability vocabulary, not production completeness or backend equivalence..
  - [Semantic conventions for generative client AI spans (commit 46d43c8)](https://github.com/open-telemetry/semantic-conventions-genai/blob/46d43c8949afb53765a202e89f4534eeb75ca3fa/docs/gen-ai/gen-ai-spans.md) — OpenTelemetry. Retrieved: 2026-08-10. Reviewed: Span lifecycle, inference attributes, token usage and content-capture warnings. Supports: The development-status convention describes logical operation spans across retries, provider and model attributes, error type, token usage and opt-in input/output content fields; it warns that content fields are likely to contain sensitive or PII data..
  - [Trace Context, W3C Recommendation 23 November 2021](https://www.w3.org/TR/2021/REC-trace-context-1-20211123/) — World Wide Web Consortium. Retrieved: 2026-08-10. Reviewed: Abstract, overview, traceparent, tracestate and privacy and security considerations. Supports: The W3C Recommendation standardizes HTTP headers and value formats for propagating request context across services. It enables distributed trace correlation; it does not define LLM quality, evaluation or prompt-retention policy..
  - [Logging Cheat Sheet (commit da4c967)](https://github.com/OWASP/CheatSheetSeries/blob/da4c967e9de854727f72bb2748dd98f76c888b06/cheatsheets/Logging_Cheat_Sheet.md) — OWASP Cheat Sheet Series. Retrieved: 2026-08-10. Reviewed: Which events to log, event attributes, data to exclude, verification, protection and monitoring. Supports: OWASP recommends risk-proportionate logging requirements, exclusion or protection of sensitive fields, sanitization of event data, protection against unauthorized access or tampering and integration with monitoring and incident response..
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-10. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation..

### [Prompt Injection: Set the Deployment Gates Before Your LLM Can Act](https://www.accessallgpt.com/research/prompt-injection-deployment-gates)
- Decision desk: Agents
- Evidence status: Security deployment guide
- Published: 2026-08-09; last material update: 2026-08-09
- Summary: Treat direct and indirect prompt injection as an authority-design problem. Decide whether to deploy, constrain or reject an LLM workflow with explicit data, tool, approval and evaluation gates.
- Method: AccessAllGPT desk review of OWASP LLM01:2025, NIST AI 100-2 E2025, the indirect prompt-injection paper arXiv:2302.12173v2 and current OpenAI agent-safety guidance, retrieved 2026-08-09. Standards guidance, research findings, vendor guidance and AccessAllGPT recommendations are labeled separately. This is not an original penetration test, security assessment, control validation or prevalence study.
- Limitations: This guide contains no original attack execution, penetration test, control validation or prevalence estimate. The cited indirect prompt-injection demonstrations use 2023-era systems and may not transfer unchanged to current configurations. OWASP and NIST provide guidance and taxonomy rather than assurance for a particular deployment; OpenAI documentation is vendor-authored. Finite adversarial suites cannot establish the absence of novel attacks, and model behavior, tools and provider controls can change. Qualified security, privacy, legal and incident-response review remains necessary.
- Primary sources:
  - [LLM01:2025 Prompt Injection](https://genai.owasp.org/llmrisk/llm01-prompt-injection/) — OWASP GenAI Security Project. Retrieved: 2026-08-09. Reviewed: Definition, direct and indirect prompt injection, impacts, prevention and mitigation strategies, and attack scenarios. Supports: OWASP distinguishes direct from indirect prompt injection, ties impact to application context and model agency, says fool-proof prevention methods are unclear, and recommends layered mitigations including constrained behavior, output validation, least privilege, human approval and adversarial testing..
  - [Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2025)](https://csrc.nist.gov/pubs/ai/100/2/e2025/final) — National Institute of Standards and Technology. Retrieved: 2026-08-09. Reviewed: Publication metadata, planning notes, abstract, keywords and documentation links. Supports: NIST describes a taxonomy spanning ML methods, attack life-cycle stages, attacker goals, capabilities and knowledge, and identifies challenges and mitigation methods. The publication provides security terminology rather than certifying a specific prompt-injection control..
  - [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (version 2)](https://arxiv.org/abs/2302.12173v2) — arXiv (authors from CISPA Helmholtz Center for Information Security and Saarland University). Retrieved: 2026-08-09. Reviewed: Abstract, version history, stated threat model, demonstrated attack scope and limitations implied by the tested systems. Supports: The authors define indirect prompt injection through retrieved data and report demonstrations against named 2023-era real and synthetic LLM-integrated applications. This establishes practical examples in that historical scope, not a current prevalence rate or a universal exploit result..
  - [Safety in building agents](https://developers.openai.com/api/docs/guides/agent-builder-safety) — OpenAI Developer Documentation. Retrieved: 2026-08-09. Reviewed: Prompt injection and private-data risks, untrusted variables, structured outputs, tool approvals, input guardrails, trace graders, evaluations and combined controls. Supports: OpenAI advises keeping untrusted variables out of developer messages, constraining data flow with structured outputs, retaining tool approvals, applying input guardrails, and running trace graders and evaluations. This is vendor implementation guidance, not independent evidence that the controls eliminate prompt injection..

### [AI Agents vs Workflows: Choose the Right Automation Architecture](https://www.accessallgpt.com/research/ai-agents-vs-workflows-automation-decision)
- Decision desk: Automation
- Evidence status: Automation architecture guide
- Published: 2026-08-08; last material update: 2026-08-08
- Summary: Do not buy autonomy for a process that can be expressed and controlled as a workflow. Use this evidence-led framework to choose deterministic code, an LLM-assisted workflow, a bounded agent—or no automation.
- Method: AccessAllGPT desk review of Anthropic engineering guidance, OpenAI’s agent-building guide and NIST AI 600-1, retrieved 2026-08-08. Vendor definitions and recommendations are labeled as such; the decision framework, gates and templates are AccessAllGPT guidance. This is not an original benchmark, agent trial, cost study, security assessment or vendor comparison.
- Limitations: This guide contains no original agent or workflow runs and has not been validated as a universal architecture scorecard. Anthropic and OpenAI sources are vendor-authored, use partly different terminology and summarize their own experience rather than controlled comparative trials. Agent behavior depends on the configured model, tools, permissions, prompts, data and runtime. Local tests cannot prove the absence of rare failures, and security, privacy, legal and operational requirements are organization-specific.
- Primary sources:
  - [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) — Anthropic Engineering. Retrieved: 2026-08-08. Reviewed: Definitions; when and when not to use agents; frameworks; augmented LLMs; workflow patterns; autonomous agents; evaluation and production considerations. Supports: Anthropic distinguishes workflows with predefined code paths from agents whose process and tool use are directed dynamically by a model, recommends the simplest sufficient design, and describes latency, cost, control and evaluation trade-offs. These are vendor-authored engineering recommendations, not comparative trial results..
  - [A practical guide to building agents](https://cdn.openai.com/business-guides-and-resources/a-practical-guide-to-building-agents.pdf) — OpenAI. Retrieved: 2026-08-08. Reviewed: Pages 4–8, 13–17 and 24–32: agent definition, use-case criteria, design foundations, orchestration, guardrails and human intervention. Supports: OpenAI defines agents as systems in which an LLM manages workflow execution and dynamically selects tools, says deterministic solutions may suffice outside complex or ambiguous use cases, and recommends incremental orchestration, layered guardrails and human intervention. These are vendor-authored recommendations, not independent evidence of production performance..
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-08. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation. It does not certify an agent or prescribe the AccessAllGPT decision framework..

### [Managed LLM API vs Self-Hosting: Make the Production Decision](https://www.accessallgpt.com/research/managed-llm-api-vs-self-hosting-production-decision)
- Decision desk: Buyer guides
- Evidence status: Architecture buyer guide
- Published: 2026-08-07; last material update: 2026-08-07
- Summary: Compare a managed model API with a self-hosted open-weight stack on accepted outcomes, full operating cost, data boundaries, control and exit—not token price or infrastructure ideology.
- Method: AccessAllGPT desk review of NIST AI 600-1, current OpenAI API data-control documentation, vLLM stable engine configuration documentation and Kubernetes GPU-scheduling documentation, retrieved 2026-08-07. Vendor statements, implementation documentation and AccessAllGPT guidance are separated. This is not an original benchmark, cost study, security assessment or legal review.
- Limitations: This guide contains no original model, infrastructure, security or cost tests and supplies no universal break-even point. OpenAI documentation is vendor-authored and applies only to the stated API controls; vLLM and Kubernetes documentation describe implementation surfaces, not workload outcomes. Prices, products, licenses, model artifacts, documentation and service terms can change. Local trials cannot prove the absence of rare failures, and labor allocation is organization-specific. Qualified security, privacy, legal, licensing, finance and reliability review remains necessary.
- Primary sources:
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-07. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation..
  - [Data controls in the OpenAI platform](https://developers.openai.com/api/docs/guides/your-data) — OpenAI Developer Documentation. Retrieved: 2026-08-07. Reviewed: Data use, abuse-monitoring retention, application state, retention controls and data residency. Supports: OpenAI states that API data is not used to train its models unless the customer opts in, describes default abuse-monitoring retention of up to 30 days, and documents feature- and eligibility-dependent retention controls. These are vendor statements about one managed service, not findings about every API or account..
  - [Engine Arguments](https://docs.vllm.ai/en/stable/configuration/engine_args/) — vLLM documentation. Retrieved: 2026-08-07. Reviewed: Model, load, parallel, cache, device, scheduler and observability configuration groups. Supports: The serving engine exposes choices including model and tokenizer resolution, data type, quantization, model length, tensor and pipeline parallelism, GPU memory utilization, CPU offload and observability. This establishes configuration surface, not a universal performance or cost result..
  - [Schedule GPUs](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/) — Kubernetes Documentation. Retrieved: 2026-08-07. Reviewed: Device plugin prerequisites, GPU resource requests and heterogeneous node selection. Supports: Kubernetes documents vendor drivers and device plugins as prerequisites for GPU scheduling, GPU resources specified through limits, and node labels or affinity for selecting accelerator types. It describes a deployment mechanism, not a complete inference operating model..

### [RAG vs Fine-Tuning: Choose the Right Adaptation Path](https://www.accessallgpt.com/research/rag-vs-fine-tuning-production-decision)
- Decision desk: Buyer guides
- Evidence status: Architecture decision guide
- Published: 2026-08-06; last material update: 2026-08-06
- Summary: Retrieval and fine-tuning solve different failure classes. Use this evidence-led decision to choose prompt-only, RAG, fine-tuning, a measured combination—or no LLM change.
- Method: AccessAllGPT desk review of the original RAG paper, NIST AI 600-1 and current OpenAI implementation guidance, retrieved 2026-08-06. Paper findings, vendor guidance and AccessAllGPT recommendations are labeled separately. This is not an original benchmark, cost study or product comparison.
- Limitations: This guide contains no original RAG or fine-tuning runs and has not been validated as a universal architecture scorecard. The RAG paper studies a particular 2020-era architecture and datasets, while OpenAI documentation is vendor-authored and product availability can change. Retrieval, training, security and economics depend on the exact corpus, model, provider, infrastructure, data rights and workload. Local tests cannot prove the absence of rare failures.
- Primary sources:
  - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (version 4)](https://arxiv.org/abs/2005.11401v4) — arXiv (authors from Facebook AI Research, University College London and New York University). Retrieved: 2026-08-06. Reviewed: Abstract, introduction, methods, experiments, results, discussion and limitations of the paper's experimental scope. Supports: The authors define and evaluate a specific RAG architecture that combines parametric sequence-to-sequence memory with retrieved non-parametric Wikipedia passages; this supports the architectural distinction, not a universal claim that retrieval improves every application..
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-06. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation..
  - [Optimizing LLM Accuracy](https://developers.openai.com/api/docs/guides/optimizing-llm-accuracy) — OpenAI Developer Documentation. Retrieved: 2026-08-06. Reviewed: LLM optimization context, prompt engineering, retrieval-augmented generation, fine-tuning and production accuracy sections. Supports: Vendor-authored guidance distinguishes context optimization for missing, stale or proprietary knowledge from model optimization for inconsistent format, style or behavior, and recommends an evaluate–hypothesize–change–evaluate cycle..
  - [Supervised fine-tuning](https://developers.openai.com/api/docs/guides/supervised-fine-tuning) — OpenAI Developer Documentation. Retrieved: 2026-08-06. Reviewed: Overview, stated use cases, evaluation prerequisite, dataset construction and data-format guidance. Supports: Vendor-authored product guidance describes supervised fine-tuning with prompt-and-response examples for classification, translation, specific output formats and instruction-following behavior, and says to establish evaluations before investing in fine-tuning..

### [Choose a Model Without Chasing the Leaderboard](https://www.accessallgpt.com/research/model-selection-without-leaderboard-chasing)
- Decision desk: Models
- Evidence status: Decision framework
- Published: 2026-08-03; last material update: 2026-08-06
- Summary: Public benchmarks can shortlist candidates; they cannot decide which configured AI system is acceptable for your workload. This guide turns model selection into a reproducible ship, trial or reject decision.
- Method: AccessAllGPT desk review of NIST AI 600-1, the HELM v2 paper, the original Chatbot Arena paper and OpenAI evaluation guidance, reverified 2026-08-06. Historical study findings, vendor guidance and AccessAllGPT recommendations are separated below. This is not an original benchmark, product test or vendor ranking.
- Limitations: This framework has not been validated as a comparative benchmark and contains no original model runs. HELM v2 and the original Chatbot Arena paper report results from their own historical scopes; model versions, leaderboards and vendor documentation can change. Local evaluation cannot prove absence of rare failures, and human or model graders can introduce bias. Teams must set workload-specific statistical, security, privacy, legal and operational requirements.
- Primary sources:
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-06. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation..
  - [Holistic Evaluation of Language Models (version 2)](https://arxiv.org/abs/2211.09110v2) — Transactions on Machine Learning Research / arXiv. Retrieved: 2026-08-06. Reviewed: Abstract, methodology summary and reported evaluation scope. Supports: The authors define scenario coverage and multi-metric evaluation as distinct parts of model evaluation, and report standardized evaluation across accuracy, calibration, robustness, fairness, bias, toxicity and efficiency..
  - [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://arxiv.org/abs/2403.04132) — arXiv (authors from LMSYS Org and UC Berkeley). Retrieved: 2026-08-06. Reviewed: Abstract, methodology description and stated validation claims. Supports: The authors describe Chatbot Arena as pairwise, crowdsourced human-preference evaluation and report agreement checks against expert raters; this establishes what the published ranking measures, not workload-specific production fitness..
  - [Evaluation best practices](https://developers.openai.com/api/docs/guides/evaluation-best-practices) — OpenAI Developer Documentation. Retrieved: 2026-08-06. Reviewed: What are evals, design your eval process, evaluator types and edge cases. Supports: Vendor guidance to define an evaluation objective, collect representative and edge-case data, combine evaluator types and continuously evaluate; used as implementation guidance rather than independent evidence of an OpenAI product claim..

### [Before You Give a Coding Agent Repository Access](https://www.accessallgpt.com/research/coding-agent-repository-access-production-readiness)
- Decision desk: Agents
- Evidence status: Decision framework
- Published: 2026-08-03; last material update: 2026-08-03
- Summary: A coding agent should earn autonomy through controls and repeatable evidence—not through an impressive demo. This framework turns repository access into a staged engineering decision.
- Method: AccessAllGPT synthesis of primary standards, security guidance, platform controls and benchmark methodology. This is not an original benchmark and does not score any vendor.
- Limitations: This framework is vendor-neutral and has not been validated as a comparative benchmark. Teams should adapt weights and stop conditions to their repositories, regulation, data classification and incident tolerance.
- Primary sources:
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-05. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation..
  - [OWASP Top 10 for Large Language Model Applications](https://genai.owasp.org/llm-top-10/) — OWASP GenAI Security Project.
  - [About rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) — GitHub Docs.
  - [SWE-bench](https://www.swebench.com/) — SWE-bench.
  - [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://arxiv.org/abs/2310.06770) — arXiv.

### [From Paper Result to Production Decision](https://www.accessallgpt.com/research/paper-to-production-evidence-ladder)
- Decision desk: Papers to production
- Evidence status: Research method
- Published: 2026-08-03; last material update: 2026-08-03
- Summary: How to decide whether a research result deserves a prototype, a controlled trial or no engineering time yet.
- Method: A reproducibility and transferability checklist.
- Limitations: This is an AccessAllGPT evidence framework, not a reproduction study. Some frontier work lacks code or compute-accessible reproduction paths; uncertainty should be stated rather than filled with inference.
- Primary sources:
  - [Artifact Review and Badging](https://www.acm.org/publications/policies/artifact-review-and-badging-current) — Association for Computing Machinery.

### [Design an Agent Benchmark That Predicts Production](https://www.accessallgpt.com/research/benchmark-design-for-agent-systems)
- Decision desk: Benchmarks
- Evidence status: Evaluation guide
- Published: 2026-08-03; last material update: 2026-08-03
- Summary: A benchmark design for teams evaluating tool-using agents on their own operational tasks.
- Method: Evergreen benchmark-design guidance.
- Limitations: Private suites improve relevance but reduce external comparability; preserve methodology and version every change.
- Primary sources:
  - [SWE-bench](https://www.swebench.com/) — SWE-bench.
  - [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://arxiv.org/abs/2310.06770) — arXiv.

### [The AI Tooling Procurement Scorecard](https://www.accessallgpt.com/research/ai-tooling-procurement-scorecard)
- Decision desk: Buyer guides
- Evidence status: Buyer guide
- Published: 2026-08-03; last material update: 2026-08-03
- Summary: A vendor-neutral scorecard covering evidence, integration, security, economics, operations and exit risk.
- Method: A vendor-neutral procurement framework. Commercial relationships should be disclosed in any applied comparison.
- Limitations: This is an AccessAllGPT procurement framework, not a vendor ranking or legal opinion. Contract, privacy and regulatory review require qualified internal or external counsel.
- Primary sources:
  - [NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence) — National Institute of Standards and Technology. Retrieved: 2026-08-05. Reviewed: Publication abstract, scope, citation and report metadata. Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation..
  - [OWASP Top 10 for Large Language Model Applications](https://genai.owasp.org/llm-top-10/) — OWASP GenAI Security Project.

## Citation and use

Link to the canonical article, preserve the stated evidence boundary, and distinguish vendor claims from independent evidence and AccessAllGPT guidance. Dates in this file are ISO 8601 calendar dates. Time-sensitive product behavior, prices and policies should be rechecked against the linked primary source before use.