Independent AI systems researchOperated by NeuralArc
How to read this index

A citation is not a transferable result.

Reviewed identifies the part of the source used. Supports states the bounded proposition we draw from it. Retrieved records when time-sensitive material was checked.

Vendor documentation establishes vendor claims and product behavior as documented, not independent performance. Papers establish results only for their stated methods and tested scope. AccessAllGPT recommendations remain labeled analysis.

Read the full methodology →
Source ledger

Evidence by decision guide

Ordered by the latest publication or material update date. Follow a source to inspect the original material.

Agents

MCP Output Validation Depends on Tool Discovery

AccessAllGPT reviewed the current MCP 2026-07-28 tools specification and CallToolResult schema, the release record and pinned client/server implementation for @modelcontextprotocol/sdk 1.30.0, Anthropic’s original MCP announcement and OWASP’s independent prompt-injection defensive guidance. I wrote and ran scripts/audit-mcp-output-schema-cache.mjs on Node v24.10.0 and Darwin arm64 with the official SDK 1.30.0, Zod 3.25.76 and linked in-memory transports. A deliberately nonconforming low-level synthetic server advertised an integer score from 0 to 100 but returned {score:"ninety-nine"}. Ten assertions compared a direct call before discovery with the same call after listTools. The pinned client source was bound to SHA-256 db6cc643f5cc095b2fd0b70bf691d1775f07a317e97555e3928559e815f5e32a. No model, production client, remote server, credential, network target or user data was used.

7 sources · 7 retrieval dates · 7 scoped claim notes
  1. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: Tool definition, output schema, structured content, calling tools, error handling and security considerations

    Supports: The current protocol defines outputSchema as an optional contract for structuredContent and says a conforming structured result must match that schema; it also separates tool execution errors from protocol errors.

  2. Schema reference: CallToolResult (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: CallToolResult content, structuredContent, isError and result metadata fields

    Supports: The schema requires structuredContent to be a JSON object when present, but the tool-specific output contract comes from the separately discovered Tool.outputSchema.

  3. @modelcontextprotocol/sdk 1.30.0 client implementation

    Model Context Protocol TypeScript SDK · Retrieved

    Reviewed: Client.callTool, cacheToolMetadata, getToolOutputValidator and listTools implementation

    Supports: The pinned client validates a tool result only when a validator is present in its metadata cache; listTools clears and repopulates that cache from discovered output schemas.

  4. @modelcontextprotocol/sdk 1.30.0 server implementation

    Model Context Protocol TypeScript SDK · Retrieved

    Reviewed: High-level McpServer tool registration, input validation, output validation and callback dispatch

    Supports: The high-level McpServer validates registered tool structuredContent against its configured output schema before returning a successful result; this differs from the deliberately nonconforming low-level fixture used in the client probe.

  5. TypeScript SDK release 1.30.0

    Model Context Protocol on GitHub · Retrieved

    Reviewed: Release identity, tag, publication date and package change record

    Supports: The official release record binds the tested package to SDK version 1.30.0, released July 27, 2026.

  6. LLM Prompt Injection Prevention Cheat Sheet

    OWASP Cheat Sheet Series · Retrieved

    Reviewed: Agent-specific attacks, thought and observation injection, least privilege, tool validation and trust-boundary guidance

    Supports: Independent defensive guidance treats tool output as untrusted content and recommends validating tool calls against user permissions and session context; it does not test this SDK cache behavior.

  7. Introducing the Model Context Protocol

    Anthropic · Retrieved

    Reviewed: November 25, 2024 announcement, architecture, SDKs and ecosystem purpose

    Supports: Anthropic introduced MCP as an open standard for connecting AI assistants to data sources and tools, establishing why discovered tool contracts enter an agent client’s trust boundary.

Agents

MCP’s readOnlyHint Is Not a Permission Boundary

AccessAllGPT reviewed the current MCP 2026-07-28 tools specification and ToolAnnotations schema, the generated types and release record for @modelcontextprotocol/sdk 1.30.0, Anthropic’s original MCP announcement and Invariant Labs’ independent tool-poisoning report. I wrote and ran scripts/audit-mcp-tool-annotations.mjs on Node v24.10.0 and Darwin arm64 using the official SDK 1.30.0, Zod 3.25.76 and its linked in-memory client/server transport. The synthetic tool declared readOnlyHint=true, destructiveHint=false, idempotentHint=true and openWorldHint=false, then wrote one temporary marker on each of two calls. Eleven assertions verified discovery metadata, pre-call state, both responses, both writes and call count. The SDK specification type source was bound to SHA-256 f612706dcbe34de49a77f2458ba4e0ef9e767efffc0f38024e3d84a8f1f66414. No model, production client, external tool, credential, network target or user data was used.

6 sources · 6 retrieval dates · 6 scoped claim notes
  1. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: Tool definition, annotations, user interaction model, calling tools and security considerations

    Supports: The current specification defines annotations as optional descriptions of tool behavior and normatively requires clients to consider them untrusted unless they come from trusted servers.

  2. Schema reference: ToolAnnotations (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: ToolAnnotations fields, defaults and semantic qualifications for read-only, destructive, idempotent and open-world hints

    Supports: The schema defines readOnlyHint, destructiveHint, idempotentHint and openWorldHint as behavioral hints, including defaults and the conditions under which some fields are meaningful.

  3. @modelcontextprotocol/sdk 1.30.0 generated specification types

    Model Context Protocol TypeScript SDK · Retrieved

    Reviewed: ToolAnnotations interface, field comments, trust warning and Tool definition

    Supports: The pinned SDK type declarations say every ToolAnnotations property is a hint, may not faithfully describe behavior, and must not drive tool-use decisions when received from an untrusted server.

  4. TypeScript SDK release 1.30.0

    Model Context Protocol on GitHub · Retrieved

    Reviewed: Release identity, tag, commit, publication date and changed-package record

    Supports: The official release page binds the tested SDK package to version 1.30.0 and tag commit 2d889f2, released July 27, 2026.

  5. MCP Security Notification: Tool Poisoning Attacks

    Invariant Labs · Retrieved

    Reviewed: Threat definition, malicious tool-description example, reported client experiments, limitations and proposed mitigations

    Supports: Independent security researchers reported that malicious instructions in MCP tool descriptions could steer tested agents toward unauthorized reads and actions; this is adjacent threat evidence, not validation of this annotation probe.

  6. Introducing the Model Context Protocol

    Anthropic · Retrieved

    Reviewed: November 25, 2024 announcement, protocol purpose, architecture, SDKs and early ecosystem description

    Supports: Anthropic introduced MCP on November 25, 2024 as an open standard connecting AI assistants to data sources and tools, establishing the chronology and intended AI integration boundary.

Agents

npm ci --ignore-scripts Still Installs Runnable Package Binaries

AccessAllGPT reviewed npm CLI v11 ci and package.json bin documentation, npm 11.6.0 ci source, bundled @npmcli/arborist 9.1.4 source and the independent OpenSSF Package Analysis design. I wrote and ran scripts/audit-npm-ci-ignore-scripts-bin-links.mjs on Node v24.10.0, npm 11.6.0 and Darwin arm64. The audit bound installed source to SHA-256 digests 76cacc0ed9bbee8a7c1e5d5656201403a4dd688d31ee141b531c9e85a7e75e9e and 0915a24508d04986869407f80ef6f793a85abd1a4785e27b4029efcb7f4921dd, packed one synthetic local dependency, created a frozen lockfile, compared three fresh npm ci flows, explicitly invoked only the synthetic binary, and passed 20 assertions over hook markers, package files, symlink targets and invocation output. No registry package executable, external service, exploit, network access or publication was used.

5 sources · 5 retrieval dates · 5 scoped claim notes
  1. npm ci documentation, CLI v11

    npm Docs · Retrieved

    Reviewed: Description, clean-install guarantees, lockfile requirements, ignore-scripts and bin-links configuration

    Supports: npm documents ci as a lockfile-frozen clean install and lists ignore-scripts and bin-links as separate configuration controls; ignore-scripts suppresses package.json scripts while bin-links governs executable links.

  2. package.json bin field documentation, CLI v11

    npm Docs · Retrieved

    Reviewed: bin field mapping, executable files, local installation links and shebang requirement

    Supports: npm documents that local installs create links in node_modules/.bin for package bin entries so commands can be run by npm scripts and other callers.

  3. npm 11.6.0 ci command implementation

    npm CLI source · Retrieved

    Reviewed: Accepted configuration, lockfile validation, node_modules removal, Arborist reify dispatch and root lifecycle guard

    Supports: The pinned command accepts ignore-scripts and bin-links independently, passes flat options to Arborist reify, and uses ignore-scripts to guard root lifecycle events after dependency reification.

  4. Arborist 9.1.4 rebuild implementation bundled with npm 11.6.0

    npm CLI source via unpkg · Retrieved

    Reviewed: Early return, build queues, lifecycle guards, binLinks branch, package metadata inspection and executable linking

    Supports: The implementation queues bins and lifecycle events together but guards them separately: ignoreScripts gates lifecycle execution while binLinks gates #linkAllBins.

  5. OpenSSF Package Analysis README at commit c5c4500

    Open Source Security Foundation · Retrieved

    Reviewed: Project purpose, behavioral signals, architecture, sandboxing and captured process and network data

    Supports: Independent ecosystem infrastructure treats commands, file access and network destinations as behaviors worth observing and dynamically analyzes packages in isolated gVisor containers; it does not test this npm fixture or establish maliciousness.

Agents

npm --ignore-scripts Is Not a Lasting Quarantine

AccessAllGPT reviewed npm CLI v11 rebuild and lifecycle documentation, npm 11.6.0 rebuild source, bundled @npmcli/arborist 9.1.4 source and an independent historical supply-chain paper. I wrote and ran scripts/audit-npm-rebuild-after-ignore-scripts.mjs on Node v24.10.0, npm 11.6.0 and Darwin arm64. The audit bound installed source files to SHA-256 digests 8a4616e70380702f7fd5c76fc9e9fd7f3e10683bdd12fd643a83c65a8d6fbcad and 0915a24508d04986869407f80ef6f793a85abd1a4785e27b4029efcb7f4921dd, packed only a synthetic local dependency, compared three fresh install/rebuild flows, and passed 14 assertions over hook traces, lockfile metadata and source control flow. No registry dependency script, external service, exploit, network exfiltration or publication was used.

5 sources · 5 retrieval dates · 5 scoped claim notes
  1. npm rebuild documentation, CLI v11

    npm Docs · Retrieved

    Reviewed: Description, synopsis, lifecycle behavior, package selection and ignore-scripts configuration

    Supports: npm documents rebuild as running matching lifecycle scripts for installed packages and exposes ignore-scripts as a command configuration, not as persisted package approval state.

  2. npm scripts documentation, CLI v11

    npm Docs · Retrieved

    Reviewed: Life cycle scripts, dependencies, working directory and npm rebuild operation order

    Supports: npm documents preinstall, install and postinstall as the lifecycle order for npm rebuild and explains that scripts run from the package root.

  3. npm 11.6.0 rebuild command implementation

    npm CLI source · Retrieved

    Reviewed: Accepted configuration, Arborist option construction, package-spec matching and rebuild dispatch

    Supports: The pinned command accepts ignore-scripts, passes current flat options into Arborist and invokes rebuild over the selected installed nodes; it does not read a previous install command or persisted approval record.

  4. Arborist 9.1.4 rebuild implementation bundled with npm 11.6.0

    npm CLI source via unpkg · Retrieved

    Reviewed: Default-node loading, build queues, ignoreScripts guards, package metadata refresh and lifecycle execution

    Supports: The implementation discovers scripts from installed package metadata, queues preinstall/install/postinstall, and suppresses those queues only when the current rebuild options set ignoreScripts.

  5. Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks

    DIMVA preprint on arXiv · Retrieved

    Reviewed: Abstract, historical malicious-package dataset, attack-tree scope and execution-stage framing

    Supports: Independent researchers documented historical malicious packages and code-execution opportunities across package ecosystems; this is risk context, not validation of npm 11.6.0 or the synthetic experiment.

Agents

npm pack --dry-run Still Executes Package Scripts

AccessAllGPT reviewed npm CLI v11 pack and scripts documentation, npm 11.6.0 pack source, bundled libnpmpack 9.0.7 source and an independent historical supply-chain paper. I wrote and ran scripts/audit-npm-pack-dry-run.mjs on Node v24.10.0, npm 11.6.0, Darwin arm64. The audit bound installed source with SHA-256 digests 7580758d2fd46c6d3998f72ad623e93e91be3f8ce1893c9c914ec4b147874de5 and 665eeb6479829e1c21699e55476edf7b020da3aac12e73d8bcf2a69c0a33951a, created only a synthetic local package, ran normal, dry-run and ignore-scripts pack modes, and passed 13 assertions over hook order, filesystem writes and reported file lists. No registry package lifecycle script, publish operation, network exfiltration or exploit was attempted.

5 sources · 5 retrieval dates · 5 scoped claim notes
  1. npm pack documentation, CLI v11

    npm Docs · Retrieved

    Reviewed: Synopsis, dry-run, ignore-scripts and description sections for npm pack

    Supports: npm documents dry-run as reporting what pack would do without making changes, and documents ignore-scripts as preventing package.json scripts from running.

  2. npm scripts documentation, CLI v11

    npm Docs · Retrieved

    Reviewed: Life cycle operation order plus prepare, prepack and postpack definitions

    Supports: npm documents prepack and prepare before tarball generation and postpack after generation; prepare has run during npm pack since npm 4.

  3. npm 11.6.0 pack command implementation

    npm CLI source · Retrieved

    Reviewed: Pack command parameters, manifest resolution, libnpmpack invocation and output path

    Supports: The command forwards flat options, including dry-run and ignore-scripts, into libnpmpack and then reports the generated tarball metadata.

  4. libnpmpack 9.0.7 pack implementation bundled with npm 11.6.0

    npm CLI source · Retrieved

    Reviewed: Directory-script guards, tarball creation, dry-run write guard and postpack execution order

    Supports: The pinned implementation gates prepack and postpack on ignoreScripts, delegates tarball creation between them, and uses dryRun only to decide whether the completed tarball buffer is written to disk.

  5. Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks

    DIMVA preprint on arXiv · Retrieved

    Reviewed: Abstract, historical package-registry attack corpus, supply-chain stages and stated scope

    Supports: Independent researchers documented historical malicious packages and execution opportunities in package ecosystems; this supplies bounded risk context, not evidence about npm pack 11.6.0 or the synthetic fixture.

Agents

A pnpm Build Approval Can Outlive the Version You Reviewed

AccessAllGPT reviewed pnpm 10.0.0, 10.1.0, 11.23.0 and 11.24.0 release records; the current approve-builds documentation; pnpm 11.24.0 source at commit cef4816dfbc9aa7ffbe67fa727c1eb9be5d5e1e7; and an independent historical supply-chain paper. The registry reported pnpm 11.24.0 with integrity sha512-vSfjRel23LC+C3oSKCF7BJqBfiGx81XJDb59xGZxiVqLwebQbCRVRQXqk+oLRfSJon7Bv7yN5qlln8oPFvoAAA==. I wrote and ran scripts/audit-pnpm-allow-builds-policy.mjs on Node v24.10.0, Darwin arm64. It fetched two commit-pinned source files, verified SHA-256 digests b0fcc6a8364c02f384115869030b51dc4643c2a7649eb2b869a05b760589079d and b4a931881f1b5c28689c47dad109e4c4e84e788c8eb63802b63cf58f196bb0bf, asserted the relevant matching branches, and exercised a six-row policy matrix with 12 passing assertions. The matrix is a source-level reproduction of the matching algorithm, not an end-to-end package installation or security exploit.

7 sources · 7 retrieval dates · 7 scoped claim notes
  1. pnpm v10.0.0 release notes

    pnpm project · Retrieved

    Reviewed: Major change that stopped dependency lifecycle scripts by default, onlyBuiltDependencies migration example and security motivation

    Supports: pnpm 10.0.0 was released on 2025-01-07 and changed dependency lifecycle scripts to blocked by default, with named-package exceptions through onlyBuiltDependencies at that time.

  2. pnpm v10.1.0 release notes

    pnpm project · Retrieved

    Reviewed: Release timestamp and additions of ignored-builds, approve-builds and ignoredBuiltDependencies

    Supports: pnpm 10.1.0 was released on 2025-01-26 and introduced commands for listing ignored builds and interactively approving dependencies whose install scripts may run.

  3. pnpm approve-builds documentation (versions 11 and 12)

    pnpm project · Retrieved

    Reviewed: Command purpose, positional approval and denial syntax, allowBuilds writes, pending placeholders and pnpm 11 migration behavior

    Supports: The command records approved package names as true and denied names as false in allowBuilds; pnpm 11 replaced and ignores several pnpm 10-era build-policy settings.

  4. pnpm v11.23.0 release notes

    pnpm project · Retrieved

    Reviewed: Release date and allowBuilds migration cleanup described for approve-builds

    Supports: pnpm 11.23.0 was released on 2026-08-23 and made approve-builds remove obsolete pnpm 10 build-policy keys when writing allowBuilds.

  5. pnpm v11.24.0 release notes

    pnpm project · Retrieved

    Reviewed: Release date, global build approvals and package-manager release scope

    Supports: pnpm 11.24.0 was released on 2026-08-24 and restored global build approvals; the registry also identified 11.24.0 as latest during this investigation.

  6. pnpm 11.24.0 build-policy and version-policy implementation

    pnpm project source · Retrieved

    Reviewed: createAllowBuildFunction precedence, package-name and name-at-version matching, identity trust, ignored-build key generation and exact-version expansion dependency

    Supports: The pinned implementation checks exact denials before approvals, then accepts either a bare package name or an exact name@version; ignored registry builds are proposed under the bare package name.

  7. Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks

    DIMVA preprint on arXiv · Retrieved

    Reviewed: Abstract, dataset scope, supply-chain stages and limitations of the historical collection

    Supports: The independent authors report a manually collected historical dataset of malicious npm, PyPI and RubyGems packages and distinguish code execution at multiple supply-chain stages; it provides context, not a pnpm 11 evaluation.

Agents

npm 12 Blocks Dependency Scripts—But a Git Dependency Still Ran prepare

AccessAllGPT reviewed npm 12.0.0 release notes, npm 12.0.2 documentation, the pacote GitFetcher source bundled in the npm 12.0.2 registry tarball, open npm issue #9783 and the independent DIMVA supply-chain preprint. I built a synthetic local Git repository pinned to commits d886768c94acb3f9fa3104645d2e294e22bb31af and d40165addbb4c8ab3df8a6cf35981657786f9d60. Its prepare hook wrote built.txt and appended its temporary working directory, process version and executable to an external trace. Baseline runs used Node v24.10.0 with npm 11.6.0. npm 12 runs used npm 12.0.2 with its supported Node v24.15.0 binary through npx. I tested default npm 12 policy, --allow-git=all without allowScripts, and npm 11 --ignore-scripts. No production package, remote repository, credential, exploit payload or private system was used. The npm 12 tarball integrity returned by the registry was sha512-uIXokLlBj6FpNUTQX1PmT5pz7BlIN9QlixX+zdaSNHsd0qUXsbDLr50xzY6Sw7cJVr0uzHKDOle0swmPW/p5Qw==.

6 sources · 6 retrieval dates · 6 scoped claim notes
  1. npm CLI v12.0.0 release notes

    npm CLI project · Retrieved

    Reviewed: Release date, breaking changes for allow-git, allow-remote, supported Node versions and dependency lifecycle scripts

    Supports: npm 12.0.0 was released on 2026-07-08 and changed git and remote dependency fetching plus dependency-script policy: git fetches require an explicit allowance and dependency scripts are blocked unless covered by the root allowScripts policy.

  2. package.json: Git URLs as Dependencies (npm CLI 12.0.2)

    npm documentation · Retrieved

    Reviewed: Git URL grammar, commit-ish resolution and the documented temporary clone, dependency install, script, pack and install flow

    Supports: npm documents that selected scripts or workspaces make a git dependency a build input: npm clones it into a temporary directory, installs its dependencies, runs relevant scripts, then packs and installs the result.

  3. Config: allow-scripts and ignore-scripts (npm CLI 12.0.2)

    npm documentation · Retrieved

    Reviewed: allow-scripts matching and scope, project allowScripts policy, override behavior and ignore-scripts semantics

    Supports: npm describes allow-scripts as covering dependency install-time hooks including prepare for non-registry dependencies, while ignore-scripts suppresses package.json scripts except explicitly invoked script commands.

  4. pacote GitFetcher implementation bundled with npm 12.0.2

    npm CLI project · Retrieved

    Reviewed: GitFetcher constructor, #prepareDir, temporary clone and tarballFromResolved implementation paths

    Supports: The bundled GitFetcher checks ignoreScripts before spawning npm inside the temporary clone, but the inspected #prepareDir path does not evaluate the root allowScripts policy before that preparation install and pack flow.

  5. Issue #9783: allow-scripts forwarded to git-dependency preparation

    npm CLI issue tracker · Retrieved

    Reviewed: Issue title, reproduction context, open state, creation date and latest activity visible through the GitHub API

    Supports: An independently filed open issue documents a neighboring npm 12 git-preparation and allow-scripts interaction. It is corroborating community evidence, not proof of the exact local result reported here.

  6. Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attacks

    DIMVA preprint on arXiv · Retrieved

    Reviewed: Abstract, dataset scope, submission history and stated supply-chain attack model

    Supports: The authors report a manually collected dataset of 174 malicious packages from npm, PyPI and RubyGems dated 2015–2019 and model code execution at different supply-chain stages; this supplies historical independent context, not an npm 12 measurement.

Agents

npm Install Is an Execution Boundary

AccessAllGPT source review of the current npm scripts documentation, Node.js permission documentation, the Node.js threat model and NIST AI 600-1, plus a bounded local reproduction on Node v24.10.0 and npm 11.6.0 on macOS 26.0.1. In a temporary sandbox, I created a file dependency with preinstall, install and postinstall hooks that append to a trace file, ran npm install, confirmed the three hook lines were written, then reran the install with --ignore-scripts and confirmed the trace stayed empty. I did not audit a production package, publish a malicious package, analyze a real exploit chain or find a third-party write-up reproducing this exact local probe.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. Scripts | npm Docs

    npm · Retrieved

    Reviewed: Description, Life Cycle Operation Order, Exiting and Working Directory for Scripts

    Supports: npm says lifecycle scripts run after package changes are applied, lists preinstall, install and postinstall in order, and documents that scripts are executed through /bin/sh on POSIX systems or cmd.exe on Windows.

  2. Permissions — Node.js documentation

    Node.js project · Retrieved

    Reviewed: Permissions overview, child-process and filesystem controls, known limitations and security notes

    Supports: Node’s permission model narrows selected runtime capabilities but does not claim to sandbox malicious code. That makes shell execution and package hooks a separate authority question, not a Node-only flag choice.

  3. Security policy and threat model

    Node.js project · Retrieved

    Reviewed: The Node.js threat model section and its statement about trusted versus untrusted code

    Supports: The Node project’s threat model says the runtime trusts the code it is asked to run and that dependencies inherit the execution user’s privileges. That is the right backdrop for treating npm hooks as code execution, not as inert metadata.

  4. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST frames AI trustworthiness as a design, development, use and evaluation problem. That general control framing supports least-privilege package installation, evidence logging and explicit rollback for agentic systems.

Agents

MCP Token Passthrough Needs an Audience Boundary

AccessAllGPT desk review of the current MCP security best practices, MCP authorization and tools specifications, RFC 8707, RFC 9728 and RFC 9700, plus a bounded local loopback reproduction on Node v24.10.0 that relays a synthetic Authorization header from one local HTTP server to another. We verified the public pages were reachable on 2026-08-22, but did not audit a production MCP implementation, a cloud auth server or any third-party deployment. The local probe demonstrates authority propagation through a relay, not a defect in MCP itself.

6 sources · 6 retrieval dates · 6 scoped claim notes
  1. Security Best Practices (version 2026-07-28)

    Model Context Protocol · Retrieved

    Reviewed: Token passthrough risks, local MCP server compromise, SSRF risks and common mistakes

    Supports: The MCP security guidance explicitly frames token passthrough as risky, warns about local server compromise, and treats SSRF and token handling as security concerns for MCP deployments.

  2. Authorization (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: Resource parameter implementation, canonical server URI, access token usage and token handling

    Supports: The authorization spec requires servers to validate that access tokens were issued specifically for them as the intended audience and describes the resource-oriented discovery flow that anchors the authorization relationship.

  3. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: Security considerations and user interaction model for tool exposure

    Supports: The tools specification emphasizes that tool exposure and invocation are security-sensitive, which matters when a server can forward user credentials to another service or trigger privileged actions on a user’s behalf.

  4. Resource Indicators for OAuth 2.0 (RFC 8707)

    RFC Editor · Retrieved

    Reviewed: Resource parameter, audience restriction and threat model

    Supports: RFC 8707 defines resource indicators so the client can ask for a token for a specific protected resource instead of receiving a bearer token that is broadly reusable across a service fleet.

  5. OAuth 2.0 Protected Resource Metadata (RFC 9728)

    RFC Editor · Retrieved

    Reviewed: Protected resource metadata and security considerations

    Supports: RFC 9728 standardizes resource metadata discovery so clients can learn which authorization server and token endpoints belong to a protected resource instead of inventing token-routing rules ad hoc.

  6. Best Current Practice for OAuth 2.0 Security (RFC 9700)

    RFC Editor · Retrieved

    Reviewed: Audience-restricted access tokens, token leakage and metadata attacks

    Supports: RFC 9700 elevates audience-restricted tokens and related countermeasures as best current practice because bearer tokens are dangerous when they can be replayed or forwarded beyond their intended recipient.

Agents

Node’s Permission Model Is a Seat Belt, Not an AI Code Sandbox

AccessAllGPT source review and bounded local experiment, first run 2026-08-16 and refreshed 2026-08-21. We cloned Node.js tag v24.10.0 at immutable commit 9b72b88f4c4565687e3a8c4d8e1232f63a501e15, inspected its permission documentation, security threat model, environment initialization and 44 permission-named tests, and checked the official release index. We then downloaded Node.js v26.7.0 for Darwin arm64, verified its SHA256 against SHASUMS256.txt, reviewed the release note and permissions docs, and reran the local probe under both --permission-audit and --permission. The script creates only temporary fixture files, starts a loopback-only Python HTTP server, runs asserted scenarios in child Node processes, kills the server and removes the fixture. It tests direct filesystem reads and writes, relative-symlink traversal, loopback fetch, default child-process denial, an explicitly allowed non-Node child, an inherited file descriptor and the new audit-mode diagnostics path. We did not execute model-generated code, test a container or VM, probe Linux or Windows, attempt undisclosed bypasses, benchmark overhead, or perform a production security assessment. Node.js project statements, source observations, local results, independent NIST context and AccessAllGPT guidance are labeled separately.

10 sources · 10 retrieval dates · 10 scoped claim notes
  1. Node.js v26.7.0 release note

    Node.js project · Retrieved

    Reviewed: Release headline and changelog bullet documenting --permission-audit audit mode behavior

    Supports: The v26.7.0 release note records the documentation change that introduced audit-mode behavior for the permission model.

  2. Permissions — Node.js v26.7.0 documentation

    Node.js project · Retrieved

    Reviewed: Operational modes, runtime API, audit-mode diagnostics-channel mapping and known issues

    Supports: The v26.7.0 docs define enforce mode and audit mode, explain that audit mode logs violations without denying access, enumerate the diagnostics-channel names, and retain the symlink and existing-descriptor limitations.

  3. SHASUMS256.txt for Node.js v26.7.0

    Node.js project · Retrieved

    Reviewed: node-v26.7.0-darwin-arm64.tar.gz checksum row

    Supports: The downloaded Darwin arm64 binary was verified against the official SHA256 before local execution.

  4. Scripts | npm Docs

    npm · Retrieved

    Reviewed: How npm handles the scripts field; shell execution behavior on POSIX and Windows

    Supports: npm documents that scripts run through /bin/sh on POSIX systems and cmd.exe on Windows, which makes shell policy part of the agent boundary whenever an agent is allowed to invoke package scripts.

  5. Permissions — Node.js v24.10.0 documentation

    Node.js project · Retrieved

    Reviewed: Scope and threat statement; stable status; restricted resources; filesystem allowlists; runtime API; npx behavior; constraints; symlink and inherited-file-descriptor limitations

    Supports: The pinned documentation calls the permission model a seat belt for trusted code, explicitly says it does not protect against malicious code, lists the resources it restricts, says node:sqlite can access files outside node:fs checks, and documents relative-symlink and inherited-file-descriptor bypass paths.

  6. Node.js security policy and threat model at v24.10.0

    Node.js project · Retrieved

    Reviewed: Responsible testing; trusted and untrusted elements; treatment of code Node.js is asked to run; inherited execution-user privileges

    Supports: The project threat model trusts JavaScript, WASM, native code and dependencies that Node.js is asked to run and says that code inherits the privileges of the execution user. This is incompatible with treating the runtime alone as a hostile-code security boundary.

  7. src,lib: stabilize permission model — pull request 56201

    Node.js project on GitHub · Retrieved

    Reviewed: Title; merge date; release lines; linked documentation history

    Supports: The permission model moved to stable through a change merged on December 12, 2024 and documented for Node.js 22.13.0 and 23.5.0. Stable API status describes support maturity; it does not revise the project’s malicious-code exclusion.

  8. Node.js v24.10.0 source tree at commit 9b72b88

    Node.js project on GitHub · Retrieved

    Reviewed: Permission initialization in src/env.cc; permission implementation; CLI documentation; permission tests for filesystem, symlinks, children, workers, addons, WASI, inspector, sqlite and file descriptors

    Supports: AccessAllGPT cloned the immutable v24.10.0 source commit. Environment initialization denies addons, inspector, child processes, workers and WASI unless explicitly allowed, then applies filesystem read and write allowlists. The test inventory exercises those controls and known symlink behavior; it does not implement a network permission boundary.

  9. Node.js release index

    Node.js project · Retrieved

    Reviewed: v24.10.0 release metadata and current release rows at retrieval

    Supports: The official index dates v24.10.0 to October 8, 2025. AccessAllGPT used that exact installed runtime; this is a version-pinned result rather than a claim about every supported or future Node.js release.

  10. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication scope; generative-AI risk-management framing; risk mapping, measurement and management context

    Supports: NIST presents the profile as voluntary, cross-sector guidance for incorporating trustworthiness considerations into design, development, use and evaluation. It supplies independent governance context for layered, measured controls; it does not evaluate Node.js or certify a sandbox architecture.

Agents

Before You Connect an MCP Server to Production

AccessAllGPT desk review of the version-pinned MCP 2026-07-28 specification, MCP security guidance, the MCP TypeScript SDK 1.30.0 README and underlying OAuth RFCs, reverified 2026-08-20. Normative MUST and advisory SHOULD language is separated from AccessAllGPT recommendations. This is implementation guidance plus a local stdio probe, not an original security test or vendor comparison.

6 sources · 6 retrieval dates · 6 scoped claim notes
  1. Security Best Practices (version 2026-07-28)

    Model Context Protocol · Retrieved

    Reviewed: Token passthrough, SSRF and local MCP server compromise

    Supports: Token passthrough is an anti-pattern; discovery can create SSRF exposure; local servers execute with local process authority.

  2. Authorization (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: Resource parameter implementation, access-token usage and token handling

    Supports: Normative resource-indicator, bearer-token transport, token audience validation and token rejection requirements.

  3. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: User interaction model, capabilities and security considerations

    Supports: Normative server controls, advisory human-approval controls, authorization-dependent tool lists and untrusted annotations.

  4. MCP TypeScript SDK README (version 1.30.0)

    @modelcontextprotocol/sdk · Retrieved

    Reviewed: Overview, quick start, servers and transports, clients, and local stdio transport description

    Supports: The SDK documents stdio as a local, process-spawned transport and says clients can connect to servers and call tools.

  5. Best Current Practice for OAuth 2.0 Security

    RFC Editor (RFC 9700) · Retrieved

    Reviewed: Redirect URI validation and audience-restricted access tokens

    Supports: OAuth security baseline used to cross-check exact redirect matching and audience restriction.

  6. OAuth 2.0 Protected Resource Metadata

    RFC Editor (RFC 9728) · Retrieved

    Reviewed: Protected resource metadata request, response and security considerations

    Supports: The discovery metadata format and security boundary underlying MCP protected-resource discovery.

Models

GLM-5.3: Trial the Coding Gains, Contain the Cyber Capability

AccessAllGPT desk review of Z.ai’s GLM-5.3 launch, Coding Plan and API documentation; the public GLM-5.2 predecessor repository pinned at revision b4734de; Terminal-Bench 3.0 pinned at tag v3.0.0 and commit 2b0442c; CyberGym v3 and its harness pinned at commit 7656b71; the ExploitBench harness pinned at commit 9d0173b; and dated official-leaderboard and Hacker News snapshots, retrieved 2026-08-15. We wrote and ran reproducible Node.js audits against the pinned GLM-5.2 Hugging Face metadata and Terminal-Bench checkout. The GLM audit paginated all 295 files, reconciled 282 weight shards between the tree and index, and reported architecture and byte counts without downloading the weights. The Terminal-Bench audit reconciled all 74 task directories with 74 unique SHA-256 manifest digests and summarized categories, GPU use, agent timeouts and expert estimates. We also inspected both pinned cyber harnesses. For CyberGym, we installed its documented development and server dependencies under Python 3.13, compiled the source, and ran targeted smoke checks for checksum binding, image mapping, timeout normalization and rate limiting; we could not run a target because the Docker daemon was unavailable and the benchmark assets were not downloaded. For ExploitBench, we inspected its canonical matrix, scoring and image-resolution code and ran the non-slow unit and golden test suite under Python 3.11. The first run omitted optional publishing dependencies and failed four publishing tests; after installing the documented publish extras, 643 tests passed, six were skipped and six were deselected. We did not run an agent on Terminal-Bench, execute either cyber benchmark or call GLM-5.3. Momentum evidence, vendor claims, predecessor-artifact measurements, independent methodology artifacts, local harness evidence and AccessAllGPT guidance are labeled separately. This is not an original model run, benchmark reproduction, repository trial, red-team exercise, GLM-5.3 weight inspection, security assessment, GLM-5.3 license review, cost study or legal analysis.

9 sources · 9 retrieval dates · 9 scoped claim notes
  1. GLM-5.3: Frontier Coding with Emergent Cyber Capabilities

    Z.ai · Retrieved

    Reviewed: Release date and status; post-training claim; coding, agentic and cyber benchmark tables; weight-release timing; API changes; Coding Plan availability; evaluation footnotes

    Supports: Z.ai announced GLM-5.3 on August 14, 2026, says it uses the same base model as GLM-5.2 with gains from additional post-training, reports vendor-run coding and cyber evaluations, documents mandatory thinking and reasoning-effort settings, and says weights are planned two weeks after launch following safety evaluation and hardening. These are vendor claims and release statements, not independent AccessAllGPT measurements.

  2. GLM Coding Plan overview

    Z.ai Developer Documentation · Retrieved

    Reviewed: Coding Plan scope; supported coding-agent integrations; subscription and quota framing; API compatibility guidance

    Supports: Z.ai documents its Coding Plan as a route for using GLM models in supported coding-agent clients. This establishes an available managed access path and integration surface; it does not establish GLM-5.3 quality, security, quota sufficiency or compatibility for a particular repository.

  3. Chat completion API

    Z.ai Developer Documentation · Retrieved

    Reviewed: Request and response contract; model selection; messages; thinking configuration; tools; streaming; errors and usage fields

    Supports: Z.ai publishes the request and response surface used to call supported chat models, including model selection, messages, tools, streaming and usage. The documentation establishes an integration contract; actual GLM-5.3 entitlement, behavior, latency, limits and billing must be verified on the adopting account.

  4. zai-org/GLM-5.2 artifact repository at revision b4734de

    Z.ai on Hugging Face · Retrieved

    Reviewed: Pinned repository tree and pagination; model card; MIT license text; config.json architecture fields; safetensors index metadata and weight map; serving-framework versions; file sizes and immutable object identifiers

    Supports: AccessAllGPT queried the Hugging Face API and small metadata files at immutable GLM-5.2 revision b4734de. The predecessor has 282 bfloat16 safetensor shards with 1,506,659,919,872 indexed parameter bytes, a 78-layer GLM MoE DSA configuration, 256 routed experts with eight selected per token, and a 1,048,576-position configured maximum. Its pinned license file is MIT. These measurements establish the scale and metadata of GLM-5.2 only. They do not establish GLM-5.3 file size, license, serving compatibility, quantization, hardware needs or release status.

  5. Terminal-Bench 3.0 release at tag v3.0.0 and official leaderboard

    Harbor / Terminal-Bench maintainers · Retrieved

    Reviewed: Tagged release and immutable commit; dataset manifest and content digests; task metadata and category distribution; agent timeouts and expert estimates; official leaderboard model-agent pairs, resolution rates, uncertainty, token totals and costs

    Supports: AccessAllGPT cloned and audited immutable Terminal-Bench 3.0 tag v3.0.0 at commit 2b0442c. Its manifest binds 74 named tasks to 74 unique SHA-256 content digests across seven categories. Task metadata spans 30- to 480-minute agent limits, with a 120-minute median, and four GPU tasks. The official leaderboard reports model-agent configurations rather than model-only scores and, in the August 15 snapshot, listed GLM 5.2 with Claude Code at 4.6% ± 1.0% but did not list GLM-5.3. These facts explain the public benchmark contract and baseline; they do not validate Z.ai’s 28.3 result or identify its agent, repetitions, task failures, tokens or cost.

  6. CyberGym: Evaluating AI Agents’ Real-World Cybersecurity Capabilities at Scale (v3)

    CyberGym authors on arXiv · Retrieved

    Reviewed: Version history; abstract; benchmark scale; task construction; proof-of-concept objective; reported baseline difficulty; limitations implied by the evaluation target

    Supports: The independent benchmark paper defines CyberGym as 1,507 historical vulnerabilities across 188 projects and primarily asks an agent to generate a proof-of-concept test from a vulnerability description and codebase. This explains what the benchmark targets; it does not validate Z.ai’s GLM-5.3 score or establish safe autonomous exploitation.

  7. CyberGym repository at commit 7656b71

    CyberGym authors on GitHub · Retrieved

    Reviewed: README data and deployment requirements; FAQ network, task, environment and scoring guidance; August 4 submission schema; server submission and verification paths; task checksum; timeout handling; rate limiting; package metadata; local lint, compilation and helper smoke checks

    Supports: At pinned commit 7656b71, AccessAllGPT inspected the current public CyberGym harness. The maintainers now request one designated final submission because any-of scoring increasingly rewards brute force, require token, time, request and artifact reporting, and warn that the partly unauthenticated PoC endpoint must remain private. The README estimates roughly 240 GB for benchmark data and roughly 10 TB for the full compilation environment. Local compilation and targeted helper smoke checks passed; the repository-wide Ruff check found one import-order error. Docker was installed but its daemon was unavailable, so no target container or PoC was run. This audit does not identify or validate Z.ai’s task set, metric, environment, network policy, trajectories or reported score.

  8. ExploitBench repository at commit 9d0173b

    ExploitBench · Retrieved

    Reviewed: README scope; canonical v8.yaml matrix; capability extraction and scoring code; image-reference resolution; model dispatch; seed and budget configuration; audit and reproduction workflow; local non-slow unit and golden tests

    Supports: At pinned commit 9d0173b, AccessAllGPT inspected the public harness and ran its non-slow unit and golden test suite locally. The canonical configuration enumerates 41 V8 environments, six models and five seeds; the scorer cumulatively ORs 16 capability flags and treats arbitrary code execution as a terminal full score. The checked-in matrix uses mutable GHCR tags, while runner code resolves the locally loaded image to an immutable image ID and the documentation recommends registry digests for publication-grade pinning. This audit explains evaluation mechanics but does not identify or validate the revision, matrix, prompts, seeds, containers, transcripts or scoring transformation behind Z.ai’s reported GLM-5.3 result.

  9. Hacker News discussion: GLM-5.3: Frontier coding with emergent cyber capabilities

    Hacker News · Retrieved

    Reviewed: Submission timestamp; linked official release; observable points and comment count in the Algolia API snapshot; discussion scope

    Supports: The August 14 submission linked the official release and showed 1,007 points and 496 comments in the snapshot retrieved August 15. This is a strong independent attention signal only; mutable community activity does not validate model capability, safety, availability, license or production fitness.

Models

GPT-5.6 Sol Ultrafast: Buy Speed Only Where Latency Changes the Outcome

AccessAllGPT desk review of OpenAI launch, model, Fast-mode and pricing documentation; Cerebras launch and benchmark disclosures; and a dated Hacker News momentum snapshot, retrieved 2026-08-15. Momentum evidence, vendor specifications, vendor-performed benchmarks and AccessAllGPT guidance are labeled separately. This is not an original API trial, latency benchmark, quality evaluation, availability test, cost study, security review or legal analysis.

6 sources · 6 retrieval dates · 6 scoped claim notes
  1. Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speed

    OpenAI · Retrieved

    Reviewed: Launch date; limited-preview status; access scope; performance framing; intended workloads; availability caveats

    Supports: OpenAI announced Ultrafast mode on August 13, 2026 as a limited preview for selected API customers and describes it as a Cerebras-powered serving option for GPT-5.6 Sol. The page supplies vendor launch and performance claims; it does not establish results for an untested customer workload.

  2. Fast mode

    OpenAI API Documentation · Retrieved

    Reviewed: Overview; configuration; response reporting; rate limits and ramp rate; usage considerations; regions; billing; data controls

    Supports: OpenAI documents Fast mode as a generally configurable service tier distinct from the new limited-preview Ultrafast offer. It documents up to 2.5x speed for GPT-5.6 Sol versus Standard, shared rate limits, possible downgrade to Standard when traffic ramps too quickly, response-tier reporting, eligibility limits and compatibility statements.

  3. GPT-5.6 Sol model

    OpenAI API Documentation · Retrieved

    Reviewed: Model identity; reasoning levels; context and output limits; knowledge cutoff; pricing; endpoints; supported features and tools; snapshots

    Supports: OpenAI identifies gpt-5.6-sol as its frontier model for complex professional work, documents text and image input, text output, reasoning-effort choices, a 1.05M-token context window, a 128K-token maximum output, supported tools and endpoint exclusions. These are vendor specifications, not independent capability measurements.

  4. Pricing

    OpenAI API Documentation · Retrieved

    Reviewed: GPT-5.6 Sol Standard, Batch, Flex and Fast short-context and long-context token prices; regional-processing note; service-tier naming

    Supports: The retrieved table lists GPT-5.6 Sol short-context Standard prices of $5 per million input tokens and $30 per million output tokens, and Fast prices of $10 and $60 respectively. Long-context and cache-write rules differ. The page does not yet publish a separate Ultrafast price in the reviewed table.

  5. Accelerating GPT-5.6 Sol Ultrafast

    Cerebras · Retrieved

    Reviewed: Launch scope; claimed output speed; HLE and GDP-Val methodology notes; customer use cases; architecture; limited-preview caveat; benchmark disclaimer

    Supports: Cerebras says it powers the limited preview and reports up to 750 output tokens per second plus internal HLE and GDP-Val comparisons. Cerebras performed and published those tests, warns observed improvements vary by workload and configuration, and is commercially involved in the service; the results are vendor evidence, not independent validation.

  6. Hacker News discussion: Accelerating GPT-5.6 Sol Ultrafast

    Hacker News · Retrieved

    Reviewed: Submission timestamp; linked release; observable points and comment count at retrieval; discussion scope

    Supports: The August 13 submission was an observable developer-community momentum signal, showing 694 points and 270 comments in the Hacker News API snapshot retrieved August 15. Popularity is not factual or technical validation, and counts can change after retrieval.

Models

LLM Model Deprecation: Migrate Without Changing Production by Accident

AccessAllGPT desk review of current model-lifecycle documentation from OpenAI, Anthropic, Google Cloud and Amazon Bedrock, retrieved 2026-08-12. Vendor schedules and lifecycle terms are reported as vendor statements; the migration gates are AccessAllGPT guidance. This is not an original model test, compatibility study, performance benchmark, cost measurement, security assessment or legal review.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. Deprecations

    OpenAI Developer Documentation · Retrieved

    Reviewed: Overview; model deprecation notice periods; notification and replacement framing; API deprecation entries

    Supports: OpenAI states that software using its models may need updates, documents recommended replacements, and publishes minimum notice periods of at least six months for generally available models and at least three months for specialized variants unless safety or compliance concerns require a faster timeline. Preview models may receive much shorter notice. These are vendor lifecycle statements, not a compatibility guarantee.

  2. Model deprecations

    Anthropic Claude Platform Documentation · Retrieved

    Reviewed: Overview; lifecycle terminology; platform scope; migrating to replacements; notifications; auditing model usage

    Supports: Anthropic distinguishes active, legacy, deprecated and retired models; says requests to retired models fail; recommends testing applications before retirement; and states that publicly released models receive at least 60 days notice. It also says partner-operated Amazon Bedrock and Google Cloud schedules can differ from Anthropic-operated platforms.

  3. Model versions and lifecycle

    Google Cloud Documentation · Retrieved

    Reviewed: Lifecycle overview; availability-period categories; retirement dates; migration framing; lifecycle definitions and date policy

    Supports: Google Cloud publishes model lifecycle categories, retirement tables and migration guidance for the documented Gemini Enterprise Agent Platform. It says listed retirement dates may be extended but will not be brought forward, and distinguishes models available for at least 12 months from short-term models that retire 45 days after a replacement release. These terms apply to the documented Google Cloud service, not every Gemini surface.

  4. Model lifecycle

    Amazon Bedrock User Guide · Retrieved

    Reviewed: Lifecycle overview; Active, Legacy and End-of-Life states; minimum availability statement; provider-date warning; migration implications

    Supports: Amazon Bedrock documents Active, Legacy and End-of-Life states, exposes lifecycle state through model APIs, and says a model remains on Bedrock for at least 12 months before its End-of-Life date. AWS warns that Bedrock dates can differ from model-provider dates and that Bedrock-specific dates govern Bedrock usage.

Buyer guides

AI API Data Retention and Residency: Set the Procurement Gates

AccessAllGPT desk review of current OpenAI API data-control documentation, Amazon Bedrock data-protection documentation, the official GDPR text and the NIST Privacy Framework, retrieved 2026-08-11. Vendor statements, legal text and AccessAllGPT procurement guidance are labeled separately. This is not an original platform test, deletion audit, contract review, privacy impact assessment, security assessment, legal analysis or vendor comparison.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. Data controls in the OpenAI platform

    OpenAI Developer Documentation · Retrieved

    Reviewed: Data use; types of API data; abuse-monitoring retention; Zero Data Retention and Modified Abuse Monitoring eligibility and limitations; endpoint-level application-state retention; regional storage and processing controls; Enterprise Key Management

    Supports: OpenAI states that API data is not used to train or improve its models unless a customer explicitly opts in, distinguishes abuse-monitoring logs from application state, documents default abuse-monitoring retention of up to 30 days, and describes feature-, project-, region- and eligibility-dependent controls. These are current vendor statements about the documented platform, not independent verification or a promise for every account and feature.

  2. Data protection — Amazon Bedrock

    Amazon Web Services Documentation · Retrieved

    Reviewed: Shared-responsibility boundary; IAM, encryption and activity-logging recommendations; warning about sensitive data in tags and free-form name fields; model deployment accounts; linked encryption, PrivateLink and retention topics

    Supports: AWS assigns customers responsibility for content controls and service configuration, warns that tags and free-form naming fields can enter billing or diagnostic logs, and states that model providers cannot access Bedrock deployment accounts, logs, prompts or completions. These are AWS service statements, not an audit of a customer configuration or a universal claim about cloud AI services.

  3. Regulation (EU) 2016/679 (General Data Protection Regulation)

    EUR-Lex, European Union · Retrieved

    Reviewed: Article 5 principles including purpose limitation, data minimisation and storage limitation; Article 28 processor requirements; Articles 44–49 transfers of personal data to third countries or international organisations

    Supports: The official regulation text establishes principles and legal obligations relevant to processing, processor arrangements, retention and international transfers of personal data. It does not prescribe this article’s procurement workflow or determine whether a particular AI deployment is compliant.

  4. NIST Privacy Framework

    National Institute of Standards and Technology · Retrieved

    Reviewed: Framework purpose, voluntary risk-management positioning, Core and Profiles, implementation resources and relationship to enterprise risk management

    Supports: NIST presents the Privacy Framework as a voluntary tool for identifying and managing privacy risk while building products and services. It provides risk-management structure, not certification, legal advice or approval of a provider.

Buyer guides

LLM Evaluation Platforms: Keep the Decision Contract Portable

AccessAllGPT desk review of current OpenAI evaluation and deprecation documentation, Anthropic evaluation guidance, HELM version 2 and the NIST Generative AI Profile, retrieved 2026-08-11. Vendor timelines, vendor guidance, research findings and AccessAllGPT procurement guidance are labeled separately. This is not an original platform test, migration exercise, grader study, cost benchmark, security assessment or vendor comparison.

5 sources · 5 retrieval dates · 5 scoped claim notes
  1. Deprecations

    OpenAI Developer Documentation · Retrieved

    Reviewed: Deprecation overview; upcoming deprecations; 2026-06-03 Evals platform entry; announcement, read-only and shutdown dates; stated migration path

    Supports: OpenAI states that it announced deprecation of its Evals platform on June 3, 2026, that existing evals become read-only on October 31, 2026, and that the dashboard and API are scheduled to shut down on November 30, 2026. This is a vendor timeline for one product, not evidence that another platform is durable or that migration is lossless.

  2. Evaluation best practices

    OpenAI Developer Documentation · Retrieved

    Reviewed: What evals are; evaluation process design; representative data and edge cases; evaluator types; human and model grading; continuous evaluation; Evals platform transition notice

    Supports: OpenAI describes evals as structured tests for variable model behavior and recommends defining an objective, collecting representative and edge-case data, combining evaluator types and continuously evaluating. This is vendor-authored implementation guidance, not independent validation of a product or grader.

  3. Define success criteria and build evaluations

    Anthropic Claude Platform Documentation · Retrieved

    Reviewed: Success criteria; test-case construction; example metrics and measurement methods; automated, human and model-based grading; evaluation workflow and limitations

    Supports: Anthropic recommends specific and measurable success criteria, multidimensional evaluation, representative test cases and a mix of grading methods. This is vendor guidance for evaluation practice, not independent evidence that Claude, a vendor console or model-based grading is accurate for a buyer’s workload.

  4. Holistic Evaluation of Language Models (version 2)

    Transactions on Machine Learning Research / arXiv · Retrieved

    Reviewed: Abstract; scenario and metric taxonomy; multi-metric design; standardized evaluation scope; transparency artifacts; reported coverage and stated limitations

    Supports: The HELM authors separate scenario coverage from metric coverage, evaluate multiple dimensions rather than accuracy alone, and publish raw prompts and completions for analysis. The reported study concerns its selected models, scenarios and historical period; it does not validate a procurement checklist or current commercial platform.

  5. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication metadata; abstract; voluntary cross-sector scope; relationship to AI RMF 1.0; design, development, use and evaluation framing

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 intended to help organizations incorporate trustworthiness considerations into the design, development, use and evaluation of generative AI systems. It does not certify an evaluation platform or prescribe this build-buy decision.

Automation

Human-in-the-Loop AI: Put Approval at the Consequence Boundary

AccessAllGPT desk review of the Model Context Protocol tools specification version 2026-07-28, current OpenAI agent-safety guidance, OWASP LLM01:2025 and NIST AI 600-1, retrieved 2026-08-11. Normative and advisory protocol text, vendor guidance, security guidance and AccessAllGPT recommendations are labeled separately. This is not an original usability study, security test, incident study, legal analysis or measurement of approval effectiveness.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: User Interaction Model, tool capabilities and tool invocation behavior

    Supports: The specification says there SHOULD always be a human able to deny tool invocations and recommends interfaces that disclose exposed tools, indicate invocation and present confirmation prompts. It does not mandate a particular interaction model.

  2. Safety in building agents

    OpenAI Developer Documentation · Retrieved

    Reviewed: Prompt-injection and private-data risks, structured outputs, tool approvals, guardrails, trace graders and combined controls

    Supports: OpenAI advises enabling approvals for MCP tool operations in Agent Builder and combining approvals with constrained data flow, guardrails and evaluation. This is vendor guidance for its products, not independent evidence that approval prevents unsafe outcomes.

  3. LLM01:2025 Prompt Injection

    OWASP GenAI Security Project · Retrieved

    Reviewed: Definition, impacts, prevention and mitigation strategies, and attack scenarios

    Supports: OWASP says fool-proof prevention methods are unclear and recommends layered mitigations that include least privilege, independent authorization, segregation of untrusted content, monitoring and human approval for high-risk actions.

  4. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into the design, development, use and evaluation of generative-AI systems.

Buyer guides

LLM Observability: Build the Evidence Layer Before You Buy the Dashboard

AccessAllGPT desk review of commit-pinned OpenTelemetry Generative AI conventions, the W3C Trace Context Recommendation, commit-pinned OWASP logging guidance and NIST AI 600-1, retrieved 2026-08-10. Standards, project guidance and AccessAllGPT recommendations are labeled separately. This is not an original platform test, incident study, cost benchmark, security assessment or vendor comparison.

5 sources · 5 retrieval dates · 5 scoped claim notes
  1. Semantic conventions for generative AI systems (commit 46d43c8)

    OpenTelemetry · Retrieved

    Reviewed: Generative AI convention status and the linked spans and metrics specifications

    Supports: OpenTelemetry labels its Generative AI semantic conventions Development and defines signal families for model and agent spans, events, exceptions and metrics. This establishes an evolving interoperability vocabulary, not production completeness or backend equivalence.

  2. Semantic conventions for generative client AI spans (commit 46d43c8)

    OpenTelemetry · Retrieved

    Reviewed: Span lifecycle, inference attributes, token usage and content-capture warnings

    Supports: The development-status convention describes logical operation spans across retries, provider and model attributes, error type, token usage and opt-in input/output content fields; it warns that content fields are likely to contain sensitive or PII data.

  3. Trace Context, W3C Recommendation 23 November 2021

    World Wide Web Consortium · Retrieved

    Reviewed: Abstract, overview, traceparent, tracestate and privacy and security considerations

    Supports: The W3C Recommendation standardizes HTTP headers and value formats for propagating request context across services. It enables distributed trace correlation; it does not define LLM quality, evaluation or prompt-retention policy.

  4. Logging Cheat Sheet (commit da4c967)

    OWASP Cheat Sheet Series · Retrieved

    Reviewed: Which events to log, event attributes, data to exclude, verification, protection and monitoring

    Supports: OWASP recommends risk-proportionate logging requirements, exclusion or protection of sensitive fields, sanitization of event data, protection against unauthorized access or tampering and integration with monitoring and incident response.

  5. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

Agents

Prompt Injection: Set the Deployment Gates Before Your LLM Can Act

AccessAllGPT desk review of OWASP LLM01:2025, NIST AI 100-2 E2025, the indirect prompt-injection paper arXiv:2302.12173v2 and current OpenAI agent-safety guidance, retrieved 2026-08-09. Standards guidance, research findings, vendor guidance and AccessAllGPT recommendations are labeled separately. This is not an original penetration test, security assessment, control validation or prevalence study.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. LLM01:2025 Prompt Injection

    OWASP GenAI Security Project · Retrieved

    Reviewed: Definition, direct and indirect prompt injection, impacts, prevention and mitigation strategies, and attack scenarios

    Supports: OWASP distinguishes direct from indirect prompt injection, ties impact to application context and model agency, says fool-proof prevention methods are unclear, and recommends layered mitigations including constrained behavior, output validation, least privilege, human approval and adversarial testing.

  2. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2025)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication metadata, planning notes, abstract, keywords and documentation links

    Supports: NIST describes a taxonomy spanning ML methods, attack life-cycle stages, attacker goals, capabilities and knowledge, and identifies challenges and mitigation methods. The publication provides security terminology rather than certifying a specific prompt-injection control.

  3. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (version 2)

    arXiv (authors from CISPA Helmholtz Center for Information Security and Saarland University) · Retrieved

    Reviewed: Abstract, version history, stated threat model, demonstrated attack scope and limitations implied by the tested systems

    Supports: The authors define indirect prompt injection through retrieved data and report demonstrations against named 2023-era real and synthetic LLM-integrated applications. This establishes practical examples in that historical scope, not a current prevalence rate or a universal exploit result.

  4. Safety in building agents

    OpenAI Developer Documentation · Retrieved

    Reviewed: Prompt injection and private-data risks, untrusted variables, structured outputs, tool approvals, input guardrails, trace graders, evaluations and combined controls

    Supports: OpenAI advises keeping untrusted variables out of developer messages, constraining data flow with structured outputs, retaining tool approvals, applying input guardrails, and running trace graders and evaluations. This is vendor implementation guidance, not independent evidence that the controls eliminate prompt injection.

Automation

AI Agents vs Workflows: Choose the Right Automation Architecture

AccessAllGPT desk review of Anthropic engineering guidance, OpenAI’s agent-building guide and NIST AI 600-1, retrieved 2026-08-08. Vendor definitions and recommendations are labeled as such; the decision framework, gates and templates are AccessAllGPT guidance. This is not an original benchmark, agent trial, cost study, security assessment or vendor comparison.

3 sources · 3 retrieval dates · 3 scoped claim notes
  1. Building effective agents

    Anthropic Engineering · Retrieved

    Reviewed: Definitions; when and when not to use agents; frameworks; augmented LLMs; workflow patterns; autonomous agents; evaluation and production considerations

    Supports: Anthropic distinguishes workflows with predefined code paths from agents whose process and tool use are directed dynamically by a model, recommends the simplest sufficient design, and describes latency, cost, control and evaluation trade-offs. These are vendor-authored engineering recommendations, not comparative trial results.

  2. A practical guide to building agents

    OpenAI · Retrieved

    Reviewed: Pages 4–8, 13–17 and 24–32: agent definition, use-case criteria, design foundations, orchestration, guardrails and human intervention

    Supports: OpenAI defines agents as systems in which an LLM manages workflow execution and dynamically selects tools, says deterministic solutions may suffice outside complex or ambiguous use cases, and recommends incremental orchestration, layered guardrails and human intervention. These are vendor-authored recommendations, not independent evidence of production performance.

  3. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation. It does not certify an agent or prescribe the AccessAllGPT decision framework.

Buyer guides

Managed LLM API vs Self-Hosting: Make the Production Decision

AccessAllGPT desk review of NIST AI 600-1, current OpenAI API data-control documentation, vLLM stable engine configuration documentation and Kubernetes GPU-scheduling documentation, retrieved 2026-08-07. Vendor statements, implementation documentation and AccessAllGPT guidance are separated. This is not an original benchmark, cost study, security assessment or legal review.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

  2. Data controls in the OpenAI platform

    OpenAI Developer Documentation · Retrieved

    Reviewed: Data use, abuse-monitoring retention, application state, retention controls and data residency

    Supports: OpenAI states that API data is not used to train its models unless the customer opts in, describes default abuse-monitoring retention of up to 30 days, and documents feature- and eligibility-dependent retention controls. These are vendor statements about one managed service, not findings about every API or account.

  3. Engine Arguments

    vLLM documentation · Retrieved

    Reviewed: Model, load, parallel, cache, device, scheduler and observability configuration groups

    Supports: The serving engine exposes choices including model and tokenizer resolution, data type, quantization, model length, tensor and pipeline parallelism, GPU memory utilization, CPU offload and observability. This establishes configuration surface, not a universal performance or cost result.

  4. Schedule GPUs

    Kubernetes Documentation · Retrieved

    Reviewed: Device plugin prerequisites, GPU resource requests and heterogeneous node selection

    Supports: Kubernetes documents vendor drivers and device plugins as prerequisites for GPU scheduling, GPU resources specified through limits, and node labels or affinity for selecting accelerator types. It describes a deployment mechanism, not a complete inference operating model.

Buyer guides

RAG vs Fine-Tuning: Choose the Right Adaptation Path

AccessAllGPT desk review of the original RAG paper, NIST AI 600-1 and current OpenAI implementation guidance, retrieved 2026-08-06. Paper findings, vendor guidance and AccessAllGPT recommendations are labeled separately. This is not an original benchmark, cost study or product comparison.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (version 4)

    arXiv (authors from Facebook AI Research, University College London and New York University) · Retrieved

    Reviewed: Abstract, introduction, methods, experiments, results, discussion and limitations of the paper's experimental scope

    Supports: The authors define and evaluate a specific RAG architecture that combines parametric sequence-to-sequence memory with retrieved non-parametric Wikipedia passages; this supports the architectural distinction, not a universal claim that retrieval improves every application.

  2. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

  3. Optimizing LLM Accuracy

    OpenAI Developer Documentation · Retrieved

    Reviewed: LLM optimization context, prompt engineering, retrieval-augmented generation, fine-tuning and production accuracy sections

    Supports: Vendor-authored guidance distinguishes context optimization for missing, stale or proprietary knowledge from model optimization for inconsistent format, style or behavior, and recommends an evaluate–hypothesize–change–evaluate cycle.

  4. Supervised fine-tuning

    OpenAI Developer Documentation · Retrieved

    Reviewed: Overview, stated use cases, evaluation prerequisite, dataset construction and data-format guidance

    Supports: Vendor-authored product guidance describes supervised fine-tuning with prompt-and-response examples for classification, translation, specific output formats and instruction-following behavior, and says to establish evaluations before investing in fine-tuning.

Models

Choose a Model Without Chasing the Leaderboard

AccessAllGPT desk review of NIST AI 600-1, the HELM v2 paper, the original Chatbot Arena paper and OpenAI evaluation guidance, reverified 2026-08-06. Historical study findings, vendor guidance and AccessAllGPT recommendations are separated below. This is not an original benchmark, product test or vendor ranking.

4 sources · 4 retrieval dates · 4 scoped claim notes
  1. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

  2. Holistic Evaluation of Language Models (version 2)

    Transactions on Machine Learning Research / arXiv · Retrieved

    Reviewed: Abstract, methodology summary and reported evaluation scope

    Supports: The authors define scenario coverage and multi-metric evaluation as distinct parts of model evaluation, and report standardized evaluation across accuracy, calibration, robustness, fairness, bias, toxicity and efficiency.

  3. Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

    arXiv (authors from LMSYS Org and UC Berkeley) · Retrieved

    Reviewed: Abstract, methodology description and stated validation claims

    Supports: The authors describe Chatbot Arena as pairwise, crowdsourced human-preference evaluation and report agreement checks against expert raters; this establishes what the published ranking measures, not workload-specific production fitness.

  4. Evaluation best practices

    OpenAI Developer Documentation · Retrieved

    Reviewed: What are evals, design your eval process, evaluator types and edge cases

    Supports: Vendor guidance to define an evaluation objective, collect representative and edge-case data, combine evaluator types and continuously evaluate; used as implementation guidance rather than independent evidence of an OpenAI product claim.

Agents

Before You Give a Coding Agent Repository Access

AccessAllGPT synthesis of primary standards, security guidance, platform controls and benchmark methodology. This is not an original benchmark and does not score any vendor.

5 sources · 1 retrieval dates · 1 scoped claim notes
  1. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

  2. OWASP Top 10 for Large Language Model Applications

    OWASP GenAI Security Project

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.

  3. About rulesets

    GitHub Docs

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.

  4. SWE-bench

    SWE-bench

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.

  5. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    arXiv

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.

Benchmarks

Design an Agent Benchmark That Predicts Production

Evergreen benchmark-design guidance.

2 sources · 0 retrieval dates · 0 scoped claim notes
  1. SWE-bench

    SWE-bench

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.

  2. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    arXiv

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.

Buyer guides

The AI Tooling Procurement Scorecard

A vendor-neutral procurement framework. Commercial relationships should be disclosed in any applied comparison.

2 sources · 1 retrieval dates · 1 scoped claim notes
  1. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

  2. OWASP Top 10 for Large Language Model Applications

    OWASP GenAI Security Project

    This earlier source record links the primary material but does not yet carry a section-level claim note. Consult the article’s methodology and limitations before transferring a claim.