Independent AI systems researchOperated by NeuralArc
AccessAllGPTRESEARCH
Menu
Evidence index

Inspect the source before the verdict.

This index exposes the material behind every decision guide: what we reviewed, when we retrieved it, the claim it supports and where the evidence stops.

158decision guides
707source records
699scoped claim notes
How to read this index

A citation is not a transferable result.

Reviewed identifies the part of the source used. Supports states the bounded proposition we draw from it. Retrieved records when time-sensitive material was checked.

Vendor documentation establishes vendor claims and product behavior as documented, not independent performance. Papers establish results only for their stated methods and tested scope. AccessAllGPT recommendations remain labeled analysis.

Read the full methodology →
Source ledger

Evidence by decision guide

Ordered by the latest publication or material update date. Follow a source to inspect the original material.

Agents

Node’s Permission Model Is a Seat Belt, Not an AI Code Sandbox

AccessAllGPT source review and bounded local experiment, first run 2026-08-16 and refreshed 2026-08-21. We cloned Node.js tag v24.10.0 at immutable commit 9b72b88f4c4565687e3a8c4d8e1232f63a501e15, inspected its permission documentation, security threat model, environment initialization and 44 permission-named tests, and checked the official release index. We then downloaded Node.js v26.7.0 for Darwin arm64, verified its SHA256 against SHASUMS256.txt, reviewed the release note and permissions docs, and reran the local probe under both --permission-audit and --permission. The script creates only temporary fixture files, starts a loopback-only Python HTTP server, runs asserted scenarios in child Node processes, kills the server and removes the fixture. It tests direct filesystem reads and writes, relative-symlink traversal, loopback fetch, default child-process denial, an explicitly allowed non-Node child, an inherited file descriptor and the new audit-mode diagnostics path. We did not execute model-generated code, test a container or VM, probe Linux or Windows, attempt undisclosed bypasses, benchmark overhead, or perform a production security assessment. Node.js project statements, source observations, local results, independent NIST context and AccessAllGPT guidance are labeled separately.

10 sources · 10 retrieval dates · 10 scoped claim notes

  1. Node.js v26.7.0 release note

    Node.js project · Retrieved

    Reviewed: Release headline and changelog bullet documenting --permission-audit audit mode behavior

    Supports: The v26.7.0 release note records the documentation change that introduced audit-mode behavior for the permission model.

  2. Permissions — Node.js v26.7.0 documentation

    Node.js project · Retrieved

    Reviewed: Operational modes, runtime API, audit-mode diagnostics-channel mapping and known issues

    Supports: The v26.7.0 docs define enforce mode and audit mode, explain that audit mode logs violations without denying access, enumerate the diagnostics-channel names, and retain the symlink and existing-descriptor limitations.

  3. SHASUMS256.txt for Node.js v26.7.0

    Node.js project · Retrieved

    Reviewed: node-v26.7.0-darwin-arm64.tar.gz checksum row

    Supports: The downloaded Darwin arm64 binary was verified against the official SHA256 before local execution.

  4. Scripts | npm Docs

    npm · Retrieved

    Reviewed: How npm handles the scripts field; shell execution behavior on POSIX and Windows

    Supports: npm documents that scripts run through /bin/sh on POSIX systems and cmd.exe on Windows, which makes shell policy part of the agent boundary whenever an agent is allowed to invoke package scripts.

  5. Permissions — Node.js v24.10.0 documentation

    Node.js project · Retrieved

    Reviewed: Scope and threat statement; stable status; restricted resources; filesystem allowlists; runtime API; npx behavior; constraints; symlink and inherited-file-descriptor limitations

    Supports: The pinned documentation calls the permission model a seat belt for trusted code, explicitly says it does not protect against malicious code, lists the resources it restricts, says node:sqlite can access files outside node:fs checks, and documents relative-symlink and inherited-file-descriptor bypass paths.

  6. Node.js security policy and threat model at v24.10.0

    Node.js project · Retrieved

    Reviewed: Responsible testing; trusted and untrusted elements; treatment of code Node.js is asked to run; inherited execution-user privileges

    Supports: The project threat model trusts JavaScript, WASM, native code and dependencies that Node.js is asked to run and says that code inherits the privileges of the execution user. This is incompatible with treating the runtime alone as a hostile-code security boundary.

  7. src,lib: stabilize permission model — pull request 56201

    Node.js project on GitHub · Retrieved

    Reviewed: Title; merge date; release lines; linked documentation history

    Supports: The permission model moved to stable through a change merged on December 12, 2024 and documented for Node.js 22.13.0 and 23.5.0. Stable API status describes support maturity; it does not revise the project’s malicious-code exclusion.

  8. Node.js v24.10.0 source tree at commit 9b72b88

    Node.js project on GitHub · Retrieved

    Reviewed: Permission initialization in src/env.cc; permission implementation; CLI documentation; permission tests for filesystem, symlinks, children, workers, addons, WASI, inspector, sqlite and file descriptors

    Supports: AccessAllGPT cloned the immutable v24.10.0 source commit. Environment initialization denies addons, inspector, child processes, workers and WASI unless explicitly allowed, then applies filesystem read and write allowlists. The test inventory exercises those controls and known symlink behavior; it does not implement a network permission boundary.

  9. Node.js release index

    Node.js project · Retrieved

    Reviewed: v24.10.0 release metadata and current release rows at retrieval

    Supports: The official index dates v24.10.0 to October 8, 2025. AccessAllGPT used that exact installed runtime; this is a version-pinned result rather than a claim about every supported or future Node.js release.

  10. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication scope; generative-AI risk-management framing; risk mapping, measurement and management context

    Supports: NIST presents the profile as voluntary, cross-sector guidance for incorporating trustworthiness considerations into design, development, use and evaluation. It supplies independent governance context for layered, measured controls; it does not evaluate Node.js or certify a sandbox architecture.

Agents

Before You Connect an MCP Server to Production

AccessAllGPT desk review of the version-pinned MCP 2026-07-28 specification, MCP security guidance, the MCP TypeScript SDK 1.30.0 README and underlying OAuth RFCs, reverified 2026-08-20. Normative MUST and advisory SHOULD language is separated from AccessAllGPT recommendations. This is implementation guidance plus a local stdio probe, not an original security test or vendor comparison.

6 sources · 6 retrieval dates · 6 scoped claim notes

  1. Security Best Practices (version 2026-07-28)

    Model Context Protocol · Retrieved

    Reviewed: Token passthrough, SSRF and local MCP server compromise

    Supports: Token passthrough is an anti-pattern; discovery can create SSRF exposure; local servers execute with local process authority.

  2. Authorization (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: Resource parameter implementation, access-token usage and token handling

    Supports: Normative resource-indicator, bearer-token transport, token audience validation and token rejection requirements.

  3. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: User interaction model, capabilities and security considerations

    Supports: Normative server controls, advisory human-approval controls, authorization-dependent tool lists and untrusted annotations.

  4. MCP TypeScript SDK README (version 1.30.0)

    @modelcontextprotocol/sdk · Retrieved

    Reviewed: Overview, quick start, servers and transports, clients, and local stdio transport description

    Supports: The SDK documents stdio as a local, process-spawned transport and says clients can connect to servers and call tools.

  5. Best Current Practice for OAuth 2.0 Security

    RFC Editor (RFC 9700) · Retrieved

    Reviewed: Redirect URI validation and audience-restricted access tokens

    Supports: OAuth security baseline used to cross-check exact redirect matching and audience restriction.

  6. OAuth 2.0 Protected Resource Metadata

    RFC Editor (RFC 9728) · Retrieved

    Reviewed: Protected resource metadata request, response and security considerations

    Supports: The discovery metadata format and security boundary underlying MCP protected-resource discovery.

Models

GLM-5.3: Trial the Coding Gains, Contain the Cyber Capability

AccessAllGPT desk review of Z.ai’s GLM-5.3 launch, Coding Plan and API documentation; the public GLM-5.2 predecessor repository pinned at revision b4734de; Terminal-Bench 3.0 pinned at tag v3.0.0 and commit 2b0442c; CyberGym v3 and its harness pinned at commit 7656b71; the ExploitBench harness pinned at commit 9d0173b; and dated official-leaderboard and Hacker News snapshots, retrieved 2026-08-15. We wrote and ran reproducible Node.js audits against the pinned GLM-5.2 Hugging Face metadata and Terminal-Bench checkout. The GLM audit paginated all 295 files, reconciled 282 weight shards between the tree and index, and reported architecture and byte counts without downloading the weights. The Terminal-Bench audit reconciled all 74 task directories with 74 unique SHA-256 manifest digests and summarized categories, GPU use, agent timeouts and expert estimates. We also inspected both pinned cyber harnesses. For CyberGym, we installed its documented development and server dependencies under Python 3.13, compiled the source, and ran targeted smoke checks for checksum binding, image mapping, timeout normalization and rate limiting; we could not run a target because the Docker daemon was unavailable and the benchmark assets were not downloaded. For ExploitBench, we inspected its canonical matrix, scoring and image-resolution code and ran the non-slow unit and golden test suite under Python 3.11. The first run omitted optional publishing dependencies and failed four publishing tests; after installing the documented publish extras, 643 tests passed, six were skipped and six were deselected. We did not run an agent on Terminal-Bench, execute either cyber benchmark or call GLM-5.3. Momentum evidence, vendor claims, predecessor-artifact measurements, independent methodology artifacts, local harness evidence and AccessAllGPT guidance are labeled separately. This is not an original model run, benchmark reproduction, repository trial, red-team exercise, GLM-5.3 weight inspection, security assessment, GLM-5.3 license review, cost study or legal analysis.

9 sources · 9 retrieval dates · 9 scoped claim notes

  1. GLM-5.3: Frontier Coding with Emergent Cyber Capabilities

    Z.ai · Retrieved

    Reviewed: Release date and status; post-training claim; coding, agentic and cyber benchmark tables; weight-release timing; API changes; Coding Plan availability; evaluation footnotes

    Supports: Z.ai announced GLM-5.3 on August 14, 2026, says it uses the same base model as GLM-5.2 with gains from additional post-training, reports vendor-run coding and cyber evaluations, documents mandatory thinking and reasoning-effort settings, and says weights are planned two weeks after launch following safety evaluation and hardening. These are vendor claims and release statements, not independent AccessAllGPT measurements.

  2. GLM Coding Plan overview

    Z.ai Developer Documentation · Retrieved

    Reviewed: Coding Plan scope; supported coding-agent integrations; subscription and quota framing; API compatibility guidance

    Supports: Z.ai documents its Coding Plan as a route for using GLM models in supported coding-agent clients. This establishes an available managed access path and integration surface; it does not establish GLM-5.3 quality, security, quota sufficiency or compatibility for a particular repository.

  3. Chat completion API

    Z.ai Developer Documentation · Retrieved

    Reviewed: Request and response contract; model selection; messages; thinking configuration; tools; streaming; errors and usage fields

    Supports: Z.ai publishes the request and response surface used to call supported chat models, including model selection, messages, tools, streaming and usage. The documentation establishes an integration contract; actual GLM-5.3 entitlement, behavior, latency, limits and billing must be verified on the adopting account.

  4. zai-org/GLM-5.2 artifact repository at revision b4734de

    Z.ai on Hugging Face · Retrieved

    Reviewed: Pinned repository tree and pagination; model card; MIT license text; config.json architecture fields; safetensors index metadata and weight map; serving-framework versions; file sizes and immutable object identifiers

    Supports: AccessAllGPT queried the Hugging Face API and small metadata files at immutable GLM-5.2 revision b4734de. The predecessor has 282 bfloat16 safetensor shards with 1,506,659,919,872 indexed parameter bytes, a 78-layer GLM MoE DSA configuration, 256 routed experts with eight selected per token, and a 1,048,576-position configured maximum. Its pinned license file is MIT. These measurements establish the scale and metadata of GLM-5.2 only. They do not establish GLM-5.3 file size, license, serving compatibility, quantization, hardware needs or release status.

  5. Terminal-Bench 3.0 release at tag v3.0.0 and official leaderboard

    Harbor / Terminal-Bench maintainers · Retrieved

    Reviewed: Tagged release and immutable commit; dataset manifest and content digests; task metadata and category distribution; agent timeouts and expert estimates; official leaderboard model-agent pairs, resolution rates, uncertainty, token totals and costs

    Supports: AccessAllGPT cloned and audited immutable Terminal-Bench 3.0 tag v3.0.0 at commit 2b0442c. Its manifest binds 74 named tasks to 74 unique SHA-256 content digests across seven categories. Task metadata spans 30- to 480-minute agent limits, with a 120-minute median, and four GPU tasks. The official leaderboard reports model-agent configurations rather than model-only scores and, in the August 15 snapshot, listed GLM 5.2 with Claude Code at 4.6% ± 1.0% but did not list GLM-5.3. These facts explain the public benchmark contract and baseline; they do not validate Z.ai’s 28.3 result or identify its agent, repetitions, task failures, tokens or cost.

  6. CyberGym: Evaluating AI Agents’ Real-World Cybersecurity Capabilities at Scale (v3)

    CyberGym authors on arXiv · Retrieved

    Reviewed: Version history; abstract; benchmark scale; task construction; proof-of-concept objective; reported baseline difficulty; limitations implied by the evaluation target

    Supports: The independent benchmark paper defines CyberGym as 1,507 historical vulnerabilities across 188 projects and primarily asks an agent to generate a proof-of-concept test from a vulnerability description and codebase. This explains what the benchmark targets; it does not validate Z.ai’s GLM-5.3 score or establish safe autonomous exploitation.

  7. CyberGym repository at commit 7656b71

    CyberGym authors on GitHub · Retrieved

    Reviewed: README data and deployment requirements; FAQ network, task, environment and scoring guidance; August 4 submission schema; server submission and verification paths; task checksum; timeout handling; rate limiting; package metadata; local lint, compilation and helper smoke checks

    Supports: At pinned commit 7656b71, AccessAllGPT inspected the current public CyberGym harness. The maintainers now request one designated final submission because any-of scoring increasingly rewards brute force, require token, time, request and artifact reporting, and warn that the partly unauthenticated PoC endpoint must remain private. The README estimates roughly 240 GB for benchmark data and roughly 10 TB for the full compilation environment. Local compilation and targeted helper smoke checks passed; the repository-wide Ruff check found one import-order error. Docker was installed but its daemon was unavailable, so no target container or PoC was run. This audit does not identify or validate Z.ai’s task set, metric, environment, network policy, trajectories or reported score.

  8. ExploitBench repository at commit 9d0173b

    ExploitBench · Retrieved

    Reviewed: README scope; canonical v8.yaml matrix; capability extraction and scoring code; image-reference resolution; model dispatch; seed and budget configuration; audit and reproduction workflow; local non-slow unit and golden tests

    Supports: At pinned commit 9d0173b, AccessAllGPT inspected the public harness and ran its non-slow unit and golden test suite locally. The canonical configuration enumerates 41 V8 environments, six models and five seeds; the scorer cumulatively ORs 16 capability flags and treats arbitrary code execution as a terminal full score. The checked-in matrix uses mutable GHCR tags, while runner code resolves the locally loaded image to an immutable image ID and the documentation recommends registry digests for publication-grade pinning. This audit explains evaluation mechanics but does not identify or validate the revision, matrix, prompts, seeds, containers, transcripts or scoring transformation behind Z.ai’s reported GLM-5.3 result.

  9. Hacker News discussion: GLM-5.3: Frontier coding with emergent cyber capabilities

    Hacker News · Retrieved

    Reviewed: Submission timestamp; linked official release; observable points and comment count in the Algolia API snapshot; discussion scope

    Supports: The August 14 submission linked the official release and showed 1,007 points and 496 comments in the snapshot retrieved August 15. This is a strong independent attention signal only; mutable community activity does not validate model capability, safety, availability, license or production fitness.

Models

GPT-5.6 Sol Ultrafast: Buy Speed Only Where Latency Changes the Outcome

AccessAllGPT desk review of OpenAI launch, model, Fast-mode and pricing documentation; Cerebras launch and benchmark disclosures; and a dated Hacker News momentum snapshot, retrieved 2026-08-15. Momentum evidence, vendor specifications, vendor-performed benchmarks and AccessAllGPT guidance are labeled separately. This is not an original API trial, latency benchmark, quality evaluation, availability test, cost study, security review or legal analysis.

6 sources · 6 retrieval dates · 6 scoped claim notes

  1. Previewing Ultrafast mode: GPT-5.6 Sol at up to 14X the speed

    OpenAI · Retrieved

    Reviewed: Launch date; limited-preview status; access scope; performance framing; intended workloads; availability caveats

    Supports: OpenAI announced Ultrafast mode on August 13, 2026 as a limited preview for selected API customers and describes it as a Cerebras-powered serving option for GPT-5.6 Sol. The page supplies vendor launch and performance claims; it does not establish results for an untested customer workload.

  2. Fast mode

    OpenAI API Documentation · Retrieved

    Reviewed: Overview; configuration; response reporting; rate limits and ramp rate; usage considerations; regions; billing; data controls

    Supports: OpenAI documents Fast mode as a generally configurable service tier distinct from the new limited-preview Ultrafast offer. It documents up to 2.5x speed for GPT-5.6 Sol versus Standard, shared rate limits, possible downgrade to Standard when traffic ramps too quickly, response-tier reporting, eligibility limits and compatibility statements.

  3. GPT-5.6 Sol model

    OpenAI API Documentation · Retrieved

    Reviewed: Model identity; reasoning levels; context and output limits; knowledge cutoff; pricing; endpoints; supported features and tools; snapshots

    Supports: OpenAI identifies gpt-5.6-sol as its frontier model for complex professional work, documents text and image input, text output, reasoning-effort choices, a 1.05M-token context window, a 128K-token maximum output, supported tools and endpoint exclusions. These are vendor specifications, not independent capability measurements.

  4. Pricing

    OpenAI API Documentation · Retrieved

    Reviewed: GPT-5.6 Sol Standard, Batch, Flex and Fast short-context and long-context token prices; regional-processing note; service-tier naming

    Supports: The retrieved table lists GPT-5.6 Sol short-context Standard prices of $5 per million input tokens and $30 per million output tokens, and Fast prices of $10 and $60 respectively. Long-context and cache-write rules differ. The page does not yet publish a separate Ultrafast price in the reviewed table.

  5. Accelerating GPT-5.6 Sol Ultrafast

    Cerebras · Retrieved

    Reviewed: Launch scope; claimed output speed; HLE and GDP-Val methodology notes; customer use cases; architecture; limited-preview caveat; benchmark disclaimer

    Supports: Cerebras says it powers the limited preview and reports up to 750 output tokens per second plus internal HLE and GDP-Val comparisons. Cerebras performed and published those tests, warns observed improvements vary by workload and configuration, and is commercially involved in the service; the results are vendor evidence, not independent validation.

  6. Hacker News discussion: Accelerating GPT-5.6 Sol Ultrafast

    Hacker News · Retrieved

    Reviewed: Submission timestamp; linked release; observable points and comment count at retrieval; discussion scope

    Supports: The August 13 submission was an observable developer-community momentum signal, showing 694 points and 270 comments in the Hacker News API snapshot retrieved August 15. Popularity is not factual or technical validation, and counts can change after retrieval.

Models

LLM Model Deprecation: Migrate Without Changing Production by Accident

AccessAllGPT desk review of current model-lifecycle documentation from OpenAI, Anthropic, Google Cloud and Amazon Bedrock, retrieved 2026-08-12. Vendor schedules and lifecycle terms are reported as vendor statements; the migration gates are AccessAllGPT guidance. This is not an original model test, compatibility study, performance benchmark, cost measurement, security assessment or legal review.

4 sources · 4 retrieval dates · 4 scoped claim notes

  1. Deprecations

    OpenAI Developer Documentation · Retrieved

    Reviewed: Overview; model deprecation notice periods; notification and replacement framing; API deprecation entries

    Supports: OpenAI states that software using its models may need updates, documents recommended replacements, and publishes minimum notice periods of at least six months for generally available models and at least three months for specialized variants unless safety or compliance concerns require a faster timeline. Preview models may receive much shorter notice. These are vendor lifecycle statements, not a compatibility guarantee.

  2. Model deprecations

    Anthropic Claude Platform Documentation · Retrieved

    Reviewed: Overview; lifecycle terminology; platform scope; migrating to replacements; notifications; auditing model usage

    Supports: Anthropic distinguishes active, legacy, deprecated and retired models; says requests to retired models fail; recommends testing applications before retirement; and states that publicly released models receive at least 60 days notice. It also says partner-operated Amazon Bedrock and Google Cloud schedules can differ from Anthropic-operated platforms.

  3. Model versions and lifecycle

    Google Cloud Documentation · Retrieved

    Reviewed: Lifecycle overview; availability-period categories; retirement dates; migration framing; lifecycle definitions and date policy

    Supports: Google Cloud publishes model lifecycle categories, retirement tables and migration guidance for the documented Gemini Enterprise Agent Platform. It says listed retirement dates may be extended but will not be brought forward, and distinguishes models available for at least 12 months from short-term models that retire 45 days after a replacement release. These terms apply to the documented Google Cloud service, not every Gemini surface.

  4. Model lifecycle

    Amazon Bedrock User Guide · Retrieved

    Reviewed: Lifecycle overview; Active, Legacy and End-of-Life states; minimum availability statement; provider-date warning; migration implications

    Supports: Amazon Bedrock documents Active, Legacy and End-of-Life states, exposes lifecycle state through model APIs, and says a model remains on Bedrock for at least 12 months before its End-of-Life date. AWS warns that Bedrock dates can differ from model-provider dates and that Bedrock-specific dates govern Bedrock usage.

Buyer guides

AI API Data Retention and Residency: Set the Procurement Gates

AccessAllGPT desk review of current OpenAI API data-control documentation, Amazon Bedrock data-protection documentation, the official GDPR text and the NIST Privacy Framework, retrieved 2026-08-11. Vendor statements, legal text and AccessAllGPT procurement guidance are labeled separately. This is not an original platform test, deletion audit, contract review, privacy impact assessment, security assessment, legal analysis or vendor comparison.

4 sources · 4 retrieval dates · 4 scoped claim notes

  1. Data controls in the OpenAI platform

    OpenAI Developer Documentation · Retrieved

    Reviewed: Data use; types of API data; abuse-monitoring retention; Zero Data Retention and Modified Abuse Monitoring eligibility and limitations; endpoint-level application-state retention; regional storage and processing controls; Enterprise Key Management

    Supports: OpenAI states that API data is not used to train or improve its models unless a customer explicitly opts in, distinguishes abuse-monitoring logs from application state, documents default abuse-monitoring retention of up to 30 days, and describes feature-, project-, region- and eligibility-dependent controls. These are current vendor statements about the documented platform, not independent verification or a promise for every account and feature.

  2. Data protection — Amazon Bedrock

    Amazon Web Services Documentation · Retrieved

    Reviewed: Shared-responsibility boundary; IAM, encryption and activity-logging recommendations; warning about sensitive data in tags and free-form name fields; model deployment accounts; linked encryption, PrivateLink and retention topics

    Supports: AWS assigns customers responsibility for content controls and service configuration, warns that tags and free-form naming fields can enter billing or diagnostic logs, and states that model providers cannot access Bedrock deployment accounts, logs, prompts or completions. These are AWS service statements, not an audit of a customer configuration or a universal claim about cloud AI services.

  3. Regulation (EU) 2016/679 (General Data Protection Regulation)

    EUR-Lex, European Union · Retrieved

    Reviewed: Article 5 principles including purpose limitation, data minimisation and storage limitation; Article 28 processor requirements; Articles 44–49 transfers of personal data to third countries or international organisations

    Supports: The official regulation text establishes principles and legal obligations relevant to processing, processor arrangements, retention and international transfers of personal data. It does not prescribe this article’s procurement workflow or determine whether a particular AI deployment is compliant.

  4. NIST Privacy Framework

    National Institute of Standards and Technology · Retrieved

    Reviewed: Framework purpose, voluntary risk-management positioning, Core and Profiles, implementation resources and relationship to enterprise risk management

    Supports: NIST presents the Privacy Framework as a voluntary tool for identifying and managing privacy risk while building products and services. It provides risk-management structure, not certification, legal advice or approval of a provider.

Buyer guides

LLM Evaluation Platforms: Keep the Decision Contract Portable

AccessAllGPT desk review of current OpenAI evaluation and deprecation documentation, Anthropic evaluation guidance, HELM version 2 and the NIST Generative AI Profile, retrieved 2026-08-11. Vendor timelines, vendor guidance, research findings and AccessAllGPT procurement guidance are labeled separately. This is not an original platform test, migration exercise, grader study, cost benchmark, security assessment or vendor comparison.

5 sources · 5 retrieval dates · 5 scoped claim notes

  1. Deprecations

    OpenAI Developer Documentation · Retrieved

    Reviewed: Deprecation overview; upcoming deprecations; 2026-06-03 Evals platform entry; announcement, read-only and shutdown dates; stated migration path

    Supports: OpenAI states that it announced deprecation of its Evals platform on June 3, 2026, that existing evals become read-only on October 31, 2026, and that the dashboard and API are scheduled to shut down on November 30, 2026. This is a vendor timeline for one product, not evidence that another platform is durable or that migration is lossless.

  2. Evaluation best practices

    OpenAI Developer Documentation · Retrieved

    Reviewed: What evals are; evaluation process design; representative data and edge cases; evaluator types; human and model grading; continuous evaluation; Evals platform transition notice

    Supports: OpenAI describes evals as structured tests for variable model behavior and recommends defining an objective, collecting representative and edge-case data, combining evaluator types and continuously evaluating. This is vendor-authored implementation guidance, not independent validation of a product or grader.

  3. Define success criteria and build evaluations

    Anthropic Claude Platform Documentation · Retrieved

    Reviewed: Success criteria; test-case construction; example metrics and measurement methods; automated, human and model-based grading; evaluation workflow and limitations

    Supports: Anthropic recommends specific and measurable success criteria, multidimensional evaluation, representative test cases and a mix of grading methods. This is vendor guidance for evaluation practice, not independent evidence that Claude, a vendor console or model-based grading is accurate for a buyer’s workload.

  4. Holistic Evaluation of Language Models (version 2)

    Transactions on Machine Learning Research / arXiv · Retrieved

    Reviewed: Abstract; scenario and metric taxonomy; multi-metric design; standardized evaluation scope; transparency artifacts; reported coverage and stated limitations

    Supports: The HELM authors separate scenario coverage from metric coverage, evaluate multiple dimensions rather than accuracy alone, and publish raw prompts and completions for analysis. The reported study concerns its selected models, scenarios and historical period; it does not validate a procurement checklist or current commercial platform.

  5. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication metadata; abstract; voluntary cross-sector scope; relationship to AI RMF 1.0; design, development, use and evaluation framing

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 intended to help organizations incorporate trustworthiness considerations into the design, development, use and evaluation of generative AI systems. It does not certify an evaluation platform or prescribe this build-buy decision.

Automation

Human-in-the-Loop AI: Put Approval at the Consequence Boundary

AccessAllGPT desk review of the Model Context Protocol tools specification version 2026-07-28, current OpenAI agent-safety guidance, OWASP LLM01:2025 and NIST AI 600-1, retrieved 2026-08-11. Normative and advisory protocol text, vendor guidance, security guidance and AccessAllGPT recommendations are labeled separately. This is not an original usability study, security test, incident study, legal analysis or measurement of approval effectiveness.

4 sources · 4 retrieval dates · 4 scoped claim notes

  1. Tools (version 2026-07-28)

    Model Context Protocol Specification · Retrieved

    Reviewed: User Interaction Model, tool capabilities and tool invocation behavior

    Supports: The specification says there SHOULD always be a human able to deny tool invocations and recommends interfaces that disclose exposed tools, indicate invocation and present confirmation prompts. It does not mandate a particular interaction model.

  2. Safety in building agents

    OpenAI Developer Documentation · Retrieved

    Reviewed: Prompt-injection and private-data risks, structured outputs, tool approvals, guardrails, trace graders and combined controls

    Supports: OpenAI advises enabling approvals for MCP tool operations in Agent Builder and combining approvals with constrained data flow, guardrails and evaluation. This is vendor guidance for its products, not independent evidence that approval prevents unsafe outcomes.

  3. LLM01:2025 Prompt Injection

    OWASP GenAI Security Project · Retrieved

    Reviewed: Definition, impacts, prevention and mitigation strategies, and attack scenarios

    Supports: OWASP says fool-proof prevention methods are unclear and recommends layered mitigations that include least privilege, independent authorization, segregation of untrusted content, monitoring and human approval for high-risk actions.

  4. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into the design, development, use and evaluation of generative-AI systems.

Buyer guides

LLM Observability: Build the Evidence Layer Before You Buy the Dashboard

AccessAllGPT desk review of commit-pinned OpenTelemetry Generative AI conventions, the W3C Trace Context Recommendation, commit-pinned OWASP logging guidance and NIST AI 600-1, retrieved 2026-08-10. Standards, project guidance and AccessAllGPT recommendations are labeled separately. This is not an original platform test, incident study, cost benchmark, security assessment or vendor comparison.

5 sources · 5 retrieval dates · 5 scoped claim notes

  1. Semantic conventions for generative AI systems (commit 46d43c8)

    OpenTelemetry · Retrieved

    Reviewed: Generative AI convention status and the linked spans and metrics specifications

    Supports: OpenTelemetry labels its Generative AI semantic conventions Development and defines signal families for model and agent spans, events, exceptions and metrics. This establishes an evolving interoperability vocabulary, not production completeness or backend equivalence.

  2. Semantic conventions for generative client AI spans (commit 46d43c8)

    OpenTelemetry · Retrieved

    Reviewed: Span lifecycle, inference attributes, token usage and content-capture warnings

    Supports: The development-status convention describes logical operation spans across retries, provider and model attributes, error type, token usage and opt-in input/output content fields; it warns that content fields are likely to contain sensitive or PII data.

  3. Trace Context, W3C Recommendation 23 November 2021

    World Wide Web Consortium · Retrieved

    Reviewed: Abstract, overview, traceparent, tracestate and privacy and security considerations

    Supports: The W3C Recommendation standardizes HTTP headers and value formats for propagating request context across services. It enables distributed trace correlation; it does not define LLM quality, evaluation or prompt-retention policy.

  4. Logging Cheat Sheet (commit da4c967)

    OWASP Cheat Sheet Series · Retrieved

    Reviewed: Which events to log, event attributes, data to exclude, verification, protection and monitoring

    Supports: OWASP recommends risk-proportionate logging requirements, exclusion or protection of sensitive fields, sanitization of event data, protection against unauthorized access or tampering and integration with monitoring and incident response.

  5. NIST AI RMF: Generative Artificial Intelligence Profile (NIST AI 600-1)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication abstract, scope, citation and report metadata

    Supports: NIST describes the profile as a voluntary, cross-sector companion to AI RMF 1.0 for incorporating trustworthiness considerations into design, development, use and evaluation.

Agents

Prompt Injection: Set the Deployment Gates Before Your LLM Can Act

AccessAllGPT desk review of OWASP LLM01:2025, NIST AI 100-2 E2025, the indirect prompt-injection paper arXiv:2302.12173v2 and current OpenAI agent-safety guidance, retrieved 2026-08-09. Standards guidance, research findings, vendor guidance and AccessAllGPT recommendations are labeled separately. This is not an original penetration test, security assessment, control validation or prevalence study.

4 sources · 4 retrieval dates · 4 scoped claim notes

  1. LLM01:2025 Prompt Injection

    OWASP GenAI Security Project · Retrieved

    Reviewed: Definition, direct and indirect prompt injection, impacts, prevention and mitigation strategies, and attack scenarios

    Supports: OWASP distinguishes direct from indirect prompt injection, ties impact to application context and model agency, says fool-proof prevention methods are unclear, and recommends layered mitigations including constrained behavior, output validation, least privilege, human approval and adversarial testing.

  2. Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2 E2025)

    National Institute of Standards and Technology · Retrieved

    Reviewed: Publication metadata, planning notes, abstract, keywords and documentation links

    Supports: NIST describes a taxonomy spanning ML methods, attack life-cycle stages, attacker goals, capabilities and knowledge, and identifies challenges and mitigation methods. The publication provides security terminology rather than certifying a specific prompt-injection control.

  3. Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (version 2)

    arXiv (authors from CISPA Helmholtz Center for Information Security and Saarland University) · Retrieved

    Reviewed: Abstract, version history, stated threat model, demonstrated attack scope and limitations implied by the tested systems

    Supports: The authors define indirect prompt injection through retrieved data and report demonstrations against named 2023-era real and synthetic LLM-integrated applications. This establishes practical examples in that historical scope, not a current prevalence rate or a universal exploit result.

  4. Safety in building agents

    OpenAI Developer Documentation · Retrieved

    Reviewed: Prompt injection and private-data risks, untrusted variables, structured outputs, tool approvals, input guardrails, trace graders, evaluations and combined controls

    Supports: OpenAI advises keeping untrusted variables out of developer messages, constraining data flow with structured outputs, retaining tool approvals, applying input guardrails, and running trace graders and evaluations. This is vendor implementation guidance, not independent evidence that the controls eliminate prompt injection.