awesomechecklists
llms.txtGitHub ↗
/

13 — AI Model and Dataset Cards Checklist#

Verify that model and dataset documentation accurately communicates intended use, capabilities, limitations, provenance, evaluation evidence, risks, and change history. A card is a living disclosure and engineering record—not proof of regulatory compliance, model safety, lawful data rights, or benchmark reproducibility.

Scope: Backend / API-specific. This file is one part of the 13_AI_Model_and_Dataset_Cards.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

  • Project / product:
  • Release / version / environment:
  • Scope and applicable jurisdictions / markets:
  • Model, dataset, provider and artifact versions:
  • Card audience, publication location, and visibility rules:
  • Documentation, evaluation, provenance and risk owners:
  • Reviewer / owner:
  • Review date:
  • Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

  1. Copy the reusable checklist into project-specific review records. Leave every checkbox in this repository unchecked.
  2. Evaluate the model card and dataset card separately where both exist; for a third-party model or dataset, verify documented provider facts independently when feasible and mark undisclosed facts as unknown instead of guessing.
  3. Mark [x] only after checking versioned model or dataset artifacts, lineage manifests, evaluation results, licenses and rights reviews, source documents, publication controls, or a recorded owner decision. Keep evidence, accepted exceptions, failed checks and justified N/A outside this shared template, keyed by CARDS ID.
  4. Apply ## API Checks and ## Frontend Checks where applicable, noting that an internal-only or offline dataset may not expose one or both surfaces.
  5. Decide explicitly what is appropriate to publish: technical summaries and meaningful limitations may be public, while source data, confidential tests, protected characteristics and security-sensitive artifacts may require restricted internal records.
  6. Re-review cards when source rights, model versions, usage boundaries, dataset versions, evaluation outcomes, safety policies, deployment conditions or known incidents change. Use the dedicated data curation, model evaluation, governance and lifecycle checklists for deeper controls.

API Checks#

  • CARDS-091 Associate every served model or dataset API identifier with the correct approved card revision and underlying artifact.
  • CARDS-092 Provide authorized clients a stable way to discover the current card or supported metadata endpoint where appropriate.
  • CARDS-093 Validate model-card and dataset-card metadata against the platform's relevant schema before publishing.
  • CARDS-094 Return actual model revision or deployment identifiers when needed for audit without disclosing protected internal details.
  • CARDS-095 Ensure card claims about available features, supported modalities, and usage restrictions match the live API contract.
  • CARDS-096 Test card/documentation links for inactive, retired, renamed, and replaced model or dataset versions.
  • CARDS-097 Enforce authentication and tenant permissions before serving restricted provenance, evaluation, or dataset details.
  • CARDS-098 Prevent public metadata endpoints from exposing proprietary prompts, secrets, private source records, or hidden evaluation labels.
  • CARDS-099 Check that model aliases and routing changes do not silently display an obsolete card for the newly deployed model.
  • CARDS-100 Version and test metadata response schemas for consumers of cards, dataset manifests, and lineage information.
  • CARDS-101 Publish machine-readable license and language fields accurately where the supported format provides them.
  • CARDS-102 Verify dataset record counts, split names, checksum metadata, and supported file types against approved exports.
  • CARDS-103 Separate evaluated model quality metrics from live-service availability and capability promises in API metadata.
  • CARDS-104 Check API caching, content negotiation, and invalidation when a card is revised or withdrawn.
  • CARDS-105 Ensure model or dataset deletion and access revocation affect card links, cached metadata, and downloadable artifacts as required.
  • CARDS-106 Log safe card revision and publication events with authorized actor and traceable artifact IDs.
  • CARDS-107 Test API behavior when card references, license metadata, or required provenance are missing or invalid.
  • CARDS-108 Validate outbound references to external model or dataset publishers rather than trusting unverified caller-supplied URLs.
  • CARDS-109 Include documentation consistency checks in CI or a controlled release workflow for material model and dataset changes.
  • CARDS-110 Confirm the production release includes approved card links or equivalent information for affected API consumers.

Release gate#

  • The released model and/or dataset has an identifiable owner, exact artifact version and approved current card with intended uses, out-of-scope uses and supported constraints.
  • Source, training and dataset provenance, relevant licensing or rights restrictions, transformations and open lineage gaps are documented without exposing protected records.
  • The card presents version-matched evaluation evidence, tested populations or modalities, material performance differences and known severe failure cases or uncertainty.
  • Meaningful safety, privacy, bias, accessibility and downstream misuse limitations have responsible mitigation or escalation guidance proportionate to risk.
  • Published metadata, references, model identifiers and dataset links have passed schema, access-control, consistency and link checks where relevant.
  • API and frontend surfaces expose appropriate, version-matched documentation, and correctly handle restricted, withdrawn or superseded cards.
  • The maintenance owner, change-triggered review, correction intake, revision history, publication controls and responsible reviewers are in place.
  • Significant missing rights, contradictory or misleading capability claims, unsupported high-impact uses or unresolved disclosure risks are blocked or formally dispositioned by qualified approvers.

Domain-specific approval rule: Block documentation approval if material claims cannot be reconciled to the actual model or dataset, a critical known limitation is concealed, a public card exposes protected information, or rights and usage restrictions remain unreviewed for the intended release. Publication does not certify a model, validate every provider claim or grant dataset-use rights.

Trusted references and scope#

The following primary specifications, publishing guides and responsible-AI references support practical model and dataset documentation. Use the parts applicable to the artifact and publication platform.

Applicability: Hugging Face model and dataset card metadata conventions apply when using that platform; an equivalent internal registry and documentation process can be appropriate for restricted artifacts. MLCommons Croissant 1.1 provides an optional machine-readable dataset metadata specification, including provenance, rather than a universal requirement. W3C PROV documents lineage structures but does not establish that training data was lawfully obtained. NIST AI RMF is voluntary guidance. Model Card Toolkit and vendor transparency notes are implementation examples, not evidence that an individual model is complete, unbiased, safe or certified. Never infer undisclosed third-party training details; document the uncertainty.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

  • Test focus: required metadata, model/dataset versions, limitations, and documentation links; include API and service behavior where applicable.
  • Cross-stack: markdownlint-cli2 + lychee.
  • Python: check-jsonschema + pytest; for pytest-based suites, run python -m pytest -q.
  • TypeScript: Ajv + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
  • JavaScript: Ajv + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
  • Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

  • doc-coauthoring — Review completeness and clarity of published cards (anthropics/skills, 193,495 repo stars).
  • evaluation-methodology — Check metrics, limitations, and evaluation disclosures (wshobson/agents, 42,437 repo stars).
  • dataset-curation — Trace dataset sources, assumptions, and splits (wshobson/agents, 42,437 repo stars).

How to check: In a project-specific copy, have the agent audit the backend / API scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.

Reference, not a certification. Check applicability, verify important claims against current primary sources, and seek expert review where necessary. Suggest an improvement.