30.5 Ecosystem

Modern machine learning projects commonly begin with pretrained checkpoints, datasets, released code, and benchmark results. Using these artifacts requires knowing where to find them, how to evaluate their relevance, and how to record their provenance. This section surveys prominent sources as of 2026, explains how to interpret leaderboards, and presents practices for managing external artifacts safely.

30.5.1 Where to Find Things

30.5.1.1 Models

The Hugging Face Hub listed more than two million public models in spring 2026 and provides Git-style versioning, model cards, and integrations with the libraries used in this book. Because popularity does not establish suitability, search by task, inspect recency and download signals, and read the model card before selecting a repository. Complementary sources include:

ModelScope, Alibaba’s hub, hosts a large part of the Chinese open-model ecosystem (Qwen and friends, 170,000+ models) — and frequently publishes new open-weight releases.
Kaggle Models hosts curated weights wired into Kaggle’s competition and notebook infrastructure (Section 30.2).
The Ollama library is a short, curated menu of local-runtime models — less a discovery surface than a convenience layer (Section 30.7).
timm provides a broad collection of vision backbones; Civitai hosts community image-generation checkpoints and LoRAs.

30.5.1.2 Datasets

Hugging Face Datasets (half a million public datasets) and Kaggle cover most supervised needs. For pretraining-scale text, the lineage runs from raw Common Crawl through filtered derivatives — FineWeb (~15 trillion tokens, plus a 1,000-language successor) provides a widely used open baseline, analogous to C4 in the period when BERT was developed. For vision–language pairs, DataComp provides an alternative to LAION with an emphasis on controlled dataset construction and evaluation. Older and very large corpora are sometimes available only through Academic Torrents. Whatever the source: datasets, like models, have versions, licenses, and documented failure modes. Review a dataset’s documentation before adopting it.

30.5.1.3 Papers and Code

New work appears on arXiv first; the community’s curated front page for ML is Hugging Face Papers, which absorbed that role when Papers with Code—for years an index from papers to code and benchmarks—closed in mid-2025. Its historical leaderboard data survives as a static archive, illustrating why project inputs should not depend on one external index. Semantic Scholar and alphaXiv help with search and discussion, and Released implementations commonly reside on GitHub. Issue history, recent commits, and documented reproduction results provide evidence beyond the paper’s abstract.

30.5.2 Choosing a Model: Benchmarks and Leaderboards

Model rankings change as models, evaluation sets, and optimization methods evolve. Static benchmark suites can also enter training data. The Open LLM Leaderboard was retired in 2025 after contamination reduced its usefulness. Model selection should therefore combine several sources of evidence:

LMArena (now Arena) — blind human pairwise preference; it measures aggregate user preferences, which may differ from the target task.
LiveBench — contamination-resistant by rotating fresh questions monthly.
SWE-bench Verified — an evaluation for coding agents on real GitHub issues; it exemplifies the benchmark-per-capability pattern (math, long context, safety all have their own).
Artificial Analysis — the quality/price/latency triangulation across hundreds of models and providers; useful when cost and latency are selection criteria.
OpenRouter rankings — revealed preference by real token volume rather than scores; instructive precisely where it disagrees with the quality leaderboards.

After forming a shortlist, build a small evaluation set from the target task and run each candidate on it. The gap between leaderboard rank and performance on your distribution is routinely larger than the gap between adjacent leaderboard entries. The evaluation discipline this book has practiced throughout — held-out data, meaningful baselines, error analysis — applies to choosing models exactly as it does to training them.

30.5.3 Staying Current

A small, deliberate set of information sources is easier to evaluate than a continuous stream of announcements. A workable minimal set, as of 2026:

r/LocalLLaMA — a community source for open-weight releases, quantizations, hardware reports, and reproduction attempts.
Hugging Face Papers daily — a curated dozen papers instead of arXiv’s daily hundreds.
One good newsletter — Andrew Ng’s The Batch, Jack Clark’s Import AI, or Sebastian Raschka’s Ahead of AI — for the weekly synthesis.
For systems depth, the GPU MODE lecture series and community, and the EleutherAI Discord for open research.

X/Twitter remains where labs announce and researchers argue; treat it as a discovery feed, not an archive. And the chapter-end resources of this book (Chapter 30) collect the durable long-form references.

30.5.4 Using What You Found

30.5.4.1 Pin the Identity

Figure 30.5.1: A reusable model artifact combines configuration, weights, preprocessing, documentation, and an immutable revision.

A model is more than its weights: the tokenizer, preprocessing, configuration, license, and revision all determine whether you can reproduce a result (Figure 30.5.1). Repositories are Git repositories — main moves. Pin the commit:

from huggingface_hub import snapshot_download

path = snapshot_download(
    repo_id="organization/model-name",
    revision="0123456789abcdef",          # an immutable commit, not main
    allow_patterns=["*.json", "*.safetensors"],
)

A small manifest records the model identity explicitly. In production, add file hashes, library versions, and an evaluation record:

from dataclasses import asdict, dataclass
import json

@dataclass(frozen=True)
class Artifact:
    repository: str
    revision: str
    task: str
    license: str
    parent: str | None = None

artifact = Artifact(repository="organization/model-name",
                    revision="0123456789abcdef",
                    task="text-generation", license="apache-2.0")
print(json.dumps(asdict(artifact), indent=2))

{
  "repository": "organization/model-name",
  "revision": "0123456789abcdef",
  "task": "text-generation",
  "license": "apache-2.0",
  "parent": null
}

The same pinning discipline applies to derived artifacts: a LoRA adapter without its base-model revision is incomplete, and a quantized conversion (GGUF, AWQ, ONNX — see Section 30.7) is a new artifact whose numerical fidelity someone should have checked against the source.

30.5.4.2 Trust and Licenses

Downloaded models are software supply-chain inputs with security and licensing implications:

Prefer safetensors — a pure tensor container. Legacy pickle-based checkpoints can execute arbitrary code on load, and the Hub’s scanners are a mitigation, not a guarantee.
trust_remote_code=True runs Python from the repository on your machine. Read and pin that code; use the flag only when the architecture genuinely requires it.
“Open” spans a wide range of licenses: permissive Apache/MIT weights, acceptable-use licenses with commercial thresholds, research-only releases, and gated models whose terms you accept per account. Record the license associated with the pinned repository revision, since repositories can relicense between revisions.
Every hub client caches aggressively (tens to hundreds of gigabytes in ~/.cache/huggingface is routine). Learn the cache tool’s scan and delete commands and monitor disk use.

30.5.5 Summary

Discovery is a skill: Hugging Face is the center for models and datasets, with ModelScope, Kaggle, Ollama’s library, and Civitai as the complements that matter; FineWeb-class corpora are the open pretraining baseline.
Papers with Code closed in 2025; arXiv and Hugging Face Papers provide paper discovery, while repository activity and reproduction reports help assess released code.
No leaderboard is trusted alone: triangulate Arena, LiveBench, task-specific benchmarks, and price/latency data — then decide on a small evaluation you built from your own task.
Use a small set of complementary sources, such as a community, a curated paper feed, and a newsletter.
Pin revisions, prefer safetensors, read licenses, treat remote code as code review, and manage your caches — treat external artifacts as supply-chain dependencies.

30.5.6 Exercises

Pick a task you care about and shortlist three models using at least two leaderboards plus Artificial Analysis. Where do the rankings disagree, and why might that be?
Inspect a model repository of your choice and list every file needed for offline inference. Which of them could execute code on your machine?
Build a 25-example evaluation set for a task you know well and run your shortlist from Exercise 1 on it. Does your ranking match the leaderboards’?
Find the license of a popular open-weight model and determine: may you deploy it commercially, fine-tune it, and redistribute the fine-tune?