Appendix C — Architecture 2.0 Resource Roles and Field Infrastructure
This appendix judges resources by the architecture work they advance. Links change, tools age, and benchmark versions move; the stable question is which part of that work a resource serves. Does it help formulate a question, explore meaningful alternatives, implement a checkable artifact, evaluate a claim, test a mechanism explanation, or defend a bounded decision? What interfaces, evidence, and failure modes does it expose, and what stronger claim remains outside its evidentiary reach?
The specific examples named below are a snapshot. The durable content is the role each resource plays; the current list of tools, datasets, and benchmarks is the kind of fast-moving record that belongs with the community forming around this topic, where it can stay current without reprinting the book. This appendix keeps the stable framework for judging whether a resource belongs in a bounded study or field-building stack and points readers to the live registry for the moving list. Named tools, datasets, and benchmarks that appear in this appendix without a bibliography citation, such as OpenROAD (an RTL-to-GDS flow), MLCommons (an engineering consortium), VerilogEval (a hardware code generation benchmark), and CVDP (Comprehensive Verilog Design Problems), are tracked with current links in that registry rather than the reference list.
Architecture 2.0 resource. A resource is useful for Architecture 2.0 when it advances one or more capabilities in a bounded architecture study, makes an architectural claim easier to test or review, or supplies shared infrastructure that lets studies become comparable and cumulative.
Read each row of the first-pass resource-role rubric in Table C.1 as an adoption test; the examples are representative rather than exhaustive. Name what the resource carries, which capability or claim it advances, what failure it makes visible, and what claim it cannot support. A synthesis wrapper that still requires manual log interpretation, for example, is missing a parsed feedback channel even if it exposes a tool API.
| Resource family | Representative examples | Capability and claim contribution | Adoption boundary |
|---|---|---|---|
| Architecture corpora and QA | Paper/manual corpora, publication records derived from DBLP (a computer science bibliography database), and QA (question answering) and reasoning data modeled after QuArch (an architecture question-answering dataset) (Prakash et al. 2025b, 2025a). | Support formulation by locating prior claims and architecture vocabulary, and support explanation by grounding mechanism hypotheses and evidence limits. | Paper text rarely preserves simulator state, failed candidates, tool logs, or review judgment. |
| Workloads and benchmarks | XRBench (a mobile extended reality benchmark), MLPerf (an industry-standard machine learning benchmark), and maintained benchmark suites (Kwon et al. 2023; Mattson et al. 2020; Reddi et al. 2020). | Bound formulation and evaluation through workload state, scenarios, metrics, rules, and comparable baselines. | Coverage, drift, update policy, and proxy validity must remain visible. |
| Evaluation harnesses and environments | ArchGym-style environments (such as an open-source gymnasium for architecture search), benchmark harnesses, simulator wrappers, and tool-calling APIs (Krishnan et al. 2023). | Connect implementation and evaluation to valid actions, observations, feedback cost, logging, and visible failure behavior. | A wrapper can hide tool semantics, unsupported actions, nondeterminism, and failure modes. |
| Mapping and DSE frameworks | Timeloop and MAESTRO-style mapping and dataflow tools (analytical models for accelerator evaluation) (Parashar et al. 2019; Kwon et al. 2019). | Expose architecture search spaces and constraints for exploration, then supply modeled observations for bounded evaluation. | Fast feedback is still a model; calibration, workload scope, and invalid candidates matter. |
| Compiler, autotuning, and codegen resources | AutoTVM and Ansor (tensor program optimizers), MLIR (a compiler infrastructure), and kernel-generation benchmarks (Chen et al. 2018; Zheng et al. 2020; Lattner et al. 2020; Ouyang et al. 2025). | Implement specialized hardware ideas through executable software paths and expose software-interface effects for evaluation. | A kernel, schedule, or IR (intermediate representation) result is not automatically a system-level architecture result. |
| RTL (register-transfer level) and verification benchmarks | Executable specification-to-RTL and Verilog design-problem tasks such as VerilogEval and CVDP (datasets for hardware code generation); see the living registry. | Test implementation claims against compile, simulation, and verification feedback. | Passing a small HDL (hardware description language) task does not establish a system-level architectural claim. |
| Full-system simulation and hardware/software harnesses | gem5 (a cycle-level architecture simulator), FireSim (an FPGA-accelerated simulation platform), Chipyard (an integrated SoC [system-on-chip] design framework), and related harnesses (Binkert et al. 2011; Karandikar et al. 2018; Amid et al. 2020). | Connect workload execution, software stacks, and generated hardware for implementation, evaluation, and mechanism testing. | Setup state, versions, nondeterminism, and workload coverage must be recorded. |
| Physical-design and EDA (electronic design automation) evidence | OpenROAD’s open RTL-to-GDS flow, CircuitNet (an open-source dataset for machine learning in EDA), ChiPBench (a benchmark for physical design), and placement or signoff-adjacent artifacts (Chai et al. 2022; Wang et al. 2025). | Evaluate candidates against physical constraints and bound which claims can survive into a defended recommendation. | Intermediate scores can mislead unless tied to downstream timing, area, power, or routability. |
| Benchmark governance and roadmaps | MLCommons rules, SPEC (Standard Performance Evaluation Corporation) suites, and roadmaps in the style of the International Roadmap for Devices and Systems (IRDS). | Maintain comparability, versioning, scenario definitions, and long-horizon evidence needed to defend claims across studies. | Governance belongs to shared field infrastructure; stale rules or hidden updates change what claims mean. |
| Evidence and provenance artifacts | Design-loop cards, released card or receipt schemas, source records, seeds, configs, tool logs, calibration records, integrity manifests, disclosure boundaries, telemetry packets, failed-run records, and rejected alternatives. | Connect formulation, execution, evidence, mechanism interpretation, and decision so a claim can be inspected and defended; suitable runnable receipts can additionally support replay or reproduction. | A schema or directory shape does not establish that the underlying evidence is complete, independent, or available. |
For a proposed resource, record its study capability or artifact, version and owner, exposed actions or feedback, visible failure modes, and the strongest claim it can help support. A missing contribution does not make the study useless; it narrows the claim. The complete bootstrap checklist belongs to Appendix A, while Appendix B owns the full card and review rubric. This appendix evaluates resources that can fill those fields rather than defining a third study checklist.
Chapter 4 sizes the public architecture-text surface with a deliberately simple scale check and sends questions about mining and maintaining that surface here. The public code surface behind the corpora and RTL rows of Table C.1 is sized in Table C.2. Broad language counts are large enough to seed artifact mining, yet each curation signal collapses them by one to three orders of magnitude, and none of them recovers the loop state that Chapter 4 shows is missing. The bottleneck is curation and linked study records, not collection.
| Search proxy | June 2026 snapshot (rounded) | What it shows |
|---|---|---|
| Verilog and SystemVerilog language counts | ~140,000 and ~43,000 repositories | A large public RTL-adjacent surface that includes small, toy, forked, stale, and non-architecture projects. |
| Language counts plus an “rtl” keyword | ~4,300 and ~1,700 repositories | A narrower proxy for RTL-oriented repositories, sensitive to keyword choice and unvalidated. |
| computer-architecture and fpga topic tags on Verilog repositories | ~350 and ~1,800 repositories | Voluntary curated labels are far smaller and less even than raw language counts. |
C.1 Missing Infrastructure
Chapter 4 shows why architecture data need represented meaning and linked trajectories rather than isolated artifacts. Chapter 10 then argues that shared infrastructure must make claims comparable, reviewable, teachable, and, when promised, replayable or reproducible. Table C.3 uses the eight layer names from that chapter and turns them into a concrete inventory.
A repository and an adjudication institution are not the same resource. A repository can store datasets, code, reports, and failed runs. It does not by itself define neutral baselines, control conflicts of interest, grant access to protected evidence, audit accepts and rejects, retire obsolete versions, or decide what resolves a disagreement. A community laboratory, consortium, or federated protocol can perform those functions when its ownership, rules, versions, disclosure boundaries, and appeal path are explicit. The protocol may use a repository, but the repository cannot substitute for the protocol.
| Missing infrastructure | Minimum concrete contents | Capability added | Failure if absent |
|---|---|---|---|
| Datasets, benchmarks, and paired multi-fidelity trajectories | Versioned architecture questions, baselines, workloads, candidate and evidence IDs, proxy and stronger-tool observations, tool and model versions, costs, uncertainty, failed stages, mechanism interpretations, bounded findings or decisions, redaction metadata, and alignment across fidelity levels. | Supports formulation, exploration, evaluation, and explanation while calibrating when cheap feedback predicts stronger evidence or reverses. | Isolated artifacts or log dumps cannot show which question was asked, when a proxy stopped tracking the design, why the result occurred, or what decision followed. |
| Versioned environments and tool contracts | Action and observation schemas, legal and invalid states, cost and fidelity labels, tool versions, dependencies, nondeterminism controls, failure semantics, proprietary boundaries, and deprecation rules. | Makes implementation and method evaluation comparable while making tool drift diagnosable. | Wrappers hide changed semantics, unsupported actions, or failures that alter the result. |
| Methods and reference practices | Role-appropriate AI and conventional methods, matched no-AI baselines, AI-role ablations, declared feedback budgets, experimental protocols, failure analysis, and reference reporting practices. | Lets a study attribute gains to the method, representation, tool path, or added feedback rather than an unmatched setup. | A method appears better because it receives more feedback, different baselines, hidden repair, or a more favorable environment. |
| Evaluation and neutral adjudication | Held-out tasks, cross-tool calibration, uncertainty and total-cost reporting, independent execution or authorized review, common challenge procedures, matched audits of accepts and rejects, conflict disclosures, retest rules, escalation, and appeal. | Lets a community compare methods and narrow or settle bounded disputed claims under shared rules. | A repository collects competing artifacts, but no accepted process can adjudicate them. |
| Reporting and claim interfaces | Compact study indexes, explicit claim and non-claim summaries, claim-to-evidence mapping, configs, seeds, commands, logs, failed and rejected alternatives, mechanism tests, integrity and disclosure manifests, accountable decisions, and runnable receipts when replay is claimed. | Supports inspection by default and replay or reproduction when the released packet is sufficient. | A plausible result has no reviewable path back to the question, execution, explanation, or decision. |
| Standards, governance, and community institutions | Stable interface and evidence semantics, named maintainers, version ownership, integrity checks, waiver expiration, drift monitoring, revalidation, retirement, incident-driven reopening, conflict rules, and ownership transfer. | Keeps records and evaluations meaningful across changing tools, models, workloads, and maintainers. | Links decay, rules drift silently, and obsolete evidence continues to be treated as support for current claims. |
| Incentives and confidentiality | Publication and artifact credit, benchmark status, industrial value for neutral evaluation, protected evaluation paths, disclosure rules, and credit for useful failure records. | Makes durable evidence contribution worthwhile while respecting legitimate intellectual-property and disclosure boundaries. | Teams rationally withhold failures and protected evidence, leaving public comparisons incomplete or distorted. |
| Education | Courses, executable labs, capstones, review exercises, and mentoring aligned to formulation, exploration, implementation, evaluation, explanation, and defense, including mechanism testing, AI-contribution evaluation, redaction, and accountable delegation. | Lets new architects conduct and challenge bounded AI-assisted studies rather than merely learn the vocabulary. | Tacit practice remains private, tools become recipes without judgment, and learners cannot transfer the discipline to a new problem. |
Not every project needs every row. At field scale, however, a missing layer bounds which cross-project comparisons and cumulative claims the community can support. The living registry can track current implementations of these roles. It cannot make missing governance or adjudication capacity exist.
C.2 Stable Book Material and Executable Practice
The stable material stays in the book because its role does not depend on a current tool URL.
- Bootstrap playbook. Appendix A owns the sequence for starting a bounded architecture study.
- Design-loop card and review rubric. Appendix B and Table B.3 provide the reusable review artifact.
- Constructed loop turn. Chapter 8 demonstrates loop mechanics with explicitly assigned proxy and stronger-stage values. It does not report a SCALE-Sim, synthesis, or silicon result.
- Runnable companion labs. Architecture 2.0 labs are a separate executable practice path. The SCALE-Sim lab produces its own tool-backed observations and receipt; it neither reproduces nor replaces the constructed loop turn in Chapter 8.
C.3 Living Registry Boundary
This appendix is not a general computer-architecture, pedagogy, or reproducibility directory. Stable resource roles and adoption boundaries stay here. Current tool, dataset, benchmark, and project links belong in the searchable registry at arch2.mlsysbook.ai/tools/, where maintainers can update or retire them without revising the book. Check versions before use and cite the primary paper or project artifact for scholarly claims.
The Architecture 2.0 GitHub repository accepts registry contributions and executable artifacts. Submission to a repository is not endorsement, independent evaluation, or adjudication. Those claims require the governed institution or protocol described in the missing infrastructure inventory.
C.4 Using and Citing This Book
Use this book as a working vocabulary for paper reviews, project proposals, class discussions, and internal design reviews. Cite the book and its edition when referring to this framework, and cite the primary resource when making a claim about a tool, dataset, benchmark, or method.
This site carries a pre-publication preview. When citing the preview rather than the published edition, include the site URL and the version identifier the site displays.