Appendix B — Design-Loop Card and Review Rubric
This appendix exists because Architecture 2.0 work will often arrive as papers before it arrives as stable infrastructure. The field is moving quickly, so a paper’s result is not the whole review question. What matters is whether the reader can reconstruct the bounded study, review its architectural claim, and judge what the evidence supports and which accountable role may authorize the next action.
The design-loop card is a compact practical index into the Architecture 2.0 study framework. The public contract retains the design-loop-card name, but the card indexes a bounded study whether or not its work iterates. It is a supporting instrument, not a definition of the discipline. The card keeps the study scope, claim boundary, execution summary, and pointers to project evidence in one adaptable summary while leaving detailed observations in a supporting evidence record and replay bindings in a runnable receipt. It is meant to be filled in for a paper, a project proposal, a class exercise, or an internal design review. The card does not replace the architecture summary or the supporting records. It indexes the study behind the report. It shows what the system is trying to do, what it can see, what it can change, how feedback is obtained, what evidence supports a claim, what was rejected, and which accountable person or organizational role holds the declared decision right.
The card should be short enough to use. If it becomes a long form, people will not fill it in. If it is too vague, it will not reveal anything. The right level is one page for a first pass and a few supporting notes for the fields where evidence is disputed.
B.1 Why a Card, Not a Paper Summary
A conventional paper summary usually asks for the problem, method, result, and limitations. That is useful, but it often hides the study structure. It may not say what simulator state was assumed, which actions were illegal, how many samples were spent, what alternatives failed, how a proxy was calibrated, or what could have rejected the result. Those omissions matter more once AI systems begin to generate candidates, call tools, choose experiments, or summarize evidence.
The card borrows the reporting discipline of a datasheet, both the component datasheet architects already trust and the “datasheets for datasets” practice that carried it into machine learning (Gebru et al. 2021), without pretending a paper is only a dataset. It also borrows the discipline of compact disclosure from model reporting. Assurance cases in Goal Structuring Notation (GSN, a structured framework for safety argumentation), Architecture Decision Records (a software-engineering record of design rationale), and reproducibility and reporting checklists carry the same compact-disclosure lesson into safety claims, design decisions, and empirical review. Benchmarking and supply-chain records add two more lessons. Claims need scenario rules and versioned provenance, while machine-readable software bills of materials such as SPDX (Software Package Data Exchange) and provenance or attestation records compatible with SLSA (Supply chain Levels for Software Artifacts) should be linked when they exist rather than recreated inside the card (SPDX Project 2021; Open Source Security Foundation 2026).
The design-loop card adapts that pattern to architecture work. It gives a reviewer a compact route into the study, requiring explicit disclosure of the claim, scope, execution path, evidence, failed alternatives, rejection checks and authority, evidence-supported claim boundary, and accountable organizational decision. The card does not make the result credible by itself.
The card captures the architectural intent being translated into work, the bounded task, and the design-space boundaries, including what is legal, invalid, or deferred. It indexes the representation and world model that make the work legible, along with the environment defining valid actions and feedback. It also records the method role, available evaluation resources, evidence supporting the claim, and failed runs or rejected alternatives captured during exploration. Finally, it identifies the rejection checks and authority, the evidence-supported claim boundary, and the accountable decision rights that remain.
The card is useful in three settings:
- Research: It helps compare papers that may use different methods on similar studies.
- Design review: It reveals whether a result has enough evidence for the commitment being made.
- Artifact review: It gives authors and reviewers a disciplined way to read Architecture 2.0 work without reducing it to a list of model names.
The borrowed pattern, summarized in Table B.1, does not mean an architecture team must fill out all of these precedent records for every artifact. It is a reminder that the card’s fields have jobs to bound a claim, make hidden assumptions visible, connect evidence to commitment, and give a reviewer a way to find omissions.
| Precedent | What it makes visible | What the design-loop card borrows |
|---|---|---|
| Model cards (Mitchell et al. 2019) | Intended use, evaluated conditions, limitations, and risks. | Claims should name the conditions under which the reported behavior was evaluated and where it should not be used. |
| Datasheets for datasets (Gebru et al. 2021) | Motivation, provenance, composition, collection, maintenance, and recommended uses. | Workload, trace, benchmark, and artifact records need provenance and usage limits, not only final metrics. |
| System cards (Meta AI 2022) | System configuration, component models, evaluations, operation, and mitigations. | A system-level record can describe what is deployed; the design-loop card records how an architecture decision moves from bounded action to evidence and commitment. |
| Assurance cases and GSN (Kelly and Weaver 2004) | Claim, context, argument strategy, evidence, defeaters, and residual risk. | A study record should say why the evidence supports the claim, what cuts against it, and what uncertainty remains. |
| Architecture Decision Records (Nygard 2011) | Context, decision, alternatives, and consequences. | A design claim should preserve the rejected or deferred alternatives that make the decision meaningful. |
| Artifact evaluation and reporting checklists (Association for Computing Machinery 2020; Pineau et al. 2021; Page et al. 2021) | Whether a submitted artifact is documented, complete, exercisable, and consistent with its paper. | Missing fields should weaken or bound the claim rather than disappear inside prose. |
| Benchmark governance and provenance records (Mattson et al. 2020) | Versioned scenarios, run rules, inputs, dependencies, and comparable reporting. | A study claim should carry workload versions, tool versions, evidence IDs, and pointers to deeper artifacts where needed. |
These records are complementary. A system card explains the configuration, evaluations, and mitigations of an operating AI system. Artifact evaluation checks a submitted artifact against the associated paper and its reported results. The design-loop card begins earlier. It records the state and allowed actions before a method acts, preserves failed runs and rejected alternatives, identifies rejection checks and authority, and states which transition from evidence to commitment is authorized. It can link to a system card or an artifact-evaluation report rather than replacing either one.
The operating pattern in Figure B.1 is to fill the card, apply the pass/warning review lens, and separately record the evidence-supported claim boundary, organizational authorization, and structural conformance level. The point is not to grade the prose of a paper. The point is to expose whether the study behind the claim is visible enough for another architect to judge.
The review outputs in the figure are not an additional status vocabulary. Use the rubric’s pass and warning signals, then record what the evidence supports, what the accountable role authorizes, and which structural conformance level the card satisfies. Organizational authority cannot strengthen the evidence, and evidence alone does not grant authority to act.
B.2 The Design-Loop Card Fields
The working card in Table B.2 orders its fields to match the study framework used throughout the book, and their display names are canonical wherever the card appears. They define the released card’s structural index, not the minimum content of a sound architectural review. When the card is used to review a claim, the Intent entry should state the explicit claim and non-claim under intent.claim_boundary, even though version 1.1 of the card’s machine contract, given later in this appendix, leaves that block optional.
| Field | Question |
|---|---|
| Intent | What architectural objective is being pursued, what exact claim and non-claim bound the review, and what constraints, non-goals, risks, or deployment assumptions matter? |
| Task | What bounded work is the study doing? Examples include generation, prediction, optimization, critique, verification, workload characterization, benchmark construction, or design-space exploration. |
| Design space | Which choices are legal, invalid, out of scope, or intentionally left for a later turn? |
| Representation | What can the study’s participants know, read, write, or assume? What state, dynamics, objectives, constraints, costs, and uncertainties are represented? |
| Environment | What can the methods act on, observe, and measure? Which actions are invalid, expensive, nondeterministic, or irreversible? |
| Method role | Is the method generating, predicting, optimizing, critiquing, verifying, planning, calling tools, coordinating, or combining several roles? |
| Feedback budget | How many evaluations are available, at what latency, cost, fidelity, model-side expense, human attention, and sample efficiency requirement? |
| Evidence | What supports the claim, and against which architecture and method baselines is it measured? When benefit from AI is claimed, where is the matched conventional or no-AI comparison or AI-role ablation? What mechanism hypothesis explains the result, and what sensitivity, intervention, or ablation tests it? Sources can include baseline replay, proxy metrics, simulation, synthesis, verification, telemetry packets, silicon data, expert review, integrity manifests, or sensitivity analysis. |
| Failed runs / rejected alternatives | What failed, was rejected, violated constraints, crashed tools, disappeared at higher fidelity, or was ruled out by an accountable review? |
| Rejection checks and authority | What result counts against the candidate or claim, which test, tool, rule, or review checks it, and what rejection or escalation follows? Examples include constraint and type checks, CDC/RDC (clock and reset domain crossing) errors, routing congestion, compiler fusion failure, formal tools, cross-tool disagreement, deployment signals, or expert review. A simulator crash is an environment failure unless a predeclared rule makes that outcome disqualifying. |
| Evidence-supported claim boundary | What is the strongest claim the evidence supports, what stronger claim remains unsupported, and what evidence would overturn the judgment? |
| Accountable decision | Which person or organizational role holds the declared authority, and what action or organizational commitment does that role authorize? |
The explicit claim, comparator, and mechanism questions above are human review requirements, not additional v1.1 schema fields. Record the claim and non-claim within Intent, and point from Evidence to the matched comparison and tested mechanism in the supporting study record. The released keys for the final four human-facing fields are negative_traces, rejection_authority, commitment_boundary, and human_decision. The commitment_boundary key records an evidence judgment. The human_decision key records organizational authorization. Neither can substitute for the other.
The released machine contract retains the key negative_traces. In ordinary language, this appendix calls those entries failed runs and rejected alternatives. In v1.1, each entry requires a reason, stage, and gate. A candidate_id becomes required at Level 2 and above. That is enough to preserve a concise disposition pointer, but it does not establish that failures were collected comprehensively or interpreted correctly. The released v1.1 contract groups rejection information under rejection_authority.gates, which is an array of free-form strings. A validator cannot distinguish the observable condition, the check that evaluates it, the actor or policy with authority, or the resulting disposition. A success-only record cannot show what the study ruled out.
B.3 Blank Template
The one-page blank form in Table B.3 is sufficient for a first pass because it prompts the study owner to name the task, design space, evidence, rejection path, evidence-supported claim boundary, and decision owner before expensive execution. For claim review, the Intent entry must also state the claim and non-claim; the blank table keeps that binding inside Intent to remain compatible with the twelve-field machine contract.
| Field | Entry |
|---|---|
| Intent | |
| Task | |
| Design space | |
| Representation | |
| Environment | |
| Method role | |
| Feedback budget | |
| Evidence | |
| Failed runs / rejected alternatives | |
| Rejection checks and authority | |
| Evidence-supported claim boundary | |
| Accountable decision |
Ready-to-fill versions and the canonical machine contract are available with the public companion materials.
- YAML template: Download the machine-checkable blank card.
- Markdown template: Download the one-page review form.
- Synthetic card example: Inspect a Level 3 schema and replay fixture. It teaches the contract with a deterministic estimator; it is not an empirical architecture result.
- Separate lab using SCALE-Sim, a cycle-accurate systolic-array simulator: Generate and review tool-produced evidence. The lab is an optional activity outside this appendix; it preserves raw reports, failed runs and rejected alternatives, and a runnable receipt.
- 30-minute Level 0 workflow: Draft and validate context and boundaries. Audit, replay, and independent review are later steps that require existing records.
After filling in the card, apply the final review questions.
Architect’s checkpoint: The Six-Capability Review
Before relying on a filled card, test the study against the book’s six learner capabilities.
- Formulate. Does the study bound the architecture question, baseline, claim and non-claim, objectives, constraints, and non-goals?
- Explore. Does it compare meaningful alternatives, justify the AI or conventional method roles, and preserve serious candidates that failed or were rejected?
- Implement. Does it connect selected alternatives to checkable artifacts, represented state, and the real tool path needed to test the claim?
- Evaluate. Do representative workloads, matched architecture and method baselines, justified fidelity, uncertainty, counterevidence, and total feedback cost support the claim? When benefit from AI is claimed, is there a matched conventional or no-AI comparison or AI-role ablation?
- Explain. Does a mechanism hypothesis connect the result to architecture, workload, and software behavior, and has a sensitivity, intervention, or ablation tested it?
- Defend. Is the recommendation bounded by tradeoffs, evidence limits, reversal conditions, and an evidence-supported claim boundary? Is the person or organizational role authorized to take the next action identified separately?
If those questions cannot be answered, the project may still be promising, but its architectural claim is not yet reviewable.
B.4 A Machine-Checkable Schema
The card is a review index first, but it also has a machine-checkable record so tools can validate its structure, supporting records can reference it, and declared fields can be compared. Semantic comparison of claims additionally requires an explicit intent.claim_boundary and evidence linked to that claim. JSON Schema version 1.1 is the canonical machine contract. The outline below mirrors its field structure for readers. The contract names the required fields and gives stable IDs for the records a study must preserve, reuses the commitment levels of Table 7.4, and compresses the fidelity ladder of Section 7.2 into eight machine-checkable rungs. It is a minimum, not a standard to freeze; the point is that “we filled the card” becomes something a validator can check. The fidelity enum is illustrative. synthesis_with_sdc means synthesis with Synopsys Design Constraints, an industry-standard format for hardware timing requirements, while shadow_traffic, canary (testing on a small live subset), and global_fleet name deployment-exposure rungs rather than architecture signoff stages.
schema_version: "1.1"
design_loop_card:
card_id: string
conformance_level: { enum: [0, 1, 2, 3] }
intent:
objective: string
constraints: [string]
non_goals: [string]
risks: [string] # optional
deployment_assumptions: [string] # optional
claim_boundary: { claim, non_claim } # optional
task:
enum: [generation, prediction, optimization, critique,
verification, characterization, benchmark, dse]
design_space: { legal, invalid, deferred }
representation:
state_schema_id: string
ir_level: string
reads: [string]
writes: [string]
uncertainties: [string]
environment:
environment_id: string
actions: [string]
invalid_actions: [string]
blast_radius_limit: string
observations: [string]
fidelity:
enum: [proxy, simulation, rtl, synthesis_with_sdc,
silicon, shadow_traffic, canary, global_fleet]
method_role:
roles:
- enum: [generate, predict, optimize, critique,
verify, plan, tool_call, coordinate]
actor_map:
- { actor_id, role, reads, writes, authority }
feedback_budget:
evaluations: integer
latency: string
cost: string
fidelity: string
model_side_cost: # optional
model_calls: integer
gpu_hours: number
energy_or_carbon: string
human_review: string
evidence:
baseline_id: string # comparison baseline
records:
- evidence_id: string
kind: string
workload_id: string
seed: integer_or_string
provenance:
tool_version: string
parameter_hash: sha256:<64-lowercase-hex-digits>
source_uri: uri-reference
resource_or_sustainability_boundary: # optional
operational_energy: string
embodied_or_manufacturing_scope: string
geography_or_time_basis: string
utilization_or_amortization_basis: string
telemetry_packet: # optional
hardware_stepping: string
software_stack: string
deployment_version: string
cohort: string
sampling_policy: string
privacy_filter: string
canary_or_rollback_label: string
incident_context: string
decision_owner: string
integrity_manifest: # conditional
dependencies: [string]
model_ids: [string]
prompt_ids: [string]
artifact_hashes: [string]
attestations: [string]
negative_traces:
- { candidate_id, reason, stage, gate }
disclosure_boundary: # optional
data_classes: [string]
redactions: [string]
reviewer_roles: [string]
release_boundary_or_compliance_review_id: string
rejection_authority:
gates: [string]
independent_of_producer: boolean
independence_basis: string
commitment_boundary:
level:
enum: [exploratory, experimental, implementation,
integration, irreversible]
strongest_supported_claim: string
would_overturn: string
human_decision: { owner, authorizes }Which fields are required depends on the structural conformance level the card claims (defined below). The conformance_level field records that claim. Level 0 requires card_id, conformance_level, intent, task, and design_space; Level 1 adds representation, environment, method_role, feedback_budget, evidence (with a baseline_id), and negative_traces; its load-bearing arrays, actor map, evidence records, and negative_traces must be nonempty, and its feedback budget must permit at least one evaluation. Level 2 adds state_schema_id, environment_id, evidence_id, workload_id, seed, and candidate_id, plus provenance containing a tool version, SHA-256 parameter digest, and replay source URI. Level 3 adds rejection_authority (independent of the producer), commitment_boundary, and human_decision. The independence_basis field records the claimed basis, such as a separate reporting chain or formal verification tool.
Level 3’s independence condition is consequence-dependent, not a universal maturity rank. Independent rejection is especially important when a decision is hard to reverse, crosses organizational boundaries, or carries high blast radius. A Level 2 card may be structurally appropriate for a replay-oriented exploratory loop, while a Level 3 card can still contain a weak or correlated independence claim. The level reports which contract fields are present, not how good the project is.
The claim and non-claim, when needed for a paper review, live under intent.claim_boundary; they do not create a thirteenth top-level card field. In v1.1 that block is optional. A card can therefore pass schema validation without stating an explicit claim and non-claim. Likewise, a Level 1 evidence.records entry requires only kind. Level 2 requires identification and provenance fields, but the record can still contain metadata only. Neither level requires an observed value, acceptance result, uncertainty, or explicit link from the observation to the claim. Schema validity does not establish that the observation supports the claim, that the evidence is adequate, that replay succeeded, or that a declared rejection path is actually independent.
The v1.1 task field validates only a task category, such as generation, verification, or design-space exploration. It cannot validate the bounded task description requested by the human-facing card. For claim review, that description must remain visible in the accompanying study record. A future schema can add it without pretending the current enum already captures it.
These are versioned contract limitations. The released v1.1 schema must remain stable. A future schema revision should require an explicit claim binding and richer claim-to-evidence and independence semantics where machine checking is intended. It should not silently strengthen v1.1 in place.
The role vocabulary is one more such limitation, and it deserves an explicit map. The eight-role teaching taxonomy of Chapter 6 is canonical for the book’s prose; the v1.1 method_role.roles enum is canonical for the machine contract and cannot name two of those roles directly. Table B.4 records how a card encodes each teaching role with the released enum, so tooling built against the schema and studies described in the chapter vocabulary remain mutually intelligible.
plan and tool_call, record work sequencing and tool invocation, loop mechanics beneath the teaching taxonomy rather than additional method roles. A future schema revision can add explicit repair and explain values without weakening released v1.1 cards.
| Teaching role | v1.1 enum encoding |
|---|---|
| Generation | generate |
| Prediction | predict |
| Optimization | optimize |
| Critique | critique |
| Repair | generate plus critique in the actor map, a bounded correction generated under the critique that motivated it. |
| Verification | verify |
| Explanation | critique, applied to make evidence, tradeoffs, and limits legible rather than to find faults. |
| Coordination | coordinate |
The optional extension paths are named explicitly in the schema above rather than implied by prose. Most cards should leave these blocks empty. They become necessary only when the claim depends on large-model cost, field evidence, resource or sustainability assertions, supply-chain integrity, mutable external evidence, or protected design state. The release_boundary_or_compliance_review_id field records the result or pointer to an authorized release, IP, or export-control review. The card does not perform that classification itself. The stable IDs (card_id, workload_id, evidence_id, candidate_id, and the other *_id fields) let one card’s supporting records reference another’s, so a rejected candidate, a workload packet, or a baseline can be cited across cards rather than re-described. For a single-actor, workflow, or multi-actor loop, actor_map records the component, tool, or human participant playing each role, what state it reads and writes, and what authority it has. Simple loops may have one entry; split-role loops may have many.
The legacy key human_decision records an accountable owner and what that owner authorizes. The owner may be a named person or an organizational role with a declared decision right. The key does not mean every parser, test, simulator, or routine gate requires manual approval. It makes accountability visible for the commitment or waiver the card says remains authorized.
B.5 The Review Rubric
The review rubric asks whether each field is strong enough for the claim being made. The standard should rise with commitment. A speculative idea can survive with weak evidence if it is labeled as such. A tool recommendation, RTL (Register Transfer Level) hardware change, or physical-design decision needs a stronger supporting evidence record. The rubric in Table B.5 makes that standard inspectable. It separates a pass signal from a warning sign so review can focus on evidence, rejection, and commitment rather than polish.
| Field | Pass signal | Warning sign |
|---|---|---|
| Intent and task | The task is bounded, measurable, and tied to an architectural objective. | The task is “use AI” or “generate a design” without a clear decision boundary. |
| Design space | Legal, invalid, and out-of-scope choices are named before the method acts. | The method can wander into unreviewed choices or silently exclude alternatives. |
| Representation | The study exposes the state needed to make valid architectural actions. | Important constraints live in hidden scripts, defaults, or informal assumptions. |
| Environment | Actions, observations, invalid states, costs, and provenance are defined. | The tool wrapper returns numbers but hides semantics, failures, or version state. |
| Method role | The method’s job is clear and matched to the feedback budget. | The method is chosen because it is fashionable, not because the task needs it. |
| Feedback budget | Latency, fidelity, sample count, and cost are explicit. | Claims ignore simulator time, EDA cost, expert review, or license limits. |
| Evidence | The evidence is relevant to the claim and calibrated to the commitment level, with a matched conventional or no-AI comparison when AI benefit is claimed and a tested mechanism account when explanation is part of the claim. Resource or sustainability basis is included when material. | A proxy metric is treated as truth without validation or uncertainty, AI benefit lacks a matched comparator or ablation, the mechanism account is untested, or carbon and sustainability claims lack time, geography, utilization, or amortization basis. |
| Failed runs / rejected alternatives | Failed candidates and rejected alternatives are captured with reasons. | Only successful runs are recorded. |
| Rejection checks and authority | The record states what result counts against the candidate or claim, which check applies, who or what may stop advancement, and what happens next. | There is no clear way to challenge or stop a plausible but invalid result. |
| Evidence-supported claim boundary | The strongest supported claim and the evidence that would overturn it are explicit. | A proxy result is treated as support for a stronger claim than it can justify. |
| Accountable decision | The person or organizational role, declared authority, and organizational action or commitment are explicit. | The record treats evidence as self-authorizing or implies that the method decides, but no role owns the action. |
The rubric is not a numerical score and does not add a third readiness status. Reviewers use the pass and warning columns to record a field-level judgment and explain its consequence for the claim. The card records the resulting evidence-supported claim boundary and accountable organizational decision, while conformance levels report structure only.
The most important review question is not whether every field is perfect. It is whether the study exposes enough structure for another architect to judge, challenge, reject, or extend the work. Attempted replay requires a separate runnable receipt.
B.6 Card Conformance Levels
A card can carry increasing structural bindings, and naming the level turns “we used the card” into a checkable contract claim rather than a gesture. Each level inherits the required structure below it. The rubric’s pass and warning columns judge content. Conformance levels report only which required fields and bindings are present.
- Level 0, context only. Card ID, conformance level, intent, task, and nonempty design-space boundaries are named. Enough to understand what the study was for; not enough to audit it.
- Level 1, inspectable study bindings. Adds nonempty representation, environment, actor map, feedback budget with at least one evaluation, evidence with a named baseline, and
negative_traces. A reader can inspect concise observation and rejection summaries and identify what needs deeper review in the supporting evidence record. - Level 2, replay bindings present. Records stable IDs and provenance bindings for workload, seed, tool version, SHA-256 parameter digest, and replay source URI. Those bindings are needed to locate inputs and attempt replay. The runnable source and replay result must be checked separately.
- Level 3, independence and decision fields present. Records declared rejection checks under
rejection_authority.gates, a claimed independent authority and its basis, an explicitcommitment_boundaryentry, and a named accountable owner underhuman_decision. Reviewers must still verify that the authority is genuinely independent of the producer; a boolean field cannot establish that relationship. This level is appropriate when the consequence requires an independent rejection path. It is not a universal maturity rank.
The contract determines which fields are required at each level. Optional fields may be omitted. When content is withheld, the disclosure boundary should name what was redacted, who may review it, and which public claims the redaction cannot support. Publish a hash only where the contract requires an integrity binding or a stable artifact can be safely bound. A hash is not universal proof of hidden content. A missing required field does not weaken conformance incrementally; it caps the whole card at the highest level whose required fields are all present.
Across all levels, schema validation checks required structure only; as the v1.1 limitations above make explicit, evidence adequacy, replay success, actual independence, and the authorized commitment remain separate judgments.
B.7 Using the Same Card in Different Contexts
The design-loop card is a compact summary and index into a broader review packet. Detailed observations belong in a supporting evidence record. Runnable inputs, environment bindings, commands, and outputs belong in a runnable receipt. Research papers and case studies use the same card. They emphasize different fields and may link different supporting records, but they do not create alternate card forms.
B.7.1 1. Using the Card for Research Papers
When publishing an Architecture 2.0 paper, the card answers the author and reviewer question that sits one level above the method. It asks what exact claim is being made, what evidence supports it, and which next action an accountable role authorizes.
Focus on:
- Claim boundary. The exact architectural claim and non-claim.
- Evidence pointers. The card names the baseline and summarizes supporting records. Detailed proxy, simulation, synthesis, signoff, hardware measurement, and uncertainty observations remain in the linked supporting evidence record.
- Mechanism and AI contribution. The supporting record states the architectural mechanism being tested and, when benefit from AI is claimed, includes a matched conventional or no-AI comparison or AI-role ablation.
- Failed runs / rejected alternatives. Crashes, invalid actions, constraint violations, and rejected alternatives. This prevents success-only reporting.
- Rejection checks and authority. The result that counts against the claim, the tests, tools, constraints, or reviews that apply it, and the declared disposition.
A paper may be technically interesting while leaving the study underdescribed. The review question is whether a reader can tell what was tried, what failed, what was measured, what could reject the claim, and what decision the evidence actually supports. How much of that evidence a paper can disclose is a separate question, taken up in the case-study discussion below.
B.7.2 2. Using the Card for Case Studies and Examples
When sharing a case study, benchmark, or industrial example, the same card summarizes the claim boundary and points to the detailed supporting evidence record. Together they turn a success story into a reusable design lesson.
Focus on:
- Task and action space. What the study was trying to do, and which actions were legal.
- Feedback and evidence pointers. What was measured, at what fidelity, with what provenance, and where the supporting evidence record can be inspected.
- Rejected alternatives. What failed or regressed at higher fidelity.
- Study judgment. The evidence-supported claim boundary and the accountable organizational decision the example demonstrates.
Not every project can disclose the same evidence. A useful report states four descriptive facts (Table B.6) rather than compressing them into one disclosure level. The access boundary, evidence depth, replay result, and independence of review are non-ordinal and non-exclusive. Confidential evidence can be replayed by an authorized reviewer, while public artifacts can still be incomplete or weak.
| Disclosure fact | What to state | Claim limit |
|---|---|---|
| Access and disclosure | Which records are public, restricted, confidential, or redacted; who may inspect protected material. | Access says nothing by itself about adequacy, replay, or independence. |
| Evidence depth | Whether the release contains context, a card, a supporting evidence record, or the underlying run artifacts, with explicit omissions. | Audit is bounded by the observations and provenance actually available. |
| Replay result | Whether replay was not attempted, attempted, or successful in a named environment, and who performed it. | A successful replay shows that the named packet ran in the declared environment; broader reproducibility or transfer needs its own protocol. |
| Review independence | Whether review was absent, performed by a separate internal role, or performed externally, including shared tools, data, and conflicts. | An independence claim extends only to the disclosed organizational and technical boundary. |
B.8 Paper-to-Study Exercise
To use the card in a reading group or class, choose a paper and fill in the fields before discussing the claimed result. The exercise usually reveals one of three patterns:
- Explicit study: The paper names the task, action space, environment, feedback budget, evidence record, rejection checks, and decision owner. Its claims are easier to explain and compare.
- Strong result with implicit study structure: The paper may report a better Pareto point or speedup without exposing the search budget, failed candidates, tool settings, or rejection checks. The card separates a useful artifact from a reviewable study.
- Broad claim from narrow evidence: A method may work for one benchmark, simulator, or proxy metric but be presented as a general design method. The card exposes the mismatch between claim scope and evidence scope.
A simple reading-group exercise is to assign two readers the same paper. One summarizes the paper conventionally. The other fills in the design-loop card. The group then discusses what the card exposed that the summary hid.
For the debrief, record how long the card took, which omission changed the evidence or commitment judgment, and, when replayability is claimed, whether the linked receipt supported an attempted replay. If the card changed no judgment and exposed no actionable omission, ask whether it earned its cost for this review.
B.9 Lighthouse Card Sketch
A deliberately incomplete card sketch for the book’s recurring mobile-XR/RISC-V lighthouse prompt appears in Table B.7. It is not a finished design. Filling even a first-pass card forces the one-sentence prompt to declare its workload slice, legal candidate space, feedback budget, and accountable owner, the loop state a generated answer would otherwise leave implicit.
| Field | Sketch |
|---|---|
| Intent | Improve efficiency for real-time mobile XR under strict power, memory, software, reliability, and deployment constraints. |
| Task | Bounded design-space exploration for a RISC-V-based compute subsystem, initially scoped to accelerator/memory organization for an XRBench-class (extended reality benchmark) workload slice. |
| Design space | RISC-V ISA options, vector width, memory hierarchy, accelerator coupling, compiler/runtime path, and voltage/frequency assumptions; technology, package, and deployment policy changes are outside this first turn. |
| Representation | Workload traces, architecture description, configurable memory/compute parameters, compiler assumptions, power model, latency targets, and uncertainty about workload drift. |
| Environment | Simulator or cost model plus workload harness, with actions such as changing vector width, memory hierarchy parameters, accelerator tiling, voltage/frequency assumptions, or dataflow choices. |
| Method role | For one loop turn, choose one bounded role, such as generating legal candidates, searching exposed parameters, predicting proxy outcomes, critiquing invalid assumptions, or summarizing evidence for accountable review. |
| Feedback budget | Many cheap proxy evaluations, fewer simulator evaluations, and only a small number of high-fidelity checks before the accountable decision. |
| Evidence | Pareto comparison over latency, energy, area proxy, memory traffic, software compatibility, and sensitivity to workload assumptions. |
| Failed runs / rejected alternatives | Configurations that violate the 3 W target, miss real-time latency, exceed memory bandwidth, require unsupported software, or fail at higher fidelity. |
| Rejection checks and authority | Declared validity and constraint checks, CDC/RDC (clock and reset domain crossing) violations, routing congestion, power/thermal limits, workload quality-of-service violations, compiler/runtime interface violations, or architect review, together with the disposition each check authorizes. Simulator and dispatch failures remain environment-failure traces unless the candidate violates a declared interface. |
| Evidence-supported claim boundary | Advance only to a stronger modeling or RTL-study question; overturn with higher-fidelity latency, power, software-path, or workload-coverage evidence. |
| Accountable decision | The named person or organizational role decides whether the candidate merits deeper modeling, different representation, stronger fidelity, or rejection. |
This sketch also shows why the book does not treat the lighthouse prompt as a one-shot generation request. The prompt is useful because it exposes the state that must be represented, not because it eliminates the loop.
B.10 Common Failure Modes
The card is most useful when it reveals failures early. Common failure modes include:
- Missing evidence. The claim is plausible, but the supporting measurement is absent, low fidelity, or unrelated to the decision.
- Missing failed runs and rejected alternatives. The study records only successful candidates, so future methods repeat known failures.
- Hidden simulator state. Defaults, flags, seeds, workload versions, and tool revisions are not recorded.
- Proxy mismatch. The method improves a metric that does not track the architectural objective, as defined in Chapter 5. For example, optimizing instruction throughput on a synthetic microbenchmark when the true objective is end-to-end tail latency.
- Untested mechanism explanation. The report offers a plausible account of why the result occurred but does not test it with a sensitivity, intervention, discriminating comparison, or ablation.
- Invalid action space. The generative method can propose configurations that cannot compile, simulate, synthesize, meet timing, or satisfy constraints.
- Unsupported autonomy. The method is allowed to make decisions whose commitment level exceeds the evidence available.
- No rejection check or authority. There is no explicit condition, check, or declared mechanism that can reject a plausible but wrong result.
- Unowned commitment. The study obscures who accepts risk and who remains accountable for the final decision.
These are not only documentation failures. They are failures in the study’s design and execution. A study that hides failed runs, rejected alternatives, invalid actions, or rejection checks is hard to improve because it cannot distinguish a weak candidate from a weak process.
Field note: Practical workflow tips for solo researchers
For failed runs and rejected alternatives, a local CSV of attempted parameters and the tool’s error code is a useful record, but it does not prove adequate exploration. Preserve the command, tool version, candidate identity, and reason the attempt failed. A segfault or compile failure is an environment failure to diagnose. It becomes a candidate-rejection check only when a predeclared rule makes that failure disqualifying for the candidate rather than evidence that the tool setup is broken. The goal is clarity and intellectual honesty, not compliance paperwork.
In a student project, the owner can be the student, advisor, reviewer, or project lead who accepts the claim boundary. The important step is naming who can reject, escalate, or stand behind the result.
For a separate, optional tool-backed receipt activity, follow the Architecture 2.0 labs. The labs exercise the card contract; they do not replace this appendix or change the canonical card.