5 Tools as Architecture Environments
“We shape our tools and thereafter our tools shape us.”
— John Culkin, Media Scholar
Author’s Note: John Culkin, an influential American media scholar, observed that our tools ultimately reshape our own behavior. In Architecture 2.0, Culkin’s “us” expands beyond the human architect; by wrapping existing EDA tools into programmatic environments for AI agents, we are fundamentally reshaping both the automated loop itself and what is possible to build.
The crux
What turns a tool flow into an environment a generative model can act in and a reviewer can trust?
Chapter 4 argued that a loop can only act on what it represents. This chapter asks where the loop acts. In Architecture 1.0, tools often sit behind the architect: simulators, scripts, compilers, profilers, spreadsheets, RTL flows, EDA tools, dashboards, and deployment logs are the means by which a human expert gathers evidence. In Architecture 2.0, those same tools must become explicit environments when they sit inside AI-assisted design loops. They define what actions are legal, what observations are returned, how expensive feedback is, which assumptions are baked in, what state is logged, what can reject a candidate, and what commitment the environment can support before the loop spends more effort.
Architecture environment. An architecture environment is the tool-facing part of the loop contract made executable. It maps the five-part execution state directly to the toolchain: it specifies the legal actions (actions allowed), observations and feedback costs (state seen), logged state and provenance (evidence), rejection authority (alternatives rejected), and commitment boundaries (ownership) for the tools a design loop acts through.
To keep these boundaries clear, we must use the tool vocabulary carefully. A tool performs an operation. A wrapper calls that tool. An environment contract states what actions are legal, what observations return, what costs and failures mean, and what evidence is logged. A harness is the maintained runnable object around that contract: workloads, versions, scripts, provenance, invalid-action records, and review state.
This shift from simple wrappers to formal environment contracts is easy to understate. A tool wrapper is not plumbing. It is a research claim about the architecture problem. It decides which parts of the design space are visible, which constraints are enforceable, which metrics are trusted, which failures are recorded, and which actions are silently impossible. A weak environment can make a strong method look useful by hiding the hard cases. A disciplined environment can make a modest method valuable by making the task bounded, repeatable, and rejectable.
The same contract is what makes comparison meaningful. Two AI-assisted methods can be compared only when they act under compatible workload definitions, action schemas, feedback budgets, invalid-action rules, provenance records, and commitment boundaries. Otherwise a result may compare private worlds rather than architecture methods.
The lighthouse prompt makes the point concrete. “Design a low-power, 64-bit RISC-V-based compute subsystem for an XRBench real-time mobile XR workload under a 3 W, 3 nm-class low-power mobile envelope” is not one call to one tool. It needs a workload harness, software stack, ISA and vector assumptions, candidate architecture representation, simulator or estimator, power model, compiler/runtime interface, validity checks, provenance log, and a way to record rejected alternatives. If any of those pieces are implicit, the prompt is not yet an Architecture 2.0 loop. It is only a sentence.
What this chapter gives you
After this chapter you can turn a tool path into a reviewable environment contract. That means you can:
- turn a tool into an environment with an explicit action, observation, cost, rejection, and commitment contract;
- read a result by naming the environment that produced it;
- explain why environment contracts make AI-method and paper comparisons meaningful;
- reason about feedback latency and fidelity as an economy of evidence;
- detect proxy mismatch and proxy gaming, simulators included, before they mislead the loop.
5.1 Tools Shape the Research Question
Computer architects have always used tools to reason quantitatively about systems. Simulators, analytic models, profilers, compilers, RTL generators, EDA flows, and measurement systems make design spaces tractable. They also shape the questions that can be asked. A simulator with a particular memory model makes some cache questions natural and others awkward. A compiler pass that exposes one schedule representation and hides another constrains what an optimizer can change. An EDA flow that returns timing and power after hours of work makes feedback precious. A runtime telemetry system that reports aggregate utilization but not per-request interference makes some deployment claims difficult to support. When tools are used by a human expert, some of these limits can remain tacit. For an AI-assisted loop, tacit limits become hidden state. The loop can act safely only on constraints, costs, and rejection rules the environment exposes.
The usual way to describe these tacit boundaries is to say that tools have limitations. That is true, but too weak. Tools do not merely limit architecture work; they define its observable world. They decide what state exists for the loop, what actions can be applied to that state, and what feedback comes back. The classic quantitative tradition in computer architecture shifted the field from intuition to rigorous simulation and measurement (Hennessy and Patterson 2017). Architecture 2.0 extends that tradition, asking for one more layer of explicitness. Where the first era measured systems, the second era makes the measurement environment itself a programmable artifact. The tool interface itself must be part of the design object.
Hennessy, John L., and David A. Patterson. 2017. Computer Architecture: A Quantitative Approach. 6th ed. Morgan Kaufmann.
This level of explicitness matters because generative and learning-based methods are literal about interfaces. A human architect can sometimes infer that a simulator result is out of distribution, that a benchmark run used a stale configuration, or that a reported improvement is not meaningful because the compiler changed. A method acting through an environment will not infer those facts unless the environment represents them. The environment must expose enough state for useful action and enough constraints for safe rejection.
The right question is therefore not, “Which tool did the paper use?” The better question is, “What environment did the loop define?” That question forces the paper or project to name its workload distribution, action schema, observation schema, feedback latency, validity constraints, cost model, provenance record, and rejection authority. Once those are visible, method claims become easier to compare.
5.2 Interfaces Are Action Boundaries
To translate these environments into practice, what makes a tool’s action and observation schema concrete is the interface it exposes. Architecture is often described as the boundary between hardware and software. For Architecture 2.0, that statement has an operational meaning. Interfaces are where actions become legal or illegal, observations become meaningful or misleading, and evidence becomes portable or trapped inside one tool script. An ISA, compiler IR, Domain-Specific Language (DSL) like SLICC for cache coherence, memory model, accelerator runtime, simulator API, benchmark harness, EDA handoff, or telemetry schema is not just a convenience for implementation. It defines what a loop can change and what can reject the change. AI agents must therefore interface with these DSLs; emitting monolithic C++ or raw RTL defeats the modularity required for a human architect to review and maintain the logic. However, an AI agent mutating a concurrency DSL like SLICC cannot rely on a cycle-level simulator as its sole rejection authority. The design loop must explicitly integrate formal model checking (e.g., Murphi) or state-space exploration to catch latent race conditions, livelocks, and deadlocks before commitment.
This is why tool interfaces belong in the architecture argument rather than in an appendix. A generator that emits a schedule must know the schedule language. A search method that changes memory hierarchy parameters must know which combinations the simulator accepts and which violate a software-visible contract. A critique system that reads a synthesis report must know which warnings are fatal, which are informational, and which require a higher-fidelity check. The interface is the boundary where method capability meets architectural validity.
Table 5.1 lists the interfaces that a credible loop often has to expose. The table is not a complete taxonomy. Its claim is that every interface has two jobs: it makes some actions possible, and it defines what evidence those actions can produce. If either side is hidden, an automated optimizer can appear capable while acting outside the architecture problem the human intended to solve.
| Interface | What it makes actionable | Evidence it makes interpretable | Failure if hidden |
|---|---|---|---|
| ISA and vector contract | Instructions, registers, vector length, exceptions, privilege, and binary compatibility. | Correct execution, portability, software-visible behavior, and compatibility tests. | The loop proposes a microarchitecture that software cannot legally target. |
| Compiler IR and schedule representation | Lowering choices, tiling, fusion, layout, vectorization, and target-specific code generation. | Compiler success, generated code, performance counters, and optimization provenance. | Performance is attributed to hardware while the software contract changed. |
| Memory and coherence model | Ordering, sharing, cacheability, consistency, DMA, and synchronization assumptions. | Correctness tests, contention behavior, latency, bandwidth, and race or ordering failures. | A candidate looks fast because it violated the program’s memory assumptions. |
| Accelerator or runtime API | Invocation, data movement, synchronization, library calls, queues, and resource ownership. | End-to-end latency, overhead, utilization, portability, and software integration cost. | Specialized hardware is efficient in isolation but unusable in the system. |
| Simulator or environment API | Legal parameters, workload inputs, observations, errors, seeds, and fidelity levels. | Comparable runs, replayable experiments, invalid-action records, and feedback cost. | The method optimizes simulator quirks or incomparable configurations. |
| EDA handoff and constraints | RTL, clocks, floorplan hints, timing constraints, power intent, physical limits, and signoff checks. | Timing, area, power, congestion, rule violations, and implementation feasibility. | A candidate survives architectural simulation but fails physical reality. |
| Benchmark harness | Inputs, versions, metrics, splits, data-leakage rules, and submission constraints. | Coverage, reproducibility, benchmark validity, and claim scope. | The loop overfits a stale or leaky benchmark slice. |
| Telemetry and deployment schema | Live workload mix, service-level objectives (SLOs), counters, interference, rollout state, and drift signals. | Field behavior, regressions, rollback triggers, and post-deployment calibration. | Production evidence is rich but too confounded to support the architecture claim. |
The EDA handoff row is worth making concrete, because it is where a wrapper does the most work. Validity comes in two tiers, and honest environments separate them. Schema legality is cheap and checkable up front, asking whether the parameters, clocks, and interfaces are well formed. Physical feasibility is late and expensive, because routability, timing closure, congestion, and IR drop often surface only at synthesis or place-and-route. A late failure must map back to the earlier action that caused it, not read as a contract violation.
Wrapping a commercial flow as an environment is therefore not a thin API. The wrapper has to emit tool-specific scripts, parse unstructured reports, survive timeouts and run-to-run nondeterminism, and attach the fidelity or confidence label itself, because the tool emits metrics, warnings, and logs, not a fidelity level. To stay auditable, its provenance record must carry more than a bare number. It records the tool and host version, thread count, license state, run-to-run variance, warnings and waivers, and a hash of the report the number came from. Two “identical” runs that differ on any of these are not comparable, and a loop that hides the difference will trust a result it should have rejected.
Lighthouse prompt: One prompt, many interfaces
Context. The lighthouse request crosses multiple interfaces at once, so no single simulator API or wrapper is the environment.
In the Lighthouse prompt. “64-bit RISC-V-based” is an ISA and ABI/software contract. The fragment “vector-capable CPU, accelerator, or SoC block” is an action boundary over compute organization and integration. “XRBench-class real-time mobile XR workload” fixes the workload harness and quality-of-service target. “3 W TDP target in a 3 nm-class LP mobile process” invokes power and physical-design constraints.
Boundary. When the optimizer proposes a SoC block instead of an isolated accelerator, it changes the action space. The environment must now expose memory attachment, coherence, interrupts, DMA, runtime ownership, verification scope, and the evidence needed to show that the subsystem works in the larger system, otherwise the loop will optimize blindly.
Takeaway. The environment for this prompt is a bundle of coherent interfaces. An AI-assisted loop can change vector width, local memory, data layout, or accelerator interface only inside contracts that can reject software-incompatible, physically implausible, or integration-breaking candidates.
In practice, concrete tools instantiate this bundle of interfaces at different fidelity levels. The point is not that Architecture 2.0 requires one canonical simulator or one vendor flow. The point is that a loop must say which environment it is acting in, what that environment can observe, and what authority its feedback has. Table 5.2 gives a compact set of examples.
| Environment instance | Action boundary | Feedback and evidence | Loop lesson |
|---|---|---|---|
| gem5-style cycle or full-system simulation (Binkert et al. 2011) | Core, cache, memory-system, ISA, and workload configuration. | Statistics, traces, timing-model behavior, simulator warnings, and scoped performance comparisons. | Strong for controlled DSE; bounded by model fidelity and configuration state. |
| Verilator-style compiled RTL simulation (Veripool 2026) | RTL modules, test benches, assertions, generated C++/SystemC models, and debug hooks. | Functional behavior, waveform/debug evidence, assertion failures, and implementation-adjacent traces. | Moves closer to implementation but narrows throughput and increases debug cost. |
| FireSim-style FPGA-accelerated simulation (Karandikar et al. 2018) | RTL target, workload image, network model, FPGA mapping, and runtime configuration. | Faster cycle-exact feedback, workload-scale behavior, instrumented counters, and deployment-like experiments. | Speed changes sample economics, but setup and observability become part of the evidence. |
| Synthesis, place-and-route, and signoff flow (Mirhoseini et al. 2021; Semiconductor Industry Association 2026; Bauer et al. 2020) | RTL, constraints, floorplan hints, clocks, power intent, libraries, and process assumptions. | Area, timing, power, congestion, rule violations, waived warnings, and closure failures. | High-fidelity samples are scarce; use them as rejection gates, not blind search targets. |
Binkert, Nathan, Bradford Beckmann, Gabriel Black, et al. 2011. “The Gem5 Simulator.” ACM SIGARCH Computer Architecture News 39 (2): 1–7. https://doi.org/10.1145/2024716.2024718.
Veripool. 2026. Verilator. https://www.veripool.org/verilator/.
Karandikar, Sagar, Howard Mao, Donggyu Kim, et al. 2018. “FireSim: FPGA-Accelerated Cycle-Exact Scale-Out System Simulation in the Public Cloud.” 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), 29–42. https://doi.org/10.1109/ISCA.2018.00014.
Mirhoseini, Azalia, Anna Goldie, Mustafa Yazgan, et al. 2021. “A Graph Placement Methodology for Fast Chip Design.” Nature 594 (7862): 207–12. https://doi.org/10.1038/s41586-021-03544-w.
Semiconductor Industry Association. 2026. Chip Design and R&D. https://www.semiconductors.org/policies/chip-design/.
Bauer, Harald, Ondrej Burkacky, Peter Kenevan, Stephanie Lingemann, Klaus Pototzky, and Bill Wiseman. 2020. Semiconductor Design and Manufacturing: Achieving Leading-Edge Capabilities. McKinsey & Company report. https://www.mckinsey.com/industries/semiconductors/our-insights/semiconductor-design-and-manufacturing-achieving-leading-edge-capabilities.
For each row, the Architecture 2.0 object is not the tool name but the maintained action/observation record: workload version, legal edits, failed runs, waivers, escalation gate, and human owner.
In practice these environments are rarely used in isolation. Open frameworks integrate them. Chipyard, an open-source framework for generating and evaluating RISC-V systems-on-chip, wraps a configuration system, Chisel-based RTL generators such as the Rocket and BOOM cores, and a FIRRTL compiler path together with the simulation, synthesis, and FPGA-emulation backends the table lists, so one configuration can be carried reproducibly from a generated design to each evaluation path (Amid et al. 2020). Figure 5.1 shows that integration. For an AI-assisted loop, the useful property is not the framework name but where the loop actually runs. The automated optimizer acts at the configuration boundary, editing one configuration under a set of legal actions; each backend, whether simulation, synthesis, or FPGA emulation, returns a receipt; and those receipts, together with invalid-action logs, escalation thresholds, and human decision points, are the loop contract that keeps a generated configuration from becoming an unowned commitment.
Amid, Alon, David Biancolin, Abraham Gonzalez, et al. 2020. “Chipyard: Integrated Design, Simulation, and Implementation Framework for Custom SoCs.” IEEE Micro 40 (4): 10–21. https://doi.org/10.1109/MM.2020.3013233.
However, even the broad categories in Table 5.2 hide internal ladders of fidelity. Post-synthesis timing, for example, is often a useful surrogate for post-route timing, but routing congestion, IR drop, clocking, and signoff checks can still reject a candidate that looked acceptable at synthesis. The lesson is not that every loop must begin with the strongest tool. It is that the loop must know which feedback is a proxy and which later check has authority to overturn it. For an AI-assisted loop, that later authority is what decides whether a candidate earns another sample, an escalation, or termination.
Physical-design environments make this staged contract particularly visible. A loop should not report “EDA feedback” as if all tool stages had the same authority. Table 5.3 separates common stages by what they can and cannot reject.
| EDA stage | Feedback returned | What it can reject | What it cannot prove alone |
|---|---|---|---|
| Logic synthesis | Mapped gates, area proxy, timing estimates, constraint warnings, and early power estimates. | Nonsynthesizable RTL, impossible constraints, obvious area/timing problems. | Routability, final timing, IR drop, clocking, and post-layout power. |
| Floorplanning and placement | Cell locations, utilization, congestion hints, timing pressure, and macro or memory placement effects. | Floorplans that create severe congestion, timing pressure, or integration problems. | Final routed timing, signoff power, manufacturability, and workload-level benefit. |
| Clock, routing, and power closure | Routed timing, clock behavior, congestion, design-rule checks, power integrity, and closure failures. | Candidates whose physical effects invalidate earlier architecture estimates. | Product commitment without verification, workload coverage, and architecture review. |
| Signoff and review | Tool signoff reports, waivers, residual risks, review decisions, and ownership records. | Unsupported claims, unacceptable waived warnings, and evidence gaps before commitment. | Generality beyond the stated workload, process, tool version, and design context. |
This arrangement is not hypothetical. Some commercial physical-implementation flows already treat synthesis and place-and-route as a search environment: tool directives and floorplan parameters become actions, full tool runs become transitions, and reports become observations about power, performance, area, and closure (Synopsys 2023; Cadence Design Systems 2021). The durable point is not which vendor leads, since the products will be renamed and replaced. It is that an EDA flow becomes an environment only when those directives are an action schema, reports are observations with provenance, closure and signoff are rejection authorities, and the architect owns the gates the loop may not cross.
Synopsys. 2023. AI-Designed Chips Reach Scale with First 100 Commercial Tape-Outs Using Synopsys Technology. Synopsys press release. https://www.prnewswire.com/news-releases/ai-designed-chips-reach-scale-with-first-100-commercial-tape-outs-using-synopsys-technology-301739936.html.
Cadence Design Systems. 2021. Machine Learning-Driven Full-Flow Chip Design Automation. Cadence Cerebrus Intelligent Chip Explorer, product white paper. https://www.cadence.com/content/dam/cadence-www/global/en_US/documents/tools/digital-design-signoff/cerebrus-wp.pdf.
5.3 The Architecture Environment Abstraction
To formalize these interfaces across different tools, we need an abstraction. The environment has an executable core, but it is not only software. It receives a proposed action, checks whether that action is meaningful, calls one or more tools, collects observations, logs provenance, and returns feedback that can become evidence. It can also enforce or record declarative interface contracts: ISA rules, memory models, telemetry schemas, benchmark rules, and EDA constraints. As illustrated in Figure 5.2, an architecture environment fundamentally reshapes this relationship. Rather than just taking inputs and returning outputs, it wraps the underlying tool with a strict contract that exposes legal actions, bounds observations, enforces physical costs, and explicitly defines rejection paths for invalid states.
The figure is a loop contract, not a software architecture diagram. Each component is useful only after the environment names the action it permits, the feedback it returns, and the rejection authority attached to that feedback. That is why two projects can use the same simulator but define different architecture environments.
Fundamentally, this abstraction can be described as a contract. The environment is literal. It is the toolchain exposed as an executable sandbox. The environment does not compute a scalar reinforcement-learning reward. It emits raw observations and cost telemetry—such as cycle counts, gate area, and test failures. The loop’s rejection authority evaluates this multi-dimensional telemetry against our claims. If we let tools compute a scalar reward directly, we invite reward hacking1 where the loop optimizes for simulator bugs instead of physical reality. The contract requires that the loop make a small set of obligations explicit. Table 5.4 lists the main fields.
1 Reward hacking occurs when an AI agent finds an unintended shortcut to maximize its reward signal without solving the underlying problem.
| Field | What it defines | Question it answers |
|---|---|---|
| Workload distribution | Inputs, traces, benchmark versions, software stack, and operating scenarios. | What behavior is the design supposed to serve? |
| Action schema | Parameters, edits, configurations, generated artifacts, or commands the loop may propose. | What can the method actually change? |
| Observation schema | Metrics, traces, logs, reports, errors, and artifacts returned after an action. | What can the loop observe after acting? |
| Constraints and validity | Type checks, feasibility rules, physical/manufacturing limits, software ecosystem compatibility, and invalid-action handling. | What makes a candidate illegal before performance is considered? |
| Feedback budget | Cost, latency, fidelity, determinism, parallel scalability, and sample limits for each source of feedback. | How much evidence can the loop afford? |
| Provenance record | Tool versions, seeds, inputs, configuration files, assumptions, and artifact hashes. | Could a human replay or audit the result? |
| Rejection authority | Conditions that stop, revise, or escalate a candidate. | What can say no? |
The contract is deliberately broader than reinforcement learning terminology. Actions, observations, and rewards are useful terms, but architecture work also needs constraints, invalid-action semantics, provenance, and human decision points. A loop that proposes a cache size, vector width, chiplet partition, compiler flag, or RTL edit needs to know not only whether a score improved, but whether the candidate is legal, reproducible, comparable, and worth committing to a higher-fidelity stage.
Applying this contract to the lighthouse prompt, the environment might expose actions such as changing vector width, local memory size, issue width, cache configuration, accelerator interface, or data layout. It might return observations such as latency, throughput, estimated power, memory traffic, area proxy, simulator warnings, compiler failures, and rejected workloads. It should also return cost: how long the run took, which fidelity level was used, and how much confidence the loop should place in the feedback. Without that cost and provenance, the loop cannot reason about sample efficiency or evidence.
5.4 ArchGym as a Worked Example
ArchGym, which frames architecture design as a gymnasium for machine-learning-assisted design where automated methods interact with architecture environments through defined interfaces (Krishnan et al. 2023), makes the environment idea concrete in the architecture domain. The name is deliberate. It borrows from OpenAI Gym, which standardized the agent-environment interface in reinforcement learning. Just as one learning algorithm can drive many Gym environments, an ArchGym user can swap a memory-controller simulator for an accelerator simulator while keeping the same optimization or learning method intact, which is exactly the composability a shared environment is supposed to buy. The Architecture 2.0 gymnasium essay makes the same broader argument. The field needs data-centric environments where architecture tasks, feedback, and evaluation are exposed systematically (Janapa Reddi and Yazdanbakhsh 2023).
Krishnan, Srivatsan et al. 2023. “ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design.” Proceedings of the 50th Annual International Symposium on Computer Architecture, ISCA ’23. https://doi.org/10.1145/3579371.3589049.
Janapa Reddi, Vijay, and Amir Yazdanbakhsh. 2023. Architecture 2.0: Why Computer Architects Need a Data-Centric AI Gymnasium. ACM SIGARCH. https://www.sigarch.org/architecture-2-0-why-computer-architects-need-a-data-centric-ai-gymnasium/.
For our purposes, the important lesson is not that every project should use ArchGym literally. The lesson is that a shared environment changes the research question. Instead of asking whether one optimizer beat another under a private script, the community can ask which task was defined, which action space was exposed, what feedback was available, what workloads were used, and which tools or methods were compared under the same conditions. That makes method claims less anecdotal. The comparison is credible only when every AI-assisted system acts against the same represented state, receives the same feedback records, and accepts the same invalid-action, escalation, and human-review rules.
Table 5.5 reads an ArchGym-style environment as a mini-contract.
| Contract field | ArchGym-style instance |
|---|---|
| Task | Search or tune a bounded architecture design space under a declared workload and metric. |
| Action space | Architecture parameters, simulator knobs, or mapping choices the automated method is allowed to change. |
| Observation and reward | Metrics returned by the simulator or tool path, with the reward derived from those observations. |
| Workload and tool path | Benchmark inputs, simulator configuration, and versioned evaluation scripts. |
| Invalid-action behavior | Rules for rejected parameters, failed runs, timeouts, or unsupported configurations. |
| Feedback cost | Approximate turnaround and sample budget for each evaluation regime. |
| Rejection and commitment boundary | Independent checks that can reject a result and the strongest claim level the environment can support. |
| Limits | Simplified simulators, cleaner action spaces, and missing industrial constraints can bound what the result proves. |
ArchGym also shows why the environment chapter cannot be collapsed into the methods chapter. Once an environment is defined, many methods can interact with it: Bayesian optimization, reinforcement learning, evolutionary search, surrogate-guided exploration, random search, heuristic search, or a human designer using the environment as an instrumented assistant. The environment is the common ground on which method comparisons become meaningful.
Evolutionary search: In this chapter, a way to propose candidate actions by mutation, crossover, and selection, useful only when the environment can reject invalid or weak candidates.
Surrogate-guided exploration: In this chapter, using a fast predictor to screen candidate actions while preserving the stronger feedback needed to reject proxy wins.
At the same time, ArchGym should not be treated as if it solves the whole Architecture 2.0 problem. A gym can still use simplified simulators. It may not include proprietary physical-design constraints, confidential workloads, tool-license behavior, workload drift, negative trace capture, or deployment telemetry. Its action spaces may be cleaner than industrial design spaces. Its feedback may be faster and more standardized than the feedback available in late-stage silicon work. Those limitations are not reasons to dismiss the environment pattern. They are reasons to make environment validity a first-class concern.
5.5 Interfaces Make Loops Composable
Validity is not the only reason to insist on an explicit interface. A single tool wrapper can support one experiment. A disciplined interface can support a research ecosystem. The difference is composability. If action schemas, observation schemas, workloads, provenance records, and validity rules are explicit, then generators, predictors, optimizers, critics, verifiers, and human reviewers can be swapped or combined without rewriting the whole loop.
This is how architecture environments can become community infrastructure. As Chapter 2 argued for benchmark governance, useful infrastructure such as MLPerf does more than name workloads. It creates rules, versions, metrics, submissions, and comparison conventions that help a community interpret results (Mattson et al. 2020). Benchmark governance governs comparisons; environment governance must also govern what AI-assisted systems may try, which actions are invalid, what feedback means, and what can stop the loop. A useful environment should not merely publish a script; it should publish the contract under which actions are legal, observations are valid, and evidence can be compared. Keeping those contracts valid after the first paper is the operating discipline Section 5.9 develops.
Mattson, Peter, Hanlin Tang, Gu-Yeon Wei, et al. 2020. “MLPerf: An Industry Standard Benchmark Suite for Machine Learning Performance.” IEEE Micro 40 (2): 8–16. https://doi.org/10.1109/MM.2020.2974843.
This transition from single-use wrappers to shared community infrastructure is why our opening vocabulary reserved the word harness. A wrapper calls a tool. A harness preserves the contract around the tool: task, workload, action schema, observation schema, cost, provenance, invalid-action semantics, negative traces, and review status. A multi-participant harness adds role boundaries: which component may propose, which may execute, which may critique, which may verify, and which human decision is required before escalation. The distinction matters because a wrapper can automate one experiment, but a harness can accumulate reusable knowledge about a design space.
A minimal implementation receipt for such a harness might include:
task identifier; workload version; input distribution; action fields; read-only constraints; tool commands; observation fields; fidelity level; runtime cost; random seed; tool versions; generated artifacts; failure status; rejection reason; and human decision.
That list is intentionally mundane. Mundane records are what make loops auditable. If a method proposes an architecture candidate, the environment should preserve not only the winning score but also the command that produced it, the workload revision, the tool version, the failed alternatives, the warnings, and the conditions under which the candidate would be rejected.
This focus on composability also changes how we interpret AI-assisted systems. A compound design system may have a planner, code generator, simulator caller, surrogate model, evidence critic, and human reviewer. Those components can coordinate only if the environment gives them a shared state representation and stable interfaces. If those components are implemented as several agents, the need for an environment contract does not shrink; it grows. The harness has to record which component read which state, which action it took, which feedback it used, and which gate could reject its output. Without that, an “agent” is merely a wrapper around a pile of scripts. With it, the loop can become an inspectable system.
5.6 Feedback Latency and Fidelity
Once an environment is structured to support these composable agents, the hardest design choice is often not the action schema. It is the feedback regime. Architecture feedback ranges from cheap and weak to slow and authoritative. A simple analytic proxy may return in milliseconds. A cycle-level simulation may take minutes or hours, because a detailed simulator typically runs many orders of magnitude slower than the hardware it models, so seconds of target execution become hours of wall-clock time. Synthesis, place and route, or signoff of a single block are commonly hours-to-days jobs. Hardware-in-the-loop, deployment telemetry, and silicon evidence may arrive only after substantial commitment.
This is why more capable AI does not remove the environment problem. In many architecture loops, the model call is fast and the feedback is slow. A method may propose hundreds of candidates before a simulator, compiler, synthesis flow, verification run, or human review can return decision-grade evidence. The environment therefore becomes the pacing item. It must decide which actions are worth spending feedback on, which cheap checks can reject early, and when the loop should escalate to a stronger but scarcer source of evidence.
Architect’s checkpoint: The Escalation Gate
When an AI method proposes a candidate, do not automatically grant it a high-fidelity evaluation. The environment must enforce an escalation gate that asks:
- Did this candidate survive all cheap proxy and validity checks?
- Is the potential improvement worth the cost of a scarce simulator or synthesis sample?
- If the high-fidelity check fails, can the environment map that failure back to a specific tool action?
This dynamic creates an economy of evidence. Cheap feedback buys breadth, pruning, and parallel scaling; expensive feedback buys stronger rejection authority. As visualized in Figure 5.3, this economy dictates the fundamental pressure of the design loop. As feedback moves toward implementation and deployment, the loop usually spends exponentially more time per sample, gains less freedom to explore, and needs dramatically clearer justification for every action it takes. The ranges are representative rather than universal. A real project should replace them with its own source receipts; they are illustrative bands, not universal empirical claims. In the early regimes, the horizontal axis mostly means tool or measurement turnaround; near prototype, silicon, and deployment, it also includes setup, queueing, fabrication, rollout, and commitment delay.
This economy is the multi-fidelity setting studied across computational science, where cheap, lower-fidelity models are combined with scarce, high-fidelity ones to keep optimization, inference, and uncertainty quantification affordable (Peherstorfer et al. 2018). Architecture 2.0 inherits that economy and adds two architecture-specific requirements: each fidelity level must carry its own rejection authority, and a move to a higher level crosses a commitment boundary, not merely a more accurate number.
Peherstorfer, Benjamin, Karen Willcox, and Max Gunzburger. 2018. “Survey of Multifidelity Methods in Uncertainty Propagation, Inference, and Optimization.” SIAM Review 60 (3): 550–91. https://doi.org/10.1137/16M1082469.
Uncertainty quantification: In this chapter, recording where an environment’s feedback is uncertain enough to force escalation rather than commitment.
This economic pressure can also be expressed as a practical design checklist. Table 5.6 extends the sample-cost regimes of Chapter 4 (Table 4.5) from what to record toward what each regime can reject, how many samples the loop can plausibly afford, and what method behavior that economics permits. A loop that confuses those regimes can search quickly and still learn the wrong lesson.
| Feedback regime | Cost / latency intuition | Sample budget | What can reject | Method implication |
|---|---|---|---|---|
| Analytic proxy or learned surrogate | Milliseconds to seconds; low direct cost but high model-risk exposure. | Thousands to millions of candidate checks, highly parallelizable. | Obvious invalidity, dominated regions, sensitivity failures, or proxy-calibration failures. | Use for pruning, active learning, and broad search, not final commitment. |
| Trace, profile, or replay | Seconds to hours depending on capture, replay, and privacy filtering. | Tens to thousands of slices or scenarios. | Coverage gaps, stale workload versions, leakage, or mismatch to intended deployment. | Use for workload state, clustering, and targeted tests. |
| Cycle-level simulation | Minutes to days depending on model detail and target workload. | Tens to hundreds of scoped architecture comparisons. | Simulator configuration errors, unsupported states, calibration gaps, and sensitivity checks. | Use staged design-space exploration with replay and escalation. |
| Compiler/runtime measurement | Compilation, build, profiling: seconds to hours; target execution, replay, or autotuning: minutes to days. | Many local measurements, fewer portable claims. | Correctness tests, portability checks, and end-to-end software-path evidence. | Use autotuning or generation only with tests and provenance attached. |
| RTL, synthesis, or physical feedback | Hours to weeks when timing, power, congestion, or signoff constraints enter. | Handfuls to tens of scarce high-fidelity samples. | Timing, power, area, congestion, design-rule checking, formal, or waived-warning review. | Use filters, surrogates, and human gates before spending samples. |
| Prototype, silicon, or deployment telemetry | High setup cost; weeks to years when field evidence or silicon commitment is required. | Few high-commitment measurements plus ongoing telemetry. | Field behavior, reliability, rollback policy, incident review, and accountable human decision. | Use for calibration, validation, and drift monitoring, not blind exploration. |
However, this table must be read with a fidelity-risk rule. Lower-fidelity feedback can prune and prioritize, but it can also be gamed or contradicted. When a proxy says “yes” and a stronger environment says “no,” the loop should record the mismatch instead of treating it as noise. The simulator-mismatch discussion below returns to that failure mode.
Compiler and runtime feedback carry their own version of this same risk. A hardware candidate can look weak because the schedule, tiling, vectorization, layout, or runtime path is poor, not because the architecture is poor. That is a false negative created by the software side of the environment. For the lighthouse prompt, “vector-capable” therefore cannot mean only that an ISA feature exists; the loop must also expose whether the compiler and runtime can generate a credible path to use it.
This fidelity regime structure therefore drives method choice. If feedback is cheap, broad search or online adaptation may be reasonable. If feedback is expensive, the loop needs sample efficiency, priors, surrogates, active learning, staged gates, or stronger human filtering. If feedback is high commitment, the loop should become more conservative. Use AI to organize evidence, critique assumptions, and narrow the search, not to make unsupported final decisions.
Active learning: In this chapter, a policy for choosing which scarce simulator, synthesis, human, or deployment feedback event to spend next, with the choice logged as part of the evidence ledger (Settles 2009).
Settles, Burr. 2009. Active Learning Literature Survey. Computer Sciences Technical Report 1648. University of Wisconsin–Madison.
The environment should therefore expose feedback budget as a first-class object. A result should report which fidelity level produced a comparison and at what cost, not only that candidate A beat candidate B. When that feedback is strong enough to count as evidence, and what provenance, uncertainty, and human decision that requires, is the bridge from this chapter to Chapter 7, which treats it in full.
5.7 Proxy Gaming and the Simulator Case
Because low-fidelity feedback can mislead a loop, an architecture environment must also defend itself against its own abstractions. Simulators, analytical models, profilers, and compiler cost models are not neutral oracles. They encode workload choices, warm-up rules, timing models, memory-system assumptions, compiler defaults, branch predictor state, cache initialization, interconnect models, and sampling choices. A method that searches aggressively can discover weaknesses in those assumptions just as easily as it can discover a good architecture candidate.
When the proxy is a simulator, proxy mismatch, the divergence between what a cheap stand-in metric rewards and what the true objective needs, takes a specific form, a gap between the behavior an environment reports and the behavior that would matter at the next stronger fidelity level, such as a more detailed simulator, synthesis, physical design, emulation, silicon, or deployment. The failure mode is not merely an inaccurate number. It is a loop that learns the wrong lesson. A mapping optimizer may exploit a memory model that omits contention. A compiler autotuner may win by relying on a backend assumption that changes under a different target. A hardware generator may improve a cycle-level metric while creating timing, congestion, or verification problems that the current environment cannot see. A workload harness may reward one benchmark version while hiding drift in the software stack. A run that does not crash is not the same as hardware validity; the environment has to define which illegal states, unsupported configurations, and silent out-of-model behaviors it can actually reject.
Failure mode: When the tool becomes the target
A common failure pattern is not that the tool is useless; it is that the loop learns exactly what the tool rewards. This is a local instance of proxy mismatch, the architecture version of what machine-learning safety calls specification gaming or reward hacking (Amodei et al. 2016). A candidate can improve a proxy by choosing an unsupported parameter corner, relying on a stale workload slice, or shifting cost into a compiler/runtime path the current harness does not measure. The environment should make that failure visible through invalid-action checks, baseline replay, sensitivity tests, and escalation to a stronger feedback source. Chapter 7 treats proxy mismatch and its calibration defenses in full.
Amodei, Dario, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. “Concrete Problems in AI Safety.” arXiv Preprint arXiv:1606.06565.
The clearest illustration comes from the field that named the failure.
Field note: The boat that won by crashing
In a boat-racing game used to study reinforcement learning, an agent rewarded on game score stopped racing and looped in a lagoon to harvest respawning targets, catching fire and never finishing the race while out-scoring the average human player by about 20 percent (Clark and Amodei 2016). Nothing malfunctioned. The loop optimized exactly the number it was given, and that number was a proxy for “win the race” that the race never actually checked.
Takeaway. An architecture environment that hands a tool a scalar reward invites the same move on a simulator’s blind spots, so the environment, not the optimizer, must own the invalid-action and baseline checks that keep the proxy honest.
Clark, Jack, and Dario Amodei. 2016. Faulty Reward Functions in the Wild. OpenAI blog. https://openai.com/index/faulty-reward-functions/.
2 Red-teaming involves intentionally attacking a system to expose vulnerabilities, flaws, and failure modes before deployment.
Environment design should therefore include red-team checks2 for the feedback source itself. A simulator-backed loop should record warm-up policy, execution- versus trace-driven mode, random seeds, sampled regions, versioned workloads, configuration files, and unsupported states. It should also include rejection tests that look for proxy gaming: cross-checks against another model, sanity constraints on bandwidth and latency, sensitivity studies, invalid-action logs, baseline replay, and escalation to stronger fidelity when the result is surprising or high commitment. The goal is not to distrust simulation. The goal is to make the simulator’s authority explicit.
5.8 Building Environments for New Subfields
Making that authority explicit is also the first move in building an environment from scratch. A useful Architecture 2.0 environment for an AI-assisted loop can start small, one bounded task where an automated tool or method can propose actions, receive observations, hit explicit invalid-action rules, and leave rejection reasons for review. The point is not to build a universal hardware-design platform. The point is to choose a bounded task where actions, observations, constraints, and rejection can be stated cleanly.
To construct this bounded task, the recipe is straightforward.
#| lst-label: lst-environment-recipe
#| lst-cap: "**Environment recipe:** Constructing a small, bounded, rejectable Architecture 2.0 environment step by step."
1. Define the task in architectural language: workload characterization, cache exploration, accelerator parameter search, compiler/runtime tuning, chiplet partitioning, reliability analysis, or design-review critique.
2. Choose the representation the loop will read and write: configuration file, architecture description, graph, trace record, report, RTL fragment, test bench, or design-loop card.
3. Define the action schema: which fields can change, which are read-only, which combinations are legal, and which changes require human approval.
4. Wrap the tool path: simulator, compiler, profiler, RTL flow, EDA stage, runtime system, benchmark harness, or telemetry pipeline.
5. Define observations and feedback: metrics, traces, logs, warnings, errors, generated artifacts, cost, and fidelity level.
6. Define invalid-action semantics: illegal parameter, noncompilable artifact, nonsynthesizable design, violated constraint, timeout, simulator crash, or unsupported workload.
7. Log provenance and negative traces: tool versions, seeds, workload versions, failed candidates, rejected alternatives, and reasons for rejection.
8. State the human decision rule: what the architect reviews, what can be accepted automatically, and what must escalate.Once built, the readiness test for this environment is simple. Could a second method, student, or research group act inside the same harness without private knowledge from the original author? Could a rejected candidate remain understandable six months later after tool versions, workload revisions, and scripts have changed? If the answer is no, the project may still contain a useful wrapper, but it has not yet produced a durable Architecture 2.0 environment.
This eight-step recipe is intentionally more operational than inspirational. It is what keeps Architecture 2.0 from becoming prompt-to-chip rhetoric. The first useful environment for a new subfield is often narrow: a bounded workload, a small action space, a clear simulator wrapper, and a disciplined log of failures. That is enough to show the loop what it can try, what it can observe, and what evidence matters.
Applying this operational recipe, the lighthouse prompt suggests a good starting point. Instead of trying to build an end-to-end processor designer, start with an XRBench workload slice and a small set of candidate architectural knobs: vector width, local memory, cache size, data layout, or accelerator interface. Define which candidates are invalid, which feedback is cheap, which feedback is expensive, and what evidence would be needed to move from proxy exploration to simulator or synthesis-backed claims. That is a real Architecture 2.0 environment even if it is far from a complete architecture design system.
The same abstraction scales down as readily as it scales up. A single hardware prefetcher makes a perfectly good environment: the actions are prefetch depth and pattern, the cheap feedback is cache hit rate and memory bandwidth from a cycle-level simulator, and the rejection rule is any configuration that breaches a bandwidth or correctness bound. The environment idea is not tied to the size of the artifact; it is tied to having legal actions, observable feedback, and something that can say no. It is complete only after it names the workload trace, simulator version, failed configurations, proxy limits, and the point at which a human escalates from hit-rate evidence to correctness or traffic review.
Design principle: Turn tools into environments
A tool wrapper is credible only when it defines legal actions, observations, costs, invalid states, and rejection paths. A command that returns a number is not yet an environment a loop can act in.
5.9 Environment Validity and Operating Discipline
An environment is useful only if it preserves the semantics of the architectural question. If the workload is wrong, the action space omits the important decision, the simulator hides a constraint, the proxy is uncalibrated, or the logging drops failed candidates, the loop may become more efficient at producing weak evidence.
To maintain this semantic link, environment validity relies on several layers. The workload layer asks whether the inputs, distributions, and software stack match the intended use. The action layer asks whether the loop can change the right architectural variables without violating hidden constraints. The observation layer asks whether the returned metrics are meaningful for the claim. The fidelity layer asks whether the feedback is strong enough for the commitment being made. The provenance layer asks whether the result can be replayed or audited. The rejection layer asks what can stop a candidate when it is illegal, unsupported, misleading, or insufficiently evidenced.
Architect’s checkpoint: The Environment Trust Gate
Before trusting an environment’s verdict on an AI-generated candidate, confirm:
- the workload, action space, and observations still match the architecture question the loop was intended to solve;
- the feedback fidelity is strong enough for the commitment at stake;
- versions, assumptions, and rejected candidates proposed by the generative method are recorded;
- a rejection authority can stop an illegal, unsupported, or under-evidenced candidate;
- another team could deploy an automated optimizer in this harness without private knowledge.
Operating discipline, the architecture counterpart of MLOps, is the practice of maintaining those layers over time. Workloads drift. Tool versions change. Compiler behavior changes. Benchmarks gain new rules. Models learn stale assumptions. A one-time environment can support a paper; a maintained environment can support a field. That is why the environment should record versions, assumptions, invalid actions, rejected candidates, and changes in the workload distribution.
MLOps: In this chapter, the operating discipline for deployed learning systems that motivates similar versioning, monitoring, drift, rollback, and ownership records in Architecture 2.0 loops.
Design principle: Maintain the environment, don't just build it
A design-loop environment is not built once; it is maintained. Workloads drift, tools change, benchmarks gain rules, and models learn stale assumptions. Keep the loop valid by versioning workloads and tools, preserving provenance and negative traces, monitoring drift, and maintaining architect-owned rejection gates.
This distinction between building and maintaining matters because a one-time environment and a maintained one look identical on the first run and diverge completely by the tenth; the maintenance burden is what turns a one-off script into shared infrastructure.
Table 5.7 states the maintenance work in operational terms. The table is intentionally prosaic because the discipline succeeds only when ordinary drift becomes visible before it invalidates evidence.
| Drift source | What to record | Failure if ignored | Operating response |
|---|---|---|---|
| Workload and benchmark updates | Versions, scenarios, excluded inputs, telemetry shifts, and coverage changes. | A loop keeps optimizing a stale or unrepresentative slice. | Re-run coverage checks and weaken or invalidate affected claims. |
| Tool and model changes | Simulator, compiler, EDA, model, prompt, policy, seed, and configuration versions. | Results become incomparable across runs without a clear cause. | Treat version changes as evidence events and preserve old receipts. |
| Action-space changes | New knobs, deprecated knobs, invalid combinations, and permission boundaries. | A method proposes actions the environment no longer supports. | Update validity checks and record invalid-action traces. |
| Evidence and rejection rules | Fidelity labels, escalation thresholds, signoff gates, waivers, and reviewer decisions. | A proxy keeps authority after the commitment level has risen. | Recalibrate evidence levels and require higher-fidelity review. |
| Human ownership | Decision owner, review date, redaction boundary, and escalation path. | Accountability disappears into scripts, tools, or automated systems. | Keep commitment decisions explicit and auditable. |
5.10 Conclusion
This chapter asked what turns a tool flow into an environment a generative model can act in and a reviewer can trust. In Architecture 1.0, simulators, compilers, profilers, RTL flows, and deployment logs sit behind the architect as ways to gather evidence. In Architecture 2.0 they have to move inside the loop and become explicit environments, places that define legal actions, return meaningful observations, price each evaluation, carry constraints and provenance, and let a rejection authority stop an illegal, unsupported, or under-evidenced candidate. That is the whole content of turning a tool into an environment.
An environment is only as good as the architectural question it preserves. If the workload is unrepresentative, the action space omits the real decision, the proxy is uncalibrated, or failed candidates go unlogged, the loop just becomes more efficient at producing weak evidence. Validity therefore has to be checked layer by layer, across workload, action, observation, fidelity, provenance, and rejection, before an environment’s verdict is worth believing.
The second, easily missed half is maintenance. Workloads drift, tools change, benchmarks gain rules, and models learn stale assumptions, so a one-time environment and a maintained one look identical on the first run and diverge completely by the tenth. Operating discipline, the architecture counterpart of MLOps, is what versions workloads and tools, preserves provenance and negative traces, monitors drift, and keeps architect-owned rejection gates alive. Building the environment earns a paper. Maintaining it is what lets a loop, and a field, keep trusting the evidence.
5.11 Open Research Questions
The concepts in this chapter establish the foundation for AI-assisted architecture environments, but they also expose major gaps in how AI-assisted loops interact with legacy tools. The following thesis-level challenges push beyond wrapper engineering toward self-learning, verifiable, and evidence-gated environment infrastructure capable of seeding high-impact research at ASPLOS, ISCA, or NeurIPS.
Can agents infer formal environment contracts from unstructured tool exhaust? Legacy EDA flows and simulators emit opaque warnings, timeouts, and unstructured logs rather than clean APIs. A major open challenge is whether language models can observe thousands of such interactions and help synthesize action schemas, constraint bounds, and rejection authorities defined in the environment contract (Table 5.4), making environment boundaries more explicit for human review.
How can generative loops extract and transfer “negative evidence” across mismatched physical and simulated environments? While positive architectural wins are highly context-dependent, failures (e.g., routing congestion, IR drop, timing violations) expose fundamental physical limits. As outlined in the EDA stage contract (Table 5.3), a critical open problem is formalizing how this “dark matter” of rejected candidates from slow, high-fidelity physical flows can be automatically distilled into reusable constraints for fast, early-stage proxies evaluating completely different workloads or technology nodes.
Can we reduce routine review with risk-aware escalation while preserving architect-defined commit gates? The economy of evidence regarding feedback latency and fidelity (Table 5.6) currently relies on static heuristics to promote candidates from cheap analytic proxies to expensive cycle-level simulators. A thesis-level challenge is building environments that dynamically quantify proxy-mismatch risk, triggering high-fidelity evaluation when a surrogate model’s uncertainty bounds breach a stated threshold and reducing proxy gaming by design.
How do we engineer self-healing infrastructure for continuous architectural claim recertification? As workloads drift, software stacks mutate, and tool versions evolve, the validity of static architectural baselines decays. Building on the operating discipline for ArchOps (see the discussion on “Environment Validity and Operating Discipline” in Section 5.9), a fundamental systems challenge is designing governance environments that detect semantic shifts, dispatch AI-assisted verifiers to re-evaluate historical data, and recertify or reject past claims under human-defined review gates.
What to carry forward
- Reader test: Could another method act in the same harness without private knowledge from the original author, judged by loop conditions rather than fashion?
- Up next: Once the environment defines what action and feedback mean, the discussion of methods for generation, prediction, and optimization (see the discussion on “Method Roles: Generate, Predict, Optimize” in Chapter 6) can ask which methods belong inside the loop.