8 Running the Lighthouse Loop
“In theory, there is no difference between theory and practice. But, in practice, there is.”
— Jan L. A. van de Snepscheut, Computer Scientist
Author’s Note: Jan L. A. van de Snepscheut, a prominent computer scientist, humorously captured the gap between theoretical models and real-world execution. In this chapter, we see this play out as we trace an illustrative end-to-end loop turn where an accelerator looks like a massive winner in a fast proxy (“theory”), but fails once a simulation-stage estimate charges data movement (“practice”).
The crux
How does an AI-assisted loop convert the lighthouse prompt into one bounded, rejectable turn rather than a one-shot design answer?
The previous chapters have described what a credible loop must contain. They have not yet shown one turn. This chapter does that. It takes the lighthouse prompt, bounds it to a task small enough to run, and walks the loop through candidate instantiation, prediction, escalation, rejection, and commitment. Every number here is illustrative and generated in code, not measured; the lesson is the shape of the loop, not the values. The numbers are constructed to be inspectable, not XRBench measurements, gem5 results, or synthesis estimates; replace them with local source receipts before using this pattern for an empirical claim. A real, measured instance of the same discipline comes in Chapter 9, a production fleet migration whose generate, reject, escalate, and commit spine runs with instrument-held rejection authority on live services. That case is a software build-and-test repair loop, not a hardware design loop; it shares the loop contract but not the proxy-to-simulation fidelity ladder this chapter walks, so read it as evidence that the discipline holds at deployment scale, not that this exact four-round sequence has been run on silicon. Here the goal is to make the shape itself legible.
Bounding the task comes first. “Design a low-power XR compute subsystem” is not a loop; it is a wish. This chapter narrows the task to one XRBench-class workload slice. The loop chooses among three compute organizations under a 3 W power envelope and an 8 ms per-frame deadline, the frame budget the slice fixes for its real-time display pipeline, then returns the surviving candidate with its evidence and its rejected alternatives. The candidates are a vector CPU extension, an accelerator deliberately left loosely coupled as a stress case, and a shared-memory SoC block. The loose accelerator is not the only accelerator realization allowed by the prompt; it is included so interface and data-movement costs have somewhere to show up. That is enough to make the loop turn. The worked example is therefore not a DSE recipe; it is a trace of what an AI-assisted architecture loop may propose, what feedback it may trust, and what residue it must leave for a human owner.
XRBench-class: XRBench is a mobile and extended-reality benchmark suite (Kwon et al. 2023). This chapter uses “XRBench-class” to mean a workload slice modeled on that benchmark family, not measured XRBench results.
Kwon, Hyoukjun et al. 2023. “XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse.” Proceedings of Machine Learning and Systems.
To ground this trace, Table 8.1 states the active slice of the lighthouse prompt. Everything outside the slice is not ignored; it is explicitly deferred to the next evidence stage.
| Loop field | Active slice in this chapter |
|---|---|
| Workload | One XRBench-class mobile-XR slice with a real-time frame deadline; a real loop would also record workload ID, input schema, scenario labels, coverage limits, distribution assumptions, provenance, and validity checks. |
| Baseline | A bounded comparison among three compute organizations, not a full product baseline. |
| ISA/software assumption | All candidates must preserve the existing 64-bit RISC-V software ecosystem to avoid rewriting the full stack from scratch; full compiler/runtime feasibility is deferred to a later stage. |
| Design space | Vector CPU extension, deliberately loose accelerator stress case, or shared-memory SoC block; no cache, chiplet, voltage, compiler-policy search, or full tightly coupled accelerator, CGRA, or processing-in-memory (PIM) search yet. PIM is the natural low-data-movement contender, deferred to a later turn. |
| Legal actions | Instantiate the three candidate organization records with prompt, generator/search version, candidate IDs, invalid-action records, and selection rule; run the proxy screen, escalate all candidates to the simulation-stage estimate because the proxy has no final rejection authority, reject only on declared deadline/power gates, and advance a survivor only to RTL-level study; changing workload scope or claiming implementation readiness is outside this turn. |
| Physical envelope | 3 W power envelope and 8 ms per-frame deadline; process and thermal assumptions are placeholders for the next stage. |
| Excluded evidence | No RTL, timing, area, thermal, physical-design, deployment, silicon, carbon-footprint, or memory-consistency/coherence evidence. |
What this chapter gives you
After this chapter you can turn a broad prompt into one bounded, rejectable AI-assisted loop turn and read the receipt it leaves. That means you can:
- instantiate and run an AI design-loop card on a concrete prompt, not merely fill it in;
- recognize proxy mismatch when the loop’s cheapest winner fails at higher fidelity;
- apply the commitment rule to stop the AI loop at an honest evidence level;
- read the residue an AI-assisted loop leaves: evidence ledger, negative traces, and the next evidence the human decision needs.
8.1 Round One: Generate and Screen on a Proxy
The four rounds that follow are not four turns. As illustrated in Figure 8.1, they make up a single complete design-loop turn and realize its five beats, grouped into four rounds because generation and proxy screening share the first. First, a generative method1 or heuristic search explores the allowed action space to propose candidate designs. Second, these candidates are quickly screened against a low-fidelity proxy, giving the loop its first observation of the design’s state. Third, because the proxy’s authority is limited, surviving candidates must escalate to stronger, high-fidelity simulation or synthesis tools that produce rigorous evidence against constraints. Fourth, any options that violate these hard constraints are definitively pruned and rejected, leaving a trail of negative traces. Finally, the human architect steps in to review the gathered evidence and make a bounded, documented commitment.
1 A generative method or model is an AI system trained to generate new data samples, such as hardware configurations, from a learned distribution.
The loop begins cheaply. A generative model or search method proposes the three organizations in a bounded candidate schema, and an analytic proxy, an operation-count or datapath-only estimate that counts arithmetic but not the traffic needed to feed it, estimates latency and energy per frame in milliseconds and millijoules. The proxy is fast and ignores most data movement, input-distribution variation, locality, runtime behavior, and coverage labels, so it flatters designs that keep arithmetic local. At this fidelity the loosely coupled accelerator looks best. It posts the lowest energy and latency because the proxy never charges it for moving data in and out.
Instead of demonstrating a full generative search to find these three organizations, this illustrative turn simply seeds the candidate records directly. A real run would store the prompt, generator version, candidate IDs, invalid candidates, and selection rule as part of the receipt.
Regardless of how candidates are generated, a proxy result is low-fidelity feedback that becomes evidence only for the narrow action of deciding what to escalate. The proxy is not evidence for an implementation commitment until stronger feedback confirms the gate. Proxy deadline or power concerns are warnings until stronger feedback confirms them. The AI-assisted loop records the proxy ranking and escalates. The loop escalated because it lacks the evidence to support a claim, and therefore lacks the authority to reject alternatives on performance or power constraints.
Architect’s checkpoint: The Escalation Gate
The automation’s role: Rank candidates using the cheap proxy to prioritize the search space. The human architect’s role: Verify that proxy results only guide escalation and are not trusted as implementation commitments.
8.2 Round Two: Escalate to a Simulation-Stage Estimate
To advance beyond the proxy’s blind spots, an illustrative simulation-stage estimate is the first stage that charges memory traffic. The exact tool is not the lesson. This stage is a stronger environment contract. It prices data movement, returns latency, energy, and power observations with provenance and cost, and has authority to overturn proxy rankings before commitment. Its power column is frame energy divided by the candidate’s active compute time, a conservative average-power screen against the 3 W budget. It is a screen, not a signoff TDP verdict; peak and transient power (di/dt, IR drop) and sustained thermal behavior are deferred to later stages. It stands in for this class of model: a dataflow analysis like Timeloop (Parashar et al. 2019), a calibrated pre-RTL estimator such as Aladdin (Shao et al. 2014) coupled to a full-system model (Shao et al. 2016) so interface and off-chip traffic are priced rather than idealized, or a cycle-accurate simulator such as gem5 (Binkert et al. 2011) or FPGA-accelerated FireSim (Karandikar et al. 2018).
Parashar, Angshuman, Priyanka Raina, Yakun Sophia Shao, et al. 2019. “Timeloop: A Systematic Approach to DNN Accelerator Evaluation.” 2019 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS, 304–15. https://doi.org/10.1109/ISPASS.2019.00042.
Shao, Yakun Sophia, Brandon Reagen, Gu-Yeon Wei, and David Brooks. 2014. “Aladdin: A Pre-RTL, Power-Performance Accelerator Simulator for Rapid Design Space Exploration.” Proceedings of the 41st Annual International Symposium on Computer Architecture (ISCA). https://doi.org/10.1109/ISCA.2014.6853196.
Shao, Yakun Sophia, Sam Likun Xi, Vijayalakshmi Srinivasan, Gu-Yeon Wei, and David Brooks. 2016. “Co-Designing Accelerators and SoC Interfaces Using Gem5-Aladdin.” International Symposium on Microarchitecture (MICRO).
Binkert, Nathan, Bradford Beckmann, Gabriel Black, et al. 2011. “The Gem5 Simulator.” ACM SIGARCH Computer Architecture News 39 (2): 1–7. https://doi.org/10.1145/2024716.2024718.
Karandikar, Sagar, Howard Mao, Donggyu Kim, et al. 2018. “FireSim: FPGA-Accelerated Cycle-Exact Scale-Out System Simulation in the Public Cloud.” 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), 29–42. https://doi.org/10.1109/ISCA.2018.00014.
It is slower and scarcer than the proxy. In this toy turn all three candidates are escalated because proxy feedback has no final rejection authority. The result is the central lesson of the chapter. The candidate that wins the cheap screen does not survive the stage that charges data movement. Table 8.2 runs the comparison.
| Candidate | Proxy lat / energy | Sim lat / energy | Avg power | Verdict |
|---|---|---|---|---|
| Vector CPU extension | 7.6 ms / 22 mJ | 9.5 ms / 24 mJ | 2.5 W | rejected: misses 8 ms deadline |
| Loosely coupled accelerator | 4.5 ms / 12 mJ | 8.8 ms / 28 mJ | 3.2 W | rejected: misses 8 ms deadline; over 3 W envelope |
| Shared-memory SoC block | 6.0 ms / 16 mJ | 6.5 ms / 18 mJ | 2.8 W | survives to RTL study |
As the quantitative results show, the accelerator that won on the proxy collapses under the simulation-stage estimate. Once data movement across the system interconnect (e.g., a Network-on-Chip or AXI bus) is charged, its energy rises above every alternative and its latency loses most of its lead. The shared-memory SoC block absorbs the same charge with little damage because its traffic stays inside the tightly-coupled on-chip hierarchy it shares with the CPU, so the proxy’s blind spot was never subsidizing it. The vector extension, which also keeps data local, fails instead on raw throughput against the deadline. This is proxy mismatch made concrete, similar to the phenomenon in ML systems where optimizing for raw FLOPs starves memory bandwidth (as seen in the Roofline model (Williams et al. 2009)). The loop was optimizing the measurement it could see (arithmetic), not the objective it cared about (end-to-end latency and energy). The true simulation-stage rejection gate correctly prices communication. The failed candidate is not deleted. It is recorded as a negative trace, with the fidelity level at which the proxy win disappeared, so a later loop does not rediscover it. As illustrated in Figure 8.2, the seemingly optimal proxy ranking is overturned once a simulation-stage estimate charges the data movement overhead, demonstrating the danger of proxy mismatch.
Williams, Samuel, Andrew Waterman, and David Patterson. 2009. “Roofline: An Insightful Visual Performance Model for Multicore Architectures.” Communications of the ACM 52 (4): 65–76.
To document this shift in ranking, Table 8.3 shows the evidence ledger left behind. The exact schema of the candidate record depends on the environment, but it must be machine-readable (e.g., JSON or YAML) to allow autonomous auditing. For example, a YAML evidence ledger entry might look like this:
candidate_id: xr_accel_loose_v4_2
status: REJECTED
rejection_gate: power_screen # also misses the 8 ms deadline
provenance:
git_hash: "a4c9f1b"
generator_version: "1.2.0"
proxy_estimates:
energy_mJ: 12
latency_ms: 4.5
simulation_estimates:
energy_mJ: 28
latency_ms: 8.8
avg_active_power_W: 3.2 # conservative screen, not a signoff TDP
failure_reason: "interconnect/DMA traffic pushes the average-power screen to 3.2 W (over the 3 W budget) and latency to 8.8 ms (over the 8 ms deadline)"Read the first two rows of Table 8.3 as feedback stages and the third as the record those stages leave; the columns are the ledger schema: the feedback source, its fidelity and budget, what it supported, the gate it could reject on, its provenance limit, and the next evidence required. The ledger is what makes the toy loop auditable rather than merely ranked.
| Stage | Feedback source | Fidelity and budget | Support | Rejection gate | Provenance / limit | Next evidence |
|---|---|---|---|---|---|---|
| Proxy screen | Operation-count or datapath-only estimate. | Three cheap evaluations; ignores most data movement. | Ranks the accelerator first on proxy energy. | No final rejection; only warnings. | Illustrative constants in this chapter. | Escalate all candidates to stronger feedback. |
| Simulation-stage estimate | Illustrative memory-aware estimate plus simple power check. | Three scarce illustrative checks; still no RTL or physical feedback. | Shared-memory SoC block clears both gates. | 8 ms deadline and 3 W envelope. | Hard-coded values; not XRBench, gem5, synthesis, silicon data, workload trace IDs, scenario coverage labels, or input-distribution receipts. | RTL, timing, area, thermal, and compiler/runtime checks. |
| Negative traces | Recorded failed alternatives. | Two rejected candidates. | Failure reasons are preserved. | 8 ms deadline and 3 W envelope; proxy mismatch is recorded as diagnostic context. | Candidate IDs and reasons only; no logs attached, and no evidence that another workload distribution would preserve the same rejection. | Prevent rediscovery and guide the next search. |
8.3 Round Three: Reject on the Envelope
Beyond simply reordering candidates, the simulation-stage estimate also exposes harder physical gates. The loosely coupled accelerator does not merely lose on energy; in this constructed diagnosis, its symptoms point to one cause the next evidence stage would have to verify. Reaching the accelerator means moving data across an interface to off-chip or distant memory, and that traffic is expensive on every axis at once. In that diagnosis it stalls the datapath, so the illustrative latency estimate misses the 8 ms deadline; it pays off-chip access energy, so per-frame energy climbs above every alternative. That charge is not arbitrary. At a representative off-chip cost on the order of a hundred picojoules per byte, moving roughly a hundred megabytes of activations and weights per frame across that interface adds about 16 mJ, which is what separates the proxy’s 12 mJ reading from the 28 mJ the simulation stage estimates. Furthermore, it keeps the memory subsystem continuously busy, so its conservative average-power screen crosses the 3 W budget. That is enough to reject it at this stage and to flag it for the sustained-thermal and peak-power (TDP) signoff the loop has not yet run. These are hard constraints, not just soft metrics, so no local proxy advantage rescues it. The rejection is the commitment rule from the trust chapter doing its work. A candidate advances only if it is valid, its evidence clears the threshold for the stage, and its residual physical risk is acceptable.
Architect’s checkpoint: The Rejection Gate
The automation’s role: Test candidates against the simulation-stage constraints (e.g., 8 ms deadline, 3 W power envelope) and reject those that fail. The human architect’s role: Ensure the automation’s rejection criteria match the physical and product constraints, and accept the negative traces produced.
This rejection gate forces us to confront the true integration cost of a proposed design, tying directly back to the constraints outlined in the lighthouse prompt.
Lighthouse prompt: Price the interface before believing the proposed accelerator
Context. This worked AI-assisted loop deliberately includes a loosely coupled accelerator as a stressed integration variant, even though the full prompt also allows a tighter accelerator path.
In the Lighthouse prompt. The “64-bit RISC-V-based” software path (which anchors the architecture to a vast, existing software investment), the “XRBench-class real-time mobile XR workload” slice, the 8 ms frame deadline the slice fixes, and the “3 W TDP target” are fixed. The AI loop varies the “vector-capable CPU, accelerator, or SoC block” choice and its interface cost.
Deferred evidence. Compiler and runtime, RTL, thermal, physical-design, and verification and reliability evidence are deferred to the next stage.
Takeaway. In the Lighthouse loop, a proposed accelerator is not a faster box attached to the side; it is an interface, memory-system, software-path, and power claim. Once data movement across that interface misses the deadline and crosses the envelope, the local arithmetic win the automation found no longer matters.
8.4 Round Four: Commit at an Honest Level
With the rejection gates applied, one candidate survives the deadline and the envelope. It is tempting to call that a final result. It is not. The evidence behind it is a simulation-stage architectural estimate standing in for cycle-level and power feedback. This supports an experimental commitment, not an implementation or a tapeout. The architect’s decision is therefore bounded. Advance the surviving organization to an RTL study where logic synthesis, place-and-route, exact timing closure, and a stronger physical power estimate can confirm or reject it. Because the survivor is a shared-memory block, it owes two formal obligations a cycle-level simulator cannot discharge, and neither is settled by the RTL timing study: its coherence protocol needs explicit-state model checking (the SLICC-to-Murphi path of Chapter 5), and its consistency model needs litmus tests escalated to formal memory-model verification (Chapter 7), since litmus tests show the presence of ordering bugs, not their absence. It won partly by keeping traffic on-chip, trading the interface data-movement cost that sank the accelerator for on-chip coherence traffic the proxy never charged, so both gates stay open beyond this turn. The other two candidates are held as recorded negative traces.
Stopping at this honest boundary is the difference between an answer and a defensible AI-assisted loop turn. The result is a replay receipt: the surviving state, the evidence that supports it, the actions and alternatives rejected, the commitment boundary, and the next evidence a human owner requires.
Architect’s checkpoint: The Commitment Gate
The automation’s role: Provide the surviving candidate alongside a replay receipt, evidence ledger, and negative traces. The human architect’s role: Accept the evidence boundary and authorize advancing the surviving candidate to RTL study, or demand stronger evidence before committing.
Halting the loop here, rather than allowing an AI generator to extrapolate beyond its authorized evidence boundary, is a fundamental rule of engineering discipline.
Design principle: Stop at an honest evidence level
An AI-assisted loop should report what its evidence supports, what it rejected, and what would overturn the decision, then stop there. The honest commitment level, not the most optimistic one a generative model might claim, is the result.
To capture this commitment, Table 8.4 records that residue as a filled design-loop card. The entries are intentionally terse. A real project would attach logs, scripts, trace identifiers, and power-model receipts, but the first test is whether the loop can state what it did and what would reject it.
| Card field | Lighthouse loop entry |
|---|---|
| Intent | Explore a low-power mobile-XR compute organization that can meet a real-time deadline under a 3 W envelope. |
| Task | Compare three bounded compute organizations for one XRBench-class workload slice. |
| Design space | Choose among vector CPU extension, deliberately loose accelerator stress case, and shared-memory SoC block; leave cache, voltage, chiplet, compiler-policy, full tightly coupled accelerator search, and RTL edits outside this turn. |
| Representation | Candidate ID, candidate record, legal action taken, workload slice ID/coverage label, proxy and simulation observations, uncertainty/provenance limits, deadline, power envelope, and verdict. |
| Environment | Analytic proxy followed by an illustrative simulation-stage estimate standing in for cycle-level feedback and a simple power-envelope check. |
| Method role | Seed candidates, predict cheap proxy scores, escalate all candidates because the proxy has no final rejection authority, and critique proxy wins against stronger feedback. |
| Feedback budget | Three cheap proxy evaluations and three illustrative simulation/power checks; no RTL, timing, thermal, or silicon evidence yet. |
| Evidence | The shared-memory SoC block is the only candidate that clears the 8 ms deadline and 3 W envelope in the illustrative run. |
| Negative traces | The vector CPU extension misses the latency deadline; the loosely coupled accelerator misses the latency deadline and exceeds the power envelope after the illustrative estimate exposes data movement. |
| Rejection authority | Deadline check, power-envelope check, and the stronger estimate that can overturn the proxy ranking. |
| Commitment boundary / would overturn | Advance only to RTL-level study; implementation commitment would require synthesis, timing, area, thermal, compiler/runtime, memory-consistency/coherence, and stronger power evidence. |
| Human decision | Advance the shared-memory SoC block to RTL-level study and keep the other two candidates as negative traces. |
Alongside the summary card, a real version of the same loop should also leave a replay receipt. The receipt does not have to expose proprietary data, but it should preserve enough structure that a reviewer can tell what was run, what failed, and what would need to be rerun. Table 8.5 gives the minimum shape.
| Receipt item | Example contents for this loop |
|---|---|
README.md |
Task slice, claim boundary, non-goals, and human decision owner. |
inputs/ |
Workload IDs, benchmark version, scenario metadata, input-distribution summary, coverage labels, trace provenance, workload-level rejection conditions, and redaction notes. |
prompts/ |
Prompt slice, model/tool versions, role assignments if multiple models act, generator/search configuration, invalid proposals, selection rule, and human overrides or approvals. |
candidates/ |
Candidate IDs, configuration records, invalid-action records, and rejected alternatives. |
runs/proxy/ |
Proxy model version, constants, scripts, logs, and cheap-screen results. |
runs/sim/ |
Simulator or stronger-feedback configuration, seeds, logs, metrics, warnings, and failures. |
evidence/ |
Evidence ledger, fidelity labels, uncertainty, baseline, and sensitivity notes. |
decisions/ |
Architect review note, escalation rule, commitment level, and next evidence required. |
To emphasize that a winning claim cannot exist in a vacuum, Figure 8.3 visualizes this receipt as a cohesive directory bundle. The winning architecture only matters because the surrounding artifacts explicitly record what evidence supported it, what alternatives failed, and what human decisions govern its deployment.
The discipline is not unique to hardware, and neither is the cost of skipping it.
Field note: Four hundred forty million in forty-five minutes
On August 1, 2012, a trading firm deployed new order-routing code, but the update reached only seven of its eight servers. The eighth still ran repurposed logic that a reused flag switched back on, and it began firing orders into the market with no position limit and nothing to halt it. No automated check verified that the right code, and only the right code, was live everywhere. In about forty-five minutes the firm lost roughly four hundred forty million dollars and nearly collapsed (U.S. Securities and Exchange Commission 2013; Knight Capital Group 2012).
Takeaway. Running the loop to a commitment is only safe if the commit step itself is gated. A deployment that cannot verify what it shipped, and cannot reject and roll back, turns one unreviewed change into an irreversible loss.
Knight Capital Group. 2012. Knight Capital Group Provides Update Regarding August 1st Disruption to Routing in NYSE-Listed Securities. Press release (Form 8-K), August 2, 2012.
U.S. Securities and Exchange Commission. 2013. In the Matter of Knight Capital Americas LLC. Release No. 34-70694. SEC.
8.5 What the Loop Leaves Behind
Ultimately, the residue of one turn is the reusable artifact.
Evidence packet: What one AI-assisted loop turn leaves behind
The evidence packet for this turn is the filled design-loop card and replay receipt above. It exposes the task boundary, evidence level, negative traces, rejection authority, commitment boundary, and next evidence owner. Another architect can read that residue and reconstruct why the surviving candidate advanced, why the others did not, and what would overturn the decision.
8.6 Conclusion
This chapter asked how an AI-assisted loop converts the lighthouse prompt into one bounded, rejectable turn rather than a one-shot design answer. The move is to shrink the prompt to a task small enough that a candidate can actually be rejected, then walk it through generation and proxy screening, escalation to a stronger estimate, rejection on the power envelope, and commitment at a level the evidence can honestly support. The numbers were illustrative. The shape was the lesson. A loop is not a single answer but a sequence of bounded, checkable transitions.
What the turn leaves behind matters as much as the candidate that survives it. The residue, a filled design-loop card and replay receipt, records the task boundary, the evidence level, the negative traces, the rejection authority, and the next evidence owner. That receipt is what lets a second architect reconstruct why the surviving candidate advanced, why the others did not, and what would overturn the decision. A win with no reviewable residue is a claim no one else can check.
The discipline the turn enforces is to stop at an honest evidence level. When a cheap proxy and a stronger check disagree, the stronger check holds authority, and the loop commits only as far as its evidence reaches, refusing to dress a proxy result as silicon-grade proof. Read this way, the worked turn becomes a template a reader can reuse on a real project, naming the loop, judging whether its evidence matches its commitment level, and saying what remains a human decision. One loop turn stays small, bounded, and reviewable, which is exactly what makes it worth trusting.
8.7 Open Research Questions
The instantiation of the lighthouse loop in this chapter demonstrates a bounded, rejectable turn, but automating this process at scale exposes profound, unsolved challenges in systems research. Resolving these challenges requires thesis-scale advances in learning, verification, and cryptography.
AI-Assisted, Multi-Fidelity Budget Allocation Policies. While the worked example statically allocates feedback budgets across proxy and simulation stages, future AI architects require learned, dynamic policies to distribute these budgets across sprawling design spaces. Research must define the state schemas and multi-fidelity training environments necessary to train reinforcement learning agents2 that recommend when to escalate, reject, or request stronger checks, while adhering to the commitment boundaries formalized by this chapter’s design principle, Stop at an honest evidence level.
Proactive Detection of Proxy Mirage and Architectural Blind Spots. As demonstrated during escalation, cheap analytical proxies often exhibit catastrophic blind spots, such as ignoring interface taxes or data movement. Rather than passively waiting for expensive simulation to catch these inversions, research must develop predictive methods and formal verification techniques capable of structurally analyzing proxy models. The goal is to mathematically bound the risk of proxy mismatch and proactively identify when a proxy is actively misleading before expending scarce cycle-level simulation budgets.
Privacy-Preserving Evidence Ledgers. Sharing the evidence ledger (Table 8.3) and replay receipts (Table 8.5) across organizational boundaries inherently risks leaking proprietary baselines and sensitive workload distributions. We require privacy-preserving formulations—such as secure multi-party computation or federated validation—that allow external tools, reviewers, or competing vendors to audit candidate lineage, fidelity, and rejection authority without exposing intellectual property.
Re-evaluating Stale Rejections. When a candidate is rejected on a strict physical constraint, it becomes a negative trace. However, treating these traces as permanent failures artificially restricts future search. Future work must design rigorous methods to differentiate localized, workload-specific failures from universally invalid architectures. This would enable automated optimizers to safely reconsider (or “resuscitate”) stale negative traces when underlying physical constraints, process nodes, or software runtimes shift, achieving continuous architectural learning.
2 Reinforcement learning agents learn to make decisions by taking actions in an environment to maximize cumulative reward.
What to carry forward
- Reader test: Could another architect reconstruct why the surviving candidate advanced and what would overturn it?
- Up next: The discussion on “Loop Patterns Across the Stack” (Chapter 9) turns this worked AI-assisted loop into stack-wide patterns.