Chapter 8 — Dialog Engineering and the CSAR Loop

Further reading

Books, guides, and papers beyond the book’s own bibliography (Appendix C, updated in errata) on structured dialog with LLMs.

• Anthropic — Prompt Engineering Guide. The vendor-neutral operational reference for structuring prompts. The chain-of-thought, few-shot, and role-context patterns are the technical building blocks CSAR composes into a four-phase protocol. Read the «Be clear and direct,» «Use XML tags,» and «Chain complex prompts» sections for the direct pairing. Free at docs.anthropic.com.
• OpenAI — Prompt Engineering. The equivalent from OpenAI, more terse but covers the same working patterns. The «Split complex tasks into simpler subtasks» strategy is Crystallize in a different voice. Free at platform.openai.com/docs/guides/prompt-engineering.
• Willison, S. — simonwillison.net. Long-running practitioner blog on LLM tooling and prompt patterns, updated near-daily. Willison co-created Django and writes with unusual precision about what LLMs actually do at the interface layer — the Assemble step’s failure modes show up here in operational detail before they show up in vendor docs. Free at simonwillison.net.
• Mollick, E. & Mollick, L. (2023). «Assigning AI: Seven Approaches for Students, with Prompts.» SSRN Working Paper. Pedagogically-framed but operationally useful catalog of seven prompt-structure patterns (AI-as-mentor, AI-as-tutor, AI-as-coach, AI-as-teammate, AI-as-simulator, AI-as-tool, AI-as-student). CSAR maps to the AI-as-teammate pattern; the other six are useful adjacent shapes for different tasks. Free at papers.ssrn.com/sol3/papers.cfm?abstract_id=4475995.
• The Verification Habit (sister book), Chapter on CSAR. The general CSAR loop lives in the sibling volume; Ch 8 of The Defensible Decision is the visualization-domain adaptation. Read the general version if you also apply CSAR to code review, technical writing, or research synthesis — the loop generalises well beyond charts.

Last curated: 2026-07-18. Reviewed quarterly. Suggestions welcome via the errata page.

Self-check quiz

Five questions on Ch 8’s load-bearing concepts. Answer them out loud before opening the reveal.

1. Q1. Name the four CSAR phases and, in one line each, the specific work each phase does.

   Show answer

   Crystallize — the analyst asks the AI what it would need to know to make the artifact defensible, before it produces anything. Scope — the AI restates the agreed scope back so the analyst can catch drift before the draft is built. Assemble — the AI produces the draft with an explicit list of decisions-made and decisions-deferred. Refine — the analyst reviews against the Part I rubric, naming corrections by principle rather than by feeling.

2. Q2. Ch 8 argues Crystallize is the load-bearing phase — the one that determines whether the loop produces defensible output. Why?

   Show answer

   Crystallize converts an under-specified request into a specification the model can be reviewed against. Without Crystallize, the model fills specification gaps with its most probable defaults — and its defaults are what Ch 4 named as default failure modes and Ch 5 named as density defaults. Even a well-run Refine phase cannot fix a fundamentally under-specified Assemble output; it can only patch symptoms. The rest of the loop compounds Crystallize’s quality — a specific Crystallize prompt makes Scope sharper, Assemble more constrained, and Refine cheaper.

3. Q3. What specifically does Scope catch that would be more expensive to catch at Assemble or Refine?

   Show answer

   Drift between the analyst’s intended specification and the model’s interpretation of it. Caught at Scope (a restatement), the fix is one prompt revision. Caught at Assemble (a generated draft), the fix requires re-generating and re-reviewing. Caught at Refine (a partially reviewed draft), the fix may require unwinding downstream Refine directions that were compensating for the drift. Scope is the cheapest checkpoint in the loop; skipping it is one of the most common ways CSAR turns into prompt roulette.

4. Q4. Ch 8 introduces the «defensibility bundle» as the audit trail each loop leaves behind. Name its six pieces and what each piece is for.

   Show answer

   (1) Crystallize prompt — what the analyst asked for. (2) Scope confirmation — what the model heard, agreed to. (3) Assemble draft — what the model produced. (4) Decisions-made list — explicit choices the model committed to. (5) Decisions-deferred list — explicit choices the model punted back to the analyst. (6) Refine corrections labeled by principle — the analyst’s changes, each named by which Part I principle justified it. The bundle takes a few minutes to assemble and saves itself the first time someone asks «why does this chart show what it shows?» six months later.

5. Q5. Ch 8 contrasts Refine with a specific anti-pattern the sibling book calls «prompt roulette.» What is prompt roulette, and how does Refine (done right) avoid it?

   Show answer

   Prompt roulette: rerolling AI output without analysing why the previous attempt failed — retyping the prompt with small variations, hoping one produces something useful. Refine done right: name the specific Part I principle the current draft violates (e.g., «Ch 5 pre-attentive-attribute rule: three colors are directing attention when one should») and direct a targeted correction («mute all bars except the top one to gray»). Refine generates information about which parts of the specification the model got right and which it did not; prompt roulette generates no information at all. Ch 13 returns to prompt roulette as one of the five diagnostics for vibe charting.

Sara’s CSAR loop, up close

The trace on the left is Sara’s cold pass: the director asked for a one-page EMEA Q3 summary before noon; Sara typed the ask into Copilot verbatim; four pages, seven visuals, one Smart Narrative paragraph, three drill-paths came back. Reading it, she has no idea what to trust. The trace on the right is the CSAR pass: Crystallize prompt asking Copilot what IT would need to know; six clarifying questions Sara answers in a few minutes with two corrections; Scope restates the corrected scope; Assemble produces a one-page draft; Refine catches an accessibility issue on brand palette; Sara ships in twelve minutes with a satisfied director acknowledgement.

1. The Crystallize move surfaces gaps Sara had not articulated. Three of Copilot’s six clarifying questions asked things Sara did not know she cared about (currency, structural-vs-temporary distinction, plan definition ambiguity). Two matched things she had thought about but not written down. One was irrelevant — and irrelevant questions cost seconds to dismiss.
2. The most valuable Crystallize move is asking the AI what IT needs. Not asking Sara to specify what she wants; asking Copilot to name what it needs to know before it builds. The cost of fielding irrelevant questions is small; the cost of shipping an artefact whose audience asks a question you did not anticipate is large.
3. Scope catches misunderstandings before Assemble builds against them. Copilot’s restatement is the last chance to correct a misheard constraint before the pipeline commits pages to it. Sara corrected one remaining ambiguity about the plan definition here.
4. Refine is where the Part I rubric shows up. Sara noticed the first chart used the brand palette without accessible contrast; Copilot revised it in one exchange. Refine is not polish for its own sake; it is the Part I discipline applied to Copilot’s output.

Inspect the trace. The full CSAR loop trace — both attempts, all six clarifying questions with Sara’s answers, and the Refine moves — is available as a flat CSV. Download sara-emea-csar.csv · Dataset README and schema.

Try this activity — walk one full CSAR loop

This activity runs one complete CSAR cycle end-to-end against a CloudRevenue prompt. Total time: 45–60 minutes; keep paper or a note-taking app open to build the defensibility bundle as you go.

1. Pick the artifact and the audience. Target: a single chart on the CloudRevenue dataset that argues services-revenue variance to Q3 plan. Audience: the CFO, who will make a Q4 investment-allocation decision from it.

2. Crystallize. Write a Crystallize prompt that specifies (a) the audience and their decision, (b) the Big Idea the chart should argue, (c) the chart family (from Ch 4), (d) the pre-attentive-attribute plan (from Ch 5), (e) the color-and-typography constraints (from Ch 6), (f) any decisions you want the model to defer back to you. Save the prompt as the first piece of the bundle.

3. Scope. Send only the Crystallize prompt to Copilot with the instruction: «Before generating the chart, restate the specification in your own words. List any details you would infer if I do not clarify, and list any decisions I have not made explicit.» Save the response as the second piece of the bundle. Read it carefully — if the restatement drifts (wrong audience-decision, wrong Big Idea, misread chart family), reply with a correction and repeat until the restatement matches your intent.

4. Assemble. Confirm the Scope restatement and let Copilot build. Take a screenshot of the generated chart as piece three of the bundle. Ask Copilot to list the decisions it made and the decisions it deferred; save both lists as pieces four and five.

5. Refine. Review the generated chart against the Ch 4 (family), Ch 5 (focus), and Ch 6 (color/type/accessibility) rubrics. For each correction you want, write it as: «[Part I principle name]: [current state]. Fix: [specific direction].» Send the corrections to Copilot as a Refine direction; take the new screenshot; iterate if needed. Save the correction list as piece six of the bundle. Bundle now has all six pieces.

6. Ship / discard decision. Look at the final chart. Ask: «Would I defend this chart to the CFO tomorrow?» If yes, the loop succeeded — ship, with the bundle attached. If no, the loop did not close the specification gap; a second loop with a sharpened Crystallize prompt is cheaper than shipping a chart you cannot defend.

7. Reflect (three prompts). Write one sentence in response to each:

   1. Did Scope catch drift you would have missed at Assemble? If so, what specifically?
   2. Which Part I principle showed up most in your Refine corrections?
   3. How long did the full loop take, versus how long you would spend hand-building the same chart?

   The reflection is the activity’s payoff. Skipping it turns the exercise into paperwork.

Optional extension: run the same specification through two separate Copilot sessions (a fresh loop each). Compare the two Assemble outputs. The Chapter 9 concept of nondeterminism will surface visibly — the two outputs will differ despite identical inputs, and the Refine directions needed will differ accordingly.

Related site resources

• Prompt Library: the seven canonical CSAR prompts from Appendix B, in Markdown and Word. The most direct pairing with this chapter.
• CloudRevenue dataset: the dataset the activity above runs against.
• Appendix E — BRD Field Guide: the BRD that the Crystallize prompt encodes as specification; if your Crystallize step feels thin, the BRD template is the scaffolding.
• Chapter 7 companion: the previous chapter — the readiness gate you should have passed before running the loop.
• Chapter 9 companion: the next chapter — applies the CSAR loop to full Copilot report generation, where the specification stakes go up.

Cited in the book

Chapter 8’s bibliography lives in the book’s Appendix C (living version in errata). The chapter’s operative references:

• The Verification Habit (sister book). The general CSAR loop originates there; Ch 8 is the visualization-domain adaptation.