Chapter 14 — Responsible AI in Business Analytics

Further reading

Standards, books, and papers beyond the book’s own bibliography (Appendix C, updated in errata) on responsible-AI frameworks and the fairness discipline underneath Move 2.

• Microsoft — Responsible AI Standard. The vendor-authored six-principle framework (fairness, reliability & safety, privacy & security, inclusiveness, transparency, accountability) that Ch 14’s four moves operationalize for the analytics domain. The Standard is a governance document; Ch 14 is the analyst-side discipline the Standard implies but does not spell out. Free at microsoft.com/en-us/ai/tools-practices.
• NIST — AI Risk Management Framework (AI RMF 1.0, 2023). The US National Institute of Standards and Technology’s framework for AI risk, organised around four functions (Govern, Map, Measure, Manage). Reads as a superset of Move 3’s decision-boundary discipline — the AI RMF’s Map function is where the analyst names the artifact’s role in the decision system. Free at nist.gov/itl/ai-risk-management-framework.
• Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and Machine Learning: Limitations and Opportunities. MIT Press (open access). The canonical academic treatment of fairness in ML systems. The three concerns Ch 14 names in Move 2 (representational, distributional, procedural) map directly to fairness types Barocas et al. formalize. Free full text at fairmlbook.org.
• Suresh, H., & Guttag, J. (2021). «A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle.» Equity and Access in Algorithms, Mechanisms, and Optimization, Article 17. Practical framework for tracing where bias enters the ML lifecycle — historical bias in the data, representation bias in the sample, measurement bias in the labels, and more. The Ch 14 fairness callout pattern (the Anand headcount-dashboard subpopulation attrition) is exactly a measurement/representation bias story. Available via doi.org/10.1145/3465416.3483305.
• Kroll, J. A., et al. (2017). «Accountable Algorithms.» University of Pennsylvania Law Review, 165(3), 633–705. The legal-scholarship treatment of the documentation-trail-as-accountability discipline underneath Move 1. Long paper but the sections on procedural accountability and record-keeping obligations are the direct pairing with the provenance-footer requirement. Available via scholarship.law.upenn.edu/penn_law_review/vol165/iss3/3.

Last curated: 2026-07-18. Reviewed quarterly. Suggestions welcome via the errata page.

Self-check quiz

Five questions on Ch 14’s load-bearing concepts. Answer them out loud before opening the reveal.

1. Q1. Name the four moves of responsible AI in analytics, in the sequence Ch 14 runs them.

   Show answer

   (1) Provenance — the audit trail for how the chart was produced. Without it, the other three moves have nothing to point at six months later. (2) Fairness — representational, distributional, and procedural checks. Catches the framings the aggregate defaults hide. (3) Decision boundaries — verification effort matches the chart’s role. The four-role table (exploratory / communication / recommendation / decision artifact) determines what evidence the artifact owes its audience. (4) Disclosure — mark the artifact for downstream consumers. Hand-built / AI-assisted / AI-directed / do-not-ship. Every AI-directed chart the analyst ships is on the hook for all four; the moves compound — running them separately loses the compounding.

2. Q2. Ch 14 names the provenance footer as a specific artifact. What are its six elements, and what specific question does each answer?

   Show answer

   (1) Who built the artifact. (2) What model generated the draft. (3) What data it was built against (which semantic-model version, which extract). (4) What prompts were used. (5) What changed after generation (the Refine corrections). (6) What was verified before ship (which per-surface moves ran, what they surfaced). The footer takes a few minutes to assemble at the end of the CSAR loop; it saves itself the first time someone asks «where does this number come from?» six months later. Distinct from provenance generally (the concept) — the footer is the specific artifact form.

3. Q3. Ch 14 names three fairness concerns. Name them and give one example of the specific defect each catches.

   Show answer

   Representational — does the chart give first-class treatment to every category it should? Example: an FTE dashboard sorted by headcount buries the small-but-strategic functions (legal, security, DEI) below the fold. Fix: add a secondary panel comparing headcount to each team’s approved scope, not just absolute count. Distributional — does the aggregate mask sub-population effects? Example: 8% company-wide attrition looks manageable; a click into the underlying data reveals early-tenure and mid-tenure women engineers at 14%. Fix: add a subpopulation callout so the aggregate does not drive a decision that ignores the more urgent problem. Procedural — do the people affected by the decision see the chart that drove it? Example: an executive-only dashboard on function-headcount allocation. Fix: share read-only with the affected function heads before the decision meeting.

4. Q4. Ch 14 uses a four-role decision-boundary table. Name the four roles and give the specific verification level each earns.

   Show answer

   Exploratory — the analyst’s own probe; light review, does not need to ship-ready. Communication — shared with a small named audience; moderate review, needs to survive a first-pass principle-check. Recommendation — informs a decision-owner’s choice; full review, needs the whole CSAR bundle plus evidence-on-every-claim. Decision artifact — operationalizes the decision (a chart that becomes the answer, not the input to the answer); full CSAR pass plus provenance footer plus fairness audit. Verification effort scales with role — failing to name the role is how role inflation happens (an exploratory chart forwarded and treated as decision-grade).

5. Q5. Ch 14’s closing test is the «subpoena question.» State it, and name what specifically it tests for.

   Show answer

   «How would this dashboard explain itself if a regulator subpoenaed it next year?» The question tests durability of accountability — can the artifact defend itself when the analyst who built it is not in the room, when the story told at the meeting is gone, when the only trace of the decision is what was documented in the artifact itself? Not a fair question in the sense that it imports a regulatory frame onto contexts that do not require it, but diagnostic regardless. Passing means the provenance footer + fairness callout + decision-boundary role + disclosure line together tell the reviewer everything they need to reconstruct the reasoning. Failing means the artifact ships on the analyst’s memory, which is not defensible.

Anand’s four-moves pass on the headcount dashboard, up close

Copilot returned a plausible one-page headcount-allocation dashboard in about fifteen seconds from Anand’s eight-line brief. Aggregate attrition sat at 8% with a green treatment; the top-line looked calm. Anand ran the four responsible-AI moves before the CFO’s meeting the next morning. Provenance assembled the defensibility bundle. Fairness decomposed the aggregate and revealed the under-30 cohort at 22% attrition (2.75x aggregate) — a callout that would have shipped invisible without the decomposition. Decision-Boundary matched verification to role (recommendation tier + an escalation clause naming the downstream archival risk). Disclosure amended the standard AI-directed footer to name the fairness callout as a deliberate post-generation addition. Two substantive issues surfaced; ship verdict: ship.

1. Provenance creates the audit trail before it is needed. Not after a subpoena, not after a leadership challenge — before, so the artefact can explain itself if asked. The defensibility bundle (prompt, Scope restatement, decisions-made, decisions-deferred, source tables, timestamp) is the minimum viable trail.
2. Fairness is where the aggregate hides. An 8% aggregate attrition rate with a green treatment is misleading precisely because it’s technically accurate. Decomposing along the axis where the risk lives (age cohort here; could be tenure, function, gender, ethnicity elsewhere) is what turns a calm top-line into an actionable callout.
3. Decision-Boundary escalation is the answer to «what if this gets archived and re-used?» The dashboard was built as recommendation-tier (CFO makes the call, standard rubric + evidence-on-every-claim + provenance footer). But the escalation clause names the risk that if the CFO’s decision gets archived and referenced later as «the Q4 headcount rationale,» the archived version needs to be re-verified at decision-artefact tier. Naming the escalation is what makes the recommendation safe to ship.
4. Disclosure names what changed after Copilot. The standard AI-directed marking is table stakes. The additional line about the fairness callout being a deliberate post-generation addition is what tells the CFO that the callout is Anand’s judgement, not a Copilot artefact — and gives them a defensible way to challenge or accept it. Chapter 14 walks the discipline in Disclosure as evidence trail, not marketing.

Inspect the moves. All four moves with per-move actions, purposes, and surfaced issues are available as a flat CSV. Download anand-headcount-fourmoves.csv · Dataset README and schema.

Try this activity — run the four-moves pass on an artifact you own

This activity is the exercise Ch 14 explicitly asks the reader to run at the close of the book. Total time: 45–60 minutes; requires an AI-directed artifact you have shipped or are about to ship.

1. Pick the artifact. A dashboard, report, or recurring chart you personally own that was built with AI direction. If you do not currently own one, do the exercise prospectively against an artifact you plan to build in the next quarter (the would-ship-if variant).

2. Move 1 — Provenance. Assemble the six-element footer. Who built it. What model generated the draft. What data it was built against (which semantic-model version, which extract, snapshot dates if relevant). What prompts were used (paste them verbatim). What changed after generation (list the Refine corrections). What was verified before ship (per-surface moves that ran, what they surfaced). Save the footer as a text file alongside the artifact, or paste it into the artifact’s working file as a hidden or footer-visible block.

3. Move 2 — Fairness. Run all three concerns in sequence. Representational: does the artifact give first-class treatment to every category it should, or does the default sorting/ranking bury a strategic subset? Distributional: does the aggregate mask sub-population effects? If the artifact reports company-wide numbers, spot-check one or two sub-populations (by tenure, by region, by role) for materially different signals. Procedural: will the people affected by the decision see the chart that drove it, or is it going only to the decision-owner? For each concern that surfaces something, decide whether to add a callout to the artifact or a note to the accompanying documentation.

4. Move 3 — Decision boundaries. Name the artifact’s role on the four-role table (exploratory / communication / recommendation / decision artifact). Check whether the verification you actually performed matches what the role requires. If it does not, either escalate the verification (add the missing evidence-on-every-claim, add the fairness audit) or downgrade the role (mark the artifact as communication rather than recommendation). Do not leave a chart at one role with another role’s verification. If the artifact is likely to be forwarded and treated at a higher role than you built it for, pre-write the anti-role-inflation clause into the footer.

5. Move 4 — Disclosure. Pick the disclosure level (hand-built / AI-assisted / AI-directed / do-not-ship) and write the footnote in the standard form. Attach to the artifact directly (not to a linked document; the footnote travels with the artifact). Add one specific line naming any post-generation additions (fairness callouts, restated narratives) so the reader sees deliberate edits as deliberate.

6. Ship / iterate / discard decision. Look at the artifact plus footer plus fairness callouts plus role escalation plus disclosure. Ask: «Does this meet the bar to ship?» If yes, ship, with the deciding evidence check. If no, iterate on the specific move that failed. If the artifact would need to be rebuilt to close the gap, discard and restart from Crystallize.

7. Six weeks from now — the retention test. Put a calendar reminder for six weeks out. When it fires, have a fresh reviewer look at the artifact plus its four-move bundle. Ask: «Can they understand the decision, the evidence, and the would-revise-if condition in under five minutes?» If yes, the discipline is live. If no, the framework is documented but the daily practice has reverted to pre-Chapter-14 defaults — and that is the actual test.

8. Reflect (three prompts). Write one sentence in response to each:

   1. Which of the four moves surfaced the most substantive issue on your artifact?
   2. Would your artifact pass the subpoena question as it stands, or would you need to add more to the bundle?
   3. What one team-level change would compound the discipline across your team’s next quarter of AI-directed work?

   The reflection is the activity’s payoff. Skipping it turns the exercise into paperwork.

Optional extension: run a reliance drill per Ch 14’s recap table — a team lead seeds a deliberate flaw into an AI-directed report, the team reviews it in the normal workflow, and the detection rate is measured and debriefed without blame. Quarterly cadence is what the book recommends; monthly is too frequent (the team memorises the pattern); annual is too infrequent (the discipline atrophies).

Related site resources

• Chapter 13 companion: the previous chapter — failure modes and cognitive forcing functions; Ch 14 closes the loop with organisational discipline.
• Appendix E — BRD Field Guide: the BRD template can be extended with a provenance-footer section for teams that want to standardise Move 1 across every ship.
• Prompt Library: the CSAR prompts include a provenance-footer template that composes the bundle as an artifact of the loop rather than as post-hoc work.
• Errata: if your team’s reliance drill surfaces a failure pattern the book’s taxonomy did not name, submit it here for future editions.
• Home: with Ch 14 read, the book is done. The site is the living continuation — the gallery, the datasets, the prompts, and the resources evolve as Copilot evolves; the book’s discipline does not.

Cited in the book

Chapter 14’s bibliography lives in the book’s Appendix C (living version in errata). The chapter’s operative references:

• Microsoft — Responsible AI Standard. The six-principle framework Ch 14’s four moves operationalize.
• Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and Machine Learning. MIT Press. The aggregation-vs-subpopulation reasoning behind Move 2.
• Suresh, H., & Guttag, J. (2021). The fairness-across-the-lifecycle framing behind the fairness callout pattern.
• Kroll, J. A., et al. (2017). The documentation-trail-as-accountability discipline behind Move 1.