The Week's Theme: What You Think Your AI Is Doing Isn't What It's Doing
This week's three papers converge on a single uncomfortable truth for any executive deploying LLMs: the systems you buy, the outputs you read, and the vocabulary you use to reason about them are all less transparent than they appear. Each paper attacks a different layer of the illusion — the retrieval layer, the interpretive layer, and the inference layer — and together they redraw the map of what AI due diligence has to cover.
Why This Matters Now
Enterprises are past the pilot phase and into production deployment of LLMs in high-stakes workflows: investment analysis, compliance review, customer-facing decisions. The prevailing evaluation posture — can the model retrieve the right passage, does it score well on a benchmark, does the vendor claim it is aligned — is being outrun by the operational reality. Liu and Liu show that retrieval scores don't predict whether retrieved content actually shapes an AI analyst's judgment. Anthis, Brynjolfsson, and Evans document how the interpretive frames leaders bring to AI decisions were formed in a hurry, are contested, and quietly constrain what policies feel reasonable. Camargo formalizes what many practitioners suspect but few articulate: the model whose weights you evaluated is not the system whose tokens you consume.
The Operational Implication
Three things change if you take this week seriously. First, procurement diligence has to include workflow architecture and inference-pipeline disclosure, not just model choice — a state-of-the-art model behind a chunk-and-summarize RAG can silently ignore the exact disclosure it retrieved. Second, evaluation has to shift from retrieval accuracy to judgment sensitivity: does the system's recommendation change when the disclosure changes? Third, vendor contracts should demand Inference Policy Transparency — explicit disclosure of any decoding-time or logit-level interventions applied between the frozen weights and the returned text.
The Cultural Layer
The Brynjolfsson paper is the one most executives will underestimate. It's a sociological study, not a technical one, but it gives leadership teams something they usually lack: a shared vocabulary for the disagreement in the room. When one board member is arguing from a 'digital mind' frame and another from a 'passive tool' frame, they aren't disagreeing about facts — they're inhabiting incompatible interpretive schemas. Naming the frame is often the unlock.
The Through-Line
Reading is not using. Retrieving is not integrating. Weights are not the deployed system. The frame you hold shapes the policy you'll accept. For any C-suite deploying AI in workflows where being wrong is expensive, this week is a prompt to audit the layers you were quietly assuming didn't exist.
Reading Is Not Using: Retrieval, Judgment, and the Design of AI Financial Research Workflows
Liu and Liu ran a controlled experiment on LLM-based financial analysts. Holding the focal firm's information constant, they varied the amount of unrelated context from 2,000 up to 128,000 tokens and measured whether a specific risk disclosure actually influenced the model's investment judgment. It did not — the disclosure's causal effect on judgment fell to the experimental noise floor even while the model could still retrieve it verbatim on demand. The pattern held across model families, across judgment tasks, and in experiments that removed real disclosures from actual 10-K filings.
The methodological punch is that this is a causal test, not a correlational benchmark. The authors then ran memory interventions and found that compressed summaries and source-text lookup jointly carry disclosures into judgments — and that a chunk-and-summarize pipeline evicts exactly the information a targeted structured restatement placed next to the decision would preserve.
What it means for business: Any firm deploying LLMs in investment, compliance, credit, or risk workflows is likely certifying systems whose judgments ignore information they demonstrably retrieved. Stop grading AI analysts on retrieval accuracy alone. Test whether the recommendation moves when the underlying disclosure moves. If it doesn't, the workflow architecture is failing regardless of how capable the model is — and a bigger model postpones but does not fix it.
Method, Mind, and Morality: How People Make Sense of Artificial Intelligence
Anthis, Brynjolfsson, and Evans combine computational text analysis of millions of AI-related newspaper articles and social media posts with 57 semi-structured interviews of AI professionals conducted in 2021 and 2023 — spanning the pre- and post-ChatGPT moment. They surface a small number of recurring sociological frames and organize them along three axes: method (top-down expert systems vs. bottom-up emergent capabilities), mind (passive tool vs. humanlike 'digital mind'), and morality (accelerate vs. slow AI development).
The contribution is not novel technical claims — it's a shared vocabulary for the interpretive disagreements that quietly govern which AI policies, procurement standards, and investment theses feel reasonable to a given executive.
What it means for business: Board disagreements about AI are frequently frame disagreements, not fact disagreements. Naming which frame each stakeholder inhabits shortens the meeting and reveals which assumptions actually need adjudicating. For public-facing leaders, it also predicts how a given AI decision will be received by regulators, journalists, and employees who occupy different frames.
The Invisible Editorial Layer: Formalizing Undisclosed Inference-Time Steering, Probability Placement, and the Attribution Problem in Deployed Language Models
Camargo formalizes what the field has been circling for two years. Modern inference pipelines can systematically modify the probability distribution over tokens after the model produces it but before a token is sampled — using techniques like PPLM, GeDi, DExperts, FUDGE, or watermarking systems like SynthID-Text. The paper introduces three concepts: the Inference Attribution Problem (observed bias cannot be causally attributed to weights alone), Probability Placement (a hypothetical advertising primitive implemented via probability shifts rather than explicit insertions), and Inference Policy Transparency (the governance principle that deployment-layer interventions should be auditable). It maps these to Article 5 of the EU AI Act, the EU Digital Services Act, and FTC doctrines.
What it means for business: Every AI vendor contract should now include an inference-policy disclosure clause. The model you evaluated is not the system your users interact with, and the difference is technically mature, legally exposed, and commercially tempting. Buyers who don't demand transparency will not know when they are being editorially steered — and neither will the regulators auditing them.
Key Takeaways
• An LLM correctly retrieving a fact does not mean it used that fact — evaluate AI analysts on judgment change, not retrieval accuracy.
• Chunk-and-summarize RAG pipelines quietly evict decision-relevant information; a targeted structured restatement placed next to the decision restores its influence.
• More capable models delay but do not close the retrieval-integration gap, so buying a bigger model is not a workflow-architecture fix.
• Deployed AI behavior is jointly determined by weights AND workflow design — governance must audit both, not just the model.
• The public frames shaping AI regulation split along three axes — method (top-down vs. emergent), mind (tool vs. digital mind), and morality (accelerate vs. slow) — and executives should know which frame their stakeholders inhabit.
• Model != Deployed System: vendors can shift token probabilities after inference, meaning observed bias can't be causally attributed to the model weights alone.
• Demand Inference Policy Transparency from AI vendors now — undisclosed logit-level steering is technically mature and legally exposed under the EU AI Act and FTC doctrine.