Read the evidence at the right scale
AI can draft an email, answer a routine question or contribute a prediction to an enrollment workflow. Each use changes a different part of the work. O.R.E. Report 002 groups the discussion around automation, prediction and engagement, then asks who remains responsible for their consequences.
Keep the evidence categories separate. The EducationDynamics article from July 2025 summarizes survey respondents' adoption and perceptions; it does not establish outcomes for every institution. The September 2021 Brookings analysis provides historical criticism of enrollment algorithms. Vendor case studies discussed in the episode describe particular implementations, often alongside other process changes. Reported gains can suggest a question to investigate, but they do not establish what AI alone caused or what another institution should expect.
In the recording · 05:23 ↗Map the tool, the data and the person responsible
Begin with a working inventory rather than a list of product names. For each AI-enabled feature, describe the communication or decision it influences, the information it receives and the person responsible for reviewing its use. Include features embedded in an existing CRM, not just separately purchased assistants.
Follow the data beyond the input screen. Record whether information reaches an outside model, who can access it, how long it is retained and what controls have actually been checked. An unknown answer belongs in the review queue. It should not disappear behind a vendor's general assurance or a familiar platform name.
In the recording · 14:04 ↗Test one workflow with a visible handoff
The report proposes a bounded AI-and-human pairing before wider adoption. Choose a routine service question that can be tested with fictional examples. Define the approved information the assistant may use, the situations it must escalate and how a person can request human help without repeating their story.
Measure more than message volume. Review answer accuracy, unresolved questions, response time and the effort staff spend correcting or completing the interaction. Compare like periods and document other changes that could explain a difference. A faster first reply is useful only if the next step is accurate and the handoff works when the question becomes complicated.
In the recording · 14:38 ↗Keep prediction open to challenge
Historical data records earlier opportunities and institutional choices as well as student behavior. A model can reproduce those patterns while appearing consistent. The episode therefore calls for equity reviews and rejects using predictive scores as the sole basis for denying admission or reducing aid.
An educational model review can examine the target, observation period, missing information and errors without making decisions about individual applicants. Ask whether a prediction is being treated as a fact, whether performance differs across groups and whether a changed process has made older data less relevant. Removing one variable does not establish fairness; related inputs may preserve the same pattern. Qualified institutional reviewers need evidence and authority to question the use.
In the recording · 15:27 ↗Give oversight a practical job
The proposed governance group brings enrollment, financial aid, technology, security, research and student perspectives into the review. Give it concrete work: examine new uses, document minimum privacy and transparency standards, review major changes and record decisions about prohibited uses.
Staff literacy belongs in that process. Let frontline colleagues critique fictional chatbot exchanges for accuracy, clarity, tone and escalation. A disclosure should describe the actual workflow. Do not promise that people review every message if they only sample outputs, or advertise human assistance without a dependable route to it.
In the recording · 16:16 ↗Write down what an improvement cannot cost
Optimization requires an objective, and objectives leave things out. The report asks leaders to name both their priorities and the outcomes they will not accept in pursuit of better enrollment or revenue figures. Those limits need an owner and a place in the evaluation.
Before expanding a workflow, write down what success means, what would trigger a pause and who can make that call. Keep evidence of student experience and access beside operational metrics. Reviewing those boundaries when a tool or policy changes helps keep institutional responsibility visible instead of treating a favorable dashboard as sufficient approval.
In the recording · 18:31 ↗