Start with the question the score should help answer
Engagement scoring turns recorded interactions into a summary that staff can use. In Chapter 1 of The Innovation Forge, David introduces events, communications, portal activity and application progress as possible inputs. The useful starting point is the purpose of the summary, before choosing a point scale.
Write down the question a colleague will ask when they see it. Are they trying to understand a recent change, prepare a conversation or find an unanswered request? Identify what action the score could help inform and what other context that person needs.
A score describes the behaviors your system can observe. It is not a complete account of someone’s interest, circumstances or needs. Keep the underlying interactions available so a staff member can understand the pattern instead of treating the total as the explanation.
In the recording · 06:31 ↗Define the behavior before counting it
An email attached to a student record is not necessarily an email from that student. The chapter specifically distinguishes student communication from counselor messages and documents, and notes that privacy protections can make engagement statistics noisy.
For every proposed input, specify who performed the action, what qualifies, which period it covers and where the evidence comes from. Decide whether the measure counts distinct events, repeated actions or the presence of an action at all. Keep automated or unrelated activity from quietly changing its meaning.
Test a few known examples before calculating the total: a student reply, an outbound message, a counselor attachment and an interaction with uncertain tracking. If two reviewers would count those differently, settle the definition first.
In the recording · 11:34 ↗Check whether the opportunity was accessible
The chapter’s event example comes with an important qualification. A campus visit requires time and resources, so a rubric that heavily rewards visiting can reflect access to the opportunity as well as engagement. David points toward virtual events, recordings and other ways to experience the institution.
List the ways a person can take part and what each activity allows them to do. Review whether meaningful participation is recognized across those routes. Do not assume that attending in person always supplies better evidence of interest than participating remotely.
Apply the same care to portal activity. The reading notes that someone may complete requirements with little online interaction. A low count should prompt a look at the available context, including whether the person had a reason to return, rather than an automatic conclusion that they are disengaged.
In the recording · 10:34 ↗Make the weights explainable and revisable
Assigning points involves a judgment about which behaviors matter. Chapter 1 recommends involving staff and other stakeholders, then informing the rubric with consistent historical data and an outcome that can be defined. A new activity may initially need a provisional judgment because no useful history exists yet.
Record the reason for each weight and the evidence supporting it. Separate a staff hypothesis from a pattern observed in prior data. Name the outcome and time period used in the comparison, and keep later activity from leaking into an earlier assessment.
Review a changed weight against several different behavior patterns. Look for an input that dominates the score through repetition or an apparent improvement caused by a recording change. A rubric needs a review date and an owner as much as it needs a calculation.
In the recording · 08:19 ↗Keep the component scores visible
The chapter proposes component scores for areas such as events and communication alongside the overall score. Those parts help explain why two similar totals can represent different journeys.
Choose components that correspond to questions staff actually ask. Show which actions contributed and when they occurred. If one person has recent replies and another has older event activity, the same total should not erase that difference.
Treat missing information separately from an observed zero. A source that has not refreshed cannot establish that no activity occurred. Keeping that distinction visible makes the score easier to question and reduces the chance of acting on an incomplete picture.
In the recording · 13:31 ↗Make refresh and review part of the design
Queries and source formats can support a scheduled scoring process, but the schedule should match the work that depends on it. Define when the inputs arrive, when the calculation runs and when staff can expect to see a current result.
Start with a small set of known examples and compare the stored results with the intended rubric. Record the version of the weights and the last successful refresh. Decide who investigates missing input or a surprising change, then revisit the score with the people using it. A dependable process keeps both the calculation and its interpretation open to review.
In the recording · 09:39 ↗