What this automation does
The work is split cleanly. The system collects and checks. The manager decides and writes.
- Manager opts in per report. At the start of the cycle, each manager chooses whether to use the assistant for each direct report. Nothing runs by default.
- Gather evidence. From approved sources only, the workflow pulls the review period’s goals and their status, completed tickets and projects from Jira or Asana, peer feedback already collected through your review process, recognition posts from a kudos channel, and any notes the manager chooses to include.
- Build an evidence digest. A model groups the evidence by goal or competency, links every item to its source, and lists periods or goals with little or no evidence so the manager knows where to look harder.
- Manager writes. The manager writes the review in a Google Doc next to the digest. If they ask, the assistant can suggest an outline or rephrase a paragraph they wrote, clearly labeled as a suggestion.
- Quality check. Before submission, the assistant reviews the manager’s draft and flags vague claims without examples, personality labels instead of behaviors, feedback drawn only from the last few weeks, and statements that contradict the evidence.
- Submit. The manager copies the final text and their rating into your review system. The assistant has no write access to ratings or the review record.
When it is worth building
Managers with six or more reports often spend several hours per review, and most of that time goes to scrolling through tickets, documents and old messages to remember what happened in March. If your cycle produces reviews that are generic, recent-only or late, an evidence digest helps more than a writing tool does.
It is not worth building if goals are not tracked anywhere, because there is nothing for the digest to organize. Fix goal setting first. It is also not worth building if your review platform already gives managers an evidence view and AI suggestions that cover your systems.
Architecture
| Layer | What we use | Why |
|---|---|---|
| Evidence sources | Jira, Asana, Google Docs activity, a Slack kudos channel, collected peer feedback | Work records that employees already know are visible to their manager |
| Exclusion rules | An allow-list of sources and fields, enforced in code | The model can only see what has been explicitly approved |
| HR context | BambooHR or Workday for reporting lines and review period only | Workday access usually needs customer-specific API approval, which affects timeline |
| Model | Claude or OpenAI’s GPT models through the API | Long context for a year of work items; API data is not used for training by default |
| Manager workspace | A Google Doc per review with the digest alongside | Managers write where they already write, and the doc history shows authorship |
| Quality checker | A model pass with a fixed list of checks and reasons | Feedback on the draft, never a rewritten draft |
| Access control | Per-manager permissions and short retention for digests | Review material is sensitive personnel data |
How to build it
- Agree on principles with HR and legal. Write down that managers author reviews, ratings are human, which sources are allowed, and how long digests are kept. Share it with employees before the cycle.
- Pick sources per role family. Engineering might use Jira and code review data; sales might use CRM activity; support might use resolved tickets and quality scores. Make sure every role has some evidence source.
- Build the evidence pull. Fetch only the review period and only the person under review. Store raw items briefly, then delete them after the digest is built.
- Write the digest prompt. Require a source link for every item, forbid adjectives about the person, and require a “thin evidence” section.
- Write the quality checks. Base them on your own review guidelines: specific examples, behaviors over traits, coverage of the full period, clear next steps.
- Pilot with a few managers. Compare digests and final reviews with the previous cycle’s reviews for the same managers. Ask employees in the pilot whether their review felt specific and fair.
- Roll out with training. Show managers what the assistant does and does not do. Emphasize that pasting a suggestion unedited defeats the purpose.
What breaks in production
- Uneven evidence. People whose work leaves a trail in tickets look more productive than people who coach, unblock or handle customers by phone. Show evidence coverage per role and adjust sources.
- Invented accomplishments. The model describes a project the person did not lead. Every item must link to a source, and the checker should confirm the link exists.
- De-anonymized peer feedback. Quoting peer feedback verbatim can reveal who wrote it. Summarize themes and keep quotes out unless your process already shares them.
- Rubber-stamped drafts. A manager accepts a suggested paragraph wholesale. Track how much managers change suggestions, and remind them when edits are near zero.
- Sensitive data leaking in. A ticket mentions someone’s medical leave. Add keyword screening on sources and drop flagged items before the model sees them.
- Retention. Digests left in shared drives become an unofficial personnel file. Set short retention and restrict access.
Timeline and cost
A pilot for one or two role families typically takes three to six weeks, including the principles work with HR. Lumeor builds this in the range of $10,000 to $25,000, depending on the number of evidence sources, the HRIS and how much access control and retention tooling you need. Model costs per review are small relative to manager time. If you are not sure whether AI belongs in your review process at all, start with a conversation about the principles before any build.
How to measure success
- Manager hours per review, from manager self-reports before and after
- Share of reviews that cite specific, dated examples
- Share of reviews covering work from the whole period rather than the last quarter
- How much managers edit suggestions, which should stay meaningful
- Employee survey results on whether the review felt specific and fair, by team and role
- Calibration disagreements caused by missing or contested evidence
Teams that already use AI meeting summaries and action items have a cleaner evidence trail, but only include meeting notes in reviews if employees have been told they will be used that way.