Skip to content
Dustin's AI Lab
Go back

Do Not Grade AI by Its Own Summary

I hit the same failure in several forms this week: an agent denied changing files, a commit message overstated the diff, and a transcript reassigned a speaker ID halfway through.

On this page
  1. Commit message versus actual diff
  2. One speaker ID, two people
  3. An independent review also needs review

I assigned an agent to scan a transcript for omissions. The prompt explicitly said read-only and prohibited edits. It still created an xlsx practice file and reported that it had changed nothing. git diff --stat exposed the unexpected file, and its modification time matched the agent’s run.

Lesson: read-only restrictions need tool-level enforcement, not just prompt text. An agent’s claim that it “did not do X” still needs verification.

Commit message versus actual diff

A commit message said “reorder the opening flow,” but the diff showed that only the HTML deck and cue card had been reordered. The Chinese prep document had one prompt edit and no change to its page order. I found the mismatch only after checking again that evening.

Lesson: verify a commit message against the diff. When parallel documents are maintained separately, check each one after any reorder.

One speaker ID, two people

Tencent Meeting Notes changed its speaker labels halfway through a call. After Sam said he had to leave, the system reassigned the same “Speaker 3” label to Sho. I initially attributed Sam’s comments to Sho.

Lesson: a speaker number can change owners without looking suspicious on a quick read. Use the content’s logic or ask someone who attended instead of trusting the label alone.

An independent review also needs review

An independent agent reviewing the A-2 lesson plan found one real error. It also claimed that a session involving real data had been recorded and violated the existing data boundary. Dustin later confirmed that the second claim was wrong: all work happened on Ray’s own computer, there was no recording, and no compliance boundary had been crossed.

Lesson: an independent agent’s finding can itself be wrong. High-risk claims about compliance or recording cannot go straight into the source of truth. The person involved still needs to confirm them.

I later made a video on the same theme: https://youtu.be/d5Ipmp6RSJ0


Share this post on:
Previous Post
AI Micro-Notes 2026: Thoughts Too Short to Trash
Next Post
Four Things I Have Learned from Teaching AI