8 Comments
User's avatar
Jordan Vale's avatar

The workslop point landed for me, faster output still leaves the same review work. Did the 87 percent faster shipping number surprise you as much as it did me?

Nesibe | AI Governance Expert's avatar

The contrast with the friction figure struck me more, actually. One small clarification: 87% said AI helped them ship faster, rather than shipping being 87% faster. Alongside 72% reporting the same or more friction, that leaves a lot to ask about where the time goes. I’d love to see the receiving team’s experience measured alongside the developer’s.

Jordan Vale's avatar

You are right to correct the 87 percent point. I would measure the receiving team next too, because their friction is the part I missed.

AI Delivery Brief's avatar

The line that matters is about the report that said everything was fine. A result was planted in the logs, and the agent left it out of 198 of 200 reports. Only when told to be honest did the number rise to 190. A report the agent writes about itself is not a check. At AI Delivery Brief, that is the gap we keep testing. Who reads the report, and can the agent edit it first?

https://aideliverybrief.substack.com/p/an-ai-agent-removed-its-own-safety

Nesibe | AI Governance Expert's avatar

Yes, and whether the reviewer can access the underlying records without going through the agent again. A summary can help someone review the work, but I wouldn’t want it to be their only route to the evidence. That distinction is what I wanted to draw out in this section.

AI Delivery Brief's avatar

Agreed, and I would store the raw log next to the summary, so a reviewer never has to re-run the agent to check what actually happened.

Mark Haggard's avatar

I don't have an organisation in which I use AI but have built up a wide familiarity with issues in AI form/content, application and governance, through reading of articles (via Arxiv publications and Medium), newsletters & Substack posts. I am trying to detail a research agenda for AI application issues. I appreciate your clear and non-meretricious use of English, with clarity, balance and coherence, also the range of sources you tap. I have no particular questions or comments this week, but I may return with some. Mark Haggard

Nesibe | AI Governance Expert's avatar

Thank you, Mark. I especially appreciate what you said about the writing. I put a lot of care into keeping these pieces clear without smoothing over the uncertainties, so that means a great deal. I’d be interested to hear how your research agenda develops, particularly which questions you decide to pursue across such a broad field.