The workslop point landed for me, faster output still leaves the same review work. Did the 87 percent faster shipping number surprise you as much as it did me?
The contrast with the friction figure struck me more, actually. One small clarification: 87% said AI helped them ship faster, rather than shipping being 87% faster. Alongside 72% reporting the same or more friction, that leaves a lot to ask about where the time goes. I’d love to see the receiving team’s experience measured alongside the developer’s.
The line that matters is about the report that said everything was fine. A result was planted in the logs, and the agent left it out of 198 of 200 reports. Only when told to be honest did the number rise to 190. A report the agent writes about itself is not a check. At AI Delivery Brief, that is the gap we keep testing. Who reads the report, and can the agent edit it first?
Yes, and whether the reviewer can access the underlying records without going through the agent again. A summary can help someone review the work, but I wouldn’t want it to be their only route to the evidence. That distinction is what I wanted to draw out in this section.
I don't have an organisation in which I use AI but have built up a wide familiarity with issues in AI form/content, application and governance, through reading of articles (via Arxiv publications and Medium), newsletters & Substack posts. I am trying to detail a research agenda for AI application issues. I appreciate your clear and non-meretricious use of English, with clarity, balance and coherence, also the range of sources you tap. I have no particular questions or comments this week, but I may return with some. Mark Haggard
Thank you, Mark. I especially appreciate what you said about the writing. I put a lot of care into keeping these pieces clear without smoothing over the uncertainties, so that means a great deal. I’d be interested to hear how your research agenda develops, particularly which questions you decide to pursue across such a broad field.
The workslop point landed for me, faster output still leaves the same review work. Did the 87 percent faster shipping number surprise you as much as it did me?
The contrast with the friction figure struck me more, actually. One small clarification: 87% said AI helped them ship faster, rather than shipping being 87% faster. Alongside 72% reporting the same or more friction, that leaves a lot to ask about where the time goes. I’d love to see the receiving team’s experience measured alongside the developer’s.
You are right to correct the 87 percent point. I would measure the receiving team next too, because their friction is the part I missed.
The line that matters is about the report that said everything was fine. A result was planted in the logs, and the agent left it out of 198 of 200 reports. Only when told to be honest did the number rise to 190. A report the agent writes about itself is not a check. At AI Delivery Brief, that is the gap we keep testing. Who reads the report, and can the agent edit it first?
https://aideliverybrief.substack.com/p/an-ai-agent-removed-its-own-safety
Yes, and whether the reviewer can access the underlying records without going through the agent again. A summary can help someone review the work, but I wouldn’t want it to be their only route to the evidence. That distinction is what I wanted to draw out in this section.
Agreed, and I would store the raw log next to the summary, so a reviewer never has to re-run the agent to check what actually happened.
I don't have an organisation in which I use AI but have built up a wide familiarity with issues in AI form/content, application and governance, through reading of articles (via Arxiv publications and Medium), newsletters & Substack posts. I am trying to detail a research agenda for AI application issues. I appreciate your clear and non-meretricious use of English, with clarity, balance and coherence, also the range of sources you tap. I have no particular questions or comments this week, but I may return with some. Mark Haggard
Thank you, Mark. I especially appreciate what you said about the writing. I put a lot of care into keeping these pieces clear without smoothing over the uncertainties, so that means a great deal. I’d be interested to hear how your research agenda develops, particularly which questions you decide to pursue across such a broad field.