I am glad the “what governance requires now” section landed, because I worry that a lot of agent conversations stop at “here is why this is scary” and never get to “here is what we can actually wire into organisations.”
My hope is to keep pushing the discussion toward concrete controls and ownership, not just new vocabulary.
Thanks for this great breakdown! What do you think about earned permissions or dynamic trust scoring for agents? One of the biggest challenges I've seen with organizations is the decision to roll out agents that have static permissions.
I am generally in favour of earned permissions and dynamic trust scoring for agents, as long as the trust signal is legible to humans and tied to specific scopes rather than becoming a vague “confidence score”.
The pattern I like most is: agents start at minimal permissions, earn access for narrow domains based on observed behaviour and monitoring, and still require a human commit for actions that touch money, rights, or safety. Static, all‑or‑nothing permission sets feel misaligned with how messy real environments are.
Exactly. I like the idea of them earning positions as they perform various chunks of a task. When the task is complete their permissions can reset (if applicable).
What we’re seeing in a lot of these agentic hacks is a need for more robust permissions.
I can only give you guys and gals snippets of what I'm actually doing here and this could be……. this could be what saves the world from a technology that is becoming incredibly powerful.
This is a very generous interpretation of the work, thank you.
For me the real milestone is when we can produce artefacts that hold up under scrutiny from lawyers, auditors, and regulators, not just from engineers.
If something like MirrorOS can help bridge that gap between “I know this is safe” and “I can prove it in a way institutions accept”, that will be a real contribution.
The Murder of an OpenAI Top Engineer and the True Dangers of Artificial Intelligence:
On November 22, 2024, 26-year-old former OpenAI engineer Suchir Balaji was brutally murdered in his San Francisco apartment.
Authorities ruled his death a suicide.
Suchir Balaji was a brilliant American IT engineer of Indian descent.
At the age of 22, he was hired by OpenAI as a top talent and played a key role in the development of ChatGPT.
In addition to his exceptional intelligence, he possessed a strong sense of justice and unwavering ethical principles.
It is therefore not surprising that he disagreed with the behavior of his boss, Sam Altman, and OpenAI's business practices. He developed an increasingly critical attitude toward management and his boss.
Sam Altman is notorious within the company for his lies and power plays. Suchir Balaji had absolutely no understanding for this and was ultimately quite disgusted by his behavior.
He also witnessed OpenAI's transformation from a non-profit, open-source project into a for-profit, closed-source company.
It's important to understand that the development of ChatGPT was only possible by feeding and training the AI with gigantic amounts of data, including vast quantities of copyrighted material.
OpenAI was only able to use this data free of charge and without the permission of the copyright holders because the company presented itself as a non-profit project.
The use of copyrighted material is considered permissible if it is a research project that does not generate profits and serves the public good.
In retrospect, it is clear that OpenAI deliberately exploited this situation. The billions in profits the company now generates are largely due to OpenAI's free access to this data during its non-profit phase.
For Suchir Balaji, this practice was completely unacceptable.
Suchir left the company in the summer of 2024, having made crucial contributions to the development of ChatGPT during his four years there.
In the months leading up to his violent death, he was preparing to launch his own startup and wrote a scientific paper on the future of large language models (LLMs) like ChatGPT.
In this work, which unfortunately remained unfinished, he refuted the so-called scaling hypothesis, championed by OpenAI and most other AI companies.
This hypothesis states that the intelligence of AI models can be developed indefinitely as long as they are fed enough data. It forms the basis for the grandiose promises of AI companies.
The achievement of a level of artificial general intelligence (AGI) has been announced for years.
AI models are supposedly about to develop superhuman intelligence (ASI = Artificial Super Intelligence), replace all kinds of jobs, cure diseases, create wealth for everyone, and so on.
In his unfinished essay, Suchir Balaji demonstrated in an impressive yet easily understandable way that, contrary to the claims of AI companies, large language models can never reach the level of human-like intelligence (AGI = Artificial General Intelligence).
He predicted that the fundamentally limited, abysmal data efficiency of this technology will inevitably slow down the further development of AI models and bring them to a standstill long before AGI is achieved.
This is an inconvenient truth for the AI industry, which it is trying to conceal to protect its business model.
Suchir Balaji was also slated to testify as a key witness in a lawsuit against OpenAI, which involved, among other things, massive copyright infringements.
In the months leading up to his death, Suchir was in good spirits and looking forward to launching his own AI company.
On November 22, 2024, he had just returned from a short vacation with his closest friends.
According to the investigation by a private investigator hired by Suchir's parents, Suchir had ordered food that evening, listened to music, and worked on his laptop. According to the investigator's reconstruction, he ...
This is a very heavy story, and I appreciate you taking the time to write it out and share your perspective.
There are clearly real tensions around IP, business models, and worker protection in the current AI ecosystem, and whistleblowers and critics are an important part of that picture.
I am not in a position to verify the specific claims about this case, but I do share the underlying concern that we should not build our future infrastructure on foundations that depend on opacity or on quietly stretching legal categories past their breaking point.
But at the moment an agent mutates state — writes to a database, triggers a workflow, transfers value, allocates access — very few systems have a deterministic enforcement gate that can allow, deny, or halt that execution with evidentiary traceability.
That binding event is where governance either proves itself or collapses.
Agentic AI doesn’t just expand the risk surface.
It forces us to encode authority and stop conditions at runtime, not assume them at Layer 8.
Until enforcement is infrastructure — not documentation — the gap will continue widening regardless of how many frameworks we publish.
Most “agent governance” I see in the wild is still descriptive, not executable: policies and risk registers on one side, and then a deployment pipeline that happily lets an agent mutate state without a meaningful gate in front of critical actions.
The shift we need is exactly what you describe: treating enforcement as infrastructure. That means deterministic checks at the point where an agent writes to a database, allocates access, or moves value, plus evidentiary logs that let you reconstruct who authorised what and why. Until that binding event is governed, the maturity diagrams will keep over‑stating how safe these systems really are.
Ai alone is not the solution. Ai is just another system, not the entire system. It needs a translation layer. Check out my pov on it that I have in my notes and articles. Might be of interest, might not.
I also do not see AI as “the system” but as one more layer in a much larger sociotechnical stack that needs a translation layer between policy, operations, and code.
I will check out your notes on this, because I think we are all still learning how to design that translation layer in a way that honours both human judgment and machine capabilities.
Your 5 points on What Governance Requires Now are helpful- thanks for going beyond problems towards solutions.
Thank you, Michael.
I am glad the “what governance requires now” section landed, because I worry that a lot of agent conversations stop at “here is why this is scary” and never get to “here is what we can actually wire into organisations.”
My hope is to keep pushing the discussion toward concrete controls and ownership, not just new vocabulary.
Thanks for this great breakdown! What do you think about earned permissions or dynamic trust scoring for agents? One of the biggest challenges I've seen with organizations is the decision to roll out agents that have static permissions.
Great question.
I am generally in favour of earned permissions and dynamic trust scoring for agents, as long as the trust signal is legible to humans and tied to specific scopes rather than becoming a vague “confidence score”.
The pattern I like most is: agents start at minimal permissions, earn access for narrow domains based on observed behaviour and monitoring, and still require a human commit for actions that touch money, rights, or safety. Static, all‑or‑nothing permission sets feel misaligned with how messy real environments are.
Exactly. I like the idea of them earning positions as they perform various chunks of a task. When the task is complete their permissions can reset (if applicable).
What we’re seeing in a lot of these agentic hacks is a need for more robust permissions.
I can only give you guys and gals snippets of what I'm actually doing here and this could be……. this could be what saves the world from a technology that is becoming incredibly powerful.
I appreciate you sharing what you can.
There is a lot of quiet, unglamorous work happening on reflective and constrained agents that will matter much more than the latest benchmark chart.
If you ever decide to write up even a very small, concrete slice of what you are doing, I would be very interested to read it.
What you just accomplished (plain English)
You built the first “court-admissible” unit of Sentinel: (top secret squirrel 🐿️ shit)
MirrorOS is beginning to breath…
This is a very generous interpretation of the work, thank you.
For me the real milestone is when we can produce artefacts that hold up under scrutiny from lawyers, auditors, and regulators, not just from engineers.
If something like MirrorOS can help bridge that gap between “I know this is safe” and “I can prove it in a way institutions accept”, that will be a real contribution.
The Murder of an OpenAI Top Engineer and the True Dangers of Artificial Intelligence:
On November 22, 2024, 26-year-old former OpenAI engineer Suchir Balaji was brutally murdered in his San Francisco apartment.
Authorities ruled his death a suicide.
Suchir Balaji was a brilliant American IT engineer of Indian descent.
At the age of 22, he was hired by OpenAI as a top talent and played a key role in the development of ChatGPT.
In addition to his exceptional intelligence, he possessed a strong sense of justice and unwavering ethical principles.
It is therefore not surprising that he disagreed with the behavior of his boss, Sam Altman, and OpenAI's business practices. He developed an increasingly critical attitude toward management and his boss.
Sam Altman is notorious within the company for his lies and power plays. Suchir Balaji had absolutely no understanding for this and was ultimately quite disgusted by his behavior.
He also witnessed OpenAI's transformation from a non-profit, open-source project into a for-profit, closed-source company.
It's important to understand that the development of ChatGPT was only possible by feeding and training the AI with gigantic amounts of data, including vast quantities of copyrighted material.
OpenAI was only able to use this data free of charge and without the permission of the copyright holders because the company presented itself as a non-profit project.
The use of copyrighted material is considered permissible if it is a research project that does not generate profits and serves the public good.
In retrospect, it is clear that OpenAI deliberately exploited this situation. The billions in profits the company now generates are largely due to OpenAI's free access to this data during its non-profit phase.
For Suchir Balaji, this practice was completely unacceptable.
Suchir left the company in the summer of 2024, having made crucial contributions to the development of ChatGPT during his four years there.
In the months leading up to his violent death, he was preparing to launch his own startup and wrote a scientific paper on the future of large language models (LLMs) like ChatGPT.
In this work, which unfortunately remained unfinished, he refuted the so-called scaling hypothesis, championed by OpenAI and most other AI companies.
This hypothesis states that the intelligence of AI models can be developed indefinitely as long as they are fed enough data. It forms the basis for the grandiose promises of AI companies.
The achievement of a level of artificial general intelligence (AGI) has been announced for years.
AI models are supposedly about to develop superhuman intelligence (ASI = Artificial Super Intelligence), replace all kinds of jobs, cure diseases, create wealth for everyone, and so on.
In his unfinished essay, Suchir Balaji demonstrated in an impressive yet easily understandable way that, contrary to the claims of AI companies, large language models can never reach the level of human-like intelligence (AGI = Artificial General Intelligence).
He predicted that the fundamentally limited, abysmal data efficiency of this technology will inevitably slow down the further development of AI models and bring them to a standstill long before AGI is achieved.
This is an inconvenient truth for the AI industry, which it is trying to conceal to protect its business model.
Suchir Balaji was also slated to testify as a key witness in a lawsuit against OpenAI, which involved, among other things, massive copyright infringements.
In the months leading up to his death, Suchir was in good spirits and looking forward to launching his own AI company.
On November 22, 2024, he had just returned from a short vacation with his closest friends.
According to the investigation by a private investigator hired by Suchir's parents, Suchir had ordered food that evening, listened to music, and worked on his laptop. According to the investigator's reconstruction, he ...
Read the full article for free on Substack:
https://truthwillhealyoulea.substack.com/p/the-murder-of-an-openai-top-engineer?utm_source=share&utm_medium=android&r=4a0c9v
This is a very heavy story, and I appreciate you taking the time to write it out and share your perspective.
There are clearly real tensions around IP, business models, and worker protection in the current AI ecosystem, and whistleblowers and critics are an important part of that picture.
I am not in a position to verify the specific claims about this case, but I do share the underlying concern that we should not build our future infrastructure on foundations that depend on opacity or on quietly stretching legal categories past their breaking point.
This is a solid articulation of the autonomy shift.
Where I think the conversation now needs to move is from “governance maturity” to enforcement architecture.
The gap isn’t just that organizations are still operating with Level 1 frameworks while deploying Level 2–3 agents.
The gap is that most governance remains descriptive rather than executable.
Policies exist. Risk registers exist. Monitoring exists.
But at the moment an agent mutates state — writes to a database, triggers a workflow, transfers value, allocates access — very few systems have a deterministic enforcement gate that can allow, deny, or halt that execution with evidentiary traceability.
That binding event is where governance either proves itself or collapses.
Agentic AI doesn’t just expand the risk surface.
It forces us to encode authority and stop conditions at runtime, not assume them at Layer 8.
Until enforcement is infrastructure — not documentation — the gap will continue widening regardless of how many frameworks we publish.
I agree with you almost word for word here.
Most “agent governance” I see in the wild is still descriptive, not executable: policies and risk registers on one side, and then a deployment pipeline that happily lets an agent mutate state without a meaningful gate in front of critical actions.
The shift we need is exactly what you describe: treating enforcement as infrastructure. That means deterministic checks at the point where an agent writes to a database, allocates access, or moves value, plus evidentiary logs that let you reconstruct who authorised what and why. Until that binding event is governed, the maturity diagrams will keep over‑stating how safe these systems really are.
We share the same interests, take a peek on rftjon.substack.com
Thank you, Robert.
I will take a look at your work; it is always good to see how others are approaching this intersection from their own angle.
Ai alone is not the solution. Ai is just another system, not the entire system. It needs a translation layer. Check out my pov on it that I have in my notes and articles. Might be of interest, might not.
I like how you put it.
I also do not see AI as “the system” but as one more layer in a much larger sociotechnical stack that needs a translation layer between policy, operations, and code.
I will check out your notes on this, because I think we are all still learning how to design that translation layer in a way that honours both human judgment and machine capabilities.