The 'ask when a control was last exercised' question is the sharpest thing in here. One layer I'd add: a kill path that's only been tested in a clean run doesn't tell you much. The real test is what the agent does in the moment right before someone has to use it, whether anyone would even catch that window in time to act.
This is a really insightful piece. I especially liked the distinction between having governance documents and actually having decision rights and controls that someone has exercised.
The distinction between an action-path failure and a model-alignment failure is spot on. The red-teaming examples really illustrate how fast things fall apart when governance is treated purely as a prompt-level safety issue rather than an operational containment issue. The point about 'exercised kill paths' will probably make a lot of CISOs very uncomfortable—in the best way possible."
The provision that moves the line is Article 25(1), and its three limbs are not equally negotiable. Putting your name or trademark on a high-risk system already on the market pulls in the provider obligations under Article 16, but that limb carries an express carve-out: without prejudice to contractual arrangements stipulating that the obligations are otherwise allocated. The other two limbs, substantial modification and changing the intended purpose so the system becomes high-risk, carry no such wording. That makes the fine-tuning example the harder one. A contract can move the badging problem. It does not move the modification problem.
You're right that the badging carve-out in Article 16 and the modification limb don't travel together, Marius — that's exactly the distinction most compliance memos flatten. Appreciate you pulling the actual article text into it.
The practical follow-on is that the modification limb now sits inside an enforced regime rather than a drafted one. The Commission's AI Office and the national authorities started enforcing the Act on 2 August 2026, and the transparency duties began applying the same day: interactive systems have to tell users they are dealing with AI, and generated or altered content has to carry machine-readable marking. A deployer that fine-tunes and rebrands inherits those disclosure duties directly, with no contractual reallocation available. How authorities will treat a first breach is not yet on the record.
Precisely. Once the modification limb is tied to an enforced regime, the question shifts from abstract applicability to evidence: who changed what, whether the change was substantial, and which actor can demonstrate compliance across the lifecycle.
Article 25(2) settles part of the evidence question by default. Once a deployer or distributor makes a substantial modification under Article 25(1)(b), the initial provider is no longer the provider of that specific system, and must cooperate with the new one, making available the necessary information and the reasonably expected technical access. The escape sits in the same paragraph: where the initial provider has clearly specified that its system is not to be changed into a high-risk system, the handover duty does not apply. Contract wording decides who can prove anything.
The 'ask when a control was last exercised' question is the sharpest thing in here. One layer I'd add: a kill path that's only been tested in a clean run doesn't tell you much. The real test is what the agent does in the moment right before someone has to use it, whether anyone would even catch that window in time to act.
This is a really insightful piece. I especially liked the distinction between having governance documents and actually having decision rights and controls that someone has exercised.
Thanks Joe! I have more like this in my newsletter
The distinction between an action-path failure and a model-alignment failure is spot on. The red-teaming examples really illustrate how fast things fall apart when governance is treated purely as a prompt-level safety issue rather than an operational containment issue. The point about 'exercised kill paths' will probably make a lot of CISOs very uncomfortable—in the best way possible."
The provision that moves the line is Article 25(1), and its three limbs are not equally negotiable. Putting your name or trademark on a high-risk system already on the market pulls in the provider obligations under Article 16, but that limb carries an express carve-out: without prejudice to contractual arrangements stipulating that the obligations are otherwise allocated. The other two limbs, substantial modification and changing the intended purpose so the system becomes high-risk, carry no such wording. That makes the fine-tuning example the harder one. A contract can move the badging problem. It does not move the modification problem.
You're right that the badging carve-out in Article 16 and the modification limb don't travel together, Marius — that's exactly the distinction most compliance memos flatten. Appreciate you pulling the actual article text into it.
The practical follow-on is that the modification limb now sits inside an enforced regime rather than a drafted one. The Commission's AI Office and the national authorities started enforcing the Act on 2 August 2026, and the transparency duties began applying the same day: interactive systems have to tell users they are dealing with AI, and generated or altered content has to carry machine-readable marking. A deployer that fine-tunes and rebrands inherits those disclosure duties directly, with no contractual reallocation available. How authorities will treat a first breach is not yet on the record.
Precisely. Once the modification limb is tied to an enforced regime, the question shifts from abstract applicability to evidence: who changed what, whether the change was substantial, and which actor can demonstrate compliance across the lifecycle.
Article 25(2) settles part of the evidence question by default. Once a deployer or distributor makes a substantial modification under Article 25(1)(b), the initial provider is no longer the provider of that specific system, and must cooperate with the new one, making available the necessary information and the reasonably expected technical access. The escape sits in the same paragraph: where the initial provider has clearly specified that its system is not to be changed into a high-risk system, the handover duty does not apply. Contract wording decides who can prove anything.