The AI governance conversation is already happening in your organisation. The question is whether quality has a seat at the table.
This is the second post in the Intelligent Quality Leadership series. It builds on the original model and specifically explores the AI + Leadership corner of the triangle. If you’re new to the series, it’s worth starting at the beginning.
Here is something I’ve noticed, both in my own experience and in conversations with others across the quality and engineering community. When organisations start talking seriously about AI governance, the people in the room tend to be legal, security, product leadership, and sometimes a CTO or engineering director. Quality engineers are rarely there. And that needs to change.
Not because we should be territorial about it. But because the skills and the instincts that quality leaders have built over years, understanding risk, thinking about edge cases, asking “but what happens when it goes wrong?”, are precisely what those governance conversations need. We’ve spent our careers thinking about failure modes. AI gives us an entirely new category of failure modes to think about, and we are well placed to lead that thinking.
The opportunity here is real. But so is the risk of missing it.
AI governance isn’t a compliance exercise. It’s a quality problem. And quality leaders are uniquely equipped to own it, if we’re willing to step into that space.
I want to be clear about where I’m drawing from in this post. Some of it comes from direct experience, working on AI-powered products and navigating the question of what “good” looks like when your system is probabilistic rather than deterministic. Some of it comes from research and from following where this conversation is heading across the industry. I’ll try to be transparent about which is which as I go.
What AI Governance Actually Means for a QE Leader
When most people hear “AI governance,” they think about regulation, policy documents, and legal risk. That framing isn’t wrong, but it’s incomplete. For a quality leader, governance means something more specific and more practical: it means having a clear, shared understanding of how AI is used within your products, what the acceptable boundaries of that use are, and how you know when something has gone outside those boundaries.
That last part is where quality engineering lives. Monitoring, observability, testing at the boundaries of acceptable behaviour, defining what “trustworthy” actually looks like for a given feature. These aren’t legal or compliance questions. They’re quality questions. And in my experience, the organisations that are handling AI responsibly are the ones where quality thinking is embedded in the governance process from the start, not bolted on at the end.
What I’ve seen more commonly, though, is AI features being shipped with the same quality process that was designed for deterministic software. You write the test cases, you check the outputs match the expected values, you sign off. But with an AI-powered feature, the outputs aren’t always predictable, the edge cases are harder to enumerate, and the failure modes are genuinely different. Running a traditional quality process against a non-traditional system doesn’t make the risk go away. It just makes it invisible until something goes wrong in production.
So the first thing AI governance means for a QE leader is this: being honest about whether your current quality process is fit for the systems you’re now being asked to ship. For most teams, the answer is “not entirely.” And that’s the opening.
The Risks QE Teams Need to Own
There are three categories of risk that I think quality teams need to take explicit ownership of when it comes to AI features. None of them are new concepts, but all of them behave differently in an AI context than in traditional software.
Hallucination and Reliability
AI models can produce outputs that are confidently wrong. In a traditional system, a bug produces an error or an incorrect value that is usually detectable. A hallucination produces something that looks correct, reads as plausible, and may only be identified as wrong by someone with domain knowledge. For quality teams, this means rethinking what a “test” looks like. You’re not just checking that the output matches an expected value. You’re assessing whether the output is within the bounds of what is acceptable, accurate, and safe for the user to act on. That requires a different kind of test design, a different kind of reviewer, and in many cases a different kind of tooling.
Bias and Fairness
Bias in AI systems is a quality problem before it is anything else. It emerges from the data the model was trained on, the way the system was designed, and the assumptions baked into both. By the time it surfaces as a user complaint or a regulatory concern, the damage is already done. Quality leaders need to be asking fairness and bias questions during development, not after release. That means working with product and data teams to understand what the training data looks like, what populations the system was designed to serve, and where the gaps might be. It means building test cases that specifically probe for differential behaviour across user groups. And it means being willing to raise a concern even when the pressure to ship is significant. That last part is a leadership question as much as a technical one.
Security and Adversarial Risk
AI systems introduce security vulnerabilities that traditional penetration testing wasn’t designed to find. Prompt injection, where a malicious input manipulates the model into doing something it shouldn’t. Data leakage through model outputs. Adversarial inputs that cause the system to behave unpredictably. These aren’t hypothetical risks. They’re being exploited in production systems right now. Quality leaders don’t need to become security specialists overnight, but they do need to understand these risk categories well enough to ask the right questions, bring the right people into the conversation, and make sure that security testing for AI features goes beyond the standard checklist.
Getting a Seat at the Table
This is the part that I think a lot of quality leaders find hardest, and I want to be honest that I’ve found it hard too. The governance conversation tends to happen at a level of seniority where quality engineering isn’t always naturally represented, and it can feel like the conversation has already moved on by the time you’re aware it’s happening.
The way in, in my experience, isn’t to ask for a seat at the table. It’s to make yourself impossible to leave out of the conversation.
The most effective way I’ve found to do that is to make the risk tangible and specific rather than abstract and general. Telling a CTO that “we need better AI governance” lands differently than showing them a concrete example of what can go wrong when quality isn’t involved in AI feature development, what it costs, and what it would have taken to catch it earlier. Risk becomes real when it has a face. Abstract governance frameworks don’t move people. Stories do.
The second approach is to come with something, not just a concern. If you walk into a governance conversation with a proposed framework for how quality should be embedded in AI feature development, even a rough one, you shift from being the person raising a problem to the person helping to solve it. That changes the dynamic considerably.
The third is to build the relationships before you need them. The people leading AI governance conversations in your organisation, whether that’s legal, security, or product leadership, need to know that quality engineering has something valuable to contribute. That credibility is built over time, in smaller conversations, before the high-stakes ones happen. If the first time you’re making the case is in the room where the decision is being made, you’re already behind.
The Bigger Picture: This Is Where Quality Engineering Earns Its Seat
I genuinely believe that AI governance is one of the most significant opportunities the quality engineering profession has had in a long time. Not because it’s easy, but because it plays directly to our strengths. We understand risk. We think in edge cases. We ask “what happens when this goes wrong?” as a matter of habit. Those instincts are exactly what organisations need right now as they navigate what it means to build and ship AI responsibly.
The teams and leaders who position themselves at the centre of that conversation, who build the frameworks, who define what trustworthy AI looks like in practice, are going to be the ones who matter most in the organisations they work in. That’s not a small thing. It’s a genuine shift in the strategic value of what we do.
But it requires us to step into a space that might feel unfamiliar. To develop a point of view on ethics and governance that goes beyond traditional test coverage. To have conversations with legal, security, and product leadership that we haven’t always been part of. That’s the challenge. And it’s one I think we’re ready for.
I want to be clear that I’m not presenting this as a solved problem. I’m still working through what a really robust Appropriate Use Framework looks like for a quality team, and I suspect that answer looks different depending on the organisation, the product, and the maturity of the AI systems involved. What I’m more confident about is the direction: quality needs to be in this conversation, and the sooner we make that case, the better positioned we’ll be when the stakes get higher.
The next post in this series will go deeper on Cognitive Automation, the QE + AI corner of the triangle, and specifically what it looks like to build an automation capability that is genuinely fit for AI-powered products. But I wanted to take this one first, because I think getting the governance and guardrails conversation right is the foundation that everything else sits on.
I’d love to hear your experience on this one. Are you in organisations where quality has a voice in AI governance? Or is it happening without you? What’s worked in terms of getting into that conversation, and what hasn’t? This is an area where the community sharing real experience is genuinely more valuable than any framework I could put together on my own.
Simon Prior 