QA to QE: Rethinking Quality in the Age of AI

I’ve been looking forward to writing this blog since I started this series. AI is not going to go anywhere, so we need to adapt our thinking in the QE/Testing world.

The rise of AI is revolutionising how we build, test, and deliver software. Traditional Quality Assurance (QA) practices, focused on validating outputs, are no longer sufficient. Quality Engineering must evolve beyond reactive testing to proactively embed quality into AI-driven systems.

In this post, I’ll explore how AI transforms the quality landscape, using real-world AI examples (which took a lot of time researching) to demonstrate the five phases of QE Transformation.

Phase 1: Assess and Plan – Understanding AI’s Unique Quality Challenges

AI introduces new complexities beyond conventional software testing and these will need to be baked into any plans and strategies being built around our solutions:

• Data bias & model drift – Example: Hiring algorithms like Amazon’s past AI recruitment tool reportedly exhibited gender bias, favouring male candidates due to skewed historical data.
• Continuous learning – Example: ChatGPT-style conversational AI models constantly adapt based on user inputs, making static test cases ineffective.
• Ethical considerations – Example: Deepfake technology raises questions about misuse, authenticity, and societal impact.

Before transforming QA into QE, organisations must assess their AI maturity, define quality objectives, and align strategies with AI’s evolving nature. A standardised test strategy won’t work, quality must be adaptive, contextual, and ethical.

It’s important to keep in mind that whatever solution is being built, it’s still for a customer who just wants a working solution, so focus the eyes on that and ensure the strategy meets what is needed to make it happen, even with new models and ways of thinking to adopt.

Phase 2: Evolve Capabilities – Expanding Quality Engineering for AI

Quality teams must shift their skillsets to engineer quality within AI models rather than just test their outputs. This is quite a shift and when building a roadmap, it needs to be considered for the timeline to bring teams on the journey with this understanding.

• AI Specific Testing – Edge Case Testing – Example: Self-driving cars (Tesla, Waymo) rely on edge case testing, ensuring models handle unexpected scenarios like pedestrians in unusual positions or extreme weather conditions. In traditional testing, edge cases would be lower priority, with self driving cars, it’s the scenarios not on the happy path that become crucial to identify.
• AI Specific Testing – Explainability Testing – Example: AI Based Loan Approval Systems will need a backbone of Explainability testing which is all about ensuring the how and why around decisions are understood.
• Model validation frameworks – Example: Financial fraud detection AI (banks, PayPal) undergoes adversarial testing (stress testing with malicious inputs) to prevent bad actors from manipulating detection patterns.
• Cross-functional collaboration – In AI-driven organisations, data scientists, developers, and testers must work together from the start. Example: Healthcare AI diagnostics (IBM Watson, Google’s DeepMind) requires interdisciplinary validation to avoid false diagnoses.

Moving beyond functional testing, QE teams must integrate AI principles into their expertise, embracing data science, ethics, and continuous monitoring to ensure long-term reliability. I come back to the term “mindset shift” again, because there is so much which needs to be considered when embracing testing in the AI world.

Phase 3: Implement Practices – Embedding AI-Driven Quality

Once QE capabilities evolve, quality must become a fundamental part of AI development and deployment. This will require educating the wider teams on the need to focus on Quality when developing AI solutions.

• Shift-left AI quality – Example: GitHub Copilot uses AI-assisted coding, but developers must embed code safety checks upfront to prevent vulnerabilities from spreading.
• Human-in-the-loop strategies – Example: Medical AI (Tandem Health, Google’s AI-powered ultrasound) relies on expert review to prevent false positives that could mislead diagnoses or in Tandem Health’s case, ensuring accuracy in medical notes taken before submitting to a patients records.
• Continuous validation pipelines – Example: Netflix’s AI recommendation system regularly updates its machine learning models to avoid irrelevant content suggestions.

A single testing phase before release will not suffice for these kinds of products. Without integrated practices, organisations risk deploying AI solutions blindly, leading to ethical failures or degraded user experiences. QE ensures AI remains trustworthy, scalable, and user-centric.

Phase 4: Foster a Culture of Quality – Rethinking AI Governance

Quality transformation in AI isn’t just technical, it’s cultural. Organisations must prioritise accountability, transparency, and shared responsibility. It’s key to ensure discussions, designs and implementations take the cultural aspects into account and everyone involved is continuously learning and evolving their thinking.

• Leadership in Responsible AI – Example: Microsoft’s Responsible AI framework ensures fairness, transparency, and human oversight in AI solutions.
• Cross-functional awareness – AI quality isn’t just an engineering problem, it impacts legal, compliance, and customer trust. Example: GDPR and AI data compliance requires teams to embed privacy safeguards at every stage.
• Continuous learning & upskilling – AI Engineers and Quality Engineers must adapt their approaches as AI evolves. Example: AI-driven cybersecurity tools (Darktrace, Palo Alto Networks) evolve their threat detection models in response to emerging cyber risks, therefore keeping on top of the latest direction or changes is absolutely imperative.

A thriving Culture of Quality ensures AI solutions don’t just function correctly, they operate responsibly.

Phase 5: Measure and Adapt – Continuous AI Quality Management

Unlike traditional software, AI never stops learning. QE for AI must be dynamic, iterative, and deeply embedded in monitoring systems. This is where it becomes even more apparent that a holistic view of quality is needed.

• New AI Quality Metrics – Beyond pass/fail, QE must track explainability, fairness, accuracy drift, and ethical compliance.
• Ongoing Adaptation – AI models must be regularly assessed to detect unintended failures. Example: Chatbot moderation AI (Meta, OpenAI) constantly adjusts its hate speech and misinformation filters.
• AI-Augmented QE – Example: AI-powered testing (Applitools, Mabl, Testim) uses AI to enhance exploratory and regression testing coverage.

In the AI era, QE isn’t a one-time transformation, it’s a continuous cycle of learning, monitoring, and adapting.

Final Thoughts: QA to QE is More Critical Than Ever in AI

The transition from QA to QE is no longer optional, it’s a necessity. AI-powered systems demand a proactive, engineered approach to quality, not just traditional validation techniques.

By aligning AI transformation with the five-phase QE Transformation model, organisations can ensure their AI solutions remain ethical, scalable, and reliable.

It’s a fun journey to be on and lots of opportunity to grow and develop as a craft. The key call to action is that we need to be proactive in this and not sit still stressing that AI will take our jobs!