#021 • 20 May 2025

Contents of this Newsletter

🤡

The Case Against Full Automation

⏰ : 7 min

📓

Recently Published Blogs

⏰ : 12 min

🗞️

This week
in AI

⏰ : 8 min

MAIN STORY

Why You Should Not Chase True Autopilot

Everyone wants in on AI. Everyone’s racing to slap “autopilot” on their AI features, from note-taking apps to CRM tools. AI badges are all over on homepages with promises of hands‑off, set‑and‑forget workflows.

And sure, the idea of letting AI run the show sounds exciting. But in practice, products built for full autopilot rarely live up to the promise and what’s worse is the fact that they’re too rigid to be course-corrected. There’s no room for guidance, no way to step in. When things go wrong, there’s no workaround. So teams just stop using them altogether. As QA teams know all too well, tests evolve, break, and demand human judgment. That’s why the real question isn’t Can AI do it? but Should it, and under what conditions?

The AI Intention Matrix: A More Thoughtful Lens

We recently came across Richard Yang’s article on Medium introducing the AI Intention Matrix, and it struck a chord. His 2×2 framework offers a refreshing way to think about AI just by looking at its intent. The matrix frames AI systems across two axes:

Augmentation ↔ Automation: Is the AI assisting a human, or acting autonomously?
Satisficing ↔ Optimizing: Is it aiming for “good enough” outcomes fast, or striving for perfect results, even if slower?

This creates four clear quadrants:

Everyday Assistants (augment + satisficing): Think autocomplete or basic reminders which are useful, lightweight support tools.
High‑Stakes Co‑Pilots (augment + optimizing): AI that helps with precision-critical work but keeps the human in the loop which are like medical diagnostics or code reviews.
Autonomous Utility (automate + satisficing): Background processes like spam filters or auto-sorting email which are fully automated, but not necessarily precise.
Autonomous Precision (automate + optimizing): Systems aiming for full automation and near-perfection, such as self-driving cars or robotic surgery.

What we appreciated most is how this framework avoids assigning value judgments. Automation isn’t always better. Augmentation isn’t always safer. Each choice depends on context, risk, and user expectations. But when we zoom in on QA and software tooling, we start to see where “autopilot” approaches begin to fall short.

Context Blindness: In many tools, autopilot works by applying the same logic across the board. That’s fine when you’re labeling images or tagging receipts. But in QA, context is everything. A failed test might mean a critical regression—or just a slow network. Treating them as the same leads to false positives, wasted time, or worse, missed bugs.
Good Enough is Not Enough: In creative or clerical tools, near-enough is often good enough. But QA is nothing like that. If an AI gets a triage decision 80% right, that remaining 20% still costs teams time and trust. And when there's no easy way to refine or course-correct the AI, users get stuck with wrong decisions they can't undo.
Fixed System in a Moving World: Many AI systems assume that what's true once will stay true. But software evolves constantly. New features, environments, and user behavior all introduce change. QA tools that can’t adjust to shifting baselines or learn from what humans are correcting quickly become outdated and ignored.

Instead of defaulting to full automation, we need AI that knows when to act and when to hand back control.

Partnership Over Autonomy

At Quash, we agree that AI should be a partner, not a ghost driver at least for now. Fully autonomous systems may get there one day, but today’s reality calls for collaboration, not delegation.

We found Yang’s matrix a bit static for the ever‑changing world of QA. That’s why we’re introducing a Static vs. Dynamic dimension to help product teams think more clearly about how their AI evolves in response to real-world usage.

The static/dynamic distinction asks:

Is the feature rigid or responsive?
Does it execute a fixed sequence of actions regardless of what’s happening around it, or can it respond to context.
Does the AI remain fixed in its behavior, or does it learn and adjust based on feedback, risk, and changing context?
Some systems follow pre-set rules no matter the outcome. But a truly dynamic system recognizes when it’s wrong, factors in user overrides or confidence scores, and evolves its behavior to better match real-world usage over time.

For instance:

A static AI system might always auto‑triage bugs the same way, regardless of past user corrections.
A dynamic AI would factor in confidence levels, your history of overrides, and the nature of the task before deciding whether to act or defer to a human.

In Quash, if you regularly step in to add a specific edge-case for test coverage, say a gesture the AI couldn’t infer from context, or a flow that needs to be tested in specific local languages because that’s how your users experience the app—our system takes note. It learns from that pattern and begins to include those nuances in future test runs.

It’s a bit like training a new team member. Even if they match the skills in the job description, they won’t instantly understand the quirks of your product or what matters most to your team. But over time, with your input, they get sharper, faster, and more in sync with your needs. That’s how we think about dynamic AI—learning not just the product, but your priorities.

TL;DR: Building AI with Intent

The race to build AI autopilots might make sense for static, repetitive tasks. But QA is different. It’s nuanced, context-heavy, and deeply connected to how real users interact with ever-changing software. What QA teams need is a reliable co-pilot. One that learns, adapts, and evolves alongside the product.

At Quash, we believe the real breakthroughs in AI won’t come from removing humans from the loop—they’ll come from designing better loops. Loops that keep product context, test coverage, and human judgment at the center. That’s how AI actually adds value, not by chasing full automation at all costs, but by knowing when to step in, when to stay out, and when to ask for help. We’re building AI that learns with you, respects your judgment, and gets better over time. QA needs a system that’s in the loop—with you.

Check out our Recent Blogs

news

This Week in AI

OpenAI launches Codex: A cloud-based software engineering agent

OpenAI has introduced Codex, a cloud-native agent built on codex-1, a fine-tuned variant of the o3 model optimized for software engineering tasks. Codex automates feature development, bug fixes, codebase Q&A, and pull request creation, all in parallel cloud environments.

Microsoft transforms GitHub Copilot into full AI coding agent

At Build 2025, Microsoft announced that GitHub Copilot is evolving from autocomplete to a fully-fledged AI coding agent. This upgraded Copilot can now reason through developer instructions and write code independently, marking a move toward what’s being called “vibe coding”—goal-oriented development through natural language.

Google adds GitHub integration to Gemini Advanced

Gemini Advanced (Google’s $20/month AI plan) now supports direct GitHub repo imports, allowing users to bring in public or private codebases for AI-assisted debugging and code generation. The feature is currently exclusive to paying subscribers. This comes shortly after OpenAI's similar GitHub-connected research tool for ChatGPT, marking a new phase in AI assistants geared toward real-world dev workflows.

CodeRabbit now integrates with VSCode, Cursor, and Windsurf

CodeRabbit brings its context-aware code reviews directly into your IDE—including VSCode, Cursor, and Windsurf. You can now write, review, and commit without breaking flow. The integration leverages deep code understanding across files and dependencies to deliver smarter, in-line suggestions.
Better context → better reviews → fewer bugs.

Alibaba introduces quantized Qwen3 models for local use

Alibaba has released Qwen3 models in quantized formats—GGUF, AWQ, and GPTQ—for local deployment. Quantization refers to reducing model size and memory usage by converting high-precision model weights to lower-precision formats (e.g., 32-bit to 8-bit), making them faster and more efficient for on-device use. Accuracy may drop slightly, but the performance trade-off is often minimal.

OpenAI releases HealthBench, a multilingual medical benchmark

OpenAI has launched HealthBench, a new open-source benchmark to evaluate AI models in healthcare settings. Developed with input from 262 physicians across 60 countries, it includes 5,000 realistic health-related prompts and structured grading rubrics. Unlike many past benchmarks, HealthBench emphasizes expert validation and real-world relevance, and it supports evaluation across 49 languages.

OpenAI adds PDF export for research reports

You can now export your Claude-style research reports from ChatGPT as polished PDFs—complete with images, tables, and source-linked citations. Just hit the “Share” icon and choose “Download as PDF.” Available for both new and old reports.

All Newsletter Issues | Security | Contact Us | Blogs | Test Case Generator

Copilots > Autopilots: The QA Lesson from 10k Test Runs