#020 • 13 May 2025

Contents of this Newsletter

⚜️

Design + AI

⏰ : 7 min

📓

From the Blog

⏰ : 8 min

🗑️

Open Source Spam

⏰ : 3 min

🗞️

AI Headlines

⏰ : 4 min

MAIN STORY

Design-Led AI for Quash

In our previous newsletter, we discussed the benchmarks we use to evaluate different LLMs in our test case generation workflows. Some models handled coverage impressively well, others offered sharp accuracy in specific domains, and a few even surprised us with emergent reasoning capabilities. But the key takeaway wasn’t just about better models; it was about creating better systems around them.

This is where design becomes essential. And it’s not just about improving the look and feel; design drives the direction of how AI tools function and how users interact with them. It’s fundamental to the way we build AI at Quash. Design isn’t decoration, it’s direction.

This includes not just the interface or the user flow, but how the AI pipeline itself is designed, to reflect the ideal user experience, not the other way around. It’s easy to start with what a model can do and build a product around that. What’s harder, and far more impactful, is to start with: what experience do users need to get their job done? And then shape the AI to fit that. This shift is what Amy Chivavibul refers to as AI’s “GUI moment.” Just as the graphical interface made computers accessible to more people, we now need interfaces that make AI collaborative—not just functional.

ChatGPT’s chatbox wasn’t just a wrapper—it was the interface breakthrough that made LLMs approachable.

When you're building tools for knowledge workers, tools that QA engineers and developers rely on daily, the stakes are higher. You're not just designing a product people enjoy, you're designing one they trust to do their job.

Designing for AI: Key Challenges and What We Did

Building an AI-native product comes with its own design challenges. Here are a few we’ve faced, and how we approached them.

1. Building User Confidence

One of the biggest challenges in AI-powered tools is user trust. It’s not enough for a system to generate outputs; you have to make sure users understand, validate, and feel confident acting on those outputs. Especially in tools designed for professionals, like QA engineers, a single wrong suggestion can lead to broken tests or misaligned coverage.

That’s why designing for validation isn’t optional—it’s fundamental. You can’t just drop an answer on a user and expect them to run with it.

In AI tools that generate test scripts, you can’t just drop an output on a user and expect instant trust. Most traditional workflows do this:

But that breaks down when the output is a technical, multi-step script. It’s too dense to review all at once.

So at Quash, we introduced an additional layer of validation:

By generating a human-readable version first, users can validate the logic easily before the script is even formed. Then the final, granular output gets a second review if needed. It’s more work on our side—but it drastically reduces error, increases trust, and makes QA teams feel in control.

2. Unpredictability of Outputs

LLMs are powerful, but not always predictable. We’ve tackled this across three areas:

Model Autoselection:
While users can choose a specific model, we offer an auto-select mode where our system picks the best-performing model for the task. This keeps evolving as we benchmark newer LLMs, saving users the effort of constant trial-and-error.
Fallbacks & Error Resilience:
During the course of API timeouts, delayed responses, or any other errors from LLMs, instead of displaying these issues and asking users to figure it out and retry, our pipelines switch to backup models and reattempt requests automatically. The goal is minimal disruption and zero dead-ends.
Structured Output Formatting:
One common LLM challenge is that the output formats vary wildly, sometimes you get 3 steps, sometimes 30. Some include preconditions, others bury test data in weird places. We’ve trained our agents to maintain a consistent structure. But since some variation is inevitable, our UI adapts: using accordions for long sections, breaking large scripts into new page views, and collapsing non-critical details to avoid overwhelming the user.

This is design responding not just to users, but to unpredictable AI behaviour itself.

Why Design Thinking Is a Core Competency at Quash

For AI to truly work, it needs to fit into people’s lives seamlessly. No matter how powerful the AI, it won’t get used if it’s confusing, unpredictable, or hard to interact with. That’s why design is everything to us at Quash. It’s not just about making things look good—it’s about making them work the way people need them to. Intuitive, adaptable, and reliable.

We’ve always looked up to tools built by founders who understood that design is not a nice-to-have; it’s the foundation. Our founders, Prakhar and Hamza, come from design backgrounds, so it’s a central part of how we work at Quash. It influences how we think about features, how we develop them, and how we integrate AI.

At the end of the day, design is about creating tools people can trust—tools that feel natural, not forced. In an AI-driven world, design is the bridge that makes complex technology actually work for the user.

Check out our Recent Blogs

Maven vs Gradle: Choosing the Right Build Tool for API-Heavy Backends 2025

When your backend juggles GitHub, GitLab, Firebase, and more, your...

Read article

Postman Alternatives in 2025: Choosing the Best API Development Tool

Postman may lead the API world, but it’s no longer the only option. This guide...

Read article

dev culture

AI Slop Is DDoS’ing Open Source

The curl project calls it what it is: a distributed denial-of-service, not on servers, but on human time.

Open source maintainers are used to the occasional invalid bug report. But Curl (cURL), a popular command-line tool used by devs worldwide, has been dealing with a flood of AI-generated vulnerability submissions, impressively formatted, grammatically flawless, and almost always wrong. Its founder, Daniel Stenberg, says he’s had it with this craziness. These reports written by AI tools and submitted via HackerOne’s bug bounty program are flooding the system and he calls it AI slop. They look convincing at a glance, but dig deeper and you'll find hallucinated functions, irrelevant references, and generic answers that feel more like prompt completions than genuine analysis. Some giveaways are easier than others: one report even included the submitter’s raw AI prompt, accidentally ending with, “and make it sound alarming.”

These fake reports waste real time. None of the AI-assisted submissions in six years have uncovered valid vulnerabilities. That’s why curl now requires submitters to disclose AI use and will ban anyone submitting “AI slop.” HackerOne agrees in theory, but the incentives still favor quantity over quality. AI might be helpful, but when it comes to bug bounties, it's currently more noise than signal.

news

This Week in AI

Gemini 2.5 Pro Outshines OpenAI & DeepSeek in Coding

Google has released the Gemini 2.5 Pro Preview (I/O Edition) ahead of its developer conference. The model now ranks #1 on the WebDev Arena leaderboard, outperforming OpenAI, DeepSeek, and Anthropic on coding tasks. Notably, it handled complex backend refactoring tasks typically done by senior developers and can now build apps directly from YouTube videos.

Figma Make: AI-Powered Prompt-to-Code Tool

Figma has introduced “Figma Make,” a new feature that allows designers to transform static designs into interactive prototypes using natural language prompts. Powered by Anthropic’s Claude 3.7 model, it enables users to add animations, interactions, and refine designs without starting from scratch, streamlining the design-to-prototype workflow.

Microsoft integrates Agent2Agent protocol to enable interoperable AI agents

Microsoft is adopting Google’s open Agent2Agent (A2A) protocol to allow AI agents to work together across platforms, clouds, and organizations. A2A enables agents to exchange goals, share state, and trigger actions in a standardized way. It will soon be available in Azure AI Foundry and Copilot Studio, with enterprise-grade security features like Microsoft Entra integration, mutual TLS, and audit logging built in.

OpenAI launches ‘OpenAI for Countries’ to help governments build local AI infrastructure

OpenAI has launched a new initiative to help countries deploy their own AI systems. As part of its larger Stargate strategy, “OpenAI for Countries” will help governments build nation-specific AI models, host them on local infrastructure, and apply them to public sector use cases like healthcare, education, and disaster response.

Hugging Face Launches Free Agentic Ai Tool For Public Use

Hugging Face has released the Open Computer Agent, a free, web-accessible AI agent that can use a cloud-hosted Linux machine to perform tasks like web searches. While it performs well on basic tasks, it struggles with complex requests and CAPTCHA challenges.

All Newsletter Issues | Security | Contact Us | Blogs | Test Case Generator

So We Had to Redesign the AI Pipeline…