I want to talk about our testing efforts for AI features, but I’ll skip speaking to your conscience, which is already worried about quality. I know you’re losing sleep. This time, I want to look at it from a different… Continue Reading…

Last week, we talked about why debugging may be frustrating. But that’s only half the battle. When you don’t understand the magic (because, hey, it’s magic), how do you explain it to your bosses? But let’s say you’ve actually done… Continue Reading…

In the old days, like five years ago, debugging was a easy. Just kidding. But if you had the right tools in place, it could be easy-ish. We looked at the logs. We found a stack trace. It pointed us… Continue Reading…

Let’s talk about your bug reports. Specifically, let’s talk about the word “Hallucination.” If you are testing AI features today, I bet your Jira is full of tickets that say, “The model hallucinated.” How can I put it? Not really… Continue Reading…

Let’s talk about the dirty secret of our industry. We’ve all been there. You come in on a Monday morning, check the nightly run, and see red. Again. Not “we broke the build” red, but that annoying, flickering, “it worked… Continue Reading…

The recording of Webinar III in the AI Quality series is up! In our previous sessions, we focused on the basics of AI testing. We learned how to paddle the “AI Kayak” in the safe harbor of development. But eventually,… Continue Reading…

Who doesn’t like asserts? We have a habit of confusing “simple” with “easy.” In traditional automation, defining quality was simple. It was binary. Assert.AreEqual(expected, actual). It either matched, or it didn’t. Green or Red. But with AI, “Good” isn’t binary.… Continue Reading…

So, you did it. You built a fantastic “AI Testing Kayak.” You followed the AI Quality Funnel. You have developer tests, sanity checks, and a “Golden Dataset” that defines “good” responses. You even have an “Automated Scorecard” that runs in… Continue Reading…

Most people think of testing an AI feature like testing a chatbot – you assess the quality of a single response. But in our real-world systems, it’s almost never a single response. We build chains of calls. The response from… Continue Reading…

Last week, I published our new strategic model: Are You Building an “AI Testing Kayak” or an “AI Testing Ship”? That post explained the “why.” It defines the “Kayak” as the collection of fast, practical tactics for the practitioner in… Continue Reading…