Building with AI: Beyond UX Focus

I remember when UX was king. We spent countless hours crafting every pixel of our interfaces. But today, everything's changed. It's all about the AI's output quality. If you're still fixated on UX, you're missing the bigger picture. In our field, we've moved from deterministic systems, where everything was predictable, to non-deterministic systems that demand an experimental approach. I've had to adapt my workflow: running internal tests, involving users right from the start, and collaborating continuously with other departments. Engineers are no longer just executors but key players in product management. Handling AI biases is a constant challenge. So, how does this shift impact our daily work? That's what I propose we dive into together.

From UX to AI Output Quality

Once upon a time, UX design was king. Every detail was polished to ensure a seamless user experience. But today, with AI stepping into the spotlight, output quality is the name of the game. I've seen this shift firsthand: we're no longer talking pixels but tokens. And trust me, no amount of good design can make up for poor AI output. I'm always aiming for that 95% accuracy baseline, a threshold I consider non-negotiable for launching AI products.

But watch out: focusing solely on UX can lead to missing the big picture. I've seen projects fail because they didn't prioritize AI performance. My advice? Always balance design with functionality.

• Prioritize AI output quality.
• Set a 95% accuracy baseline.
• Be cautious not to overlook AI performance for UX design.

Embracing Non-Deterministic Systems

Transitioning from deterministic to non-deterministic systems is like going from an automatic to a manual gearbox. You need to understand the role of randomness and probability in AI outputs. It's not straightforward, but it opens up incredible flexibility. I've had to adjust my expectations and accept an 80% accuracy threshold for some AI features.

Non-deterministic systems require more robust testing. Every time I work on a new model, I dedicate time to exhaustive testing to ensure we achieve the desired results. It takes more resources, but it's worth it.

• Understand the role of randomness in AI systems.
• Accept an 80% accuracy threshold for certain features.
• Prepare for more extensive testing.

Adopting an Experimental Mindset

In AI product development, being ready to experiment is crucial. Personally, I run up to 12 experiments in parallel. It might sound crazy, but it's essential for achieving statistically significant results. I use "dogfood users" within our team to test new features before releasing them.

But be careful not to get stuck in analysis paralysis. Set clear goals for each experiment and don't waste time on endless hypotheses.

• Conduct multiple experiments simultaneously.
• Use significant results to guide decisions.
• Avoid "analysis paralysis" by setting clear objectives.

Collaborative Product Management

I've shifted from hierarchical product management to a collaborative approach. This means involving engineers early in the development process. Believe me, it's a radical but necessary change. Communication and collaboration are key to successful AI projects.

A warning, though: ensure all team members understand AI limitations. Otherwise, you risk creating unrealistic expectations that can derail the project.

• Adopt a collaborative approach to product management.
• Involve engineers from the start.
• Ensure the team understands AI limitations.

Setting the 'Non-Embarrassment Bar'

Before releasing an AI product, I always ensure it passes the 'non-embarrassment bar'. This means achieving a 90-95% confidence level before rolling it out to a wider audience. Gathering user feedback is crucial for refining the product.

But beware, don't rush to market without thorough internal testing. I've seen products fail because they were launched prematurely. Better safe than sorry.

• Achieve a 90-95% confidence level before release.
• Use user feedback to improve the product.
• Be wary of premature launches.

So, shifting from UX to AI output quality, I've really embraced working with non-deterministic systems. First, you have to get comfortable with the messiness of it all. Precision isn't the goal; it's all about experimentation. Then, product management becomes a team sport. We need to collaborate to set the 'non-embarrassment bar' high and ensure AI development success. Don't rest easy: aim for a 95% accuracy for your AI, but know that you can ship at 80%. And, run a dozen experiments in parallel, that's where the real breakthroughs happen.

Looking ahead, this paradigm shift is a game changer. Ready to rethink your product development approach? Start involving your engineers and users today. For deeper insights, check out the full video: YouTube. It's worth a watch.