What Shipping Agentic Products Taught Me This Year

In the last twelve months, I've shipped three agentic systems into production cybersecurity and observability environments. Mobot — a 4-agent autonomous SIEM investigation system at Sumo Logic. Six weeks from hypothesis to working prototype. Two months to the AWS re:Invent stage. MTTR dropped from 60 minutes to under 3. Forrester validated 166% ROI. Mo Copilot — the NL-to-SIEM interface that originally shipped at a 40% hallucination rate; I rebuilt the RAG infrastructure and drove it to under 2% at enterprise scale. And now at Hackerdogs.ai — attack surface in 3 clicks, shipped through thin-slice iteration with users as design partners rather than audiences.

Here's the honest lesson from all three: traditional product management fails, utterly, to produce effective solutions in the age of AI. The playbook most of us learned — specify the feature, align the stakeholders, hand off to engineering, launch in the quarter — was built for a world where the ground didn't move underneath you every week. That world is gone for security and observability.

What actually shipped these products was a different operating style. It's what I've started calling the AI MacGyver product leader — and it's not optional anymore. The rest of this piece is what that looks like in practice, the six non-negotiables that make it work, and why the window to adopt it is closing faster than most organizations realize.

Why This Is More Important Than Ever

Breakout times are in seconds, not hours.

eCrime breakout times are at 27 seconds. Average end-to-end dwell time for an agentic attack is 29 minutes — a 65% speed increase in a single year. Humans don't operate at that cadence. Committees don't. Quarterly planning doesn't.

OpenClaw weaponized agentic attack chains.

The same agentic infrastructure we're using to build defensive systems is being used offensively. Adversaries are orchestrating multi-step attack chains at machine speed, exploiting the assumptions built into static playbooks and policy-based defenses. Your SOAR automation is now part of your attack surface.

Claude Mythos is the crystal ball.

Last week, Anthropic announced Claude Mythos — a frontier model they elected not to release because of what it could do. In controlled testing, Mythos found a 27-year-old vulnerability in OpenBSD that had survived every prior audit. Fewer than 1% of the bugs it discovered have been patched.

If Anthropic is holding a model like this back, state actors will have equivalents within 18 months. Some already do. The defensive window isn't closing — it's already half closed.

This is the environment we're now shipping into. And most security product organizations are not built for it.

What a MacGyver Product Leader Actually Does

MacGyver energy is the ability to improvise working solutions from what's in front of you — fast, with confidence, without waiting for permission to experiment.

In practice: customer conversations Monday, working code Wednesday, real feedback Friday, iteration the next week. No committees deciding whether something is worth trying. No handoff between product and engineering that costs two weeks in translation. No waiting for the threat intelligence database to be complete before building against what you already know.

It means shipping the partial version on the thin slice. Treating users as design partners. Letting the agent build the experience as you watch what works. Treating failure as the fastest form of learning.

And there's a deeper shift most PMs haven't processed yet:

Agents don't need specs. They need direction and guardrails.

A well-orchestrated agent doesn't require you to define every edge case upfront. It builds the experience as it goes, given good intent and good constraints. The PM job is no longer writing the PRD that covers every scenario. It's architecting the intent space the agent operates within, watching what it actually does, and adjusting the guardrails when reality teaches you something the spec didn't.

Less perfection. More orchestration. Less "is the feature complete?" More "is the loop learning?"

The Six Non-Negotiables for Shipping Agentic Systems

AI MacGyver energy only works if it's disciplined. Shipping fast without guardrails isn't MacGyver — it's reckless. Here are the six non-negotiables I hold every agentic build to. Each one is executable within a working week, not a quarter.

1. Use-case validation and the data reality check.

Before engineering cycles get committed, answer two questions. Is this the right job-to-be-done for the actual customer? And is the data available to solve it?

If the data isn't there, you need a short / medium / long-term plan. Short-term: pivot the framing so you can solve a real adjacent problem with the data you already have. Medium-term: land-and-expand — ship the thin slice, earn the right to collect the data you need through actual customer usage. Long-term: build the full vision, once the data flywheel is running.

The PMs who skip this step spend six months building beautiful demos for problems the data can't actually support. The ones who nail it are the ones whose products get renewed.

2. Risk/ROI evaluation using the Value Matrix.

Whether to automate, where to put guardrails, how much human oversight a given agent action requires — not philosophical. It's math. Severity × Time Sensitivity in the numerator, Blast Radius × Irreversibility in the denominator. Score the proposed action. If the math says go, go. If it says hold, hold. Full breakdown here: https://uxforai.com/p/should-the-agentic-soar-playbook-pull-the-trigger-the-math-is-simpler-than-you-think

With practice, this evaluation takes about an hour. Not a week-long design review. Not a quarterly planning cycle. 1 hour. The discipline is building the framework once and running it routinely on every significant agentic decision.

3. Judge-in-the-loop is not optional for enterprise deployment.

Agents are not safe to deploy inside the enterprise without a judge model evaluating every autonomous action against context, permissions, and authorized data sources. A cheap, well-prompted Haiku-class judge beats a fine-tuned "judge model" for 99% of use cases and ships this week instead of next quarter. Full architecture here: https://uxforai.com/p/kids-matches-agents-judges-and-the-simplest-soc-agent-safety-layer-nobody-built-yet

If your product leader can't articulate the guardrails, they shouldn't be shipping the agent.

4. Bounded research, relentless ROI.

"Let the developers explore" is Valley bullshit when the window is quarters, not years. Research is fine. A two-week spike is fine. Research disconnected from customer-visible ROI for six months is not. Pull the team back to execution sooner. If it doesn't convert to a shippable capability on a defined horizon, it wasn't research — it was stalling.

5. Cross-functional swarm accuracy meetings, weekly, non-negotiable.

Sales brings the customer issues from the week. SMEs explain what the agent got wrong. UX and engineering dissect failure modes in one room. Fixes go into RAG, instructions, and guardrails that afternoon. Tests get added to Phoenix Arize or your eval harness before the meeting ends.

This is the only way AI accuracy actually compounds. Quarterly reviews don't work. Monthly is too slow. Ship weekly, or the accuracy curve flattens.

6. GA is the starting gun, not the finish line.

The pre-AI playbook said GA meant you shipped; then you went to polish the next feature. That's over.

For agentic systems, GA is the price of admission. Real work starts the day after: sit with actual customers, ride along on real workflows, watch what breaks, fix it aggressively. Product leaders who still treat GA as a final celebration will lose to those who treat it as Monday.

Which Side of the Sort Are You On?

The market is already separating product organizations into two buckets. The ones still running six-month cycles will find the threat landscape finished three cycles while they were still on one. The ones building AI MacGyver teams will be the ones with products worth buying when Mythos-class capabilities land in adversarial hands.

We need to change the way we ship. We need to create a culture of safety where AI MacGyver product leaders thrive.

And here's the really crazy part: making it safe to experiment is actually the safer long-term strategy. The six-month heroic bet is where careers and companies quietly die. Rapid iteration is insurance. This isn't about giant personal teams or moonshot initiatives — it's about steady delivery, continuous rapid iterations, and a quiet operating rhythm that compounds into something the heroic model can't touch.

Prove me wrong.

Coming Next: The Step-by-Step Guide

I'm publishing a multi-part step-by-step guide for building agentic systems that walks through exactly how to execute on all six non-negotiables — in an hour per decision, not a quarter. You'll see the frameworks, the templates, the Value Matrix scoring in practice, the judge-in-the-loop architecture, the weekly accuracy meeting agenda, and the data reality check.

Greg

P.S. Ready to Become an AI MacGyver?

That's exactly what we teach in the UX for AI Professional Certification. Cohort 1 sold out. Cohort 2 is opening soon.

→ Get on the waitlist: https://uxforai.com/c/certification