Essential UX for AI Techniques: Vision Prototype

IMAGE CREDIT: Midjourney "rusty tangled industrial pipes in the factory in a style of arcane anime."

Several years ago, a Vision Prototype saved one of Greg’s IoT AI projects after it’s been stuck rusting in the corner for years. In this installment of our newsletter, we’ll explain what makes a Vision Prototype such an essential tool for AI projects and cover the practical "dos and don'ts" for building one, so buckle up for a fast ride. (BTW, this is the first installment of our “Essential UX for AI techniques” series. If you find this practical technical stuff helpful, let us know, and we’ll write more of it. This Newsletter is meant to be an interactive conversation – otherwise, we’d just write this as a book, OK? All right, enough talk, on to UX for AI!)

Before we talk about Vision, we need to talk about rust.

Typical industrial pipes look gnarly in the field, more like the Undercity background in a Fortiche Arcane anime than anything you see in shiny glossy factory advertising. Of course, rusted-out pipes are a safety hazard, but it is also a giant expense. You might not think it, but pipes make up as much as 40% of the investment in a typical factory. An investment that wears out quite fast in most cases – in about 5-10 years. Understanding the remaining life of factory piping (and extending its life as much as possible with special coatings) is a lucrative business. 

Unfortunately, measuring the remaining thickness of a section of pipe is a lengthy manual process that requires accurate measurements with expensive, sensitive ultrasound equipment performed by a highly trained technician (https://en.wikipedia.org/wiki/Ultrasonic_testing)

A few years ago, Greg was working with a large industrial company that wanted to produce a harness of cheap “set it and forget it” sensors that would be installed with the pipe and continuously monitor the thickness of the remaining material, alerting the factory owner when a particular section of the pipe was wearing faster than normal, or when a section of pipe was approaching the end of its life so it could be replaced or coated with a special corrosion-resistant material.

Now the company worked for years trying to perfect small chape sensors to be as good as a complex, expensive, and sophisticated manual machine because all they had to go on was this TACTICAL UI showing a simple graph, showing time on the X axis and pipe thickness on the Y axis:

There was no vision for what to do AFTER we figured out how to accurately measure the thickness with cheap “leave on” sensors.

While many factors were responsible, the overall outcome was clear: although the company spent years developing this technology, no matter how hard they worked, accurate readings of the absolute pipe thickness using cheap sensors eluded them. But the company just kept trying because they assumed that what they needed was the ability to accurately measure the thickness of the pipe, exactly like the manual process did because of all of the simple mockups for the system UI. 

The company never bothered to put together a vision prototype… Because, why bother – everyone knew how the system worked. They had their use case, SMEs (Subject Matter Experts), Salespeople, and Product Managers who knew their stuff, so why would anyone bother with (and I quote) “a #$%&@ picture”?

Sigh. 

With just a bit of research, it turned out that the customers did not care that much about this automated solution accurately determining the absolute thickness of their pipes. 

That’s because the government inspection protocol already mandated accurate periodic manual inspection readings. 

They cared most about the RATE of degradation and what to do about the corrosion, such as applying the internal pipe coating to slow down the corrosion, pre-processing methods like altering the solution's acidity, etc.

So there was no value in producing those accurate readings. 

What customers cared about then was Precision, NOT Accuracy:

And those small, cheap sensors the company developed were already very good at measuring the rate of change (e.g., the sensors were precise, just not very accurate, shown in the picture on the right). 

In the additional lucky twist of the Law or Universal Attraction, the company in question was already a leading supplier of internal pipe coatings specifically designed to stop corrosion, so there was a clear target and value in what AI could predict. 

It was a match truly made in heaven. 

So Greg came up with the idea of the UI displaying different scenarios of pipe’s life expectancy using various pipe coatings, pre-processing, etc., and having AI predict what would happen with a particular manually entered scenario while also suggesting the best course of action given the user’s constraints. This UI earned the company several patents, including this one: https://patents.google.com/patent/US10976903B2/en?oq=10976903, and moved the company way ahead of the competition. The Vision Prototype demonstrated the company’s vision, leadership, and commitment and helped close several key deals and enter into valuable partnerships. 

Now, if you’ve been reading the previous installments of our newsletter, you’ve already read about the importance of picking the right use case, and this was certainly another case of picking a bad one. However, there was something more at work in this particular problem – unwillingness to ask uncomfortable questions. You see, in order to build an effective vision prototype, it is vitally important to release yourself to some degree from the status quo. Permit yourself to dream about what could be, using uncomfortable questions as a guide.

Here are some “dos” to help you facilitate your vision prototype process: 

Do:

1. Goof around and find out. Be willing to play a bit! You don’t have to be a real asshole about it (you can remain an artisan professional), but it helps if you are willing to ask some pretty goofy questions. Use humor and fun and play at tipping the sacred cows. Give yourself permission to have a little fun. For example, Greg’s breakthrough in the corrosion case above came when he asked, “Just what the hell is the AI supposed to predict in this case?  The corrosion graph is so simple, a middle-schooler can predict when the pipe will burst – it’s a simple linear equation!” This willingness to play around a bit and challenge the status quo got the discussion going in the right direction, and the rest, as they say, is all patents and history.

2. Imagine and walk through the various scenarios. There is an essential brainstorming technique called “book-ending,” in which you take a solution and draw it out as far as it would go, then be willing to put that aside and do it again for a different solution (Want to read more on this essential brainstorming technique? Let us know in the comments below!) Be willing to do this in a lighthearted and fun, casual way to avoid pissing people off with your probing questions. While the team did not hit upon various scenario analyses right away, using simple drawings to quickly document various ideas without getting stuck on one particular solution, Greg was able to guide the team to a solution within an hour.

3. Assume realistic constraints. The question “Just what the hell is the AI supposed to predict?” exposed what data was available and what was merely wishful thinking. While the thickness measurement was unavailable because the technology was not there, the rate of change was there for us to use. Design is not art! Constraints are fuel – they are what makes the design move forward.  You do not need all the pieces to solve the problem; in fact, creativity often kicks in when the information is limited, and that is often exactly where AI shines.

4. Play the “Omnipotent AI” game. Often, during brainstorming, it can be a useful technique to assume that your AI will be omnipotent out of the gate and then figure out how to train it after you figure out the all-important question of where value comes from. Questions such as “What if you had the almighty AI? What would you be able to do with it? How would you deliver value to the customer?” In the corrosion case study above, it was this very question that yielded the idea of trying different scenarios with pipe coating and pre-processing and having AI predict what would happen with a particular manually entered scenario while also suggesting the best course of action given the user’s constraints.

5. Ask, “What would make it valuable?” This question is absolutely key for AI projects. Failing to ask that question early and continue asking it at every opportunity is why many companies pursue “shiny pennies” while ignoring piles of gold within their easy reach. Ask, “Why is this information valuable?” Then ask it again and again. Then follow it up with, “What would give us that information?” In the use case above, simply knowing the thickness of the pipe is valuable for compliance. However, that information was not available, given the level of technology. So by asking, “What would make this solution valuable as is, with only the rate of degradation, not the absolute thickness?” Greg uncovered that there is even more value in understanding how to decrease the rate of corrosion in order to extend the life of the pipes. And if we understand that, then we can ask, “OK, what would accomplish this decrease in the rate of corrosion?” The answer from SMEs would be “pre-processing and adding pipe coating.” From this level of problem definition, it is easy to step up to a UI that would display different scenarios for doing that. Likewise, from this problem definition, it is also quite easy to see exactly where AI comes into play and adds value.

6. Follow the Data. Another powerful question for the later stages of the vision prototype discussion is to focus on who would supply the data by asking questions like “Who has the data we need to train our model (Machine Learning)? How do we get it?” Have you heard the expression “follow the money?” Well, “follow the data” is the same thing, but for AI. 

7. Go into the field. Field studies are critical to really understand all of the moving parts, artifacts, and challenges different crews encounter while doing the installation, inspection, and workflows. There is just no substitute for first-hand field research.

8. It takes a village. As a UX person, your effectiveness is about the quality of your questions. It is also above whom you ask. In the words of Winston Churchill, “Never let a good crisis go to waste.” If the product development is stuck, use this crisis as leverage to bring together multiple perspectives through research or invite warring SMEs from different BUs to come together and bury the hatchet together over some pizza, beers, and AI brainstorming discussion “to help invision the future of the company.” Leverage co-creation and participatory design with customers and vendors. Use your favorite brainstorming techniques (like Six Hats, google it) if you need help creating more formal brainstorming exercises to provide more structure for the meeting.

While the best practices are important, we would be remiss if we didn’t also mention some severe mistakes to avoid in vision prototyping.

DON’T:

1. Don’t aim too close. Vision prototype time horizon should be 1-2 years or never. Anything designed to be released in 2-3 sprints is a tactical prototype, and as such, it is subject to different constraints (which we will cover in more detail in future installments of the newsletter.) 

2. Don’t just show a bunch of screens – leverage the use case! Build your vision prototype as a flow to solve a specific problem and address the actual use case in question to the very end, e.g., don’t stop your flow too soon, but keep going to the final screen where the customer benefits from your solution.

3. Don’t lorem ipsum – content is vital and needs to be treated with the respect and attention it deserves. Spell out the actual steps in the flow and make them as authentic as possible. This means numbers have to add up, names have to be realistic and match industry standards, measurements have to be reasonable, etc. (More on prototype content in the future installments, which means that you should sign up below so you don’t miss it) 

4. Don’t worry about every possible corner case. Go after the meat! Be sure to come to the primary use case first and only then consider going beyond that. Having a complete list of use cases covered is not the goal – the vision is! Most vision prototypes cover just 1-2 use cases.

Finally, one last word of warning: when it comes to building the MVP of your solution: don’t delve too deep. In the words of J.R.R. Tolkien, “You fear to go into those mines. The dwarves delved too greedily and too deep. You know what they awoke in the darkness of Khazad-dum... shadow, and flame.” When building out your idea, you, too, should be afraid. It is OK to dream as big as possible, but when it comes to building, consider the actual cost of implementation and think deeply about how to lower it! Make the first pass of the MVP as simple as possible, and then update your solution later. First, make sure customers use your idea. (More on this in later installments...)

We hope you enjoyed the latest installment of our newsletter: “UX for AI: How to Measure Rust in Your Pipes,” ahem… Actually, our newsletter is “UX for AI: UX Leadership in the Age of Singularity,” and you should smash the [sign up] button below so you don’t miss more hands-on UX techniques and IA design patterns like this in the future. 

If you enjoyed this installment and found this useful, please let us know and tell us what you’d like to see us cover in the future.

Rust out,

Greg Nudelman & Daria (Siegel) Kempka