AI is Revolutionizing Interaction Models

How things have been

Most applications — whether websites, mobile apps, or desktop apps — are largely hierarchical and static.

Hierarchical (Information Architecture)

Apps generally have screens with at least somewhat of a hierarchy: a root/home, and some combination of menus, buttons, and links to navigate through and to other screens that contain content; perhaps a screen dedicated to showing details for a book, movie, product, discussion, profile, or video. It’s usually not strictly hierarchical, with multiple paths and shortcuts, so it’s more of a web than a hierarchy. Regardless, this is referred to as the Information Architecture, or IA.

Sometimes the IA is great and it’s easy to find what you want. Sometimes it can be difficult or onerous to have to navigate through to get to what you want.

Search engines (whether 3rd-party like Google, or integrated into the app) can sometimes provide a shortcut, an easy way to bypass the architecture to get to the content you want. That’s the intention, at least.

Static

Whether an app’s Information Architecture was the product of thoughtful design and research, or came about naturally or by accident, that IA is part of how the app behaves and is used — what you have to use in order to get to what you want. That’s even more true with mobile apps where there’s less ability to use deep links and bookmarks to jump to what you want.

Examples: Discussion Boards and Support Sites

When searching for reviews of something — let’s say, for a refrigerator — you may end up on discussion boards where people talk about “What is the best refrigerator,” or “What do you think of this model of refrigerator?” There are often many pages of discussion, which can be onerous to page through. And that’s just one discussion thread; there are often dozens. Sometimes there can be hundreds of pages of content scattered across many threads and many sites (much of it not relevant) to sift through to find what what you’re looking for.

I often have a similar experience when troubleshooting something. For example, when trying to figure out “Why isn’t my WiFi working?” I’ll often end up at the product’s support site. I then have to browse through various support articles to find what I’m looking for, often piecing together information from various pages.

AI is changing how we access such information

As I wrote about in AI-augmented “Reading,” AI can find and extract the information we’re looking for and distill it down to a few paragraphs, often relieving us from the tedious and time-consuming task of sifting through tons of pages to find out which one has an answer to our question.

Creating and maintaining support sites take work

Support sites have consisted of writing support articles (which entails deciding which articles to write, collecting the necessary information from prior support cases, drafting it, editing it, etc.), setting up the support site (which entailed UI and IA development at some point), publishing articles, and keeping them organized and up-to-date.

But AI is essentially allowing us to bypass the support UX that was built. So, why do all that?

Rethinking that work

Companies can instead use AI to draw from the underlying knowledge base (support discussions, support cases, etc.) to support each user’s specific situation.

Beyond support sites, nearly all other apps (web, mobile, and desktop) are similarly constraining; their UI/UX are also static. “Creating an app” (designing and coding an app) is a middleman between the service’s information and actions (“backend”) and the user.

The Future: Outgrowing UI and IA with AI

While the conversational interaction model (“chatbot”) may be great for some things (like information gathering and troubleshooting), it won’t always be. Sometimes we do want a UI (a screen with laid-out information, buttons, links, etc), we’ll see interaction model changes there, too.

On-the-fly UI

The UI of apps — usually designed by Product Manager and User Experience designers, then implemented by software engineers — are often too limiting (and sometimes frustrating or onerous to use), or even just not optimized for what you want to do.

So, we’ll gradually move away from designed, limiting, static UI/UX and IA for many cases, and to on-the-fly, AI generated UI. Like cutting out the work of creating and maintaining curated support articles, the work of designing and implementing UIs can also be cut out.

And then we won’t be beholden to what PM/UX decides is most likely the best UI/UX for most people.

How AI can produce dynamic UX

These are a few of the different ways AI could generate UIs:

1. Use a UI toolkit that AI can use at runtime, such as HTML or React.
   • I think this is what we’d see first, as it lets UI toolkits — which are fast and efficient — do most of the heavy lifting, and gets us to good, consistent UIs more easily and quickly, and AIs already have a good understanding of these.
2. Uses a lower-level rendering engine (such as Vulkan or Skia).
   • While impractical today, it’s conceivable that AI can be trained to use these effectively to produce more custom and rich user experiences.
3. Generates images (raster) of the visual UI.
   • We’re pretty far from an AI being able to generate UIs this way quickly (e.g., up to 30+ FPS that’s needed for the interactivity of some UIs). However, in the more distant future, this may be how we create the most unique and interesting interactive experiences for games and fictional world exploration.

Yes, it will take time

Not only does change take time (as a whole, people are resistant to change), there are many things that need to be addressed and improved, such as:

• Compliance: AI too often does things it wasn’t supposed to.
• Context: AI can drift far off topic (causing it lose the ability to be compliant).
• Consistency, Predictability, Confidence: Just because the AI did the right thing the first time, many variables can cause its behavior to change and not do what we intended it to, which often isn’t acceptable.
• Cost & Efficiency: AI-generated, dynamic UI might not be cost effective enough to justify its benefit.
• Portability: Such UI may also be best generated on the client side, so user’s devices may need more capable, efficient, specialized hardware.
• Capability: UI-generation is a use of AI that isn’t as well developed as other uses.

Beyond

Conversational AI for information gathering (for support and troubleshooting) and accomplishing some tasks (e.g., “AI, set up this smart speaker for me. I want it added to the ‘kitchen’ [room] and to be part of the ‘downstairs’ group.”), and AI-generated UI on the fly are but a few ideas.

Think about how you could interact with a hotel concierge: it would be a combination of conversation, being presented with brochures, flipping through pages of a book of local history, and them making calls for you. We should expect AI to be similarly multi-modal.

Given time we’ll see many more new ideas that are hard to foresee… 🤔