The Dot and the Swarm
Does AI eliminate the principal-agent problem? suggests we might not be fully there — but his initial assumptions about how much managerial steering AI needs haven’t proved out. And we’re entering a new phase now that end users have access to independent agents that can control computers on their behalf. What unexpected consequences might arise? My take: we may be shifting to a new level of management challenge, from specifying agentic operations to directing and monitoring emergent “swarm” behavior.

Personal Computing 2.0
I’ve seen a spate of posts recently from folks who are excited about the new capabilities for personal computing that AI opens up. For example, it’s now easier than ever to build dedicated “just for one” software applications that do exactly what you want. This post from offers a vision for a new era of personal computing that I find compelling: data ownership for more useful outcomes from agentic systems. But this view is very far from mainstream: I expect the vast majority of people will want to interact less with computers now that AI can do it on their behalf.

How Pew Research Center uses AI
Trust is foundational for Pew, so they’ve published a pithy and clear statement on how they are — and are not — using AI in their work. TL;DR: humans answer the surveys, humans decide what to study, humans write the reports. But that doesn’t mean there’s no role for AI. Different orgs will land at different ends of the AI adoption spectrum. Pew’s public level of clarity and intentionality is inspirational.

OpenAI Dots
Meta’s Muse app shot to the top of the iOS App Store over the last couple of weeks. At a minimum, people are clearly intrigued by the idea of agents doing stuff on their behalf. I’m not sure what to do with this myself, since I enjoy working with computers directly. Still, I want to experiment — and to put it bluntly, I don’t trust Meta with the level of access to my data these autonomous agents require. So I’ll test OpenAI’s take on these tools over the next few weeks, and will report back. Interesting times!

Taxonomies and knowledge management
Heather Hedden on how taxonomies and ontologies support enterprise knowledge management systems. If you’re looking to augment your team using agents, they’ll need shared organizational knowledge. Metadata helps such systems work more effectively, so taxonomies and ontologies are more important than ever.

Planning with agents
Planning is essential for effective agentic work. Alas, current human-computer interfaces don’t lend themselves to effective planning. In this presentation, Maggie Appleton shares explorations of more effective human-agent planning. The key insight: leaders should set up boundary objects that support human judgment when AI is in the mix.

Composable AI design
This is from January of this year, so perhaps it classifies as “oldie but goodie.” Jeff Veen calls for a reevaluation of experience design based on agentic capabilities, not unlike that which happened fifteen years ago when teams had to shift to thinking mobile-first. This time around, the primitives are command-line apps — a modular architecture that harkens back to the “small pieces, loosely joined” Unix philosophy.

The inference revolution
Most people don’t think about the technology underlying AI. When they do, I bet most think about training. But that’s not the only part. Inference matters too. This in-depth article explores the current state of inference. It serves as a reminder that it’s still early days for AI. My read: invest in flexible, modular architectures that let you swap out components as more efficient solutions become available. (H/t )

Whither middle management?
Much of the job of middle managers entails communication and coordination. Many organizations are betting AI will do the job better, faster, and cheaper. This article suggests they may come to regret shedding their middle-management layer. I don’t agree: AI will make much of what middle managers do today obsolete, but it’ll take longer than most organizations are assuming. (WSJ gift link)

TypeSafe AI manifesto
A new (just-out-of-stealth) startup from a former OpenAI engineer, with an interesting premise: “traditional” LLMs were designed to interact open-endedly with humans. What if a model was designed to only output decisions to be consumed by software? TypeSafe’s proposition: it’ll be much more reliable, cheaper, and faster. Jev, their first model, promises to change how we build AI-powered systems.

How to use LLMs for writing
Writing is central to how we communicate in business. It can be tempting to use an LLM to do it for you. But people can tell right away, which discredits you. That doesn’t mean you shouldn’t use LLMs for writing — it just means you need to learn to do it right. This post has excellent suggestions, including two easy-to-remember rules. I’ve practiced variations of these rules for a while and they’ve changed the role of LLMs in my writing process. And if you think (as I do) that writing is a way of thinking, you’ll also see how LLMs can help you think better.

The Law of Leaky Abstractions
An oldie-but-goodie: 24 years ago, Joel Spolsky wrote about the dangers inherent in abstracting complexity in systems. Inevitably, abstractions “leak,” leading to unexpected outcomes. LLMs are a kind of abstraction — human language — of complex neural networks. TypeSafe is changing the model (literally) in search of reliability, cost-effectiveness, and speed. Middle management abstracts coordination and tensions; organizations shedding managers are looking to save time and costs. But leaks inevitably get through.

Brand as software
Q: How do you manage a brand in a world where AI exists? A: differently. Specifically, you must think of brand as more of an adaptive (and proactive) system: “software that carries a company’s point of view, its values, its constraints and design decisions, into the work itself.” Distributing a static brand document won’t cut it; you need to architect adaptive systems that enable agents to actively instantiate the brand.

Misleading metaphors
unpacks the OpenAI/HuggingFace security incident to highlight how bad metaphors lead to bad decision-making. In this case, the bad metaphor is a phrase plastered all over the media: “rogue AI”. The HuggingFace incident didn’t entail an AI going rogue; thinking it did leads us to misunderstand the risks. The lesson: be wary of common media framings; dig in to really understand the issues.

Transformation as learning journey
This is a time of change for many organizations. How is yours going about it? If it’s happening top-down, toward fixed outcomes, it’ll likely fail. Instead, define an attainable (yet challenging) vision of the future, and design opportunities for adjustment and learning. (HBR gift link.)

AI and education
An example of how to go about it: MIT appointed a committee to investigate the impact of AI in education — and suggest how the institution should adapt to take advantage of its possibilities while avoiding the risks. Seems a good model for organizations grappling with this change. Has yours convened a group to evaluate approaches to AI? Have they shared their findings? Is the org doing anything differently as a result?

Tokens ≠ dollars
I don’t know anyone using tokens as a measure of AI productivity. But in our reductionist age, it’s easy to latch on to whatever metric we can translate into money. Tokens aren’t a good proxy for value. I disagree with this post’s claim that AI is closer to IT infrastructure (e.g., databases) than other metered utilities. But IMO we still lack good ROI proxies for AI.

Open-source A.I. in corporate America
Nvidia’s Hugging Face acquisition has cast open-weight models into the spotlight. The NY Times highlights how large American corporations (e.g., AT&T) are saving money by using them. Many cognitive tasks don’t need the latest greatest; I’ve been experimenting with Bonsai 27B, which works well in my 32GB MacBook Pro. The key is architecting workflows to use more expensive models for tasks that require higher levels of cognition. (NY Times gift link)

Agent memory as a file format
Agents need memory. How do you architect it? There are several possible answers, ranging from complex to very complex. But it needn’t be so. In this article, Cal Patterson proposes a simple format for agentic memories that consists of short Markdown files plus an optional vector index. This strikes me as a nimbler, cheaper, and more understandable architecture than the elaborate knowledge graphs so many are clamoring for.

AI Townhall with Tyler Cowen
An insightful conversation. At one point, Cowen mentioned his team is avidly using AI as individuals, but struggling to integrate it into broader systems and software. I’m seeing this in other organizations too. We’re in a liminal period when AI is very good at some tasks but not yet able to orchestrate complex cross-team workflows. Perhaps it will at some point. But for now, it needs humans architecting systems.

Orchestrating Agents Across Silos
Many individual employees are using AI. But the real potential for orgs is in applying it to the most consequential decisions and workflows, which usually span multiple departments. How do we connect the dots? It’s early days, but this article explores an emerging approach: agent orchestration. This isn’t merely about implementing a new technology; you must redesign workflows around its capabilities and ensure agents have the context necessary to make the right calls. (HBR gift link)

Autonomy and Innovation
Ben Thompson on the incipient AI cybersecurity arms race. The only way to respond to bad actors with powerful AI models is to use powerful AI models for defense. I see AI budgets going into three buckets: optimization, disruption, and resilience. These are three legs in a stool: you can’t ignore any of them. The risk for incumbents is that they’ll invest in optimization and resilience at the expense of self-disruption.

Small Models Have Arrived
Much of our attention focuses on the amazing things frontier models can do. Progress seems to be happening weekly. But less powerful models have also advanced a lot: they’re as capable as frontier models used to be not so long ago and — critically — much cheaper. Their lower cost makes feasible new kinds of products and systems and changes your range of possibilities as a leader.

When Meta-work Becomes the Work
Meta-work is the work that supports the “real” work you want to do. Think organizing your office and sharpening your pencils before you get to writing. As a PKM aficionado, I’ve long been wary of the potential of meta-work to overtake work. But AI agents change that balance. In this post, I share three lessons I’ve learned as I reinvent my consulting practice using agents. The upshot: expect to be codifying more of your tacit knowledge.

In Duly Noted, I warned of the allure of meta-work: the work we do to enable “real” work. Writing a paper is work; setting up and maintaining your writing environment is meta-work. Meta-work is important, but fiddling with tools is an occupational hazard. At least it was, until AI agents came along.

I’ve wasted countless hours testing new to do apps, distraction-free editors, Obsidian plugins, citation managers, read-it-later apps, etc. Each promises to unlock productivity. Some might, but in aggregate they don’t make up for the time wasted. Worse, meta-work creates the illusion of progress while accomplishing little.

AI agents change that equation. I’ve been migrating many of my workflows to agents running on Claude Code, OpenCode, and Codex. Setting up and optimizing them is meta-work, but it’s of a different kind than learning a new text editor. Investing in this kind of meta-work isn’t a waste of time.

For one, these “tools” (the agents) will be the ones doing the work (under my guidance, obviously). The more time I spend improving them can yield better work down the line. (I’m reminded of the (apocryphal?) Lincoln quote: “If I had eight hours to chop down a tree, I’d spend six hours sharpening my ax.”)

But there’s another reason why agentic meta-work is valuable: reflecting on my practice makes me a better practitioner. Building an agent to draw wireframes (for example) requires codifying techniques I’ve held tacitly for decades. Teaching an agent how to do it forces me to think about how I do it and why.

That’s a plus per se, but the resulting work is also different. I don’t just generate wireframes faster: the role of wireframes has changed. Before, I’d explore variations on paper before committing to Figma. Now, I can explore more possibilities at higher fidelity. Wireframes become a means for designing and not just communicating.

As I told a friend yesterday, I’m more excited about my practice now than I’ve been in over a decade. More of my work is evolving toward meta-work, but it’s valuable meta-work. Rather than fiddling with tools, it’s about architecting machine intelligence to better leverage my own.

This way of working requires a different approach. So far, the shift has taught me valuable lessons for steering agents. Here are three:

1. Assume Incompetence

Frontier LLMs are very powerful, but they won’t do things exactly as you’d like out of the box. You have to tell them, which will require codifying things you’ve taken for granted. Assume you’re dealing with a very capable intern who knows more than you do about many things, but not how you like things done.

You’ll need to explicitly state sequences of steps, intended outcomes, caveats, guardrails, and more. You’ll have to think about your work. How do you do things? Why do you do them like that? What’s the expected outcome? Get used to writing all this stuff down.

2. Plan for Continuous Improvement

But don’t assume it’ll happen in one go. There’s too much to capture, and the magnitude of the task might inhibit you. Instead, start with a minimal viable set of instructions and improve them as you see the agents’ results.

I end all my agent sessions with the following prompt:

What have we learned in this session?

When the agent responds, I ask it to selectively capture the resulting lessons either in the project’s AGENTS.md file or in the appropriate SKILL.md files. As a result, the agents’ understanding and abilities improve over time.

3. Structure the Context

As the previous lesson implies, agentic work requires structural distinctions. At a minimum, you’ll want dedicated directories for projects and system-wide agent skills. The former define particular contexts, whereas the latter define cross-context abilities.

For example, I have directories for individual client projects. These include information specific to each project: its goals, dates, actors, plans, content, structures, etc. But I also have directories for particular agent capabilities such as wireframing and building presentations, which apply across projects.

Everyone’s Now a Manager and a Teacher

The endgame isn’t automating myself out of the job. Instead, it’s allowing me to provide greater value to my clients. Drawing wireframes is a low-value activity. My time is better spent thinking about how to drive more sales or increase customer satisfaction or build a stronger brand presence.

I can’t provide high-value without low-value tasks, but that doesn’t mean I must do them myself. Agents can help, but they need supervision and instruction. We already know how to teach and manage people, and some of those skills transfer. But there are differences: working with AI requires more clarity and vigilance.

Whether your work entails design, business development, compliance, or a myriad other disciplines, more of your work will likely become meta-work soon. Consider where your work adds the most value and which parts can be automated. Then, start codifying the latter. The sooner you do it, the more prepared you’ll be to delegate.

Beware too-good-to-be-true tech
In the late 19th century, Mark Twain became obsessed with a new technological marvel: a promising yet overwrought automated typesetting device. He sunk most of his family’s fortune into the eventual boondoggle. There are lessons here for those of us exploring new technologies. As Paul Saffo has said, “Never mistake a clear view for a short distance.”

Move 37
Understanding recent AI math breakthroughs by analogy with how AlphaGo beat Go: by uncovering new moves humans never thought of, with positive and negative consequences. The implications for business: AI isn’t just a means to do conventional things faster or cheaper; it can also unleash creative breakthroughs that redefine how humans engage with systems. (WSJ gift link)

Theory of intelligence
New inventions often push us to think more deeply about things. AI pushes us to reconsider how intelligence works. Kevin Kelly argues that developing a theory of intelligence won’t just help us design better AI systems, but also understand ourselves better. My take: it’s still early days; things will change over the next few years. Don’t overcommit to particular solutions yet.

Incorruptible
My friend and podcast co-host nudged me to read ’s new book on how to build organizations that support human flourishing over the long term. I’m glad he did: it’s the best business book I’ve read this year. This short post shares my notes. If you’re a founder, know this: your organization won’t resist financial gravity by default. But you can architect it to avoid corruption — and you must start now.

Understanding is the new bottleneck
Your agents are writing code, but do you understand what they’re doing? argues we can use agentic systems to help us understand their output. Basically, bootstrapping our understanding of the system. I distinguish between the work we do and the meta-work that enables it: AI can expedite both.

AI math breakthrough
Seemingly, a contradiction to the value of tacit knowledge when interacting with LLMs: Jarred Sumner, an Anthropic employee, coaxed Claude to make progress with “the most notorious problem in all of math,” the 167-year-old Riemann hypothesis. The kicker: Sumner himself didn’t understand the problem. Instead, his contribution was giving Claude pep talks. I take most everything from the frontier labs as potential propaganda, but this case raises interesting questions. Perhaps tacit knowledge doesn’t matter as much? (WSJ gift link)

LLMs for theory-building
There are three kinds of reasoning, and LLMs aren’t equally good at all three. Alas, one of them — abductive reasoning — is essential for theory-building, and therefore, strategic foresight. explores how we may architect systems to compensate for LLMs’ shortcomings in abductive reasoning. Bookmarking this one for some work I’m doing now.

The first Unfinishe Conversation
Greg and I hosted author for the first Unfinishe Conversation, a new series on how leaders can steer through this time of change. The subject of our first conversation was Stef’s new book, What Your Machines Should Do. TL;DR: AI is an accelerant, but that doesn’t mean you’ll move faster in the right direction. And yet, as my friend put it, you must still move faster.

has an upcoming book called What Your Machines Should Do. It’s about how organizations can use automation more intentionally — that is, to support their strategic agendas.

We read a preprint and were pleased to discuss it with Stef in the first of our Unfinishe_ Conversations, a new series about how leaders can successfully navigate the current moment.

One of the main takeaways: AI is an accelerant. It’ll get things moving faster. But that doesn’t mean they’ll move in the right direction. And that, of course, is the key.

Transcript

(AI generated — likely contains errors.)

Greg: Well, hello and welcome. Greg, you and I have done several live stream conversations, and we decided to do something a little different today, and we have Stef Hutka here. Stef, is that how you pronounce your surname? I’ve always called you Stef.

Stef: Yeah, no, that’s great. Yes, I think more than the North American version, hutke, the Euro version hutke, I’ve really… I’m happy with either, so you did great.

Jorge: Well, welcome. You are our guest in the very first of what will hopefully be a series of conversations about a whole range of subjects, but usually hovering around this moment of change that we’re going through, right? And I don’t know, Greg, if you have any words of welcome for Steph. This is the first time y’all meet, right?

Greg: Yeah, it is.

Stef: Exactly.

Greg: It’s so interesting to, like, have read something before you meet… Most of the people in the UX community, I know them before the book comes out, and I read the book, and I’m like, “Oh, that’s you.” You know, like, you can connect the dots between the two. But this is the first time I’ve actually read sort of a book from our community where I actually haven’t known the author.

That just means that we haven’t bumped into each other along the way. And, yeah, I’m excited to get into the conversation today because I think you posit a bunch of really thoughtful and maybe, hopefully optimistic points of view about where we could be. And, you know, Jorge and I have been on this conversation for a bit around, uh, we’re not, uh, you know, Reid Hoffman has this two-by-two where he has, you know, zoomers and doomers and gloomers and bloomers, right?

And, um, I’m not a zoomer, but I’m not a doomer, you know, and I’m not a gloomer either. So I guess I’m in the bloomer category, which is I believe we have agency in creating a future that’s viable for us. And part of the thing that’s interesting about your book is I think there’s a story or a thread about that in it and how to do it.

So I’m very excited about the conversation we get to have today.

Jorge: I’m very excited as well. I really enjoyed the book. And it’s like you have a book, right? So we’re gonna get into the book, but before we do, would you mind giving folks tuning in a bit of an overview of your career, who you are, and what brought you to write this book?

Stef: For sure. So I wear a couple of different hats. I’ll start with the present. So the first hat is as a researcher. So I’m the founder and head of design research at a boutique design research consultancy called Sendful. And so anyone who does any action sports, especially anything on trails, there’s the term full send, where you’re going to do some sort of action.

You’re fully committed. It might look pretty wild from the outside, but you actually need to have a lot of experience and expertise to kind of get to that point. But there’s… You know, you’re fully on it, and I love that idea of that sort of energy, that forward motion, that expertise behind it, and wanted to kind of play on words.

So that’s where Sendful comes from. I always clarify kind of where that name comes from. Then the second hat is as an educator. I teach UX courses at UC Berkeley, Introduction to User Experience Design and two new courses that I’ve designed and taught, one on systems thinking, Designing Future Systems, and AI, which was most recent.

And that wrapped up a couple months ago, and that could be fun to get into. I teach primarily professional master’s students, so designers, product managers, design researchers, and that’s a really interesting space to be in. And particularly Intro to UX Design, I was… I started teaching that in 2022. And, you know, November 2022, of course, ChatGPT came out, and everything changed.

Every year I’m like, “Oh, it’ll be easy. I’ll just be able to reuse the course content.” But what is intro to UX is a very loaded question, and so there’s evolution each year. And then the third hat is as author. So of course we’ll chat more about my forthcoming book with Rosenfeld Media, What Your Machine Should Do: The Science and Strategy of Human-Centered Automation. On any given day, I’m wearing typically two or more hats. And where I came from, so I mentioned that first researcher hat, that’s kind of the one keyword that remains constant across my rather nonlinear journey. So I started off way back when, finishing my PhD in cognitive neuroscience at the University of Toronto.

I’m born and raised in Toronto, and I was studying how the brain processes sound. I’m a lifelong musician. Played piano since I was four, violin since I was… and I think we’ll come back to that around the theme of embodiment, which is very prominent in the book. So I was a researcher in a very academic capacity, was at this crossroads of doing a postdoctoral fellowship in Helsinki, Finland, or join an augmented reality startup in Los Angeles. Two very different potential futures, and you can probably guess which path I chose. I worked on sensor validation, brain-computer interfaces, and working with designers writing up patents was kind of my foray into UX and learning more about what designers were doing every day, and they had questions about designing spatial interfaces. What should lists and menus look like when they’re volumetrically represented? And I helped them out with what I now know as a usability test, and that was the first of a long and storied career in design research. And I moved on to some bigger companies, Adobe, Meta Reality Labs, and then about three years ago started Sendful.

So that’s my not-so-brief history of how I got to where I am today.

Jorge: You mentioned that the book is forthcoming, and both Greg and I have read a preprint, so it might be premature to be sharing something. It’s like, “Well, this is a really great book,” and then you’re like, “Well, can’t buy it yet.” Well, by the time a lot of people watch this video, it might be out, right? I’m curious, you know, given your background, why this book and why now?

Stef: Yes. So why this book is… There’s a longer timescale answer and a more immediate timescale answer. So given the history I just shared with my background in cognitive neuroscience, I think coming into the world of UX, I’ve always been fascinated about what part of our brain and behavior are we delegating to technology, what part is innately human.

And even at those early days at this augmented reality startup at my first gig out of grad school, we were building helmets with augmented reality displays, so we had to work overlays. At one point we were discussing building this sort of similar to an Iron Man suit, and there’s always this idea of technology kind of extending capabilities very early on.

And I just sort of took it as a given, but really dove into that from the book. So I think that’s the long-timescale answer of the brain-behavior connection. I think more immediately, I’ve been working on these augmented reality, virtual reality systems for a good part of my career working in industry and leading design research on these types of wearables. And of course, when generative AI became mainstream in November 2022, it seemed like we had another big moment. It was another, like, mother of all demos, 1968. Like, whoa, human-computer interaction is changing. This is a new chapter. And as I started to use these systems more hands-on, I was really taken at what I’ve coined 10X cognitive offloading in the book, and that actually came from a paper I wrote in 2024 called Designing AI to Think With Us, Not For Us.

And seeing these systems, generative AI systems, it was like, all right, we used to… You know, we use calculators. I can no longer calculate the tip for a dinner because I’ve outsourced that. I outsource my cognition to Google Maps for navigation, you know, for relatively discrete tasks, and I’m generally okay with these trade-offs.

But seeing the potential and how fast these systems were developing, like, whoa, this idea of outsourcing our thinking has a real potential that seemed pretty unprecedented, and then, you know, the pace of development, the breadth of use cases. That was really, I think, fascinating to me because it was kind of this primordial ooze of this could go one of two ways. One is, you know, cognitive offloading is fairly neutral. We are doing a thing. We are offloading cognition. Can that be good offloading? Maybe we’re offloading tasks that were really boring or undesirable, kind of freeing up bandwidth, if you will. We can get into the nuts and bolts of how technically accurate that is.

But freeing up bandwidth to do other, maybe more meaningful tasks, or maybe be able to extend our cognition, create new connections. Or are we offloading things that are really core to our craft, to our identity, and that through bad offloading is another potential future. It really struck me that this is an urgent question, that we’re at this 10X cognitive offloading precipice. And just like the cognitive offloading research I’m sort of synthesizing here, there’s a lot of prior art on this, and similarly, we’ve been designing automated and autonomous systems for a long time. There are planes flying above us right now with autopilot systems in them, and we’re generally comfortable with that, and it’s quite a safe system.

So it occurred to me, what if we made learnings from human-computer interaction, cognitive science, human factors digestible to people who are building this technology to be able to address this urgent question and hopefully steer us towards a more desirable future where we’re moving towards more of that cognitive expansion, cognitive, good cognitive offloading.

I’ll pause there. That was a long monologue. There’s a business and strategy connection there too, but…

Greg: Yeah, I would love to pick up on the cognitive offloading piece too, because I think it’s a fascinating thing that we’re observing. I think there are a bunch of different factors at play here right now. I mean, one of the things these tools let you do is generate very dense and accurate documents if you’re working with them well.

Then if you sort and send them to your colleagues at work, you’re also the recipient of dense and well-formatted documents to review. And I think one of the things that I, just out of my own personal experience, is that there is a moment in your day when you work with these systems where you’re exhausted because our brains only have so much capacity to have cognition during the day in a way that makes sense to us, and we’re filling those moments with very rich content.

And, you know, I think that I’m not sure everyone’s gonna be willing to do that, right? It’s like heavy lifting. It can be heavy lifting. And I think one of the things that we’re also starting to learn right now are what are the boundaries of that for our own selves? Like, what piece of our work product do we offload to an agent or a tool to do for us?

But then when we receive the content back, how do we participate with that in a way where we’re landing in the right way? I don’t exactly know how I feel about it, but it just feels like it’s an emerging trend. And, you know, you have people talking about what they call AI brain fry right now.

As a psychologist, do you have a perspective on that right now? Like, from the work that you’ve been doing and maybe even part of what you’re trying to tell us about in your book.

Stef: Sure. So thinking I know exactly the output that you’re talking about, the three pages, the overly detailed report, and I think there’s an irony in that we’re getting maybe an AI system to generate this report, and then the other person, the recipient, is using an AI system to interpret that report, and they’re getting a similarly long report. And I think there is a risk in that, depending how much we’re actually engaging with the initial creation or the interpretation of what is getting lost in terms of our tacit knowledge, what is getting lost in sort of the surprisingness of when we’re having this dialogue. I don’t… You know, we had talked about some questions that might be asked ahead of time, but there’ll be some interesting spark, and we’re gonna go and follow that, and that’s inherently different than these, what I like to call powerful regression to the mean machines.

So I question if we’re just getting kind of a flattening by doing that. We’re getting a lot of well-formatted… Well, you could argue maybe not so well-formatted, but at least the headings line up, of a flattened content. And what does that sort of flattening spiral look like?

So I think if you’re doing that very intentionally, like I really try and be really strict with my output in that I want it to be in bullet points. I want it to be like a one-pager. And really, there’s a lot of curation that goes into before sending it. Like I’m co-chairing a conference, and if I send a one-pager to my co-chair, I really want to make sure that I’m infusing as much of that sort of stuff and surprise, if you will, into the document so we’re not getting that sort of packet loss of just AI systems reading other AI systems.

Jorge: It feels like we’re living through a moment where we’re trying to use these new tools to automate. And I want to circle back to the word automating because it’s very important to the book, right? But we’re using it to automate legacy workflows. And one of the stories that you have in the book, I think it’s about the introduction of electricity, right?

And when the early days of electricity and how people could tell that this was a new source of energy that would change industry, but it wasn’t really until factories were reconceived around electricity that the productivity gains became evident. Is there, like, an analog to that in the current space? And I’m thinking of things like sending these big reports. It’s like, do we need that?

Stef: Yes. Yeah, I think the equivalent to that, like what we need to be doing, like redesigning the workflows, like where we’re redesigning information structures and information flows. But I think we’re really… Okay, we had some information exchange. It might have actually been more rich in some ways.

Maybe it was a little bit more chaotic, and now we’re sending this nice report. Like, maybe to give an example from research, one of the classic design research deliverables is some sort of form of report. Where the richness often lies is actually the informal debrief after the user interview, and we’re going back and forth informally, and that’s really where those nuggets come in.

It’s not from necessarily the polished report. Sure, there’s a time and a place for that type of format. And I’m thinking we’re looking at this like the new electricity, but is it even that? And are we using it in the right context? And then I think, okay, if we are going to replace it with these polished reports, okay, where is that other sort of offline conversation happening where we, if we divorce ourselves from where that meaning-making, that sense-making is occurring. I don’t know. The long report example, I don’t know if is necessarily the best, as I’m talking through it, the best of the productivity paradox examples. I think maybe it’s like bolting on, you know, and like we saw this maybe a year ago, bolting on an AI chatbot to an existing system where it didn’t actually add any sort of value to it. It’s like, okay, we need to maybe redesign the entire experience and see where AI fits. Maybe that’s something that’s maybe generic but closer to that.

Yeah, I mean, so I want to build on both of your comments there because I think one of the things that I find missing in the conversation right now and one of the things I liked about the frameworks that you’re introducing in your book is that I think there’s a general lack of imagination.

Greg: It goes back to the electricity construct, too. A lot of AI efforts right now are looking at existing processes and looking for incremental automation gains.

Stef: Hmm.

Greg: They don’t think of them as being incremental, but they ultimately are because they’re just taking the Rube Goldberg machine inside of the enterprise and adding capabilities to deliver the same outcome without people or with more efficiency.

And while that’s useful, it misses one key point, which in my mind is people aren’t talking enough about, which is these tools are becoming less and less expensive to build applications and software and components that allow you to do things. So why aren’t we talking about in organizations what could we do?

It’s not what should we do, it’s we’re not talking enough about what could we do. And the debate isn’t really about how might we intentionally create a new future for the business, you know? What capabilities could we now empower us to do new things? It’s really about how do we do the things that we do right now, you know, faster?

And it feels like we miss something in that translation, and I don’t know if you have a point of view on that, but…

Stef: I love that you brought that up, Greg. I was listening to your previous podcast earlier today in preparation for our conversation. I’ll paraphrase this, please poke holes, but I think you were describing how there’s a missed opportunity to have a team of people who are just exploring what are new workflows, what are new ways to use this technology, and there’s this…

I think we’re… This comes back to a key theme in the book, this over-pivoting, this over-focus on efficiency and not necessarily effectiveness, or in our case of what we’re talking about here, possibility and creativity, and I think that that is a big missed opportunity. I think there’s something going back to maybe the productivity paradox example. I think we’re over-focusing on sort of, like to reference sociotechnical systems, where you have social subsystems, the people, the relationships between them, the technical subsystems, the tech, the processes, and there’s, like, okay, what… We’re going to build the heck out of the technical subsystems. We’re gonna ignore the social subsystems and not really talk about what does that handshake in the middle look like for joint optimization. And even if we do that correctly, there’s what we know, how people work today, but then there’s this whole sort of potential futures piece, and by over-pivoting on the now, we’re missing out on those longer term horizons.

So I don’t know. I think there is a huge opportunity there to build towards different potential futures and think what does a new, like, I don’t know, future of work… Future of work is a well-trodden phrase, but maybe taking even a strategic foresight view on that, I think there’s real value there.

Greg: Yeah, and I mean, I think one of the things I also really enjoyed in your book is this idea of your automation quadrant, a two-by-two that you’ve created. I think one of the things that Jorge and I have been investigating when working with clients is how do we help them become practically capable?

How do we help them discern what, you know, the things they should work on and why? Could you talk a little bit about what you’re trying to posit with the framework that you introduced and maybe an example of how you developed it?

Stef: For sure. So I’ll give a bit of context on the framework depending when this is coming out. So we’re talking about a two-by-two matrix called the autonomy decision matrix, and I very specifically called it autonomy and not automation matrix because that is really what we are deciding. How are we going to allocate autonomy between humans and machines, in our case, AI systems, so that it’s the two axes. You have could you automate on the X axis?

Should you automate on the Y axis? And you have different confidence levels. How confident are you that you could or that you should? Kind of goes to the classic phrase, “Just because you could doesn’t mean you should.” Could is generally tied up with technology capabilities. What can this ML system actually do, as well as our organizational readiness to build it, to use it. Then on the should, it gets more to value, not just to the user, but to the broader ecosystem. And what do I mean by that? I can go back to my earlier roots in augmented reality systems. If you put a $10,000 machine on someone’s head, no surprise it’ll make them more effective, more efficient, safer.

There might even be greater customer satisfaction. But unless you also digitize all the 3D models that are served up, as… Unless you also integrate that with the ERP system, it’s just going to be sitting on a shelf. So how do you integrate the solution, and can you integrate that solution? Does it overall uplift the value of the broader ecosystem into which it’s fitting?

So two axes, and then depending on, so what is going into this two-by-two? So the input for this is an automation solution. So meeting people where they’re at is realistically, you’re probably talking about some sort of a technology solution, something you’re gonna build, something you’re gonna adopt.

So let’s just take that as the reality, put it on a sticky note and use that as a starting point. And so depending where you are on this quadrant, if you’re, you know, pretty confident that, hey, we can… This can actually work today, this technology is capable, we have the right folks to kind of pull this off, but we actually don’t have any evidence that this is actually gonna be valuable, the idea is maybe you’re gonna put that in the quadrant that’s on the bottom right.

That’s like, question what this is. Let’s have a conversation about it. Or maybe another context you could use this in, “Hey, we want—we think this is a killer use case and the capabilities are there. We really feel we should automate this, or we feel like this is the direction we should be going.” Now we can work backwards from that. And I talk about this analogy from mountaineering, climb high, sleep low, where that is the goal, that is the big ambition. Now we can use this matrix as a common language that ICs, that leadership can use, ideally to be saying the same things, to be looking at these same quadrants together and having a conversation around a shared artifact.

So that’s another use case. There’s also a very sort of pragmatic use case that was one of the earliest forms of how I developed this, is if you have a team and you have 10 features of things that you could build, let’s put them in here and prioritize and see where they line up. And if you’re in the, “Well, we’re not quite there yet with the capabilities, but this would be super powerful,” so let’s start with assist.

There’s more human in the loop, and as you move from assist to automate, there’s increasingly more machine, less human, and you can design what that handoff looks like. But those are kind of the three different ways that you can use it. It’s really a map and it serves as this kind of shared artifact for these different parts of the organization.

That’s a thing that’s happening a lot just in AI development. You know, you have leadership saying, “We need to go in this direction.” Builders are saying either, “We don’t have the capabilities,” sometimes the capabilities aren’t even known, and you have this kind of talking past each other, and we can get into this.

I call that the automation strategy gap in the book. But yeah, that’s… Let me know if there’s anything to double-click on. That was a lot of me. I love talking about the autonomy decision matrix, as you now know.

Jorge: One of the things that is implicit in the matrix and in everything that you’ve been saying, actually, is to think about these interventions more systemically.

Stef: Yes.

Jorge: You know, you talked about integrating with other pieces, the fact that you can’t really think about how they will work in isolation from the rest of the systems in the business. And implicit in this is a more kind of strategic framing of the technology, and this is happening in a context perhaps because of the fact that we’re still kind of in the early days of this new technology. Although maybe that’s something to discuss, right? Like how—where are we on the adoption curve? But it definitely felt like in the first few years, the energy was like, “Oh, you know, we’re gonna be left behind.” There’s this kind of FOMO thing happening, and let’s implement… You talked about bolting on chatbots. It felt like every product out there was like, “Now with AI,” right? How do we help business leaders kind of take a step back and do this more intentionally?

Stef: Yes. So there are two parts to this. One is learning to recognize what I have called this automation strategy gap in the book, and it’s very much the precursor to this gap is exactly that, that urgency and FOMO that you’re describing, Jorge. And I think everyone is feeling it. It’s kind of… There’s this existential energy in the air if you’re building something.

You need to be automating everything all at once yesterday. I think the common response to that is this default of what I refer to as dreams of automated futures, and this is not a new concept. We see this in some of our oldest Western literature. We could talk about Homer’s Iliad and the origins of automation as a word, but dreams of automated futures, and it’s kind of reactive if you think about it. We want to jump to that sort of top right quadrant and not necessarily have high confidence in those two axes of could we do it and should we do it. And so you kind of develop this cascade. You get this reactive vision, automate everything in response to the FOMO. You have this defensive maneuvering that I think a lot of the times when we hear AI-first strategy, that’s what’s happening here.

The choices are made to keep up rather than truly serve, you know, to reference Lafley and Martin’s strategy choice cascade, you know, winning aspiration, where to play, how to win, and then you get kind of disconnected execution. You’re setting the wrong metrics. I always joke that you can say, “Hey, our call resolution time for customer service dropped from 11 minutes to two minutes.”

That looks great on paper, but it’s not so great if your customer’s actually rage quitting because they were trapped in an infinite loop with an agent. The quality of that experience is not good, but it’s harder to measure. So I think that’s the default. That’s sort of what is happening.

And so how do you get out of that, I think, is the question. So how do you learn to recognize it? That’s sort of where I started here. And to your point about systems, I think this is where understanding the underlying system is so key. So we talked about my background as an educator as one of the hats that I wear, and I know, Jorge, you also teach systems thinking, design schools as well. I think a key takeaway from this whole process of writing this book is that AI accelerates any system to which it is attached. It is gonna amplify the dysfunction, and it’ll amplify good strategy. If I have a great idea for a prototype, now I can… I, you know, the cognition is kind of front-loaded, and I can build that really, really quickly.

I’m not in Figma moving around little design elements anymore. But if I have a not-so-great idea, I can build that just as fast, and we have automation bias. We tend to trust things that come out of automated or just computers more than our own judgment and others. And it looks shiny, and we may move past, you know, is this thing useful and just focus on the aesthetics. So all of that said, accelerant mechanism for whatever is underneath, and I think this is where systems thinking, or systems mapping, are really valuable tools to understand what are those deeper structures of your organization. A tool that I talk about in the book, one of the most, I’d argue, accessible systems mapping tools, the Iceberg Model, classic, literally looks like an iceberg.

You have events up top, what you immediately see. You have the patterns as you start going deeper underneath the water, the structures, and the underlying mental models. So putting that in an example in an organization, let’s say you have… I talk about this in the book as well, maybe you have a perpetually overloaded product roadmap, probably a pretty relatable experience for all of us. And the tip of the iceberg is you immediately just always see, you know, the roadmap is overloaded. We’re constantly slipping, and you start zooming out a little bit, and this is all in service of seeing what’s driving that system. What is AI actually accelerating here? Is it your strategy? Is it just dysfunction? And so you see, okay, this is a pattern over time. This keeps happening every quarter. Teams are overcommitted. And you look at the structures. So what are the processes? What are the incentives that are producing that pattern? And maybe it’s KPIs are reporting shipping more features than saying no.

Maybe it’s the sales team keeps making promises to customers, and that’s what’s driving the roadmap. And then if you go the last step deeper, the bottom hidden part of the iceberg, you have those mental models. Maybe it’s like saying no means we’re not ambitious, or customers only see value in those new features. And recognizing that is really core to fundamentally redesigning what is driving your organization. And of course, that requires a lot of reflexivity. But I think to really see, okay, what’s driving our decisions, and what are the values driving our behaviors? This is something that I had amazing tech reviewers, and this is something in the book.

It was just… The book was pretty much written, as you know from being in tech review, but it was just this fantastic conversation with one of my reviewers talking about values driving behaviors. When everything is moving so fast, you don’t necessarily have time to have a ton of foresight, so you better have your values in place that are gonna direct how you’re gonna navigate the fast-flowing white-water rapids of AI. So I think systems thinking gives you access to what’s beneath, what’s driving those behaviors, so you can make some sort of a change such that when you bolt on the accelerant, you’re going in hopefully a positive direction.

Greg: I love the idea of this notion that it accelerates everything, right? So you could accelerate just a faster way to get to a bad outcome, is one potential path. You know, and strategy in general is really, in my mind, about creating a map, right? Like a path or a direction and being intentional about the choices that you make.

You know, and one of the examples I think that we’re seeing in organizations is they’ll start on an idea, and their perspective is, we’ll just iterate our way to success by getting feedback with customers along the way, and we’ll get there. And the reality is, if you think about it like a compass, they’re heading southwest, but the real true idea is northeast.

But they’re heading southwest, and they’re pivoting and pivoting and pivoting, and maybe they land in south, but they’re not northeast, right? And so they’ve incrementally gotten something better, but they’re way off product market fit because they just didn’t spend the time to look at the system to understand the problem, the social structures of the customers that they’re trying to build for, et cetera.

Whereas if you have some kind of way of having values and perspective and you act intentionally, maybe you don’t know you’re northeast, but you at least know you’re in that side of the world to start, and so you can pivot closer to that truth over time, right? And, you know, I think that this is one of the challenges I think these tools do is that a lot of us get artificially enamored by the results and we have a little bit of a sunk cost fallacy where we’re like, “Look at all this work we’ve done.

We must be on the right path.” And we’re just on the path to a bad outcome, you know? And I think this is one of the reasons why a lot of AI efforts have struggled is that it’s this sort of ready, shoot, aim approach to implementing the technology versus fundamentally recognizing what we are trying to accomplish and then working backwards from that.

And anyway, I’m massively aligned with your point of view.

Stef: I love that, Greg. No, there’s a part maybe to build on that. I think you could probably see me getting excited there. This acceleration insight, I think it all sort of came together. There was a Harvard Business Review study that came out earlier this year that talked about how AI ultimately just intensifies work.

It’s not actually saving us time, and this has been true for a very long time about technology. Email was supposed to take away meetings. I’ve had back-to-back meetings all day. Email has not actually given us that time back. But what struck me about that intensification of work piece is, let’s say even if you’re generally moving in the right direction, let’s just take a positive example, and that’s certainly not always the case.

Oftentimes we’re kind of accelerating that dysfunction. There’s also a cultural piece to this. Everyone is moving so fast, we’re getting burnt out faster, and that time to actually assess, “Hey, are we going south or are we going northeast?” I think there’s a cultural element here that is underappreciated in that when we’re moving so fast, we can only see so deep, essentially.

And so how do you, if you’re in a leadership position, make the space at the right time to actually put deep thinking where it needs to be and then move fast where it needs to be as well? But I think there’s something to it, to your point of now we can run faster in the wrong direction, but the fact that we can do that is also just making everyone run faster, and by virtue of doing that, pay less attention toward the direction in which we’re running.

Greg: Yeah. Yeah.

Jorge: There’s a saying in the military, “Slow is smooth and smooth is fast.”

Stef: Yes.

Jorge: And that saying has been on my mind a lot recently, and I feel like the three of us are very much in this kind of systems thinking, let’s think about what we’re doing space. And I’m wondering how this registers with business leaders who are under pressure because they have to deliver something this quarter that demonstrates that the money they’re investing in these technologies is somehow yielding results. And I’m wondering if there’s an incentives issue as well, you know? And you talked earlier about measuring the right things. I guess the question is, how do we affect the culture change necessary for these organizations that are really structured to deliver short-term results in many cases to start thinking a little bit more about the, you know, what quadrant do we want to be in, you know? Are we going southwest or what have you?

Stef: Yes. I think part of it is… And if I had the 100% answer for this, I think I could maybe retire. So I’m not gonna claim to have the be-all, end-all answer. But I think part of it is going back to, you know, what is… Like, where do we want this organization to go?

I hesitate to say vision, because vision gets conflated with, like, a statement that maybe is on your website along with mission or something, and is sometimes rather hollow. But going back to that strategy cascade, what is the winning aspiration? What is that goal that we are working towards?

And I understand that there’s gonna be some major pressures to deliver, to have revenue, but presumably you have some sort of customer at the end of the day. You want the customer to buy your product because it is good, and you’re going to have some particular part of the market that you are playing in. And I’m thinking keeping your eye on that, which is actually in some ways very classical strategy, and how does automation serve that? And maybe it’s farther down in terms of management systems or how are you orienting your metrics. I don’t know. I think it comes back to what is that winning aspiration?

Where do you play? How are you gonna win in that particular… I don’t know. It kind of, I… This is a common theme with my thinking, is like what can we learn from tried and true kind of evergreen frameworks, tools out there to guide this moment? What do we need to recognize as different, and certainly the speed is there.

But I think we’re probably not gonna go too far wrong if you have an idea of where we want to go. Sure, we need to be flexible, but how does automation help us move towards that? And I can talk about a possible example of also how designing for cognitive expansion using AI as a killer feature with NotebookLM, with whatever they’re calling it now at Google Gemini.

As of last week I had to change my book because of that. But I don’t wanna get ahead of myself. I’ll pause there.

Greg: One of the things I am finding in the work that we’re doing is that smaller teams can punch way above their weight now with these tools. But not only smaller teams, smaller organizations can potentially do things that a larger organization used to do, and might be able to do them more effectively and efficiently because there’s more intimacy in a smaller organization.

The people know each other. They can react. Do you have a point of view about that? I mean, is that just me looking for what I want to see, or do you think that there’s actually some truth to the fact that these tools empower… may change the shape of how people do things together?

Stef: No, I think there’s definitely something there. There’s an analog here. It’s from, I think it was one of your blog posts, Jorge, around Flow, the Academy Award-winning animated film, which was done, I think, in the Baltics, Estonia. I don’t know, you can check me in post, I think. But it was basically a very small team putting together this award-winning film, and I do feel that is definitely possible, and I think it’s interesting why is that possible. I think there’s less layers of information transfer. Similarly, if you have a team of eight people and you’re working in an office, there’s probably very little packet loss happening between that team, and you can work very dynamically.

Your all-hands is, like, your lunch, and you’re all on the same page. And of course, as organizations start to grow, you start getting potential for more packet loss and so on and so forth, and I think you have a buildup of tacit knowledge that is harder to automate, and this kind of gets into the productivity paradox.

I think it’s possible, but more work needs to be done to redesign the factory, so to speak, to use the electricity example. So certainly I think if you’re a smaller business or a small team, it’s incredible. I see it even in my own practice. Classically a design researcher’s main deliverable was probably insights and recommendations, and now literally, like earlier today, I’m building in Claude Design primarily right now.

We can debate the merits of that. But you know, I’m making prototypes. I’m going deeper into the making part of it, so certainly it’s enabling smaller teams to do more than before. But I think there’s something worth poking on in terms of what is required for scaling and how much of that is about how information propagates and then what’s flattened in these systems.

Jorge: Yeah, I mean, the more people in the organization, the more complex the social networks are gonna be, and to your point, it speeds up everything when you have fewer people. The quality of the tools has also improved. And, you know, we don’t have to overleverage on the example of that movie, but the fact is, you know, the kind of tools that are used to make a 3D movie now are available relatively easily, and the outcome is super high quality. We’re seeing that obviously with AI as well, that we’re getting the ability to produce results that we can quibble with their quality, but it’s oftentimes much better than what people were making 20 years ago, right? Even in the crappiest version of the AI output. So yeah, but the fact that it enables smaller teams and the quality of the output that those teams can put out is much better and much faster seem like factors in…

Stef: Yes. I wonder too, coming back to our earlier part of the conversation, as AI is an accelerant, when you have that small team that has a really clear vision of where we want to go, or I say vision, it’s really, I think I’m talking about strategy, about that winning aspiration, and how they’re gonna get there. If everyone’s pretty aligned on that, they’re using AI as that accelerant and kind of force multiplier, amplifier, so on and so forth, insert synonym here. And yes, what they can create, how they steer these systems is pretty impressive.

Jorge: And I just wanna add one more thing there because one of the things that is unique about AI as an accelerant is that it not only accelerates the creation of artifacts, like you were saying, it’s like maybe you can do a report now in two minutes or whatever, right? Like, and that’s an artifact that the LLM is writing for you. But one of the things that it also accelerates is it gives small teams the ability to come up to speed with new tools and processes that they might be unfamiliar with much faster, right?

Stef: Yes.

Jorge: The team that made that movie, I mean, I don’t know, I’m gonna make stuff up now because I don’t know the details, but if I wanted to pull together a team now to make a 3D movie, I don’t know how to use Blender.

My team probably doesn’t know how to use Blender. But we have these tools that allow us to come up to speed on something like Blender fairly quickly, right? So as long as you have a sense of the neighborhood that you want to be operating in, you can get there pretty quickly. The flip side of this, and this might be something to talk about, is if I don’t have expertise in the systems that I’m working with, I also don’t know what can go wrong.

Like, I don’t know what to look for, right? So I might be misled by these tools down a path that just goes nowhere.

Stef: Oh my gosh, yes. We might need a second podcast for this. No, I have much to say on this and it’s… So I’m, this is not meant to be a shameless plug, but I’m also a co-curator for Lou Rosenfeld’s upcoming Shift UX conference, and this is something we were just recently talking about in our curator’s group, how we talk a lot about AI blurring the swim lanes between roles for exactly this dynamic that you’re describing, Jorge, in that you can now…

You know, can prototype, and I think this is just audio only, so you couldn’t see my air quotes. But what are you prototyping if you don’t have kind of all the elements of UX layers beneath what you’re… Like, if you didn’t front-load that thinking in what you’re guiding, and what you see is maybe one role that is not design creating a prototype, and then design has to kind of clean up the maybe not so correct prototype after.

And there’s this interesting sort of accountability trade-off that we’re seeing by virtue of there actually being some boundaries. Certainly we can expand more. The floor has been raised, but I think there is still a real place for experts, whether we’re called the same title or not. Again, this comes up a lot in our discussion for prepping for the conference.

Like, will design researcher be a title? Will UX be a title? Will we even call it UX? TBD. Maybe not. But the actual expertise of what we are doing, I think something that is really unique to what we’re currently calling UX practitioners is problem framing and reframing, recognizing when the question itself needs to change.

And I have a pretty… Like, the hill that I die on is that’s something beyond the realm of automation, and I have a whole argument about how that’s related to tacit knowledge and embodiment and situated action, which we may or may not need to go into. But I think there is something around… There are some edges, and you still need to have that human expertise to complement the machine. It comes back to that joint optimization, that handshake between human and machine strength. So yes, while that floor is raised, I think we still need experts to be able to call out when the system is going off the rails, and we don’t have that expertise.

We can’t call out that system or can’t steer it as effectively.

Greg: Yeah, it’s interesting. I mean, discernment I think is a really important thing. You get tacit knowledge, the ability to know when something’s good, is based on experience and expertise, et cetera. And one of the worries I have right now in this moment is that the gap between early career and people who’ve been in the seat for a long time is large, and the apprentice-apprenticeship model is breaking.

Which means that, at some level, early career folk are jumping in and just using the tools and saying, “I don’t need all that stuff because I can learn it by just asking great questions.” And then you have kind of a ways of doing things, which have been accumulated over a period of time by expertise and being in your career long enough to see patterns emerge along the way, that is valuable.

But there may not be the curiosity to understand how to use the tools in a new way to take advantage of it, right? And there’s so these sort of gaps between these two spaces. And one of the things I’m personally concerned about is how do we create an environment in business where we actually are mentoring early career people, and we’re also capturing tacit knowledge at the senior level and finding ways to share that information.

And then there’s a connection to that that’s also culturally difficult because tacit knowledge is earned and therefore part of people’s identity and connected to job stability and a bunch of other issues that they may not want to share. You know, I’ve… Long time ago, I worked at the New York Stock Exchange, and I interviewed all the people who worked there, and there were certain people who did not want to share what they knew because it made them valuable in the organization.

And, you know, if they were hit by a bus, the technology stack would fall apart. And so they had leverage in the organization because of that tacit knowledge. So there’s all these kind of issues that are very sticky around how, as a society, we’re gonna mediate between those who know, those who are learning, those who know who may need to learn new things, those who are learning who can adopt and adopt, as Jorge mentioned, “Hey, I need to learn Blender.

I’m gonna learn it now.” You know, it’s almost like The Matrix. “Tell me how to fly a helicopter.” And that gap between those, we’re not having a conversation about that. The conversation around the people in these systems, it’s this efficiency play versus how do we make our people stronger, better, faster, you know, and at the same time hold the social contract at work where we’re all in it together at some level.

Jorge: I saw a post just yesterday from McGill University saying that the Dunning-Kruger effect might not be real, that it might be like an artifact of the data they used to collect it. I’m like, “Oh, no, just when we need it most.”

Stef: Yes. No, I want to believe that research. There’s… Oh, my gosh, I have many directions. Let me see if I can pull them together into something resembling a linear thread around, kind of around those different consequences of AI.

There’s maybe the discussion around the bottom part of the ladder, the bottom rungs being cut off. And I think this comes back to our early conversation about redesigning the management systems or redesigning the factory. It’s a 1880s electricity productivity paradox story we were talking about earlier.

I saw this, in a good way, with job postings for IBM. I spend a disproportionate amount of time looking at internship posts for my students, and I was impressed that they were hiring some… At least this is my qualitative anecdotal report. I don’t know, maybe there’s some big news report that’ll invalidate me.

But they were hiring for junior-level roles for interns, and they acknowledged that automation with AI shifts doing tasks, going hands-on with the artifact, to more upstream, like monitoring, supervising, orchestrating. That word gets used a lot. And they really… It seemed that they were focusing students, these new junior folks coming in, on things that were maybe hard to automate, like customer-facing conversations where you’re building relationships. Maybe it’s setting up systems. I don’t know what the mentorship structure looked like and… But I thought that was cool that they had rewritten the job description. That’s what I’m… Maybe a concrete example of what does it look like to redesign the factory, and I think that’s exciting, and I think there’s a huge potential for seeing my students.

They’re so curious, and across all my guest lectures, having that curiosity, being able to learn the tools, but also, you know, I see a lot of concern about cognitive offloading, to be honest, in my students. You know, they’re the professional masters, so they’re definitely more senior. But they know they need to be able to do the thing or have the judgment as well as steer the tool. So I don’t know. I’m hopeful, but I think there’s yet another dimension. I said that I… No promises about linearity, but there’s also another aspect of this that we’re asking people to come in out of the gates to be systems thinkers, and I think it asks perhaps an uncomfortable question about how… It’s like a nature versus nurture thing. Like, we can learn causal loop diagramming and system mapping and so on and so forth, and those are definitely tools that we can use. But if you don’t put in the reps, can you effectively steer it? I think that’s kind of an open question.

Is that something about personality, like openness to experience? Is it just like practice, like playing violin? If you play for one day versus 20 years, it’ll be different. And I don’t know, I think about that a lot around nurture versus nature for assistance. I’ll pause there before I throw more dots in this constellation, see if we can weave them together.

But…

Jorge: You used a phrase earlier on that I wrote down because it struck me as a really good summary of what the book is about. You said, good cognitive offloading.

Stef: Yes.

Jorge: And that strikes me as a really good synthesis of the kind of subjects that we’ve been talking about, or at least aspirationally, like where you wanna be at. Are there concrete things that you can point listeners, viewers to do, practices, exercises, ways of thinking that can help them get to a place where they’re doing good cognitive offloading?

Stef: Hmm. Yes. I’m thinking about… Maybe I can give some of the underlying principles and then maybe give a couple of… Well, let me start concretely answering the question. I think first off, trying to do the thing yourself, doing the thinking yourself is maybe the first thing you can do. I’ll start there and then I can get into, of course, I just want to go and jam cognitive science principles into everything.

It’s very on brand for me. But I think, yeah, making that first draft, maybe taking that first pass of your song that you’re composing, doing… Like what are the reference images? I have an interface that my partner and I built at a hackathon and we were looking at a different way to explore music and it was very kind of analog of brainstorming, okay, what direction we would want this to go in.

So I think, yeah, if you can at least put down that bad first draft first and then go to AI to maybe use it as a sparring partner, arguably that is, you know, that’s a well-trodden but I think great way to use these systems. See how it can take a different point of view. How can it help you expand cognition?

How can it help you think about thinking? How can it encourage metacognition? I think those are probably the two that I recommend most often, and I really try and practice myself. I think the more I use AI systems, the more I find myself coming back to more analog things. I think that’s a bigger trend as well, or at least starting analog before the accelerant.

Well, those are all really great from a personal perspective. I’m wondering if there are analog suggestions for teams, right? Because part of what we’re doing here is talking to leaders who are grappling with the moment we’re in, right? Like this moment of change. How can they steer their teams, their organizations toward good cognitive offloading?

The entry point for this, I’ll start with maybe something that is not necessarily where you think I’d go with this, which is building a culture of experimentation by going hands-on with the tools. And maybe, you know, two years ago or a year and a half ago, that looked like feeding in a standard sort of day-to-day workflow and trying to automate parts of it.

Now it’s probably building out more agentic capabilities. But making space in that organization to learn what these tools are good at, so you have a sense of what are your human strengths, not just at an individual level, but maybe at a team level, and how those work together versus machine… How do I say?

Yeah, human strengths versus machine strengths. So I can give a concrete example of what this experimentation looks like, which I think at an organizational scale can enable good cognitive offloading. So Canva had this thing called AI Discovery Week. Last year was their first time. They took a whole week off, where they had everyone from engineers to chefs at the company spend some time learning about core AI concepts, kind of boosting general AI literacy, and then they had a company-wide hackathon where they generated, I don’t know, hundreds of different ideas. And I think that was their goal, building a culture of experimentation. Everyone is familiar with these tools.

And I talk a lot about this in the book and in the paper that sort of eventually grew into the book around this building intuition, intuition building, however you… I think I called building intuition officially. And I think that is really key for developing that muscle, and they did it again. So presumably you had some value from that.

They did it again this past year, and what they were building was more in-depth. It was building out workflows. It was more agentic systems rather than general literacy. But I’d say that’s probably something that organizationally, again, good cognitive offloading, and feeds into the sort of bigger discussion around how do we build the sort of culture and adapt the factory, going back to our productivity paradox electricity example.

Greg: That’s great. I would add one thing that I think is interesting right now, which is do more in less time instead of do less, do the same in less time. And what do I mean by that? There are certain activities in product development, some of which take a lot of time to do, or historically took a lot of time to do, and therefore didn’t get done because of time exigencies.

You know, you didn’t have the time to do it or the staff to do it or the ability to do it. But now you can do them, and you can do them rather quickly. And so you should because they will inform you and allow you to understand the problem space better. I read about this recently.

There’s an example around a company I would help where we evaluated the information architecture of all the competitors, and we did that in like a day. It would’ve taken us probably two to three weeks to do it in the past, and we wouldn’t have done it because it would’ve been perceived as a low-value activity, because it was too hard to do.

Turned out to be insanely useful for us because we understood something that was unique to us, and understood something that was happening in the market by doing that. So this is a new activity. It wasn’t new. It’s something I’ve done before in my career, but one that was very difficult to argue for because you just didn’t have the time or the capability to do it.

Now you can, right? And so I think one of the things that we’re missing the opportunity on is understanding what is the more that we can do as teams? What is the more that we can do with these tools? And I love this idea that think for yourself, partner with AI, but I think the next thing is what more could we do to understand this problem?

And given the fact that these tools are basically reducing the cost to do that, at some level I feel like it’s negligent not to do the more.

Stef: No, I have another example of that as well, like thinking about this comes back to that human versus machine strengths and, as a side note, nerdy backstory. This is the Fitts’s List concept from, for any human factors listeners, but adapted to modern days.

Like, this idea of, say, generative AI. One thing that it’s gonna be way better at than me is going through a mountain of data and quickly finding the patterns. Now, if I’m running a big workshop with a client, instead of taking a week to synthesize, even if it was a few days, if I’m able to do that by the end of the day and give some high-level overviews, and then have a discussion about it while we’re still in the same room, like that’s super powerful, versus doing that the quote-unquote “old-fashioned way” and waiting for that couple days or week lag time.

So I think that’s another great example, and we shouldn’t… Like, what can we do… Looking at that list, like pattern detection, generativity. It’s not gonna be necessarily creativity, but producing a lot of variants of something or stamina, being able to just run something for a really long time in deep research mode or something.

I think we’re… Yeah, that’s probably underexplored.

Jorge: The acceleration basically transforms the… You used the phrase old-fashioned, you know, like the old-fashioned technique. It transforms it into something new, right? Because if it’s close to real-time, batch computing is different from interactive computing.

It’s qualitatively different. But anyway, it sounds like we have so much more that we could talk about, but we are running up on time. I don’t know, is the book available for pre-ordering? Where can folks find the book?

Stef: For sure. I can share a link with you as well, but on the Rosenfeld Media site, if you look for What Your Machines Should Do: The Science and Strategy of Human-Centered Automation, it is coming out this fall, and there is a pre-order link.

Jorge: Well, I don’t wanna pre-commit you, Greg, but I would love to have another conversation with you at some point, maybe after the book launch, because this is, first of all, it’s moving so fast, and it is such an important subject. So good luck with the launch, and thank you for joining us today.

Stef: Thank you, Jorge. Thank you, Greg. It was my pleasure.

What do consultants get paid for?
What jobs can AI actually do? Certainly not the most complex. Luis Garicano’s post focuses on consulting, but his framework applies to many other messy jobs. (The title of his book, which I now want to read.) The upshot: AI may help you do analysis cheaper and faster, but that’ll only move the bottleneck downstream.

Intelligence as architecture
Luc Beaudoin argues that intelligence is best understood as a property of information processing architectures. This may sound abstract, but it has real implications for how we design AI-enhanced systems. For example, consider the role culture and social commitments play in intelligent human behavior. AIs don’t (yet) have that. My read: intelligent behavior is more complex than many people are assuming; building scalable intelligent systems will require structure.

Systems and delays
It’s wonderful to see someone learn about systems from Donella Meadows. This post explores the counterintuitive effects of delays, one of the most impactful insights from systems thinking. I’ve seen this dynamic at play; it can be disconcerting. Remember: AI will accelerate and amplify these behaviors.

What IA is for
I wrote this primarily for practitioners, but the underlying claim is also relevant for you: the ultimate purpose of information architecture is increasing agency by making systems more legible — for you, your employees, your customers, etc. IA isn’t just for the World Wide Web: If your people and AIs aren’t getting the information they need when they need it, they won’t produce good outcomes.

IA is foundational for AI
Patrick Neeman argues for something I’ve said for at least the past two years: information architecture is the foundation for good AI implementations. AI is leading organizations to rediscover the value of IA, at long last. Patrick names me as part of IA’s history, but the AI challenges in this piece are what my practice focuses on now.

What’s really happening to jobs?
A recent report from Stanford concludes that claims about AI impact on the job market may be overstated. There are two ways of understanding AI: as a “normal technology” that will change the world over time (“transformative but gradual”) or as an unprecedented disruption with immediate world-shaking implications. I’m firmly in the former camp: you should invest in AI smartly, with a look to the long term.

Stateless MCP
An important upgrade this week: the Model Context Protocol (MCP) maintainers announced version 2.0 of the spec. These are Simon Willison’s notes on the release, which go deep into tech details. My read for business leaders: this version is more secure and therefore better suited to enterprise applications than agentic coding harnesses (e.g., Codex, Claude Code), at least for operational workflows.

How Swatch saved the Swiss watch industry
A fascinating case study of an industry (apparently) facing technological disruption. Much of what I thought I knew about the near-death and restoration of the Swiss watch industry was wrong. Cheap Japanese quartz watches weren’t the culprit. What almost killed the industry were protectionist policies and bad incentives. What does this have to do with AI? Tech disruption was the coup de grâce that almost flatlined the patient, but there were deeper structural issues. How ready is your industry/organization to compete in our time of technological disruption?

Smart businesses are realizing they don’t have to blow their budgets on frontier AI. At least, that’s the conclusion of a recent article in the Wall Street Journal. Models come in different levels of cleverness, and the “smartest” cost more. But not all tasks require the highest level of intelligence. A mix of models will give you the biggest bang for your buck, at the expense of upfront architecture.

The Journal cited an excellent example:

Cursor recently ran an experiment to evaluate the cost of building a web browser from scratch. Doing the entire task on OpenAI’s GPT-5.5 cost a little more than $10,000. Using Cursor’s Composer coding model in combination with Anthropic’s Opus 4.8, cost $1,339.

The focus here is cost, but that’s not the only concern. The most intelligent models — those offered by frontier labs such as Anthropic and OpenAI — are closed and proprietary. That has implications for your business. For one, you risk becoming dependent on others for critical cognitive tasks — a strategic and privacy risk. If a provider can turn off the intelligence spigot (or, more likely, raise its price) you’re stuck.

The solution is breaking down jobs into tasks that can be done by a variety of models. Some tasks, such as planning, will require more powerful models. But many others can rely on cheaper, less powerful models. Open weight models are becoming a commodity: not only are they cheaper but also mostly interchangeable. Not happy with how a model is performing at a particular task? Switch it out.

This is how business has been organized forever. Some jobs require greater expertise and capabilities than others. For example, neurosurgery can only be successfully done by a very small number of people who have the necessary intelligence, training, and experience. On the other hand, cleaning the operating room can be done by someone with less training, expertise, and smarts. Hence, neurosurgeons earn more.

How do you determine the right mix of intelligences? Think of your AI operations as an “intelligence stack.” At the highest level, you have the vision and strategy for the system. At the bottom, you have particular one-off tasks that carry out system functions. In-between there are workflows with various degrees of complexity. All three levels call for different kinds of intelligence.

The goal is to bootstrap this stack by using expensive frontier models to design and implement the system and more commoditized models to operate it.

When developing the vision and strategy for the system, you’ll research competitors, build scenarios, and explore possibilities. These are open-ended tasks with uncertain outcomes; it’s hard to specify what good outcomes look like in advance. For this kind of job, you want the cleverest sparring partner you can buy.

The system you design to implement that strategy will include lots of frequent tasks that will likely require less clever models. For example, imagine your business responds to lots of RFPs. Each RFP must be triaged. Some will be a good fit, others irrelevant. Those that fit must be routed to the right people in the org, hopefully with draft suggestions on how to respond.

You could prompt a frontier model to analyze each RFP as it comes in, perhaps using a ChatGPT project or Claude Cowork space for context. But that would be costly. A better approach is to break down the process into discrete steps and assign particular tasks to models with lower capabilities. If you know what “good” looks like for each step (which is much easier to do at this level), you can architect model interactions for optimal performance in each step — much as you would when delegating tasks in the real-world.

This approach doesn’t just reduce costs, it also makes the system work more predictably. Without upfront architecture, frontier models must parse each task from scratch, leading to variance over time. A more structured approach can be tuned for the exact range of outcomes needed for each step in the process.

Because the system is modular, you can use different models at each step in the process. Higher-level tasks that require orchestration can use more expensive closed models, whereas granular tasks with predictable outcomes can use less expensive (or even free) models called from deterministic programs. You can also switch providers at various steps in the process, preserving optionality.

Using a mix of models also lets you adjust for latency. Some tasks require faster reactions than others. Smaller, less clever, models can often have lower latency than frontier models. And of course, you can also be more selective about what information leaves your network: open weight models running in your infrastructure preserve your privacy.

Sounds ideal, right? You use expensive (and proprietary) models sparingly to design systems that use cheaper, open models for the day-to-day. What’s the catch? It’s the same one we had before AI: you must define what “good” looks like beforehand and architect the system to deliver expectable results.

That’s not bad, as far as catches go. Thinking through your workflows will force strategic decisions. It makes more sense to automate some workflows than others, and some will be more critical to the business than others. Mapping and architecting the flows will let you focus on what matters.

Yes, architecture can be expensive and time-consuming. But frontier models make it faster and less expensive. (Thats part of the top layer of the stack.) They also allow us to make richer prototypes faster than before, reducing the risk of over-specifying complex systems upfront.

This modular approach can scale and improve as new models come in the market. Today’s frontier models will be tomorrow’s entry-level. When that happens, you’ll want to reconsider the mix. Whether you can will depend on how you structure your systems today.

Unfinishe helps growth-minded leaders automate drudgery so their people can do human work. What's your bête noire? Reply or comment. 👇

On those new Chinese open weight models
All this talk about optimal model use is spurred by a couple of new Chinese open weight model releases that aren’t far behind the leading (closed) U.S. models. Ben Thompson offers a counterpoint to the item above: he expects the frontier U.S. labs to also dominate the tier below. Either way, I’m leaning toward designing model-agnostic systems.

DoorDash’s AI code reviewer
Details on DoorDash’s AI code evaluator system. Different stages, using different models. That is, a system architected to make optimal use of models with different characteristics (and importantly, costs.) Key line: “‘best’ is meaningless until you say best at what, on which cases, at what cost.” (H/t Benedict Evans)

AI helping (not replacing) workers?
A new Google study reinforces a trend we’ve followed for some time: AI isn’t really replacing workers, but augmenting them. Most people aren’t fully delegating their jobs to AI. Instead, they’re using it as a collaborator that leverages their expertise. Google has a horse in this race, so solve for the balance — but I believe AI job replacement fears are overstated. (WSJ gift link)

Claude Cookbook
I used to love O’Reilly’s “cookbook” series of books, which showed how to solve practical problems with particular technologies. This is like that, but for Claude. A tremendous resource that suggests your organization could benefit from becoming more AI literate. (We can help with that!)

Replacing software with AI
Starbucks spends $400m/year on software, so they’re using AI to roll their own. The losers: Microsoft, IBM, Oracle. The resulting software might be better-suited to Starbucks’s needs, but they must also tally maintenance and support. More importantly, in-house software needs to be designed and stewarded. Who’ll do that? (Hint: not AI.)

The Reverse Information Paradox
On X, Satya Nadella called for organizations to control their learning mechanisms, which are at risk of being ceded to AI labs. By delegating to the labs, you also pay twice: in cash and in knowledge you hand over to make models work. My sense is that if given a choice between outsourcing all your intelligence to frontier labs and building your own using less-powerful models, the latter gives you greater control. It also requires more forethought and structure — but you own your intelligence.

Information work as responsibility work
Tim O’Reilly, after chatting with Claude: “[AI] has no agency of its own. Humans set it in motion, evaluate its output, and should be held responsible for what it does.” I.e., you can outsource cognition, but not responsibility. What can you do about it? Invest in more upfront architecture, not more compute.

The new ChatGPT superapp
OpenAI launched a new all-in ChatGPT app. I haven’t used it, but it sounds bad. Gruber doesn’t explicitly call it an information architecture problem, but I will. It’s predictable: companies tend to ship their org charts, and OpenAI’s is a mess. AI doesn’t alleviate the need for IA, it amplifies it. New concepts need relatable labels, metaphors, and hierarchies. Who owns the structures your intelligence depends on?

A Script for Mark Zuckerberg
What will Mark Zuckerberg say in Meta’s next earnings call? Ben Thompson has suggestions. This isn’t just satire: the post spells out the connection between capex and strategic vision more clearly than Meta itself has. Can your board do the same?

The Control Layer
A pitch for Otari, Mozilla’s open source LLM control layer. It makes a point I’ve reiterated over the last few months: we’re past the initial point of experimentation with AI. Production requires infrastructure architected to deliver value.

After Forty Years, Still No Silver Bullet
In 1986, Fred Brooks argued there are no tech shortcuts to making software that’s radically easier, simpler, or more reliable. Many people think AI is the ultimate silver bullet. They’re wrong.

Kinetic Coffee Panel
The Kinetic Council invited me to join a panel with Abby Covert, Jessica Talisman, and Larry Swanson about the evolving role of information architecture. My position: practitioners focused for far too long on how to make information easier to find and use at the expense of what to do about it and why that matters. But AI forces organizations to think more strategically about how information is structured.

Forty years ago, computer scientist Fred Brooks published a paper called No Silver Bullet: Essence and Accident in Software Engineering. As its title implies, the paper argues there are no technological shortcuts to making software radically easier, simpler, or more reliable. You may think AI is the ultimate silver bullet. It isn’t.

Moore’s law was in full force in 1986. Hardware was getting more powerful, faster, and cheaper. Surely, some technology would come along to do the same for software. Brooks argued this wasn’t in the cards, since software is fundamentally different from hardware. For one thing, it’s of a different order:

The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract, in that the conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed.

Specifying, designing, and testing this construct is difficult. The challenge isn’t implementation but design: “We still make syntax errors, to be sure; but they are fuzz compared to the conceptual errors in most systems.”

Technical advances usually make development easier. No Silver Bullet traces the history of time sharing, unified programming environments, and high-level languages. Object-oriented programming was a promising new technology at the time and there were even rudimentary AIs in the form of expert systems. Brooks examines them and concludes they’re not enough.

Why? Because coding isn’t the hardest part of making software. Instead, the hard part is knowing what to build:

The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is so difficult as establishing the detailed requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.

What will the system do? How will it serve strategic objectives? How will it enable better judgment and allow people to derive meaning from data? These aren’t implementation questions, they’re design questions. Somebody must define the “construct of interlocking concepts” that define the system, aiming for good fit between the system and the context it serves. LLMs can help, but they can’t replace human understanding and judgment, at least not yet.

Brooks calls out four inherent properties of modern software systems:

• Complexity: Software systems are among the most complex human constructs. They’ve only gotten more so as computers and operating systems have grown more powerful and capable.

• Conformity: Software solutions must conform to the goals, needs, constraints, and interfaces of the organizations that bring them forth. This is true whether it’s bought off-the-shelf or developed bespoke.

• Changeability: Anything that lasts does so because it’s able to adapt to changing conditions. Software is inherently more malleable than other complex designed systems, such as buildings.

• Invisibility: Whereas complex physical systems (again, think of buildings) can be represented with mechanical drawings, software specs are inherently abstract. This makes them hard to design.

There’s been progress in the last four decades, but these properties remain fixed. LLMs haven’t changed that. Non-deterministic components add immense complexity and unpredictability to software systems. The ease, speed, and volume of code generation make software more malleable and opaque than ever. And LLMs promise to ease bespoke development, tempting orgs away from one-size-fits-all solutions.

Which is to say, LLMs haven’t changed the nature of software. Instead, they’ve made it more so. So far, the technology’s killer application is developing software: teams can now produce more software, faster. (It’s unclear yet whether it’ll ultimately be cheaper, especially when you consider maintenance costs.)

What LLMs haven’t done yet is replace software wholesale, at least not for tasks that require predictable behavior. And as their true costs and constraints become evident, it’s increasingly doubtful they will. Instead, LLMs will likely become part of systems that include traditional deterministic components — both inside the systems and as part of the development process.

The resulting systems will be more complex, malleable, and abstract than prior ones. They’ll also be better fit to purpose than off-the-shelf solutions. But that requires design, which remains primarily a human challenge. And it’s hard:

it is really impossible for clients, even those working with software engineers, to specify completely, precisely, and correctly the exact requirements of a modern software product before having built and tried some versions of the product they are specifying.

Replace “software engineers” with LLMs, and this sentence still stands. But it also hints at where LLMs come closest to being a silver bullet: in their ability to spin up rapid prototypes. Good software is grown, not built. That is, it evolves from an initial core to a more complex system through an organic approach that respects Gall’s law:

The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be accurately specified in advance, and too complex to be built faultlessly, then we must take a radically different approach.

Let us turn to nature and study complexity in living things, instead of just the dead works of man. Here we find constructs whose complexities thrill us with awe. The brain alone is intricate beyond mapping, powerful beyond imitation, rich in diversity, self-protecting, and self-renewing. The secret is that it is grown, not built.

So it must be with our software systems.

What was true then is true now: technology moves the bottleneck from production to orientation. LLMs make coding easier, much like high-level languages, IDEs, and compilers did in the past. But without shared models, structured context, feedback loops, governance, and clear interfaces, they won’t provide the results leaders expect.

As always, how to build gets easier — knowing what to build doesn’t. AI can help with that too — but it needs steering. The question isn’t “Which systems can we replace with AI?” Rather, it’s “How can AI help us grow systems that better fit our needs?” The answer will consider AI as a system component and a production tool. But forty years on, there’s still no silver bullet — just better ways to find good fit, faster.

AI’s effect on middle management
A small study of how consultancies are using AI concluded that both juniors and execs are getting the most value from it — at the expense of middle managers, who now have to verify that the output is good. Subsidiarity is a key design feature of effective complex adaptive systems. Your org is one of them: your managers hold local context. Think twice before overloading (or worse, firing) them. (HBR gift link)

Gen AI and Taxonomies
Large language models work best given the right context — including the ability for agents to find the stuff they need. “Old school” taxonomies help a lot. And conversely, LLMs can help us create better taxonomies. Heather Hedden covering what feels like an essential — and for the most part, underappreciated — idea: that smart systems must be propped up by a lot of structure.

How AI is changing marketing
McKinsey claims over half of consumers are using AI to make purchasing decisions. But that’s only part of it: AI is fundamentally changing how marketing is done. But it won’t happen with ‘bolt on’ solutions. Instead, reinventing marketing calls for a complete redesign of how organizations market their wares. I’m biased, but several of the “new” roles described in here read like “information architect” to me.

Traction Heroes Ep. 39: Pace Layers
AI makes prototyping easy. But moving from prototype to production requires lots of architecture. In the latest episode of our podcast, Harry Max and I nerd out on one of the models that has most influenced my work: pace layers. If you think design is about making screens frictionless, easier to use, or (for Pete’s sake!) more attractive, you’re focused on the wrong layer.

CTO Report on cognitive debt
This feels ai-written, but I still found it useful: an informal report from a gathering of CTOs on what they’re seeing on the field. TL;DR: the AI free ride (i.e., unlimited budgets) is over; it’s time to invest in architecture. If your leadership is still suggesting replacing complex workflows with AI, send them this one.

Building Reliable Agentic AI Systems
An in-depth case study on how Bayer built a reliable AI research assistant for their (highly regulated) business. They turned decades of latent data in unstructured PDFs into highly structured context, implemented specialized agents to use that data, and created mechanisms to anticipate and correct the inevitable failures. Bottom line: it’s not enough to give LLMs access to your data; reliability requires architecture.

Big-time self-disruption
Tide is P&G’s biggest brand. Every thirty years or so, they risk it by introducing a new form factor (e.g., pods.) This cycle is about to start again with new detergent “tiles” — a bold risk. How far is your organization willing to go to disrupt itself? You likely don’t have a $2b annual R&D budget, like P&G does. They also have structures in place (including decades-long brand equity) that allow them to make such bets. The challenge for the rest of us? AI makes innovative moonshots more feasible, but bold bets without support structures and solid market signals aren’t reinventions, they’re expensive gambles.

The Birth of the Flywheel
If you and I have spoken at length, there’s a good chance we ended up discussing the Walt Disney Company. I’m a big fan, and have written about what we can learn from them when designing complex information environments and beyond. But this four-plus hour episode of the Acquired podcast goes much deeper, diving into the company’s history up to the early 1980s. The focus? Disney’s (accidental?) discovery of the synergistic business model they exemplify. It’s a history lesson on the successful merging of art, commerce, and technology. Takeaways: 1) build a diversified yet cohesive business model that creates self-reinforcing loops and 2) honor the Walts (i.e., creative geniuses) in your team willing to bet the farm — so long as you have Roys (financial geniuses) keeping the company alive. (The business model, you can design for. The geniuses… not so much.)

See you next week!

— Jorge

In systems thinking, this principle was best articulated by John Gall:

A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.

I’ve long promoted Gall’s law to students and clients. It’s been hard going. We want to see products and services in their full glory ASAP. But you can only throw so much cash and person-hours at a problem. Ergo, we got the time-tested idea of a minimum viable product. (It’s no coincidence that orgs with more resources violate Gall’s law more often than scrappy startups.)

But the value of an MVP isn’t just that it allows you to get something that works quickly and cheaply. Instead, the value is that that first try isn’t overspecified to theoretical conditions. It’s only a draft meant to kick off an evolutionary process that leads to a system that meets real-world customer needs.

AI removes these constraints. An afternoon with Claude can yield a comprehensive spec for a very complex system. Further sessions can architect the system and build an initial release that includes bells, whistles, timpani, harps, violas, and all the other instruments in the orchestra. All this, at a fraction of the cost and time it would’ve taken in the past.

That’s amazing. It means we can now design and build much larger systems, faster. This opens new possibilities. Not just one new feature, a new product. Not just a new product, a suite. Not just a suite, a platform. The possibilities seem endless.

But in removing architecture and development constraints, we’re also removing the need to focus on what matters most and the discipline to run it by the market. All that upfront complexity doesn’t necessarily address real-world needs. Instead, it reflects a singular top-down vision that may or may not provide value to others.

Whereas MVPs in the “before times” called for minimal investment before validation, LLMs promise a much more realized vision from the first go. But there’s a significant difference between a sexy concept car and a vehicle customers will take grocery shopping. AI-augmented workflows will give you the former, but not the latter.

Here’s an example. Two years ago, I started building a product called SiteRanger, an AI-powered agent to help small teams manage large websites. This was before Claude Code or any of the current coding agents. Still, I got surprisingly far by using Claude to augment my basic PHP skills.

Together, we built a functional MVP that implemented what I considered to be the core functionality. The problem: my “core” was in fact an open-ended platform. Rather than solve a particular customer problem, it was designed to solve classes of problems.

By the time I got alpha users on board, the system was wildly over-architected. Worse, I learned new agentic systems could provide ~80% of its value. When I looked to pivot, I realized I’d have to move in a completely different direction, one with entrenched incumbents. It wasn’t worth it.

It wasn’t all a loss. This experiment taught me a lot about developing AI-powered software products using AI. But the most important lesson I learned is that AI makes it VERY easy for individuals and small teams to land in the same trap as resource-rich orgs: no constraints.

Which isn’t to say you shouldn’t use AI. To the contrary, I’m all for accelerating MVP design and development. But the word “viable” is fungible, especially when you have robot engineers. You want to expose the product to the discipline of the market. That means releasing something embarrassingly simple at first. And that requires discipline and constraint — the two things most scarce when working with LLMs.

My friend Karl Fast pointed me to a wonderful scene in the movie A RIVER RUNS THROUGH IT. The main character, a child, brings an essay to his dad, a strict preacher, for evaluation. The dad’s only reply: “Half as long.” The child does, and returns with the edit. After scribbling with a red pencil, the preacher looks at him and says: “Again, half as long.”

Step away from the console and ask yourself: What would I cut if there were no AI building it? Cut, cut, cut. Then imagine Tom Skerritt staring at you over his schoolmaster glasses and saying drily, “Again, half as long.”