Unfinishe Conversations: What Your Machines Should Do

A conversation how organizations might use automation more strategically.

Stef Hutka, PhD has an upcoming book called What Your Machines Should Do. It’s about how organizations can use automation more intentionally — that is, to support their strategic agendas.

We read a preprint and were pleased to discuss it with Stef in the first of our Unfinishe_ Conversations, a new series about how leaders can successfully navigate the current moment.

One of the main takeaways: AI is an accelerant. It’ll get things moving faster. But that doesn’t mean they’ll move in the right direction. And that, of course, is the key.

---

---

Transcript

(AI generated — likely contains errors.)

Greg: Well, hello and welcome. Greg, you and I have done several live stream conversations, and we decided to do something a little different today, and we have Stef Hutka here. Stef, is that how you pronounce your surname? I’ve always called you Stef.

Stef: Yeah, no, that’s great. Yes, I think more than the North American version, hutke, the Euro version hutke, I’ve really… I’m happy with either, so you did great.

Jorge: Well, welcome. You are our guest in the very first of what will hopefully be a series of conversations about a whole range of subjects, but usually hovering around this moment of change that we’re going through, right? And I don’t know, Greg, if you have any words of welcome for Steph. This is the first time y’all meet, right?

Greg: Yeah, it is.

Stef: Exactly.

Greg: It’s so interesting to, like, have read something before you meet… Most of the people in the UX community, I know them before the book comes out, and I read the book, and I’m like, “Oh, that’s you.” You know, like, you can connect the dots between the two. But this is the first time I’ve actually read sort of a book from our community where I actually haven’t known the author.

That just means that we haven’t bumped into each other along the way. And, yeah, I’m excited to get into the conversation today because I think you posit a bunch of really thoughtful and maybe, hopefully optimistic points of view about where we could be. And, you know, Jorge and I have been on this conversation for a bit around, uh, we’re not, uh, you know, Reid Hoffman has this two-by-two where he has, you know, zoomers and doomers and gloomers and bloomers, right?

And, um, I’m not a zoomer, but I’m not a doomer, you know, and I’m not a gloomer either. So I guess I’m in the bloomer category, which is I believe we have agency in creating a future that’s viable for us. And part of the thing that’s interesting about your book is I think there’s a story or a thread about that in it and how to do it.

So I’m very excited about the conversation we get to have today.

Jorge: I’m very excited as well. I really enjoyed the book. And it’s like you have a book, right? So we’re gonna get into the book, but before we do, would you mind giving folks tuning in a bit of an overview of your career, who you are, and what brought you to write this book?

Stef: For sure. So I wear a couple of different hats. I’ll start with the present. So the first hat is as a researcher. So I’m the founder and head of design research at a boutique design research consultancy called Sendful. And so anyone who does any action sports, especially anything on trails, there’s the term full send, where you’re going to do some sort of action.

You’re fully committed. It might look pretty wild from the outside, but you actually need to have a lot of experience and expertise to kind of get to that point. But there’s… You know, you’re fully on it, and I love that idea of that sort of energy, that forward motion, that expertise behind it, and wanted to kind of play on words.

So that’s where Sendful comes from. I always clarify kind of where that name comes from. Then the second hat is as an educator. I teach UX courses at UC Berkeley, Introduction to User Experience Design and two new courses that I’ve designed and taught, one on systems thinking, Designing Future Systems, and AI, which was most recent.

And that wrapped up a couple months ago, and that could be fun to get into. I teach primarily professional master’s students, so designers, product managers, design researchers, and that’s a really interesting space to be in. And particularly Intro to UX Design, I was… I started teaching that in 2022. And, you know, November 2022, of course, ChatGPT came out, and everything changed.

Every year I’m like, “Oh, it’ll be easy. I’ll just be able to reuse the course content.” But what is intro to UX is a very loaded question, and so there’s evolution each year. And then the third hat is as author. So of course we’ll chat more about my forthcoming book with Rosenfeld Media, What Your Machine Should Do: The Science and Strategy of Human-Centered Automation. On any given day, I’m wearing typically two or more hats. And where I came from, so I mentioned that first researcher hat, that’s kind of the one keyword that remains constant across my rather nonlinear journey. So I started off way back when, finishing my PhD in cognitive neuroscience at the University of Toronto.

I’m born and raised in Toronto, and I was studying how the brain processes sound. I’m a lifelong musician. Played piano since I was four, violin since I was… and I think we’ll come back to that around the theme of embodiment, which is very prominent in the book. So I was a researcher in a very academic capacity, was at this crossroads of doing a postdoctoral fellowship in Helsinki, Finland, or join an augmented reality startup in Los Angeles. Two very different potential futures, and you can probably guess which path I chose. I worked on sensor validation, brain-computer interfaces, and working with designers writing up patents was kind of my foray into UX and learning more about what designers were doing every day, and they had questions about designing spatial interfaces. What should lists and menus look like when they’re volumetrically represented? And I helped them out with what I now know as a usability test, and that was the first of a long and storied career in design research. And I moved on to some bigger companies, Adobe, Meta Reality Labs, and then about three years ago started Sendful.

So that’s my not-so-brief history of how I got to where I am today.

Jorge: You mentioned that the book is forthcoming, and both Greg and I have read a preprint, so it might be premature to be sharing something. It’s like, “Well, this is a really great book,” and then you’re like, “Well, can’t buy it yet.” Well, by the time a lot of people watch this video, it might be out, right? I’m curious, you know, given your background, why this book and why now?

Stef: Yes. So why this book is… There’s a longer timescale answer and a more immediate timescale answer. So given the history I just shared with my background in cognitive neuroscience, I think coming into the world of UX, I’ve always been fascinated about what part of our brain and behavior are we delegating to technology, what part is innately human.

And even at those early days at this augmented reality startup at my first gig out of grad school, we were building helmets with augmented reality displays, so we had to work overlays. At one point we were discussing building this sort of similar to an Iron Man suit, and there’s always this idea of technology kind of extending capabilities very early on.

And I just sort of took it as a given, but really dove into that from the book. So I think that’s the long-timescale answer of the brain-behavior connection. I think more immediately, I’ve been working on these augmented reality, virtual reality systems for a good part of my career working in industry and leading design research on these types of wearables. And of course, when generative AI became mainstream in November 2022, it seemed like we had another big moment. It was another, like, mother of all demos, 1968. Like, whoa, human-computer interaction is changing. This is a new chapter. And as I started to use these systems more hands-on, I was really taken at what I’ve coined 10X cognitive offloading in the book, and that actually came from a paper I wrote in 2024 called Designing AI to Think With Us, Not For Us.

And seeing these systems, generative AI systems, it was like, all right, we used to… You know, we use calculators. I can no longer calculate the tip for a dinner because I’ve outsourced that. I outsource my cognition to Google Maps for navigation, you know, for relatively discrete tasks, and I’m generally okay with these trade-offs.

But seeing the potential and how fast these systems were developing, like, whoa, this idea of outsourcing our thinking has a real potential that seemed pretty unprecedented, and then, you know, the pace of development, the breadth of use cases. That was really, I think, fascinating to me because it was kind of this primordial ooze of this could go one of two ways. One is, you know, cognitive offloading is fairly neutral. We are doing a thing. We are offloading cognition. Can that be good offloading? Maybe we’re offloading tasks that were really boring or undesirable, kind of freeing up bandwidth, if you will. We can get into the nuts and bolts of how technically accurate that is.

But freeing up bandwidth to do other, maybe more meaningful tasks, or maybe be able to extend our cognition, create new connections. Or are we offloading things that are really core to our craft, to our identity, and that through bad offloading is another potential future. It really struck me that this is an urgent question, that we’re at this 10X cognitive offloading precipice. And just like the cognitive offloading research I’m sort of synthesizing here, there’s a lot of prior art on this, and similarly, we’ve been designing automated and autonomous systems for a long time. There are planes flying above us right now with autopilot systems in them, and we’re generally comfortable with that, and it’s quite a safe system.

So it occurred to me, what if we made learnings from human-computer interaction, cognitive science, human factors digestible to people who are building this technology to be able to address this urgent question and hopefully steer us towards a more desirable future where we’re moving towards more of that cognitive expansion, cognitive, good cognitive offloading.

I’ll pause there. That was a long monologue. There’s a business and strategy connection there too, but…

Greg: Yeah, I would love to pick up on the cognitive offloading piece too, because I think it’s a fascinating thing that we’re observing. I think there are a bunch of different factors at play here right now. I mean, one of the things these tools let you do is generate very dense and accurate documents if you’re working with them well.

Then if you sort and send them to your colleagues at work, you’re also the recipient of dense and well-formatted documents to review. And I think one of the things that I, just out of my own personal experience, is that there is a moment in your day when you work with these systems where you’re exhausted because our brains only have so much capacity to have cognition during the day in a way that makes sense to us, and we’re filling those moments with very rich content.

And, you know, I think that I’m not sure everyone’s gonna be willing to do that, right? It’s like heavy lifting. It can be heavy lifting. And I think one of the things that we’re also starting to learn right now are what are the boundaries of that for our own selves? Like, what piece of our work product do we offload to an agent or a tool to do for us?

But then when we receive the content back, how do we participate with that in a way where we’re landing in the right way? I don’t exactly know how I feel about it, but it just feels like it’s an emerging trend. And, you know, you have people talking about what they call AI brain fry right now.

As a psychologist, do you have a perspective on that right now? Like, from the work that you’ve been doing and maybe even part of what you’re trying to tell us about in your book.

Stef: Sure. So thinking I know exactly the output that you’re talking about, the three pages, the overly detailed report, and I think there’s an irony in that we’re getting maybe an AI system to generate this report, and then the other person, the recipient, is using an AI system to interpret that report, and they’re getting a similarly long report. And I think there is a risk in that, depending how much we’re actually engaging with the initial creation or the interpretation of what is getting lost in terms of our tacit knowledge, what is getting lost in sort of the surprisingness of when we’re having this dialogue. I don’t… You know, we had talked about some questions that might be asked ahead of time, but there’ll be some interesting spark, and we’re gonna go and follow that, and that’s inherently different than these, what I like to call powerful regression to the mean machines.

So I question if we’re just getting kind of a flattening by doing that. We’re getting a lot of well-formatted… Well, you could argue maybe not so well-formatted, but at least the headings line up, of a flattened content. And what does that sort of flattening spiral look like?

So I think if you’re doing that very intentionally, like I really try and be really strict with my output in that I want it to be in bullet points. I want it to be like a one-pager. And really, there’s a lot of curation that goes into before sending it. Like I’m co-chairing a conference, and if I send a one-pager to my co-chair, I really want to make sure that I’m infusing as much of that sort of stuff and surprise, if you will, into the document so we’re not getting that sort of packet loss of just AI systems reading other AI systems.

Jorge: It feels like we’re living through a moment where we’re trying to use these new tools to automate. And I want to circle back to the word automating because it’s very important to the book, right? But we’re using it to automate legacy workflows. And one of the stories that you have in the book, I think it’s about the introduction of electricity, right?

And when the early days of electricity and how people could tell that this was a new source of energy that would change industry, but it wasn’t really until factories were reconceived around electricity that the productivity gains became evident. Is there, like, an analog to that in the current space? And I’m thinking of things like sending these big reports. It’s like, do we need that?

Stef: Yes. Yeah, I think the equivalent to that, like what we need to be doing, like redesigning the workflows, like where we’re redesigning information structures and information flows. But I think we’re really… Okay, we had some information exchange. It might have actually been more rich in some ways.

Maybe it was a little bit more chaotic, and now we’re sending this nice report. Like, maybe to give an example from research, one of the classic design research deliverables is some sort of form of report. Where the richness often lies is actually the informal debrief after the user interview, and we’re going back and forth informally, and that’s really where those nuggets come in.

It’s not from necessarily the polished report. Sure, there’s a time and a place for that type of format. And I’m thinking we’re looking at this like the new electricity, but is it even that? And are we using it in the right context? And then I think, okay, if we are going to replace it with these polished reports, okay, where is that other sort of offline conversation happening where we, if we divorce ourselves from where that meaning-making, that sense-making is occurring. I don’t know. The long report example, I don’t know if is necessarily the best, as I’m talking through it, the best of the productivity paradox examples. I think maybe it’s like bolting on, you know, and like we saw this maybe a year ago, bolting on an AI chatbot to an existing system where it didn’t actually add any sort of value to it. It’s like, okay, we need to maybe redesign the entire experience and see where AI fits. Maybe that’s something that’s maybe generic but closer to that.

Yeah, I mean, so I want to build on both of your comments there because I think one of the things that I find missing in the conversation right now and one of the things I liked about the frameworks that you’re introducing in your book is that I think there’s a general lack of imagination.

Greg: It goes back to the electricity construct, too. A lot of AI efforts right now are looking at existing processes and looking for incremental automation gains.

Stef: Hmm.

Greg: They don’t think of them as being incremental, but they ultimately are because they’re just taking the Rube Goldberg machine inside of the enterprise and adding capabilities to deliver the same outcome without people or with more efficiency.

And while that’s useful, it misses one key point, which in my mind is people aren’t talking enough about, which is these tools are becoming less and less expensive to build applications and software and components that allow you to do things. So why aren’t we talking about in organizations what could we do?

It’s not what should we do, it’s we’re not talking enough about what could we do. And the debate isn’t really about how might we intentionally create a new future for the business, you know? What capabilities could we now empower us to do new things? It’s really about how do we do the things that we do right now, you know, faster?

And it feels like we miss something in that translation, and I don’t know if you have a point of view on that, but…

Stef: I love that you brought that up, Greg. I was listening to your previous podcast earlier today in preparation for our conversation. I’ll paraphrase this, please poke holes, but I think you were describing how there’s a missed opportunity to have a team of people who are just exploring what are new workflows, what are new ways to use this technology, and there’s this…

I think we’re… This comes back to a key theme in the book, this over-pivoting, this over-focus on efficiency and not necessarily effectiveness, or in our case of what we’re talking about here, possibility and creativity, and I think that that is a big missed opportunity. I think there’s something going back to maybe the productivity paradox example. I think we’re over-focusing on sort of, like to reference sociotechnical systems, where you have social subsystems, the people, the relationships between them, the technical subsystems, the tech, the processes, and there’s, like, okay, what… We’re going to build the heck out of the technical subsystems. We’re gonna ignore the social subsystems and not really talk about what does that handshake in the middle look like for joint optimization. And even if we do that correctly, there’s what we know, how people work today, but then there’s this whole sort of potential futures piece, and by over-pivoting on the now, we’re missing out on those longer term horizons.

So I don’t know. I think there is a huge opportunity there to build towards different potential futures and think what does a new, like, I don’t know, future of work… Future of work is a well-trodden phrase, but maybe taking even a strategic foresight view on that, I think there’s real value there.

Greg: Yeah, and I mean, I think one of the things I also really enjoyed in your book is this idea of your automation quadrant, a two-by-two that you’ve created. I think one of the things that Jorge and I have been investigating when working with clients is how do we help them become practically capable?

How do we help them discern what, you know, the things they should work on and why? Could you talk a little bit about what you’re trying to posit with the framework that you introduced and maybe an example of how you developed it?

Stef: For sure. So I’ll give a bit of context on the framework depending when this is coming out. So we’re talking about a two-by-two matrix called the autonomy decision matrix, and I very specifically called it autonomy and not automation matrix because that is really what we are deciding. How are we going to allocate autonomy between humans and machines, in our case, AI systems, so that it’s the two axes. You have could you automate on the X axis?

Should you automate on the Y axis? And you have different confidence levels. How confident are you that you could or that you should? Kind of goes to the classic phrase, “Just because you could doesn’t mean you should.” Could is generally tied up with technology capabilities. What can this ML system actually do, as well as our organizational readiness to build it, to use it. Then on the should, it gets more to value, not just to the user, but to the broader ecosystem. And what do I mean by that? I can go back to my earlier roots in augmented reality systems. If you put a $10,000 machine on someone’s head, no surprise it’ll make them more effective, more efficient, safer.

There might even be greater customer satisfaction. But unless you also digitize all the 3D models that are served up, as… Unless you also integrate that with the ERP system, it’s just going to be sitting on a shelf. So how do you integrate the solution, and can you integrate that solution? Does it overall uplift the value of the broader ecosystem into which it’s fitting?

So two axes, and then depending on, so what is going into this two-by-two? So the input for this is an automation solution. So meeting people where they’re at is realistically, you’re probably talking about some sort of a technology solution, something you’re gonna build, something you’re gonna adopt.

So let’s just take that as the reality, put it on a sticky note and use that as a starting point. And so depending where you are on this quadrant, if you’re, you know, pretty confident that, hey, we can… This can actually work today, this technology is capable, we have the right folks to kind of pull this off, but we actually don’t have any evidence that this is actually gonna be valuable, the idea is maybe you’re gonna put that in the quadrant that’s on the bottom right.

That’s like, question what this is. Let’s have a conversation about it. Or maybe another context you could use this in, “Hey, we want—we think this is a killer use case and the capabilities are there. We really feel we should automate this, or we feel like this is the direction we should be going.” Now we can work backwards from that. And I talk about this analogy from mountaineering, climb high, sleep low, where that is the goal, that is the big ambition. Now we can use this matrix as a common language that ICs, that leadership can use, ideally to be saying the same things, to be looking at these same quadrants together and having a conversation around a shared artifact.

So that’s another use case. There’s also a very sort of pragmatic use case that was one of the earliest forms of how I developed this, is if you have a team and you have 10 features of things that you could build, let’s put them in here and prioritize and see where they line up. And if you’re in the, “Well, we’re not quite there yet with the capabilities, but this would be super powerful,” so let’s start with assist.

There’s more human in the loop, and as you move from assist to automate, there’s increasingly more machine, less human, and you can design what that handoff looks like. But those are kind of the three different ways that you can use it. It’s really a map and it serves as this kind of shared artifact for these different parts of the organization.

That’s a thing that’s happening a lot just in AI development. You know, you have leadership saying, “We need to go in this direction.” Builders are saying either, “We don’t have the capabilities,” sometimes the capabilities aren’t even known, and you have this kind of talking past each other, and we can get into this.

I call that the automation strategy gap in the book. But yeah, that’s… Let me know if there’s anything to double-click on. That was a lot of me. I love talking about the autonomy decision matrix, as you now know.

Jorge: One of the things that is implicit in the matrix and in everything that you’ve been saying, actually, is to think about these interventions more systemically.

Stef: Yes.

Jorge: You know, you talked about integrating with other pieces, the fact that you can’t really think about how they will work in isolation from the rest of the systems in the business. And implicit in this is a more kind of strategic framing of the technology, and this is happening in a context perhaps because of the fact that we’re still kind of in the early days of this new technology. Although maybe that’s something to discuss, right? Like how—where are we on the adoption curve? But it definitely felt like in the first few years, the energy was like, “Oh, you know, we’re gonna be left behind.” There’s this kind of FOMO thing happening, and let’s implement… You talked about bolting on chatbots. It felt like every product out there was like, “Now with AI,” right? How do we help business leaders kind of take a step back and do this more intentionally?

Stef: Yes. So there are two parts to this. One is learning to recognize what I have called this automation strategy gap in the book, and it’s very much the precursor to this gap is exactly that, that urgency and FOMO that you’re describing, Jorge. And I think everyone is feeling it. It’s kind of… There’s this existential energy in the air if you’re building something.

You need to be automating everything all at once yesterday. I think the common response to that is this default of what I refer to as dreams of automated futures, and this is not a new concept. We see this in some of our oldest Western literature. We could talk about Homer’s Iliad and the origins of automation as a word, but dreams of automated futures, and it’s kind of reactive if you think about it. We want to jump to that sort of top right quadrant and not necessarily have high confidence in those two axes of could we do it and should we do it. And so you kind of develop this cascade. You get this reactive vision, automate everything in response to the FOMO. You have this defensive maneuvering that I think a lot of the times when we hear AI-first strategy, that’s what’s happening here.

The choices are made to keep up rather than truly serve, you know, to reference Lafley and Martin’s strategy choice cascade, you know, winning aspiration, where to play, how to win, and then you get kind of disconnected execution. You’re setting the wrong metrics. I always joke that you can say, “Hey, our call resolution time for customer service dropped from 11 minutes to two minutes.”

That looks great on paper, but it’s not so great if your customer’s actually rage quitting because they were trapped in an infinite loop with an agent. The quality of that experience is not good, but it’s harder to measure. So I think that’s the default. That’s sort of what is happening.

And so how do you get out of that, I think, is the question. So how do you learn to recognize it? That’s sort of where I started here. And to your point about systems, I think this is where understanding the underlying system is so key. So we talked about my background as an educator as one of the hats that I wear, and I know, Jorge, you also teach systems thinking, design schools as well. I think a key takeaway from this whole process of writing this book is that AI accelerates any system to which it is attached. It is gonna amplify the dysfunction, and it’ll amplify good strategy. If I have a great idea for a prototype, now I can… I, you know, the cognition is kind of front-loaded, and I can build that really, really quickly.

I’m not in Figma moving around little design elements anymore. But if I have a not-so-great idea, I can build that just as fast, and we have automation bias. We tend to trust things that come out of automated or just computers more than our own judgment and others. And it looks shiny, and we may move past, you know, is this thing useful and just focus on the aesthetics. So all of that said, accelerant mechanism for whatever is underneath, and I think this is where systems thinking, or systems mapping, are really valuable tools to understand what are those deeper structures of your organization. A tool that I talk about in the book, one of the most, I’d argue, accessible systems mapping tools, the Iceberg Model, classic, literally looks like an iceberg.

You have events up top, what you immediately see. You have the patterns as you start going deeper underneath the water, the structures, and the underlying mental models. So putting that in an example in an organization, let’s say you have… I talk about this in the book as well, maybe you have a perpetually overloaded product roadmap, probably a pretty relatable experience for all of us. And the tip of the iceberg is you immediately just always see, you know, the roadmap is overloaded. We’re constantly slipping, and you start zooming out a little bit, and this is all in service of seeing what’s driving that system. What is AI actually accelerating here? Is it your strategy? Is it just dysfunction? And so you see, okay, this is a pattern over time. This keeps happening every quarter. Teams are overcommitted. And you look at the structures. So what are the processes? What are the incentives that are producing that pattern? And maybe it’s KPIs are reporting shipping more features than saying no.

Maybe it’s the sales team keeps making promises to customers, and that’s what’s driving the roadmap. And then if you go the last step deeper, the bottom hidden part of the iceberg, you have those mental models. Maybe it’s like saying no means we’re not ambitious, or customers only see value in those new features. And recognizing that is really core to fundamentally redesigning what is driving your organization. And of course, that requires a lot of reflexivity. But I think to really see, okay, what’s driving our decisions, and what are the values driving our behaviors? This is something that I had amazing tech reviewers, and this is something in the book.

It was just… The book was pretty much written, as you know from being in tech review, but it was just this fantastic conversation with one of my reviewers talking about values driving behaviors. When everything is moving so fast, you don’t necessarily have time to have a ton of foresight, so you better have your values in place that are gonna direct how you’re gonna navigate the fast-flowing white-water rapids of AI. So I think systems thinking gives you access to what’s beneath, what’s driving those behaviors, so you can make some sort of a change such that when you bolt on the accelerant, you’re going in hopefully a positive direction.

Greg: I love the idea of this notion that it accelerates everything, right? So you could accelerate just a faster way to get to a bad outcome, is one potential path. You know, and strategy in general is really, in my mind, about creating a map, right? Like a path or a direction and being intentional about the choices that you make.

You know, and one of the examples I think that we’re seeing in organizations is they’ll start on an idea, and their perspective is, we’ll just iterate our way to success by getting feedback with customers along the way, and we’ll get there. And the reality is, if you think about it like a compass, they’re heading southwest, but the real true idea is northeast.

But they’re heading southwest, and they’re pivoting and pivoting and pivoting, and maybe they land in south, but they’re not northeast, right? And so they’ve incrementally gotten something better, but they’re way off product market fit because they just didn’t spend the time to look at the system to understand the problem, the social structures of the customers that they’re trying to build for, et cetera.

Whereas if you have some kind of way of having values and perspective and you act intentionally, maybe you don’t know you’re northeast, but you at least know you’re in that side of the world to start, and so you can pivot closer to that truth over time, right? And, you know, I think that this is one of the challenges I think these tools do is that a lot of us get artificially enamored by the results and we have a little bit of a sunk cost fallacy where we’re like, “Look at all this work we’ve done.

We must be on the right path.” And we’re just on the path to a bad outcome, you know? And I think this is one of the reasons why a lot of AI efforts have struggled is that it’s this sort of ready, shoot, aim approach to implementing the technology versus fundamentally recognizing what we are trying to accomplish and then working backwards from that.

And anyway, I’m massively aligned with your point of view.

Stef: I love that, Greg. No, there’s a part maybe to build on that. I think you could probably see me getting excited there. This acceleration insight, I think it all sort of came together. There was a Harvard Business Review study that came out earlier this year that talked about how AI ultimately just intensifies work.

It’s not actually saving us time, and this has been true for a very long time about technology. Email was supposed to take away meetings. I’ve had back-to-back meetings all day. Email has not actually given us that time back. But what struck me about that intensification of work piece is, let’s say even if you’re generally moving in the right direction, let’s just take a positive example, and that’s certainly not always the case.

Oftentimes we’re kind of accelerating that dysfunction. There’s also a cultural piece to this. Everyone is moving so fast, we’re getting burnt out faster, and that time to actually assess, “Hey, are we going south or are we going northeast?” I think there’s a cultural element here that is underappreciated in that when we’re moving so fast, we can only see so deep, essentially.

And so how do you, if you’re in a leadership position, make the space at the right time to actually put deep thinking where it needs to be and then move fast where it needs to be as well? But I think there’s something to it, to your point of now we can run faster in the wrong direction, but the fact that we can do that is also just making everyone run faster, and by virtue of doing that, pay less attention toward the direction in which we’re running.

Greg: Yeah. Yeah.

Jorge: There’s a saying in the military, “Slow is smooth and smooth is fast.”

Stef: Yes.

Jorge: And that saying has been on my mind a lot recently, and I feel like the three of us are very much in this kind of systems thinking, let’s think about what we’re doing space. And I’m wondering how this registers with business leaders who are under pressure because they have to deliver something this quarter that demonstrates that the money they’re investing in these technologies is somehow yielding results. And I’m wondering if there’s an incentives issue as well, you know? And you talked earlier about measuring the right things. I guess the question is, how do we affect the culture change necessary for these organizations that are really structured to deliver short-term results in many cases to start thinking a little bit more about the, you know, what quadrant do we want to be in, you know? Are we going southwest or what have you?

Stef: Yes. I think part of it is… And if I had the 100% answer for this, I think I could maybe retire. So I’m not gonna claim to have the be-all, end-all answer. But I think part of it is going back to, you know, what is… Like, where do we want this organization to go?

I hesitate to say vision, because vision gets conflated with, like, a statement that maybe is on your website along with mission or something, and is sometimes rather hollow. But going back to that strategy cascade, what is the winning aspiration? What is that goal that we are working towards?

And I understand that there’s gonna be some major pressures to deliver, to have revenue, but presumably you have some sort of customer at the end of the day. You want the customer to buy your product because it is good, and you’re going to have some particular part of the market that you are playing in. And I’m thinking keeping your eye on that, which is actually in some ways very classical strategy, and how does automation serve that? And maybe it’s farther down in terms of management systems or how are you orienting your metrics. I don’t know. I think it comes back to what is that winning aspiration?

Where do you play? How are you gonna win in that particular… I don’t know. It kind of, I… This is a common theme with my thinking, is like what can we learn from tried and true kind of evergreen frameworks, tools out there to guide this moment? What do we need to recognize as different, and certainly the speed is there.

But I think we’re probably not gonna go too far wrong if you have an idea of where we want to go. Sure, we need to be flexible, but how does automation help us move towards that? And I can talk about a possible example of also how designing for cognitive expansion using AI as a killer feature with NotebookLM, with whatever they’re calling it now at Google Gemini.

As of last week I had to change my book because of that. But I don’t wanna get ahead of myself. I’ll pause there.

Greg: One of the things I am finding in the work that we’re doing is that smaller teams can punch way above their weight now with these tools. But not only smaller teams, smaller organizations can potentially do things that a larger organization used to do, and might be able to do them more effectively and efficiently because there’s more intimacy in a smaller organization.

The people know each other. They can react. Do you have a point of view about that? I mean, is that just me looking for what I want to see, or do you think that there’s actually some truth to the fact that these tools empower… may change the shape of how people do things together?

Stef: No, I think there’s definitely something there. There’s an analog here. It’s from, I think it was one of your blog posts, Jorge, around Flow, the Academy Award-winning animated film, which was done, I think, in the Baltics, Estonia. I don’t know, you can check me in post, I think. But it was basically a very small team putting together this award-winning film, and I do feel that is definitely possible, and I think it’s interesting why is that possible. I think there’s less layers of information transfer. Similarly, if you have a team of eight people and you’re working in an office, there’s probably very little packet loss happening between that team, and you can work very dynamically.

Your all-hands is, like, your lunch, and you’re all on the same page. And of course, as organizations start to grow, you start getting potential for more packet loss and so on and so forth, and I think you have a buildup of tacit knowledge that is harder to automate, and this kind of gets into the productivity paradox.

I think it’s possible, but more work needs to be done to redesign the factory, so to speak, to use the electricity example. So certainly I think if you’re a smaller business or a small team, it’s incredible. I see it even in my own practice. Classically a design researcher’s main deliverable was probably insights and recommendations, and now literally, like earlier today, I’m building in Claude Design primarily right now.

We can debate the merits of that. But you know, I’m making prototypes. I’m going deeper into the making part of it, so certainly it’s enabling smaller teams to do more than before. But I think there’s something worth poking on in terms of what is required for scaling and how much of that is about how information propagates and then what’s flattened in these systems.

Jorge: Yeah, I mean, the more people in the organization, the more complex the social networks are gonna be, and to your point, it speeds up everything when you have fewer people. The quality of the tools has also improved. And, you know, we don’t have to overleverage on the example of that movie, but the fact is, you know, the kind of tools that are used to make a 3D movie now are available relatively easily, and the outcome is super high quality. We’re seeing that obviously with AI as well, that we’re getting the ability to produce results that we can quibble with their quality, but it’s oftentimes much better than what people were making 20 years ago, right? Even in the crappiest version of the AI output. So yeah, but the fact that it enables smaller teams and the quality of the output that those teams can put out is much better and much faster seem like factors in…

Stef: Yes. I wonder too, coming back to our earlier part of the conversation, as AI is an accelerant, when you have that small team that has a really clear vision of where we want to go, or I say vision, it’s really, I think I’m talking about strategy, about that winning aspiration, and how they’re gonna get there. If everyone’s pretty aligned on that, they’re using AI as that accelerant and kind of force multiplier, amplifier, so on and so forth, insert synonym here. And yes, what they can create, how they steer these systems is pretty impressive.

Jorge: And I just wanna add one more thing there because one of the things that is unique about AI as an accelerant is that it not only accelerates the creation of artifacts, like you were saying, it’s like maybe you can do a report now in two minutes or whatever, right? Like, and that’s an artifact that the LLM is writing for you. But one of the things that it also accelerates is it gives small teams the ability to come up to speed with new tools and processes that they might be unfamiliar with much faster, right?

Stef: Yes.

Jorge: The team that made that movie, I mean, I don’t know, I’m gonna make stuff up now because I don’t know the details, but if I wanted to pull together a team now to make a 3D movie, I don’t know how to use Blender.

My team probably doesn’t know how to use Blender. But we have these tools that allow us to come up to speed on something like Blender fairly quickly, right? So as long as you have a sense of the neighborhood that you want to be operating in, you can get there pretty quickly. The flip side of this, and this might be something to talk about, is if I don’t have expertise in the systems that I’m working with, I also don’t know what can go wrong.

Like, I don’t know what to look for, right? So I might be misled by these tools down a path that just goes nowhere.

Stef: Oh my gosh, yes. We might need a second podcast for this. No, I have much to say on this and it’s… So I’m, this is not meant to be a shameless plug, but I’m also a co-curator for Lou Rosenfeld’s upcoming Shift UX conference, and this is something we were just recently talking about in our curator’s group, how we talk a lot about AI blurring the swim lanes between roles for exactly this dynamic that you’re describing, Jorge, in that you can now…

You know, can prototype, and I think this is just audio only, so you couldn’t see my air quotes. But what are you prototyping if you don’t have kind of all the elements of UX layers beneath what you’re… Like, if you didn’t front-load that thinking in what you’re guiding, and what you see is maybe one role that is not design creating a prototype, and then design has to kind of clean up the maybe not so correct prototype after.

And there’s this interesting sort of accountability trade-off that we’re seeing by virtue of there actually being some boundaries. Certainly we can expand more. The floor has been raised, but I think there is still a real place for experts, whether we’re called the same title or not. Again, this comes up a lot in our discussion for prepping for the conference.

Like, will design researcher be a title? Will UX be a title? Will we even call it UX? TBD. Maybe not. But the actual expertise of what we are doing, I think something that is really unique to what we’re currently calling UX practitioners is problem framing and reframing, recognizing when the question itself needs to change.

And I have a pretty… Like, the hill that I die on is that’s something beyond the realm of automation, and I have a whole argument about how that’s related to tacit knowledge and embodiment and situated action, which we may or may not need to go into. But I think there is something around… There are some edges, and you still need to have that human expertise to complement the machine. It comes back to that joint optimization, that handshake between human and machine strength. So yes, while that floor is raised, I think we still need experts to be able to call out when the system is going off the rails, and we don’t have that expertise.

We can’t call out that system or can’t steer it as effectively.

Greg: Yeah, it’s interesting. I mean, discernment I think is a really important thing. You get tacit knowledge, the ability to know when something’s good, is based on experience and expertise, et cetera. And one of the worries I have right now in this moment is that the gap between early career and people who’ve been in the seat for a long time is large, and the apprentice-apprenticeship model is breaking.

Which means that, at some level, early career folk are jumping in and just using the tools and saying, “I don’t need all that stuff because I can learn it by just asking great questions.” And then you have kind of a ways of doing things, which have been accumulated over a period of time by expertise and being in your career long enough to see patterns emerge along the way, that is valuable.

But there may not be the curiosity to understand how to use the tools in a new way to take advantage of it, right? And there’s so these sort of gaps between these two spaces. And one of the things I’m personally concerned about is how do we create an environment in business where we actually are mentoring early career people, and we’re also capturing tacit knowledge at the senior level and finding ways to share that information.

And then there’s a connection to that that’s also culturally difficult because tacit knowledge is earned and therefore part of people’s identity and connected to job stability and a bunch of other issues that they may not want to share. You know, I’ve… Long time ago, I worked at the New York Stock Exchange, and I interviewed all the people who worked there, and there were certain people who did not want to share what they knew because it made them valuable in the organization.

And, you know, if they were hit by a bus, the technology stack would fall apart. And so they had leverage in the organization because of that tacit knowledge. So there’s all these kind of issues that are very sticky around how, as a society, we’re gonna mediate between those who know, those who are learning, those who know who may need to learn new things, those who are learning who can adopt and adopt, as Jorge mentioned, “Hey, I need to learn Blender.

I’m gonna learn it now.” You know, it’s almost like The Matrix. “Tell me how to fly a helicopter.” And that gap between those, we’re not having a conversation about that. The conversation around the people in these systems, it’s this efficiency play versus how do we make our people stronger, better, faster, you know, and at the same time hold the social contract at work where we’re all in it together at some level.

Jorge: I saw a post just yesterday from McGill University saying that the Dunning-Kruger effect might not be real, that it might be like an artifact of the data they used to collect it. I’m like, “Oh, no, just when we need it most.”

Stef: Yes. No, I want to believe that research. There’s… Oh, my gosh, I have many directions. Let me see if I can pull them together into something resembling a linear thread around, kind of around those different consequences of AI.

There’s maybe the discussion around the bottom part of the ladder, the bottom rungs being cut off. And I think this comes back to our early conversation about redesigning the management systems or redesigning the factory. It’s a 1880s electricity productivity paradox story we were talking about earlier.

I saw this, in a good way, with job postings for IBM. I spend a disproportionate amount of time looking at internship posts for my students, and I was impressed that they were hiring some… At least this is my qualitative anecdotal report. I don’t know, maybe there’s some big news report that’ll invalidate me.

But they were hiring for junior-level roles for interns, and they acknowledged that automation with AI shifts doing tasks, going hands-on with the artifact, to more upstream, like monitoring, supervising, orchestrating. That word gets used a lot. And they really… It seemed that they were focusing students, these new junior folks coming in, on things that were maybe hard to automate, like customer-facing conversations where you’re building relationships. Maybe it’s setting up systems. I don’t know what the mentorship structure looked like and… But I thought that was cool that they had rewritten the job description. That’s what I’m… Maybe a concrete example of what does it look like to redesign the factory, and I think that’s exciting, and I think there’s a huge potential for seeing my students.

They’re so curious, and across all my guest lectures, having that curiosity, being able to learn the tools, but also, you know, I see a lot of concern about cognitive offloading, to be honest, in my students. You know, they’re the professional masters, so they’re definitely more senior. But they know they need to be able to do the thing or have the judgment as well as steer the tool. So I don’t know. I’m hopeful, but I think there’s yet another dimension. I said that I… No promises about linearity, but there’s also another aspect of this that we’re asking people to come in out of the gates to be systems thinkers, and I think it asks perhaps an uncomfortable question about how… It’s like a nature versus nurture thing. Like, we can learn causal loop diagramming and system mapping and so on and so forth, and those are definitely tools that we can use. But if you don’t put in the reps, can you effectively steer it? I think that’s kind of an open question.

Is that something about personality, like openness to experience? Is it just like practice, like playing violin? If you play for one day versus 20 years, it’ll be different. And I don’t know, I think about that a lot around nurture versus nature for assistance. I’ll pause there before I throw more dots in this constellation, see if we can weave them together.

But…

Jorge: You used a phrase earlier on that I wrote down because it struck me as a really good summary of what the book is about. You said, good cognitive offloading.

Stef: Yes.

Jorge: And that strikes me as a really good synthesis of the kind of subjects that we’ve been talking about, or at least aspirationally, like where you wanna be at. Are there concrete things that you can point listeners, viewers to do, practices, exercises, ways of thinking that can help them get to a place where they’re doing good cognitive offloading?

Stef: Hmm. Yes. I’m thinking about… Maybe I can give some of the underlying principles and then maybe give a couple of… Well, let me start concretely answering the question. I think first off, trying to do the thing yourself, doing the thinking yourself is maybe the first thing you can do. I’ll start there and then I can get into, of course, I just want to go and jam cognitive science principles into everything.

It’s very on brand for me. But I think, yeah, making that first draft, maybe taking that first pass of your song that you’re composing, doing… Like what are the reference images? I have an interface that my partner and I built at a hackathon and we were looking at a different way to explore music and it was very kind of analog of brainstorming, okay, what direction we would want this to go in.

So I think, yeah, if you can at least put down that bad first draft first and then go to AI to maybe use it as a sparring partner, arguably that is, you know, that’s a well-trodden but I think great way to use these systems. See how it can take a different point of view. How can it help you expand cognition?

How can it help you think about thinking? How can it encourage metacognition? I think those are probably the two that I recommend most often, and I really try and practice myself. I think the more I use AI systems, the more I find myself coming back to more analog things. I think that’s a bigger trend as well, or at least starting analog before the accelerant.

Well, those are all really great from a personal perspective. I’m wondering if there are analog suggestions for teams, right? Because part of what we’re doing here is talking to leaders who are grappling with the moment we’re in, right? Like this moment of change. How can they steer their teams, their organizations toward good cognitive offloading?

The entry point for this, I’ll start with maybe something that is not necessarily where you think I’d go with this, which is building a culture of experimentation by going hands-on with the tools. And maybe, you know, two years ago or a year and a half ago, that looked like feeding in a standard sort of day-to-day workflow and trying to automate parts of it.

Now it’s probably building out more agentic capabilities. But making space in that organization to learn what these tools are good at, so you have a sense of what are your human strengths, not just at an individual level, but maybe at a team level, and how those work together versus machine… How do I say?

Yeah, human strengths versus machine strengths. So I can give a concrete example of what this experimentation looks like, which I think at an organizational scale can enable good cognitive offloading. So Canva had this thing called AI Discovery Week. Last year was their first time. They took a whole week off, where they had everyone from engineers to chefs at the company spend some time learning about core AI concepts, kind of boosting general AI literacy, and then they had a company-wide hackathon where they generated, I don’t know, hundreds of different ideas. And I think that was their goal, building a culture of experimentation. Everyone is familiar with these tools.

And I talk a lot about this in the book and in the paper that sort of eventually grew into the book around this building intuition, intuition building, however you… I think I called building intuition officially. And I think that is really key for developing that muscle, and they did it again. So presumably you had some value from that.

They did it again this past year, and what they were building was more in-depth. It was building out workflows. It was more agentic systems rather than general literacy. But I’d say that’s probably something that organizationally, again, good cognitive offloading, and feeds into the sort of bigger discussion around how do we build the sort of culture and adapt the factory, going back to our productivity paradox electricity example.

Greg: That’s great. I would add one thing that I think is interesting right now, which is do more in less time instead of do less, do the same in less time. And what do I mean by that? There are certain activities in product development, some of which take a lot of time to do, or historically took a lot of time to do, and therefore didn’t get done because of time exigencies.

You know, you didn’t have the time to do it or the staff to do it or the ability to do it. But now you can do them, and you can do them rather quickly. And so you should because they will inform you and allow you to understand the problem space better. I read about this recently.

There’s an example around a company I would help where we evaluated the information architecture of all the competitors, and we did that in like a day. It would’ve taken us probably two to three weeks to do it in the past, and we wouldn’t have done it because it would’ve been perceived as a low-value activity, because it was too hard to do.

Turned out to be insanely useful for us because we understood something that was unique to us, and understood something that was happening in the market by doing that. So this is a new activity. It wasn’t new. It’s something I’ve done before in my career, but one that was very difficult to argue for because you just didn’t have the time or the capability to do it.

Now you can, right? And so I think one of the things that we’re missing the opportunity on is understanding what is the more that we can do as teams? What is the more that we can do with these tools? And I love this idea that think for yourself, partner with AI, but I think the next thing is what more could we do to understand this problem?

And given the fact that these tools are basically reducing the cost to do that, at some level I feel like it’s negligent not to do the more.

Stef: No, I have another example of that as well, like thinking about this comes back to that human versus machine strengths and, as a side note, nerdy backstory. This is the Fitts’s List concept from, for any human factors listeners, but adapted to modern days.

Like, this idea of, say, generative AI. One thing that it’s gonna be way better at than me is going through a mountain of data and quickly finding the patterns. Now, if I’m running a big workshop with a client, instead of taking a week to synthesize, even if it was a few days, if I’m able to do that by the end of the day and give some high-level overviews, and then have a discussion about it while we’re still in the same room, like that’s super powerful, versus doing that the quote-unquote “old-fashioned way” and waiting for that couple days or week lag time.

So I think that’s another great example, and we shouldn’t… Like, what can we do… Looking at that list, like pattern detection, generativity. It’s not gonna be necessarily creativity, but producing a lot of variants of something or stamina, being able to just run something for a really long time in deep research mode or something.

I think we’re… Yeah, that’s probably underexplored.

Jorge: The acceleration basically transforms the… You used the phrase old-fashioned, you know, like the old-fashioned technique. It transforms it into something new, right? Because if it’s close to real-time, batch computing is different from interactive computing.

It’s qualitatively different. But anyway, it sounds like we have so much more that we could talk about, but we are running up on time. I don’t know, is the book available for pre-ordering? Where can folks find the book?

Stef: For sure. I can share a link with you as well, but on the Rosenfeld Media site, if you look for What Your Machines Should Do: The Science and Strategy of Human-Centered Automation, it is coming out this fall, and there is a pre-order link.

Jorge: Well, I don’t wanna pre-commit you, Greg, but I would love to have another conversation with you at some point, maybe after the book launch, because this is, first of all, it’s moving so fast, and it is such an important subject. So good luck with the launch, and thank you for joining us today.

Stef: Thank you, Jorge. Thank you, Greg. It was my pleasure.