Name: How to roadmap with decisions rather than dates
Uploaded: 2026-09-21T18:33:13.099Z
Duration: 55 min 56 s
Description: How to roadmap with decisions rather than dates

Transcript for "How to roadmap with decisions rather than dates": Looks like we are live. Hello. Welcome, everyone. Good morning, good afternoon, depending on where you're logging in from. I'm Peter Saulitis. I work on content and digital programs for developers and engineering leadership here at Anthropic, and I will be your moderator today. Thank you for everyone for joining us. It's always so fun to see all these country flags pop in. I see we have folks from India. Thank you for staying up late. I think we have a a dog here, so it's not all humans. So it's fun that everybody is joining us for today's session. You are joining the very first and hopefully first of many sessions of our guest lecture series. Right now, we're focusing on leadership and engineering. It's a series that's built for engineering leaders whose roles are changing very rapidly with AI, from how they plan roadmaps to how you make decisions. We've invited some really amazing leaders on who have built and scaled engineering organizations. They've done it themselves. These are leaders from the field to share the lessons that have helped them lead through that shift and every, session is gonna be twenty five minutes. Will's gonna go through his presentation and it's gonna end with an open discussion and we're gonna answer your questions afterwards. After today's session, we have two more sessions coming up. Sadir Ann from, from Stripe on October 8 and then Jean Sue and Kate Huston on October 13. I've seen a sneak peek of their sessions. They are amazing. So if you have registered for this entire series, you will be getting calendar invites, and we hope you can join through the entirety of the series. As I said before, questions can be submitted, at any time using the q and a tab in the webinar portal. This is essential. Please, this is your time to ask Will quite literally anything about leadership and engineering. I know you guys have questions. So please use this opportunity to drop in questions throughout the presentation. We have a quite a few people on the line right now, so we will absolutely not be able to get to every question, but we are gonna do our best. And if we can't get to your question, we're gonna try to follow-up with either blog content or creating FAQs. So we're gonna do our very best to answer all your questions. Very common question, will this session be recorded? Yes. The record... This session is being recorded, and we're gonna send out a, we're gonna distribute the recording via email within twenty four hours. Great. Now it's time for... To hear from Will Larson. So Will Larson is currently CTO at Imprint. He has led engineering at Stripe, Carta, Calm, and Uber. He is the author of four books on how engineering organizations work, an elegant puzzle, staff, staff engineer, engineering executives primer, and crafting engineering strategy. He is a prolific writer at latheine.com. I was actually, compelled to reach out because I... While I was reading your article on the revised rules of engineering leadership, it is such a great succinct distillation of how engineering leaders are adjusting to this crazy brave new world of agentic development. And I did a little bit of digging. You've posted 770 articles since 2007, including a 192 in 2008. We thank you for your contribution to the community. That is amazing consistency. Thank you for all you've done. We're so honored to have you, here today and and and giving your lesson. So, without further ado, I'm gonna pass it off to to Will Larson to to share how to roadmap map with decisions rather than dates. Take it away, Will. Thank you, Peter. Excited excited to be here and and chatting, about about a topic I've been thinking about a lot recently, which is as you, Peter, mentioned, the world's changed just a tremendous amount in the last the last year, in the last two years, and I think a lot of how we're working hasn't changed much at all. And I will say as we go into this, that one of the challenges with these kind of presentations is that you get, kind of the very best version of the truth. So I'll say that pretty much everything I'm about to say here is is pretty accurate. And so, I think that I hope will encourage you to think that this is not just us and then making things up for for a talk, but actually really how we're trying to work. Obviously, the details do get a little bit messy though in reality. So I'm Will, CTO at Imprint. We do loyalty platforms and payments for some of the the biggest commerce companies out there in the world. And I've written four different books, most recently, Crafting Engineering Strategy that came out last year. And ignoring all of that, though, what I really wanna talk about is how do we work. And so I came into the industry in 2007, 2008. And in my first meeting as kind of a a junior engineer, I saw one of these, Gantt charts. And I've subsequently seen a lot of Gantt charts over the the last, you know, while, but I've never I've never really felt a whole lot of love for the Gantt chart. And I suspect that's true for for pretty much everyone. You can really spend this massive amount of time organizing how things should get done, but I've often found how things actually work and how we kind of laid things out in these Gantt charts to work. It doesn't have, like, a ton of a correlation between the two. And that's feeling even more kind of true for me as I look at this is Imprint's, kind of report from DX looking at the pull request per engineer per week. You can see over the last seven months or so, we've four x'd the number of pull requests per engineer per week. So that's four times more kind of pull requests. We've also increased the size of engineering by, about, you know, like, 70 to to 80%. So I think you could say, you know, four times and then also, like, another double. That's, like, eight x. So, basically, an order of magnitude increase in the amount of changes we're making each week. And and I'd say that the, you know, the ambition of these changes is at least constant. I would think it's actually, like, a little bit up. So it's not that we've just started splitting PRs into much smaller chunks. There was no, no mandatory PR increase per week or something driving this, Just like a a real kind of behavior change. And so the thing I've been thinking about, if we've four x the PRs per engineer and we've doubled the number of engineers, order of magnitude improvement, shouldn't we be planning differently? And this this is, you know, one order of magnitude improvement, but it doesn't have to just be one order of magnitude. I think as you start drawing the trend, I think we're gonna get even more productive from here as we get better at adopting these different tools that we have and as we get more efficient and kinda how we're using tokens. So shouldn't how we plan change, isn't that Gantt chart that we had kind of at the beginning, representative of this old way of running an organization? And if so, you know, I certainly believe that the way that we're planning should change. Although, admittedly, I I hated how we used to plan with the Gantt chart, so I'm, you know, a little bit self motivated in that belief. But if if we do think that, how should we think about changing how we plan? And so I really believe that the old constraint that a lot of companies are oriented around, which was kind of engineering velocity or engineering bandwidth has moved. That's not where the bottleneck and velocity for development is anymore. How we plan hasn't changed, though. And so what should we be doing differently? And that's what I wanna spend some time here this morning kind of walk walking through. So we're gonna talk about, first, recent handoffs across teams as one major way that we can increase velocity. Two, we're gonna talk about how we can when we do have required approvals, especially kind of cross functional approvals and strategies that we've been using to pull the human out of those. Then looking at a lot of times when we have gotten stuck waiting for someone to weigh in, it's actually been because there's a a really key decision about how we might approach a certain problem for our customer base. And often, though, we're finding that we can simply avoid getting blocked on these by prototyping and how does that work for us. Then finally, an oldie, but a goodly. Even if you change how your company operates, you still have to have exceptions for how to to figure out how to get past the places where you will inevitably kinda get stuck, and that is escalating quickly. No, no agentic slant on that one. Just good practice. So let's let's dig in a little bit. First one is reducing handoffs. Something Something I was thinking about a couple months ago is we had three really valuable features for our members that we were trying to ship. But we kinda have, like, two different large buckets of things that we ship. We ship things for our partners. We ship things for our partner that's gonna be a brand, a brand like Shell, a brand like Crate and Barrel. When we ship something for them, there's gonna be a date. There's gonna be a lot of stakeholders who care a lot about it shipping on time because it really matters for their businesses as well. Also, there are commitments we've made to them, and so we we wanna honor our commitments. But there's other features that we ship that don't have strong, loud stakeholders, usually things that make the member experience better. So as one really concrete example, we have not historically linked with something like Plaid to get data out into kind of financial aggregators to contract your credit. But we've built that, and we're in the process of releasing it. And that was one of these three features along with two other things. So they were ready to ship except just like one little missing piece, two two little missing pieces. We, you know, hadn't built it on native yet. And so there's kind of an interesting trade offs like, hey. Could we just ship it on web and not build it on on native? And I think for this case, maybe you can make that case. Two, it's like, hey. Could we just get a few more hours, on the the native engineers? But they were already working really hard on these things that were kind of partner facing releases that really had to go out in a timely fashion. And then there's, like, a totally different lens that some people ask me, like, hey. Should we just add more headcount? Should we are we just under resourced here? And and so those were all really good questions. But, you know, what I did at first was nothing. I was like, hey. This this might self resolve if I just wait long enough. Turned out waiting was not really the strategy I was looking for in terms of getting this stuff released to our our members, And that mattered to me a lot because it's not just, like, idly good. It's stuff that'd be making people's lives better every single day if we got it out. And so one day, I kind of got away from my fear of native development, which is kind of funny. My my very first professional project was actually being an iOS developer. But, admittedly, in the last two decades, a lot has changed with iOS development. But I just buckled in, and tried it myself. And then 500 tokens 500 of of tokens later, it worked. And then I, you know, got some feedback from the team that definitely is a lot better at token. Sorry. I had a iOS and and Android development than me, did a little bit of revving to improve it, and then then shipped it. That was that was super, like, motivating for me because I've been thinking in my head, like, native development, you know, is hard. And so I've been doing, you know, hundreds of front end pull requests. I've been doing hundreds of of back end Go pull requests. But somehow, like, in my head, native development was just, like, a a step too far. You know, too too risky. If you do a really bad native deploy, you can't roll it back in the same way as they kinda like a a front end web change. So I just never quite got there. But then $500 later, plus, you know, another, like, $50 or so for fixing some of the issues that came up in review, it was done. And then a couple days later, it it was shipped for for all three of these projects. That was a really powerful moment, and it really kind of made me think about, you know, one of the classic books of software, which is Mythical Man Month and kind of the core premise that adding more people to a project makes it later because of communication costs. And even as we as an industry have kind of understood that adding more people is bad, we still have kind of carve outs. Right? And so the carve outs are often like, okay. We can't just add more people, but we really need a designer. So we're gonna pull that designer in. We can't not pull in the designer. Oh, in my case, hey. I need, like, a an Android developer. Like, Android development is just too tricky. I really don't wanna add someone to the project, but I have to, because I I just need that additional capacity to get it done because I don't know Android well enough. But what this project really taught me is something that I've learned over and over and over again the last, you know, eighteen months, which is that in a lot of cases, when you have a great model and you have a great harness, judgment's increasingly transferable across skills. So someone who has great judgment and is able to verify their work is increasingly able to do work in other domains. And that doesn't mean that judgment is infinitely transferable, but, also, it doesn't mean the people we thought were really good actually have good judgment. I think, actually, we're in this kind of a crisis of availability of good judgment now where we can see folks that have great judgment and their ability to transfer across so many problems versus folks who are experts but don't necessarily have transferrable judgment. And so the rule that we really focused on is not that teams have to ship everything everywhere. That probably wouldn't be practical for us. But instead, because sometimes, you know, people do in fact need guidance. I think legal, guidance is, like, a great example. You don't really trust even myself to make the right legal call on a lot of things. I wanna pull in the the legal team to get help there. And on native, for a really complicated feature, I'd wanna pull them in too in terms of understanding what are the implications, not just to the code, but to the design system that we're trying to build there around how our native app feels cohesive across many different features that we're building. But, increasingly, we're asking our teams to try it themselves first. If they get stuck, that's okay, but try it first. And then for teams who are getting these pieces of work from other teams, all of a sudden, this this native team is getting, you know, their their CTO is trying to write, Android and and iOS diffs for them. They get to decide a few different things. So first, they could decide, like, hey. This is actually, shockingly and confusingly, the the CTO wrote something worth worth accepting in and incorporating. That that's very desirable, and it can happen. Sometimes, as was the case in in this specific story, they're gonna ask for some specific changes. And even when your judgment's good, I think there's all this this context that you're missing from working in a code base so infrequently about the types of prior errors that have happened here. And in a really well loved code base, that have has a lot of contributors, Those sort of things get codified pretty well. But in a code base that only has a couple people working on it, often things are implicit rather than explicit because they just taught each other or they learn something. They don't need to put it into rules that need to make sure the linters are updated to capture that. They simply know. And so when you kind of work in these places that you previously thought maybe weren't accessible to you, sometimes it's really hard to do that. And so you have to get a little bit of feedback and incorporate that. And then finally, the receiving team can can say, like, hey. This change was actually such a misguided attempt that we don't we don't trust you to to work on this, and we're gonna take that on. And by doing that, though, they understand that they're actually in the hot path. And then the the biggest thing when they think about it is not how do they take more dependencies or how do they get faster. It's rather how do they set up the system so that more and more things get to the the second outcome here where, like, they can automatically give feedback or they can, like, give feedback by hand, but it's still a reasonable thing they could accept. Or, ideally, getting into the first bucket where the automated feedback coming through is strong enough, the linters are specific enough, the typing narrow enough, that when I put up a pull request that works, it's actually something they're they're willing to accept. And if this idea seems just kind of, like, unreasonable, like, you know, the idea that you should actually go a lot faster and have much, much less cross team dependency seems unreasonable because, hey. Like, only they know about how the payment system works. Only this team understands how the the ledgering software works. You just have to try it. And look. If you find out that it doesn't work, the thing you take away is not that this idea is poor. Thing to take away is, hey. Your your systems are not safe enough. You're missing automation. You're missing guardrails to make it easy to do complicated things. Because, look, even if you don't accept it, the way increasingly you're gonna be working on your software is using a harness to kinda drive the implementation. So any work you do, to help these folks is also gonna be helping you and your team very directly. Getting started on this can be a little bit tricky, though. And so one of the things that I've tried to roll out in the organization that I work with is this idea of asking people to show their work. So if they put up to a proposal or if they ask for something, I ask them like, hey. Like, what have you looked at so far? What have you tried so far? What have you done so far? What's the prototype you've tried to put together? Just pushing back on folks that ask questions about doing some of the research, that they should in fact be doing it and pushing them to do more. Hey. You you did some search on on Notion or whatnot. That's that's a great starting point. And if you didn't do that, that's that's pretty bad. You need to at least do that before you kinda ask for for other folks in your company to do that much. But how do you start kind of pushing them to go further and further? And I think just starting here can start moving towards this world where people understand the expectation to really try to solve the entirety of it and then bring the problems where they got stuck. Just like if you're in grade school and you're doing, you know, algebra or division or something like that, you wouldn't just tell your teacher you can't do something. You'd show them working the problem every step of the way until you got to a place you don't know what to do. And then they can understand kind of how you got there and help educate you rather than simply performing a unit of work on your behalf, which is not a very interesting way to be getting better as a company or or as a person. So that's the first one. That that one is thinking about how do we get faster by having fewer and fewer cross team dependencies. Another version of that, though, is automating approvals. And so I think many times, you can simply get rid of cross team dependencies, but you can't in every case. And so a very real example for us is marketing approvals. And so when you do a marketing for a credit card, there's a lot of things like Reg z that you have to be in compliance with. You can't simply kinda YOLO and send something out. That's a really bad situation for the business, for our partners, and also for the the, you know, prospective cardholders getting an offer that's not appropriate. So we need to make sure that every single time that we send out something in marketing, it's been approved. And there's kinda two different approval steps. So the first one is we have this internal kind of review from our team. And then second, we actually work with the sponsoring bank who has to approve it ourselves as well. And so looking at this, though, we we actually have, like, a really great corpus of all these different approvals that have happened in the past. And we look at any individual one, and it's very easy to see for a specific piece of marketing. Is this gonna be trivial to approve? And it's gonna be trivial to approve because it's the exact same language as an existing asset that's been approved before? Or is it gonna be non trivial to improve? And basically, anything that's not been approved exactly before probably won't be trivial or acquire a little bit of human effort. But simply that realization and then managing the corpus of kind of everything that's been approved before starts to get you into a place where you can actually automate the easy the easy versions of this. Similar for security approvals for, you know, requests for comments that your company writes when you're doing new architecture. Fully automating security review is is, I think, very, very challenging. And if you've tried some of these skills around doing automated kind of security review, I find them, like, a little bit inconsistent. Particularly when you go broad, I find find they tend to go little bit like low value. If they go very specific, they tend to be super helpful. But, if again, thinking about the easy case, there are so many security approvals that are extremely easy to automate. And so it's gonna look something like an existing API using an existing security and authentication mechanism authorization mechanism to add some parameters. Almost certainly, as long as those are not like PII, there's no, like, PCI kind of data getting involved. Almost certainly, you can automatically approve that because it's simply, consistent with what's been done before. And so in each of these cases, the marketing approval, but also security approval, so it's gonna be simple kind of basics that you can automate looking at the history of what you've done. And we found this really powerful in terms of make it possible to do the easy stuff really, really fast and reserving humans to be focused instead on the more kind of complicated decisions. Also, I think, it's really empowering for for me as a person. So let's say that my job is reviewing RFCs for security concerns. All of a sudden, I'm much more of a policy expert who's kinda working on extending our policy rather than simply doing rote review. Right? Systems like this let us really elevate, what our colleagues do and what we do into something that's more scalable, but, also, like, vastly more satisfying to be working on. And I think that's that's really, really exciting. So how do we do approval automation? First thing for us, the thing we did was unifying ticket tracking. So we completely moved on to a new ticket tracker, which had fewer permissions, and had less isolation, kind of less fragmentation across the different sources of tickets. This has worked really, really well for us, and I think it's been really kind of the first step. I won't get into a ticket tracker politics on on this one, But I find that previously, when tickets were, like, very constrained by permissions, we often couldn't even see the the different approvals happening. So we just struggle to know where approvals were happening, and many people couldn't tell, why approvals were happening when they did happen. And so it was just a really challenging moment to be in. But once you have kind of a unified ticket tracker that has strong kind of APIs or MCP integration, you're then able to very quickly start to find recurring approval flows. And just find one that for us, I think security review was like a very straightforward one. The marketing review, like another straightforward one. Again, not easy to do in the hard cases, but easy to do in the easy cases. And from that, once you start seeing those approvals and the rationale, you start to get, like, a corpus of kind of all these decisions that have been made and the asset it's been made on. And, look, in the very best case, each time you made a decision, someone would be looking at, like, a policy and extending your policy about the whys and the whats. But in a lot of companies, like, that's not really how it works. Right? Like, someone makes a decision, and they're super busy, so they move on to making the next decision. But this is sort of place where a harness that can see all of the approves and all of the non approves or all of the kind of please, like, address these open questions can actually extract the rules for you really well. And, again, you can't take it at face value. You can't just say, like, oh, what the the harness says is right. But it can do such amazing analysis for for basically almost no cost to build up a perspective policy, which will be equally good with something I think you would be able to put together without spending a ton of time on it. And then, you can start using that policy to auto approve the simplest cases. And, again, I just think this is so important because each of our companies has, you know, dozens, hundreds, maybe thousands of people who are doing relatively rote jobs, to just run the system. But increasingly, by using this sort of approach, especially with unified ticket tracking, we can measure and have visibility to all the work and then developing the corpus and the automation on top of it, really elevate those roles into something, I think, much, much more appealing to to work on. And once you've done that, you just keep extending iteratively and kind of making it more and more complicated over time as you go. And where to start today is I I really think you can auto approve one real case. And so for us, the security reviews actually were the first one we did because the simple ones were just, like, no new no new patterns introduced whatsoever, not PCI data, not PII data, just something as simple as that. Doing review was very straightforward in that case. Even if it's not perfect, even if it's wrong, as long as you're, like, focused on kind of the the pain of the wrong, or kind of like a note grates on you a little bit, still, this will start pulling you into a really interesting place. Even if the first real case is is pretty basic, I think it's a really powerful kind of milestone, a flag to kind of plant that this is how you wanna be working going forward. Then the next one is prototyping rather than getting blocked. And so we've talked a little bit about how to reduce dependencies across teams, talk a little bit about how to automate the dependencies you have to keep with, you know, approval frameworks. And finally, there's a lot of places where I think you can get blocked because there's just a lot of ambiguity. And one example that I ran into was adding passkey support. So I had a pretty clear sense of how passkey support worked before I I started adding this, and this is a feature that we launched, I let's say, four months ago, something like that. But it turned out, as I dug into how passkeys work in the details, they didn't quite work the way that I thought they worked. In particular, how does, you know, web kind of support for web often actually work in these different browsers and how the browsers are changing? There's a lot of proposals that are out there about improving the usability of passkeys. There's also a lot of concerns about those proposals because they tend to make it easier to fingerprint a user because you could actually detect if they have a passkey if some of these proposals went through without the user performing an action to actually trigger the the password manager. And so looking through all of that before well, before I looked through it, I thought I knew how passkeys works. As I researched, I increasingly thought I knew how passkeys worked. But I didn't actually figure out how passkeys works until I I kind of messed up in terms of I built, like, this full process, but didn't actually have the usability that I wanted to do. So I built these product requirements. I I wrote the architecture. I I implemented a version of it. Then I started playing around with it and actually just didn't behave the way I wanted it to to have, like, a great experience for for our members. And so if I assigned that to someone, we actually would have gotten stuck because I think that person would have gotten these requirements for me that I felt very strongly about, but but were, like, subtly wrong. And they might have built it and stopped. Like, hey. It's good enough. Or they might have, you know, told me that it was totally impossible, and I might have been skeptical of that. But by prototyping, I was actually able to figure out what the reality was underneath my beliefs about how passkeys worked in the in the details and my beliefs about, you know, Chrome three a releasing with certain new features to allow auto prompting versus what Safari had versus what's available on kind of the the native environments for Android and iOS. But simply with prototyping it, all of these questions I might have gotten stuck on disappeared because there was just a truth underneath all of it. And then we were able to ship that in less than a month. And so really prototyping it, then getting around to feedback, going through security review in particular, was able to ship it. And the thing that I was most excited about is that we do have a road map historically. Historically, it has had dates. Historically, it's not had just decisions on it. But this was something that we're able to ship from zero to one to all of our users, and it was never on a road map. And we're able to do that by having a very small team for this specific feature. It was me prototyping our way through. And when I got stuck, I just kept prototyping until I figured out what the answer was. The general process, though, of how we've used the prototyping approach for a number of features is brain dump. They take the initial thinking out. We build the back end and the the web behind the feature flag. So for us, web is just much, much faster to iterate on native than native because of the release kind of cadence and deployment structure. With the feature flag, the actual risk of doing the back end and web development is quite quite low. Then we iterate until it feels good to the person or one or two people working on it. At that point, we actually document our real understanding of the project now that we understand it and if there's any major lingering trade offs that we polish based on the feedback we get on that final doc, and then we release the iOS and Android. And this has worked really, really well for us. Almost all of the biggest member experience improvements we're making over the course of this year have been done this way. And so, again, I I talked with these two different buckets of work. Look. Work that evolves, like, our partners, like, they have, like, a strong point of view as well they should about, like, what we're building and why and the details of how it works. There, we have to be really, really careful and really collaborative about it. But for things for our members, like, the the real way to get educated, that is like listening to support calls, reading support tickets, looking at the the utilization data and how people are actually behaving, like, the the revealed preference versus the status preferences that come from kind of the the calls that come in. In that way, we can pull all this data and work super, super quickly. And this is, like, one of the joys of kinda working consumer in in 2026. And then once we've done all of that, we we can ship. And that's that's really what the the funds about. And so the the place you can start is that, again, for prototyping, I think if I looked at it, I would have thought maybe some things I wanted to do would have been hard to prototype. And so I started with a document. I think in a lot of cases, you can just bring the prototype today. And And I think there's two different types of prototypes you can talk about. And so there's a prototype, which is like a a lovable or something like that, which is kind of like a clickable thing. Then I'm really talking about a step further, a prototype that actually integrates into your data model, so prototype that actually works. Because that's that's really how you flush out all the real challenges. Because there's gonna be things if you just do kind of the simple clickable prototype where an engineer is gonna look at it and knows your data model. It's good. Okay. This is impossible to ship for these seven reasons, and this is a terrible idea for these other three reasons. But by actually forcing the integration into the code, you get to figure out what are the real things you can actually ship versus the things that seem like they should be shippable if you don't understand the constraints underneath. So if you do all of that, though, you still have to escalate quickly. And so I think about the the ticket tracker migration, and the actual effort to migrate between these two different ticket trackers is pretty low. Basically, we set up a an MCP on one hand to an MCP on the other hand, and the MCPs just work with the harness to kinda carry the the right tickets over, dropping a lot of the tickets that didn't need to come. But, actually, getting alignment on doing this was substantially more than no human effort. Right? And it took a lot of energy because this wasn't just a change for engineering or for products or for the tech organization. We actually moved the entire company over because that's how we knew we'd get to the place where we could actually see these decisions happening in in a consistent way and start automating not just, like, the technology function, but the company's function, which is really the the reward on the other side of this. But it took a lot of time to get that alignment through. And if I just escalated earlier, I would have been able to get the decision versus, trying to convince everyone one one person at a time. And so, you know, really, the only way to get effective as a company and just move quickly as you move away from this date oriented kind of execution. And so the the illusion you get with dates is that you're actually making all the decisions upfront then rolling through. That obviously doesn't work in practice. And so just ignore ignore all that. Instead, escalate whenever these other techniques don't get you there. And, you know, the the place to start is that you should just escalate something today. I think escalations are always perceived as a breach in kind of the relationship you have around you. But often, I think with a bilateral escalation where you and the kind of the other perspective are escalating together, this is just a much easier place to make progress quickly. And so recapping a little bit of what we talked about, we talked about reducing handoffs, talked about automating approvals, prototyping rather than getting blocked and escalating. And I'd really encourage you as you look at these, I think this can be fundamentally the framework you use for moving quickly, and starting to move away from dates. And I think it's it's scary to think about a world where a road map is really just about cross team coordination points. But in a world where we're going 10 x faster than a year ago or or really, like, seven months ago, you have to start thinking, like, how are things gonna have to change when you're no longer constrained on this idea of engineering velocity, but you're actually constrained on just your internal process. And these are some of the ways that we've found to get a lot faster. And if you do this well, you you don't actually get to not be constrained on things right. You just end up being constrained on, a different set of things, which is, you know, your your internal processes and how you make decisions. And with that, that's that's the deck. And I think we get to go into some q and a now. Thank you, Will. I think, judging by the emojis and reactions, a lot of folks on the line really enjoyed the presentation as did I. Thank you all for a ton of amazing questions. Unfortunately, we're not gonna get to all of them, but we're gonna do our best. Some really good ones off the bat. I I actually really liked and there's a couple recent one that... Ones that came in. Well, you know, you're thinking about not adding the... Thinking... Rethinking the roadmap and not having the dates and I almost see the freaking... Well, we have to present. You know, we have to present to leadership. We need to get this approved. So I think it's a couple of really good questions. And then one from Eric, how do you think about communicating the roadmap to other stakeholders when it's more open... When you take a more open ended approach like you did? So I think I might be thinking about a lot in this context is leadership has to be more present and more clear about the decisions and what we're doing and not let people just kind of infer what's important. And so I I think, executive teams, senior groups with engineering, we should be making more explicit decisions. And this is a lot of what my my last spoke about was, hey. You wanna use, a different back end programming language? You can't. You wanna use a new front end programming language? You also can't. You wanna change your database? Nope. Not allowed to do that. And I think by being really clear about what you're not allowed to do, you can then be clear about what people are allowed to do and channel them into, like, here are the areas where you're allowed to make forward progress. And for each team, here is the problems that matter to me the most. And and, importantly, I think when people hear this idea of, like, what matters to teams, they think, what's on my roadmap and what's on Peter roadmap? Or, like, which things are is Peter allowed to do? And I actually, like, have been trying to push back on this idea of, like, really strict ownership. If if you're in consulting, there's, like, the MECE, like, mutually exclusive, collectively exhaustive. I forget if if I'm not sure if c that's the right c. But where people will basically try to figure out how do we make sure there's no overlap between teams. I don't think that's the idea at all. Right? I actually think about if you have a different priority, then you'll get to different tasks at different times. And so that that's actually what I care about is priorities, not kind of exclusive ownership, and being clear about what the priorities are and things no one's allowed to do so they don't spend time trying to do them. But in those cases, I think you increasingly don't have these sorts of, like, conflict points. But there there are still, places where there's, like, a a partner commit or something like that that is just very time sensitive. And people really top down just have to say, like, if the partner needs something, that's that's the p zero, and that's always gonna be more important than your kind of, like, independent exploration of of this thing over there. So, again, I think leadership has to have a higher standard of of detailed clarity, and people have to get comfortable that there's gonna be more top down decisions about where they're allowed to play and where they're not. I think this is a place where a lot of companies have kind of gotten stuck in the wrong spot over the last decade. That's great. I think we we saw a lot of questions about, your prototyping approach, a bunch of questions about managing the influx of PRs. Let's tackle the first first, I think. Second is a very popular topic today, but, one question that came in prototyping instead of being blocked is a good philosophy, but how do you reconcile that with the standards like ISO two seven zero zero one, which expect documented risks assessments and security requirements before deployment? Yeah. I I think a lot of the stuff comes down to kinda like definition of deployment. Right? And so, like, my my approach and our goal is to push compliance of that sort, and there's, like, different types of compliance. Right? But push that sort of compliance as late in the process as possible because often the thing you're actually trying to build isn't what you think you're trying to build, and you don't realize that until you get late. So then a 100% to to the point about whether it's ISO or or something else, You do have to go through all of those steps, but trying to get them so late that you really have conviction on what you're building. I think oftentimes, people have, like, inadvertently this, like, philosophy of getting stuck. It's It's like, how do I get stuck? But you often don't have to get stuck. You just have to be comfortable throwing your work away if it's wrong. And I think in the past, the idea of this, like, loss aversion to, like, I've spent, like, two weeks or six weeks building something, and then compliance told me I have to throw it away. And it's just like, I've just I've just lost, like, six weeks of my life I'll never get back. But now it's like, hey. I lost, like, I lost two days. And but the the flip side of it is now I know exactly what we need to build. And now I've actually gained, like, four weeks of time. And so I think if you if you wanna go fast, you have to be willing to throw away your work a lot more frequently than than maybe you were used to in the past. And I it's uncomfortable, but I I do think I throw away a lot more stuff. And I it's a lot less personal because I put a lot less time into it than I I did in the past. It's a great segue. You're talking about throwing out the work, but what what about when you're when you're managing a team that is raising 40 PRs a day? And this question came in from Jay. How how is an engineering lead? Can I manage them and move my team faster? I think, first, like, the the whole pull request thing, code review thing is is pretty interesting right now. Right? And I think the first problem is, like, I think a lot of companies have people that are just putting up, like, trash, effectively. And then, they they gotta learn or they gotta stop. And so, like, for for us, the rule is, you have to stand behind your PR. You can your your agent's output can never be first shown to someone else. Like, you actually have to go see it and work through it. Because I I think a year ago, you you'd actually see people, like, putting up PRs that they never looked at, which kind of, like, makes sense, but there there's sort of this, like, I mean, you know, AI psychosis where people are like, oh, it's probably great work because the, you know, Anthropic's Claude, like, Opus five or or Fable or, like, GPT six Aster did it. But often, like, it's just really misguided work that you get out of the harnesses too, and so you do actually have to review it. And if you don't, it's, like, disrespectful. And so setting that baseline that it's, it's not rude to ask if people have looked at the work before they show it. And sometimes you don't even have to ask. Sometimes just the errors are so kind of egregious that you know someone's not looked at it, and that preferred that person's not performing. So that's like bucket one. And I think there are a lot of teams where people are in bucket one. Then bucket two is, like, what do you do when you have, like, really good people who are generating a ton a ton of software? And so I I've always felt code reviews are largely kind of, reputation, like, system more so than, like, purely, like, a quality review system. I think you see this more and more true where we're actually thinking, in some ways, like, the the best performance management system you could have today would be time to get, like, an approval on your PR. Because I just see when the volume is going up, you can just really tell the people who are struggling to get reviewed versus the people who are not struggling to get reviewed. And and this is, like, to me, actually, like, increasingly a lot of the reviews. It's the reviewer has a point of view about whether the change is safe or not. And so when I do reviews, like or when I put pull requests up personally, like, every pull request I do is, like, behind a feature flag or it's not used or there there's some mechanism where even if it's catastrophically terrible, like, the the impact to our partners to to our members is is contained. And some people you you learn from reviewing PRs that some people don't set up their PRs that way, where they actually, like, if this goes wrong, they have no mechanism to revert it other than doing a, you know, a a fail forward, additional release, right, which is not not particularly fast relative to flipping a feature flag. And so, again, people kind of earn a reputation, but even what I just said, like, couldn't, like, a harness be actually giving that feedback automatically? Couldn't the harness be steering towards this sort of behavior? And it could. And increasingly, it should be. Although, again, like, the details are always then you end up with, like, hundreds of feature flags, many of which you don't need. And so there there there's always, like, a constant iteration on it. But I I see a lot more ad, like, reviews, I guess, I'd say, for folks who have demonstrated a lot a lot of success and safely shipping changes. Now folks who haven't, I I the the social pressure of the slow review is is alive and well. And I think that if you're struggling to get reviews, I think that, like, a one of the few different things is happening. Like, your PRs are confusing and scary, that you're not managing risk well. I don't know. You you've alienated your colleagues in a different way, but but it is it is tricky. And, also, as an example of a place where I don't think we've figured it out, we're getting a ton of very large PRs, like, three months ago, like, thousands and thousands of lines. So we put together some rules to prevent that, and then we started getting a lot of very confusing, like, sliced PRs. And so it's it is it is still tricky. Right? It's not obvious that there's any, like, golden rule that makes this easy, but I think continuing to iterate on the reputation piece, continuing to push on folks who aren't following practice well and hold them accountable, those have been the highest kind of value for us. I do wonder I do wonder what the world will look like in a year, but I I don't think I understand that one super well yet. I feel like we could do a whole session on on managing PRs. Maybe maybe upvote that. Maybe we can suggest that in the comments. But, I I think take... Taking a pivot away from some of the process oriented harness related questions. So I thought this one was exceptional question. It was on, you you were mentioning transferable judgment. If transferable judgment beats hard skills when leveraging AI models for implementation, how do tech interviews change to bias superior judgment skills? I thought that was a great one. Yeah. I think interviewing is really hard right now. Right? And so I think and there's many different types of interviewing. Right? So I I think when you have you know, part of the transferable judgment piece is, like, how do you interview someone when you don't understand the work they're doing? And so I think executive jobs are the best example of this because people that interview executives never know what that executive does. It's typically, like, if you're gonna be a CTO, engineers won't really have, like, much of a say in whether to hire you or not, which sounds sort of, like, ludicrous. Right? It's like, well, obviously, you want, like, the functional experts to be interviewing their executive. But almost almost universally, that's not true. Right? Almost universally, you hire a head of marketing, and no one in marketing or maybe one person marketing is interviewing them. Instead, it's, what does the CEO think? What does the CTO think? And, you know, what does, like, the CFO know about marketing and they get pulled in. Right? And so for for a lot of these roles, it's it's really kind of like vibes and back channel. Right? And I think is that concerning? Like, a a little bit. This is just fundamentally how, like, a lot of interviews work. And I think the the challenge with the AI tooling is like, it's increasingly kind of true that more interviews are our vibes and backchannel, which is like discriminatory against, you know, like folks come from smaller markets. You know, someone who went to school in Kentucky and then came out to San Francisco, I I remember that. Like, I didn't have the background to, you know, get get into the companies I wanted to. I had to kind of, like, stack prestige over a course of, like, two decades to kinda get to to where I wanted to be. But I I think what we found is that just watching people actually do things as opposed to talk about doing things has been, like, the the the biggest source we have right now. And so watching people not talk about building something, but watching them build something. And even watching someone use a harness, I think you can see how they engage. You can see the feedback they're giving. You can see, the the level, the altitude they're operating at. You You can see if they're, like, adding, like, mechanisms to prevent the mistake from happening again. You can see just the state of their their setup. And if they've, you know, put a lot of thought into their setup, you start to see just how they work and how they think. That that's not the full answer, though. Right? And it is, like, overanchored on, I think, attention to detail, which, although I would say attention to detail is, like, always been a pretty underrated skill and is increasingly an underrated skill in the current age. Or that that's really, you know, the right word can change the the output of the the harness pretty quickly. And I it's just hard hard to measure that, particularly folks coming right out of school. I think it's incredibly hard to to do. Totally. And and another good one that we've seen with a lot of CTOs that we've that we've interviewed. You know, one of the cool things about, you know, agentic engineering, it's what it's allowed engineering leaders who maybe it's maybe it's been years since they've stepped in and and merged PR themselves. Right? And right... And get back into that arena. And I think that's, you know, one of the cool things that's happened with the advent of, you know, Claude Code and some of the better coding models. So, you know, one question I thought was really interesting is, how do you think, Will, about jumping in yourself to ship something versus making sure the team can handle that same situation when you're not around? I I think it's a great question. I I think it's it's tricky. Right? Like, I think so before I joined Imprint, I joined Imprint, let's say, a year and a half ago. I hadn't really contributed code since I left Uber. So I left Uber in 2016. And so that is basically a decade where I wasn't contributing on on production software. And I had, like, side projects. I was kinda doing things myself. But I I was I was very rusty. And so, personally, though, I I was still, like, hands on kind of, like, playing with stuff on on my own time. The the thing I found is, like, when the quality of these agents hit kind of that, like, Opus four or five kind of moment, like, roughly a little less than a year ago, like, the the fact that I was very rusty at things, like, started to matter less. For example, react react hooks, I think, are, like, super annoying and kind of managing the React hooks properly. The the nuances of doing that without spending time, like, educating myself will started started to kinda go away. Similarly, we're we're a going back in shop, and I think going is lightly joyless in terms of, like, things one might choose to write. But it's strongly typed incredibly fast. It, like, works really, really well for harness driven development. I don't I don't really love writing it personally, but it worked incredibly well for us. And so I think got to a point where I was able to be productive then. And and I've shipped more PRs at Imprint than I did in my entire career. But before that, which is kind of wild to think about. But then, like, even if I can, like, should I be doing it or not? And and that's a question that I think about a lot. I think in twenty tens, there was sort of this, like, virus that managers shouldn't think about technology. That was actually a sign of bad management was having a strong educated point of view on technology. It was like a kind of a put out idea. I think a lot of people subscribe to. And so, like, the first thing to do as a new manager is you have to get out of the details. It's kind of like the the argument. And there's like some, it makes sense, right? Because people are basically getting told to stop micromanaging your team. But instead of saying stop micromanaging your team, you're you're pissing them off. We said get out of the details, which is only one solution for getting to stop mic stopping micromanaging your team. There's other ways to stop micromanaging your team as well that are not not so destructive to to the kind of companies around them. And this really forced us as, like, leaders to then over anchor on this kind of, like, more processy kind of set of solutions to a lot of things. But we're and then I think an entire generation of managers, a a generation that I kinda grew up in in a lot of ways, I think was, like, inadvertently poisoned with with these ideas that they shouldn't be technical. It's actually bad for them. And folks now, this is the hot take you didn't ask for. A lot of them are kind of, like, stuck now because now what they want from, like, c levels is actually, like, very detail oriented execution and kind of, like, management across these fields. But where they came up in, they were basically told not to do the former. And so they're really struggling to get to that that last step because the the industry, like, wanted one thing and now want something else. And that's, like, a really a really frustrating moment for for folks. It makes makes sense. See a lot of great people from the last decade get stuck in this this moment that way. But the way I try to make this decision and then what I try to tell the managers I work with is the things you can just do versus asking for help are continuing to expand. And your goal is, like, instead of, like, asking for help and interrupting someone, to just do those things. And the problem is you don't know what you can just do without asking if you don't try. And so you do have to just have, like, kind of a steady stream of stuff you're trying. And the amount of things that I can do without trying is, like, increasingly, like, very, very broad. And this brings us to other questions, like, what should a product manager be shipping into your production code base, which I think is, like, another another interesting question that we can either choose to or not choose to, engage with that one. Okay. Third third session on top of this session. But, yeah, it's a great question. I think we only have time for what... One more, and maybe we'll make this one, Will, make just a little brief. I I think it's it's a really interesting question. It's it's come up a lot in terms of metrics. You know, is PR per engineer per week considered a good metric? Maybe just your thoughts as a CTO on, the changing metrics that you care about in, you know, sort of the new agentic engineering world. Yeah. I think look. All metrics that you measure are basically bad. And so I think the the reason I think that dashboard or the, the chart I showed is interesting is we don't have, like, a goal for folks. Like, there's no performance like, we don't pull in your PRs per week into your performance review or something. Like, the we don't measure it in a weaponized way. And so I think it remains, like, fairly honest as a result. But it but I definitely think there there's no super interesting measure of productivity other than business outcomes. Right? And so then the challenge is, like, that's like a lagging indicator, an output metric. What are you supposed to create people on from an input metric perspective? And I I really don't care about pull requests. I really don't care about deploys. I really don't care about, like, any of it. I just care about seeing the stuff they're doing that's translating into things that actually ship. And so I think shipping software and then shipping software that has impact on our members and our partners, like, that's the only thing I care about. And if you're doing stuff that doesn't contribute to that, that's, like, ultimately not very interesting to me. A giant caveat, like, unless there's also, like, operational kind of rules in in companies. Right? And so making sure your up uptimes are good, making sure your security is appropriate. But even, like, security security is, like, a core driver of, like, having partners. Right? Like, partners work with us because they trust our security. Members apply for a credit card with us because they trust our security. Right? And so I think you can always reframe work you're doing compliance, security, infrastructure in the context of driving, like, the actual business. And if that sounds like, unlike unbearably boring way to think about it, it is like, unfortunately, how, like, investors think about businesses. It is, unfortunately, how, like, CEOs and CFOs think about businesses and how public markets think about businesses. So even if that doesn't sound exciting, that that's fine. I don't think it's super exciting way to think about life sometimes either. It is, in fact, like, how people think about us. No. I I I love it. I think it's emphasizing the craft rather than just the sheer volume of output, and and I think that was perfectly framed. Will, do you mind, sharing your screen again? I think we need to jump back into the slides, before we wrap up. Will, thank you so, so, so much for joining. Presentation was fantastic. Thank you all on the line. Those questions were incredible. Of course, we got to a fraction of them. We will do our very best to answer those via blog or via FAQ. We will do whatever we can to get some answers to your questions because there were some amazing questions that got left unanswered. Up next in the series, AI as an engineering leadership, multiplier. Sudhir writes a lot about the skills that he uses, for his daily reviews. He is, an amazing writer, has an amazing perspective. I got a sneak preview of that that presentation. Incredibly valuable. Please jump in for that session that's on October 8. And then there was a couple questions about, evaluating the next step in your career. I think this is also a really interesting, you know, point to make as a software engineer at any stage. You know, what is next for you? What should you be doing? What what what skills do you have that are transferable to maybe a different role? So it's gonna be a really, really, really, really good pertinent talk as well, and Jean and Kate are amazing presenters. So that should be an incredible session. Next slide, I think it's... I actually think it's... There we go. There's the branding slide. But thank you again for joining. Be sure to to fill out. We're gonna be sending you a survey, just to get your thoughts on the session. And a couple other resources that we've pinned to the docs are things that, Will has written. He has shared with us to share with you. I know, Will, I think there's a career site on Imprint. Anything else that you would wanna share off the top of your head? Yeah. That that's the most exciting one. Come come work with us Imprint. You can also buy one of my many books. They're all they're all, worth at least $25 that that they charge. But, you know, up to you. Done. That's great. Again, Will, thank you so much. Thank you everyone on the line. Take the survey. We love the feedback. We need the feedback, and have a great rest of your day. Thanks, everyone.