AI and Software Engineering — Martin Fowler Cleaned transcript of the uploaded edited video excerpt NOTE: This is a lightly cleaned transcript. Repetitions, obvious false starts, and filler words have been reduced for readability. The opening montage is summarized because it consists largely of introductory narration and on-screen biographical material. The edited video jumps among portions of the longer interview, so the section headings below follow the edited clip rather than the timestamps of the original full interview. Generated by GPT-5.6 Sol. ---------------------------------------------------------------------- INTRODUCTION ---------------------------------------------------------------------- [Opening montage / narrator] Martin Fowler is introduced as a highly influential author and software engineer whose work spans agile software development, software architecture, refactoring, and design. He was one of the authors of the Agile Manifesto in 2001, is the author of Refactoring, and regularly publishes articles about software engineering. The excerpt focuses on how AI and large language models are changing software development and what that means for software engineers. ---------------------------------------------------------------------- AI AS A MAJOR SHIFT IN SOFTWARE ENGINEERING ---------------------------------------------------------------------- GERGELY OROSZ: Looking back on your career, what similar changes have you seen that could compare, to some extent, to AI in the technology field? MARTIN FOWLER: It's the biggest, I think, in my career. If we looked back at the history of software development as a whole, the comparable thing would be the shift from assembly language to the very first high-level languages, which is before my time. When people first started coming up with COBOL and Fortran and the like, I would imagine that would be a similar level of shift. GERGELY: You started to work with Fortran, and you probably knew people who were still doing assembly, or at least knew people from that generation. What did you pick up in terms of what needed to change, and how it changed the industry, moving from mostly assembly to mostly higher-level languages? MARTIN: For a start, things were very specific to individual chips. The instructions were different on every chip. You had things like registers and accessing memory. You had these very convoluted ways of doing even the simplest thing because your instruction was something like, move this value from the memory location to this register. You were always thinking in these very, very low-level forms. Even a relatively poor high-level language like Fortran at least lets me write things like conditional statements and loops. In Fortran 4 I could at least go "if." I couldn't do a block of statements; I had to use GOTOs. But it was better than what you could do in assembly. So there was a definite shift away from the hardware toward thinking in terms of something a bit more abstract. Once I'm using Fortran, I can be insulated to some degree from the hardware I'm running on. Am I running this on a mainframe? Am I running it on a minicomputer? There are issues because languages varied a little from place to place, but you've got a degree of decoupling there that was really quite significant. With LLMs, it's a similar degree of mind shift, although the interesting thing is that the shift is not so much an increase in the level of abstraction, although there is a bit of that. The biggest part is the shift from determinism to non-determinism. Suddenly you're working in an environment that's non-deterministic, which completely changes how you think about it. ---------------------------------------------------------------------- IS NATURAL LANGUAGE A NEW PROGRAMMING ABSTRACTION? ---------------------------------------------------------------------- GERGELY: Can we talk about that shift in abstraction? One way of looking at it is that we've had assembly, where you need to be intimately aware of the hardware. Then we have high-level programming languages, where you don't need to be aware of the hardware; you're aware of the logic. You might say we now have a new abstraction: the English language, which will generate the code. You're saying you don't think it's that large an abstraction jump. Why? MARTIN: I think it is a bit of an abstraction jump. I think the abstraction jump is smaller than the determinism-to-non-determinism jump. One of the key things about high-level languages is the ability to create your own abstractions in that language. That's particularly important when you get to things like object orientation or more expressive functional languages like Lisp. With Fortran and COBOL you could do it to some extent. In Fortran you can create subroutines and build abstractions out of that. But you've got so many more tools for building abstractions with more modern languages, and that ability to build abstractions is crucial. There's an old Lisp adage that what you really want to do is create your own language in Lisp and then solve your problem using the language that you've created. I think that is a good way of thinking in any programming language. You're both solving the problem and creating a language to describe the kinds of problems you're trying to solve. If you can balance those two nicely, that is what leads to very maintainable and flexible code. The building of abstractions is a key element of high-level languages. AI helps us a little bit because we can build abstractions a bit more easily, a bit more fluidly. But now we have the problem that we're talking about non-deterministic implementations of those abstractions. That's an issue, and we've got to learn a whole new set of balancing tricks to get around it. My colleague Unmesh Joshi has been exploring using the LLM to co-build an abstraction and then using that abstraction to talk more effectively to the LLM. I find that a really interesting way of thinking about it. There's an example involving chess. Apparently, if you describe a whole load of chess matches to an LLM in plain English, the LLM can't really understand how to play chess. But if you describe those same matches using chess notation, then it can. That's interesting. You're shrinking the token size, but you're also using a much more rigorous notation to describe the problem. Maybe that's an angle for how we use LLMs. We have to come up with a rigorous way of speaking, and we can get more traction that way. That has great parallels with domain-driven design, ubiquitous languages, domain-specific languages, and language workbenches. GERGELY: Is this the first time we're seeing a tool that is so widespread in software engineering that is non-deterministic? MARTIN: It's a whole new way of thinking. It has some interesting parallels to other forms of engineering. In other forms of engineering, you think in terms of tolerances. My wife's a structural engineer. She always thinks in terms of, what are the tolerances? How much extra do I have to do beyond what the math tells me because I need it for tolerances? We probably need some of that kind of thinking ourselves. What are the tolerances of the non-determinism that we have to deal with? We have to realize that we can't skate too close to the edge, because otherwise we're going to have some bridges collapsing. I suspect we're going to do that particularly on the security side. We're going to have some noticeable crashes because people have skated way too close to the edge in terms of the non-determinism of the tools they're using. ---------------------------------------------------------------------- RAPID PROTOTYPING ---------------------------------------------------------------------- GERGELY: What are some new workflows or software-engineering approaches you've observed that sound exciting, things we can now do with LLMs that would have been difficult with our old deterministic toolkit? MARTIN: One area that's already got lots of attention is being able to knock up a prototype in a matter of days. That's way more than you could have done previously. This is the vibe-coding thing, but it's more than just that because it's also the ability to try explorations. People can say, "I'm not really quite sure what to do with this, but I can spend a couple of days exploring the idea much more rapidly than I could have before." For throwaway explorations, disposable little tools, and things of that kind, including things made by people who don't think of themselves as software developers, I think there's a whole area there. We can, with good reason, be very suspicious of taking that too far because there's danger there. But as long as you treat it within the right bounds, it's a very valuable area. ---------------------------------------------------------------------- UNDERSTANDING LEGACY APPLICATIONS ---------------------------------------------------------------------- MARTIN: At the completely opposite end of the scale, one area that's really interesting is helping to understand existing legacy systems. My colleagues did a good bit of work on this. The basic idea is that you take the code itself, do semantic analysis on it, populate a graph database with that information, and then use the graph database in a RAG-like style. You can begin to interrogate it and ask, "What happens to this piece of data? Which bits of code touch this data as it flows through the program?" It's incredibly effective. If I remember correctly, we actually put understanding legacy systems into the Adopt ring of the Thoughtworks Technology Radar because we said that if you're doing any work with legacy systems, you should be using LLMs in some way to help you understand them. GERGELY: There are very few things in the Adopt ring, and one of them is using GenAI to understand legacy code. That tells me you've seen real success. MARTIN: It came from the fact that some of the people who had done really interesting work on legacy-code problems tried this and found it to be very effective. Modernizing legacy systems is an ongoing interest for many of us at Thoughtworks because we have to do it all the time. Every big company that's older than a few years has this problem. Those are two areas where we're clearly seeing success with LLMs. Then there are areas we're still figuring out. We're seeing signs that, if you're working one-to-one with an LLM to build decent-quality software, you have to work in very thin, rapid slices. Small slices. You've got to treat every slice as a pull request from a rather dodgy collaborator who's very productive in the lines-of-code sense of productivity, but you can't trust what it's doing. You've got to review everything very carefully. Using it well, you can definitely get some speed-up in your process. It's not the kind of speed-up that the advocates are talking about, but it's non-trivial. It's certainly worth learning how to make some use of this. We're still learning how to do it. Most of the experience we're gaining is in greenfield environments. That leaves big questions in brownfield environments. We know that LLMs can help us understand legacy code. Can they help us modify legacy code safely? That's still a question. And another area that's really up in the air, both greenfield and brownfield, is what happens when you've got a team of people. Most software has been built by teams and will continue to be built by teams. Even if AI makes us an order of magnitude more productive, we might still need a team of ten people to build what a team of a hundred people used to build. There's no sign of demand dropping for software. The question is: how do we best operate with AI in a team environment? We're still trying to figure that out. ---------------------------------------------------------------------- VIBE CODING AND THE LEARNING LOOP ---------------------------------------------------------------------- GERGELY: You mentioned vibe coding. What is your understanding and take on vibe coding? MARTIN: When I use the term "vibe coding," I try to go back to the original meaning, which is basically that you don't look at the output code. Maybe you take a glance at it out of curiosity, but you really don't care. And maybe you don't know what you're doing because you've got no knowledge of programming. It's just spitting stuff out for you. I think it's good for explorations. It's good for throwaways, disposable stuff. But you don't want to be using it for anything that's going to have any long-term capability. I had a small example recently. A colleague used an LLM to create an SVG graph. The graph looked fine, and I wanted to tweak it a little bit. I opened the SVG and it was astonishingly complicated and convoluted for something that should have been about a dozen lines. That's the thing with vibe-coded stuff. It may produce God knows what, and often you cannot tweak it a little bit. You have to basically throw it away and hope you can regenerate whatever you're trying to change. The other important difference is the learning loop. When you're using vibe coding in this way, you're removing a very important part of software development. If you're not looking at the output, you're not learning. So much of what we do is that we come up with ideas, try them out on the computer, and have this constant back-and-forth between what the computer does and what we're thinking. We're constantly going through that learning loop. You cannot shortcut that process. LLMs can skim over all of that, and then you're not learning. When you're not learning, when you produce something you don't know how to tweak it, modify it, evolve it, and grow it. All you can do is nuke it from orbit and start again. ---------------------------------------------------------------------- CODE REVIEW AND LEARNING WITH AI ---------------------------------------------------------------------- GERGELY: There is a lot more code going out there and a lot more code to review. Engineers are asking, how can I be vigorous at code reviews when there are more and more of them? Have you seen approaches that help people keep learning with these tools? MARTIN: Not a huge amount. I am paying close attention to what Unmesh is doing. His approach is very much this notion of: let's build a language to talk to the LLM. Work with the LLM to produce a language that communicates more precisely and carefully what we're looking for. I feel that's a promising line of attack: creating our own specialized language for the problem we're working on. That also brings up another thing that LLMs are useful for: understanding an unfamiliar environment. If you're working with a language, game engine, API, or technology you're unfamiliar with, an LLM can help you explore. You can ask how to do something, get a starting project or skeleton project, and use it to learn your way around an unfamiliar set of APIs and coding ideas. It can be quite handy for that. GERGELY: This reminds me a little of Stack Overflow. People used to copy snippets and see whether they worked. As engineers became more experienced, they would tell junior developers, "You need to understand this first. Even if it works, you need to understand why it works." MARTIN: It's a similar kind of thing, but boosted and on steroids. GERGELY: So maybe the lesson is that you need to care about the craft. You need to understand the LLM's output. It's there to help you, but if you don't understand the output, eventually you're no better than someone who just prompts it mindlessly. MARTIN: Exactly. I have no problem with taking something from the LLM and putting it in to see if it works. But once you've done that, understand why it works. Also look at it and ask, "Is this really structured the way I'd like it to be?" Don't be afraid to refactor it. And then, of course, there's testing. Anything you put in that works, you need to have a test for. ---------------------------------------------------------------------- TESTING AND VERIFYING AI-GENERATED CODE ---------------------------------------------------------------------- MARTIN: One of the people I particularly focus on in this space is Simon Willison, and something he stresses constantly is the importance of tests. Testing is a huge deal for making these things work. Birgitta is from Thoughtworks, and we're very much an extreme-programming company, so she's steeped in testing as well. She'll say the same thing. You've got to focus a lot on making sure the tests work together. And this is where LLMs struggle. You tell them to do the tests, and I'm constantly hearing about problems or experiencing them myself. The LLM tells me, "I ran all the tests. Everything's fine." Then you run NPM test and get five failures. GERGELY: Sometimes they can lie to you, which is weird. MARTIN: They do lie to you all the time. In fact, if they were truly a junior developer, which is how people sometimes characterize them, I would be having some words with HR. GERGELY: I had an example where I asked an LLM to add a configuration item and put the current date in a comment. It copied the previous date. I told it that wasn't today's date. It apologized and then put yesterday's date. For even the simplest things, if you're a professional working on important software, you shouldn't trust it. MARTIN: Absolutely. Never. Don't trust, but do verify. ---------------------------------------------------------------------- SPEC-DRIVEN DEVELOPMENT AND INCREMENTAL DEVELOPMENT ---------------------------------------------------------------------- GERGELY: One interesting area Birgitta is exploring is spec-driven development. The idea is: what if we define pretty well what we want an LLM to do and give it a really good specification? It can run with it and iterate. Does that give you any déjà vu from waterfall development? MARTIN: The similarity to waterfall is when people say, "Let's create a large amount of specification and not pay much attention to the code." To me, the key thing is that you want to avoid the waterfall problem of trying to build the whole spec first. Do the smallest amount of specification you can possibly do to make some forward progress. Cycle with that. Build it. Get it tested. Get it into production if possible. Then cycle again with these thin slices. What role a spec plays in driving that could be argued to be a form of spec-driven development. But to me what matters is the tight loops, the thin slices, that kind of thing. And you have to be the human in the loop, verifying every time. That's clearly crucial. ---------------------------------------------------------------------- DO WE BEGIN TO PROGRAM IN NATURAL LANGUAGE? ---------------------------------------------------------------------- MARTIN: Where spec-driven development becomes interesting again is when it ties back to building domain languages and domain-specific languages. Can we craft some kind of more rigorous specification to talk about the problem? Eventually what we're saying is that LLMs may give us the ability to build and express abstractions in a slightly more fluid form than we could if we were building them purely inside the code base. But we still don't want those abstractions to deviate too much from the code base. We still want the ubiquitous-language notion that it's the same language in our head as it is in the code. We're seeing the same names and they're doing the same kinds of things. The structure is clearly parallel. The way we think is a bit more flexible than the way the code can be. Can we blur that boundary a little bit by using the LLM as a tool? That's an area I think is interesting. GERGELY: It feels new because we've never been able to use language this close to representing code, or representing business logic. MARTIN: Although there are plenty of people who take that kind of domain-specific-language thinking into their programming. I know people who would get to the point where they could write certain parts of business logic in a programming language like Ruby, show it to a domain expert, and the domain expert could understand it. They might not feel able to write it themselves, but they could understand it enough to point out what was wrong and what was right. That's still programming code, but it requires a certain way of projecting the language to get that kind of fluidity. GERGELY: And DSL means domain-specific language. If you're working with accountants, for example, you're going to have the terms they use and the way they use them. MARTIN: Exactly. What you're trying to do is create a communication route where a non-programmer can at least read what's going on, understand it enough to find what's wrong, and suggest changes. The suggestions may not be syntactically correct, but a programmer can easily fix that. Some people have reached that goal in some places. The interesting question is whether LLMs will enable us to make more progress in that direction and see it happen more widely. ---------------------------------------------------------------------- ENTERPRISE SOFTWARE AND RISK ---------------------------------------------------------------------- GERGELY: This must be particularly important in enterprises, where software developers might be only ten or twenty percent of the staff and you have accounting, marketing, and specialized business divisions that all know what they want. Historically there have been layers of people translating between them. Could LLMs make that easier? MARTIN: That is the world I'm most familiar with. The corporate enterprise world is a whole different kettle of fish. Software developers are a small part of the picture, and there's very complex business stuff going on that we've got to somehow interface with. And, of course, there's usually a much worse legacy-system problem as well. GERGELY: There's regulation, history, exceptions, and lots of accumulated knowledge. Banks are an obvious example. MARTIN: And remember, banks tend to be more technologically advanced than most other corporations. You're looking at the good bit when you're talking about banks. I mean retailers, airlines, government agencies, things of that kind. I was chatting with some people working in the Federal Reserve in Boston, and they have to be extremely cautious. They were not allowed to touch LLMs because the consequences of error when you're dealing with a major government banking organization are pretty damn serious. You've got to be really, really careful about that kind of stuff. Their constraints are very different. To understand how a software-development organization works, you have to look at the core business of the organization and see what they do. If the organization handles money under extremely strict controls, that mindset percolates into software development. If you're involved in an airline, you're very concerned about safety. That affects your whole way of thinking, or it ought to. GERGELY: Startups have very different risk tolerances from mature enterprises. MARTIN: Exactly. GERGELY: AI also seems to have been adopted extraordinarily rapidly. Even very cautious organizations are at least evaluating it. MARTIN: It is everywhere. We see it all over the place, but with more caution in the enterprise world, where they're saying, "Yes, but we also see the dangers here." And the important thing to remember with any big enterprise is that they are not monolithic. Small portions of these companies can be very adventurous, and other portions can be extremely cautious. You'll find some small groups in almost any big organization doing aggressive, experimental things. The variation within an enterprise is often bigger than the variation between enterprises. ---------------------------------------------------------------------- REFactoring — BEGINNING OF NEXT TOPIC ---------------------------------------------------------------------- GERGELY: Speaking about refactoring, you've written the book Refactoring. Why did you decide to write the book? Can you bring us back to what the environment was like? MARTIN: I first came across refactoring at Chrysler when I was working with Kent Beck early in the project. I remember him showing me how he would refactor some Smalltalk code. I was always someone who liked going back to something I'd already written and making it more understandable. I've always cared a lot about something being comprehensible. That's true in my prose writing and in my software writing. What Kent was doing was taking these tiny little steps. I was astonished at how small each step was. But because they were small, they didn't go wrong, and they would compose beautifully. You could do a huge amount with a sequence of little steps. [The uploaded excerpt ends as the discussion moves into refactoring.] ---------------------------------------------------------------------- END ----------------------------------------------------------------------