I believed I’d supply a number of takeaways and reflections based mostly on final week’s first AI Codecon digital convention, Coding with AI: The Finish of Software program Improvement as We Know It. I’m additionally going to incorporate a number of brief video excerpts from the occasion. If you happen to registered for Coding with AI or if you happen to’re an present O’Reilly subscriber, you’ll be able to watch or rewatch the entire thing on the O’Reilly studying platform. If you happen to aren’t a subscriber but, it’s straightforward to begin a free trial. We’ll even be posting further excerpts on the O’Reilly YouTube channel within the subsequent few weeks.
However on to the promised takeaways.
First off, Harper Reed is a mad genius who made everybody’s head explode. (Camille Fournier apparently has joked that Harper has rotted his mind with AI, and Harper truly agreed.) Harper mentioned his design course of in a chat that you just may need to run at half pace. His greenfield workflow is to begin with an thought. Give your thought to a chat mannequin and have it ask you questions with sure/no solutions. Have it extract all of the concepts. That turns into your spec or PRD. Use the spec as enter to a reasoning mannequin and have it generate a plan; then feed that plan into a distinct reasoning mannequin and have it generate prompts for code technology for each the appliance and exams. He’s having a wild time.
Agile Manifesto coauthor Kent Beck was additionally on Crew Enthusiasm. He instructed us that augmented coding with AI was “probably the most enjoyable I’ve ever had,” and stated that it “reawakened the enjoyment of programming.” Nikola Balic agreed: “As Kent stated, it simply introduced the enjoyment of writing code, the enjoyment of programming, it introduced it again. So I’m now producing extra code than ever. I’ve, like, 1,000,000 strains of code within the final month. I’m taking part in with stuff that I by no means performed with earlier than. And I’m simply spending an obscene quantity of tokens.” However sooner or later, “I feel that we gained’t write code anymore. We are going to nurture it. It is a imaginative and prescient. I’m certain that a lot of you’ll disagree however let’s look years sooner or later and the way all the things will change. I feel that we’re extra going towards intention-driven programming.”
Others, like Chelsea Troy, Chip Huyen, swyx, Birgitta Böckeler, and Gergely Orosz weren’t so certain. Don’t get me improper. They suppose that there’s a ton of wonderful stuff to do and be taught. However there’s additionally a whole lot of hype and free pondering. And whereas there might be a whole lot of change, a whole lot of present expertise will stay vital.
Right here’s Chelsea’s critique of the latest paper that claimed a 26% productiveness improve for builders utilizing generative AI.
If Chelsea will do a sermon each week within the Church of Don’t Consider All the things You Learn that consists of her displaying off varied papers and giving her dry and insightful perspective on how to consider them extra clearly, I’m so there.
I used to be a bit shocked by how skeptical Chip Huyen and swyx have been about A2A. They actually schooled me on the notion that the way forward for brokers is in direct AI-to-AI interactions. I’ve been of the opinion that having an AI agent work the user-facing interface of a distant web site is a throwback to display screen scraping—absolutely a transitional stage—and whereas calling an API might be one of the best ways to deal with a deterministic course of like fee, there might be a complete lot of different actions, like style matching, that are perfect for LLM to LLM. After I take into consideration AI searching for instance, I think about an agent that has discovered and remembered my tastes and preferences and particular targets speaking with an agent that is aware of and understands the stock of a service provider. However swyx and Chip weren’t shopping for it, at the very least not now. They suppose that’s a good distance off, given the present state of AI engineering. I used to be glad to have them convey me again to earth.
(For what it’s price, Gabriela de Queiroz, director of AI at Microsoft, agrees. On her episode of O’Reilly’s Generative AI within the Actual World podcast, she stated, “If you happen to suppose we’re near AGI, strive constructing an agent, and also you’ll see how far we’re from AGI.”)
Angie Jones, alternatively, was fairly enthusiastic about brokers in her lightning speak about how MCP is bringing the “mashup” period again to life. I used to be struck particularly by Angie’s feedback about MCP as a sort of common adapter, which abstracts away the underlying particulars of APIs, instruments, and knowledge sources. That was a robust echo of Microsoft’s platform dominance within the Home windows period, which in some ways started with the Win32 API, which abstracted away all of the underlying {hardware} such that software writers now not needed to write drivers for disk drives, printers, screens, or communications ports. I’d name {that a} energy transfer by Anthropic, apart from the blessing that they launched MCP as an open commonplace. Good for them!
Birgitta Böckeler talked frankly about how LLMs helped cut back cognitive load and helped suppose by way of a design. However a lot of our every day work is a poor match for AI: massive legacy codebases the place we alter extra code than we create, antiquated know-how stacks, poor suggestions loops. We nonetheless want code that’s easy and modular—that’s simpler for LLMs to grasp, in addition to people. We nonetheless want good suggestions loops that present us whether or not code is working (echoing Harper). We nonetheless want logical, analytical, vital excited about downside fixing. On the finish, she summarized each poles of the convention, saying we’d like cultures that reward each experimentation and skepticism.
Gergely Orosz weighed in on the continued significance of software program engineering. He talked briefly about books he was studying, beginning with Chip Huyen’s AI Engineering, however maybe the extra vital level got here a bit later: He held up a number of software program engineering classics, together with The Legendary Man-Month and Code Full. These books are a long time outdated, Gergely famous, however even with 50 years of software growth, the issues they describe are nonetheless with us. AI isn’t more likely to change that.
On this regard, I used to be struck by Camille Fournier’s assertion that managers like to see their senior builders utilizing AI instruments, as a result of they’ve the talents and judgment to get probably the most out of it, however typically need to take it away from junior builders who can use it too uncritically. Addy Osmani expressed the priority that primary expertise (“muscle reminiscence”) would degrade, each for junior and senior software program builders. (Juniors could by no means develop these expertise within the first place.) Addy’s remark was echoed by many others. No matter the way forward for computing holds, we nonetheless must know how one can analyze an issue, how to consider knowledge and knowledge buildings, how one can design, and how one can debug.
In that very same dialogue, Maxi Ferreira and Avi Flombaum introduced up the critique that LLMs will have a tendency to decide on the commonest languages and frameworks when making an attempt to unravel an issue, even when there are higher instruments accessible. It is a variation of the remark that LLMs by default have a tendency to provide a consensus resolution. However the dialogue highlighted for me that this represents a threat to ability acquisition and studying of up-and-coming builders too. It additionally made me marvel about the way forward for programming languages. Why develop new languages if there’s by no means going to be sufficient coaching knowledge for LLMs to make use of them?
Virtually all the audio system talked in regards to the significance of up-front design when programming with AI. Harper Reed stated that this appears like a return to waterfall, besides that the cycle is so quick. Clay Shirky as soon as noticed that waterfall growth “quantities to a pledge by all events to not be taught something whereas doing the precise work,” and that failure to be taught whereas doing has hampered numerous initiatives. But when AI codegen is waterfall with a quick studying cycle, that’s a really totally different mannequin. So this is a vital thread to drag on.
Lili Jiang’s closing emphasis that evals are way more complicated with LLMs actually resonated for me, and was in step with lots of the audio system’ takes about how a lot additional we have now to go. Lili in contrast a knowledge science undertaking she had performed at Quora, the place they began with a rigorously curated dataset (which made eval comparatively straightforward), with making an attempt to cope with self-driving algorithms at Waymo, the place you don’t begin out with “floor reality” and the proper reply is extremely context dependent. She requested, “How do you consider an LLM given such a excessive diploma of freedom when it comes to its output?” and identified that the code to do evals correctly will be as massive or bigger than the code used to form the precise performance.
This completely matches with my sense of why anybody imagining a programmer-free future is out of contact. AI makes some issues that was once onerous trivially straightforward and a few issues that was once straightforward a lot, a lot more durable. Even if you happen to had an LLM as decide doing the evals, there’s an terrible lot to be discovered.
I need to end with Kent Beck’s considerate perspective on how totally different mindsets are wanted at totally different levels within the evolution of a brand new market.
Lastly, an enormous THANK YOU to everybody who gave their time to be a part of our first AI Codecon occasion. Addy Osmani, you have been the right cohost. You’re educated, an amazing interviewer, charming, and a whole lot of enjoyable to work with. Gergely Orosz, Kent Beck, Camille Fournier, Avi Flombaum, Maxi Ferreira, Harper Reed, Jay Parikh, Birgitta Böckeler, Angie Jones, Craig McLuckie, Patty O’Callaghan, Chip Huyen, swyx Wang, Andrew Stellman, Iyanuoluwa Ajao, Nikola Balic, Brett Smith, Chelsea Troy, Lili Jiang—you all rocked. Thanks a lot for sharing your experience. Melissa Duffield, Julie Baron, Lisa LaRew, Keith Thompson, Yasmina Greco, Derek Hakim, Sasha Divitkina, and everybody else at O’Reilly who helped convey AI Codecon to life, thanks for all of the work you set in to make the occasion a hit. And because of the virtually 9,000 attendees who gave your time, your consideration, and your provocative questions within the chat.
Subscribe to our YouTube channel to observe highlights from the occasion or grow to be an O’Reilly member to observe your entire convention earlier than the following one September 9. We’d love to listen to what landed for you—tell us within the feedback.
