Adventures in Claude Code
Strengths and Weaknesses
I wanted to build a sophisticated demo application, with a database, an api server, and a web page. I’ve built at least 20 of these in my time, so this was nothing new for me. It would be mostly boilerplate stuff, with some specialized capabilities (which were the point of the demo).
Around the same time that I decided I wanted to build this, I got an email about Claude Code. I was told it would help me build my applications. I’ve had some experience with using Copilot to generate some simple functionality, so I decided it was worth the $20 to give Claude Code a try.
Beginning
I ask claude to build a solution. I gave it a straightforward prompt, pretty much what I would give to a jr developer who was helping me.
DB
It started with the db, and did an ok job designing the schema. It did a decent job extending the schema (adding, renaming and adjusting the type of columns) per my additional instructions. It even did a decent job creating sample data to fill in the schema with rows.
It made a couple of mistakes, some of which I caught immediately, some of which I didn’t catch until later. Claude actually caught one mistake (the way it created the primary key) hours later, when I described a use case.
API Server
It did a reasonable job with the API server, although it struggled more here. What I observe is that, much like a Jr programmer, it has no awareness of the complexities of enterprise software development. It kept using “clever” shortcuts, probably because most of its source data comes from guides and articles about coding for particular solutions, where (for space & readability reasons) adding all of the structure that’s useful for enterprise apps is actively avoided.
So it kept making Jr Programmer mistakes about how to build methods, how to set up authentication and authorization, how to return data in a thoughtful way.
The good news - it was quite quick - it usually had something that was about 90% correct in less than a minute. And since this was just a demo, I was able (with some effort) to let some of the bad practices go. So I was knocking out database models and services and controllers with ease.
API Server Unit Tests
The worst part were the unit tests I asked it to create. They were some of the most fragile tests I’ve ever seen. Again, probably because the source material is nothing but fragile tests that are demonstrating the capabilities of the testing framework, rather than thoughtful implementations. All in all, I spent more time trying to get the unit tests working than I did trying to get the code right.
At one point, Claude actively gave up on the unit tests and asked me if it could just disable them. So I spent a good hour fixing them.
Web App
I am not a front-end developer. I’ve written some React and lots of HTML, but it’s not my area of speciality. Claude did a decent job here, especially at the beginning. The stuff it created looked much better than anything I could produce, and generally had the functionality I needed.
Middle
Once I had the basics working, I started extending capabilities in a variety of ways that were unique to my demo. In other words, there wasn’t a lot of referenceable training data for Claude to use.
In this situation, Claude made lots of mistakes. Having said that, a Jr Programmer would also have made lots of mistakes here. The advantage of Claude over a Jr Programmer is that it made its mistakes over the course of 30 - 90 seconds, instead of 2 - 8 hours. Which makes it much easier to address right away. Claude struggled with authorization, with password hashing, and with spawning processes. But those were manageable problems.
As I worked through the code, I noticed a lot of duplicate code, which I asked it to refactor, and it did a reasonable job. The only problem was, every time I asked it to produce a new endpoint or service, it would re-duplicate that code, so I kept having to refactor it over and over again.
End
Once I had all of the structural stuff in place, I needed the final parts, which were the most specific to my problem domain (Policy as Code) and my particular vision of something fairly innovative. Claude was almost helpless here, because it just didn’t have anything referenceable for my scenarios. Jr Programmers would have been equally helpless here, so that’s not really Claude’s fault.
Implications
All in all, this was a better experience than trying to ride herd over some Jr programmers. Claude is fast, resourceful and enthusiastic. It is also sometimes hilariously wrong, but then, so are Jr programmers.
The big difference is that Jr Programmers will learn from their mistakes, but Claude will not, at least not in any consistent and/or focused way.
I will not be working forever. How do we train new Sr Developers if we use Claude instead of Jr Developers?
Claude, et al, will save time by producing all of the tedious boilerplate, but it is simply not capable of being trusted with a scalable solution that has even the slightest bit of novelty. If you want to build a pet shop API, Claude will probably build the scaffolding of a decent prototype. But if you want to build a rock-solid, scalable, maintainable, secure pet shop? You’ll have to hire Sr Developers to identify all of the rookie mistakes.
To be clear, once you tell Claude where it has made mistakes, it can generally fix all instances of that mistake in a minute or two. As long as you didn’t have it build tests, because almost inevitably, the tests are written in such a way that fixing the mistake breaks the tests
How To Train Your Jr Programmers
I think the right way to fix this is the way that med students gather around a doctor performing surgery in an operating theater. Have the Sr Programmer working with Claude, and share his screen(s). Have the Sr Programmer keep a running dialog of what he is doing, and when Claude produces code that is suboptimal, have the Sr Programmer explain *why* the code is suboptimal. Explain some of the theory and expertise that makes the Sr Programmer make this assessment. Allow the Jrs to ask questions along the way.
The challenge here is that the Jr Programmers aren’t providing a lot of value if they’re just watching Sr Programmers talk to Claude. But they will likely ask useful questions, and potentially remind the Sr of side effects or edge cases. They might also be useful in, after the code is written, writing actually useful test cases that aren’t ridiculously fragile.
The other benefit is that your team will have a better understanding of the overall code architecture. They will (in theory) be better prepared to take on more sophisticated tasks faster than if they were thrown into a subsystem with an hour of overview.
I can’t promise that this will work, and there’s always a danger that, if you hire 5 Jr Programmers, and use this method to teach them how to be Sr Programmers, 4 of them will leave for other opportunities. But that’s generally a problem now as well.
Conclusion
What do you think? Any interesting thoughts on how to avoid the collapse of the Jr Developer market? I would love your ideas: johnbr@paclabs.net