I am going to admit something rather embarassing in this blog post. However, I expect that quite a few people have had this experience and don’t want to talk about it. My embarassing admission? I was robbed by my AI Copilot.

I don’t know how much I trust these numbers, but a cursory Google search shows that over 90% of software engineers are using some sort of Generative AI to assist them in writing code. I count myself in that group. At first, I was reluctant to use it too much and, frankly, up until about 6 months ago, the output of these tools was pretty bad. I generally only used it when I was prototyping or to fill in methods or boiler plate. It was small isolated changes I could easily review and fix by hand.

This year, a few things happened that changed the way I utilize AI in my day to day software engineering life. First, I failed a job interview because because of AI. During the interview, I was given a challenge that was obviously too big to complete in the alloted time. I was told I could use any tool I wanted, including AI. As an interviewer, If I gave problem that is obviously big for the alotted time it’s because I wanted to see how the candidate thought through the problem - not because I wanted to see a completed solution. So that is how I approached this particular interview problem. I explained my reasoning, I used AI to help fill in methods and such, but I was the one using my brain and engineering experience to come up with the solution. I learned later that this was not what the interviewer was looking for. They did not want to test the engineering skills that I had honed over the years. Instead they were interested in how quickly and efficiently I could produce code with the help of generative AI.

This was an eye-opening experience for me. While I don’t believe this has become the norm for software engineering interviews, I would not be surprised if it is becomming more common. There are a large number of businesses that rely on tech but are not necessarily “tech” companies. All of the AI companies have been selling the idea that you can use AI to dramatically increase your output while using fewer engineers. For companies that need some custom tech but are not really positioning themselves as “tech” companies why wouldnt they lean into this? It makes sense to me why they might start interviewing for the abiltiy to ship large amounts of code via AI instead of the more traditional path of trying to evaluate the overall skill level of the engineer as a proxy for their future ability to ship code.

Im not one who likes to be left behind by tech, so after this interview I made an effort to lean into the AI copilot a bit more than I had been. I increasinly used it in different ways and for increasingly complex tasks. I found that the newer generations of LLMs where far more capable than they ever had been. The code produced, while not what I would have written, was pretty solid most of the time. I noticed it would get lost a bit in bigger projects and the solutions tended not to consider the project as a whole. However, I never went as far as vibe coding anything. I always worked in small chunks and carefully reviewed the output.

A little later, a friend asked me to help him with a project hed been working on. He was not an engineer, but was using Replit to build out his idea. He had built a huge industry specific CRM using vibe coding alone. He was getting stuck trying to implement new features - nothing worked out the way he was expecting anymore and he said he spent most of his time arguing with the Replit Agent. When I looked under the hood, I realized the project was way to large for me to quickly get a handle on what was going on by viewing code alone. So, I engaged the help of my ai co-pilot. I asked very specific questions and had it examine the source code and produce summaries and point me to the files that contained the code I was looking for. Using this method I was able to quickly break down the project and get a handle on why things were not working for him. The agent had created 3 parallel authentication and authorization systems and was confusing itself and my friend. He asked me to help him get back to a state where he could continue working. There was no way I had the time to be able to fix up the problem in my normal coding style. I had to lean into vibe coding. I had a long conversation with the AI agent working out a design and the changes needed. The design was specified in descriptive text and ascii art diagrams. Then, I let the agent write the code. Somehow it managed to produce something reasonable and after a few refinement passes I was able to hand the project back to my friend so he could continue.

From this experience, I started to understand both the power and dangers of vibe coding. I started using vibe coding for testing out new ideas in my personal projects. Being able to have a random idea pop into my head and then vibe code a prototype in a single afternoon is quite a powerful tool. I started using it more often to experiment with different aproaches to solving a problem before moving forward. It became one more tool in my AI assisted coding toolbox.

Recently at work, I needed to make a change to the session management system. This is not something that I really trust AI to do so I decided to do the change 100% by hand. When I tried to get started, my first impuls was to flip the co-pilot into ask mode and start exploring various ways of solving the problem. I found when I was just thinking through things on my own, it was far more painful than I remember. My brain did not want to work. It had gotten used to evaluating options, not producing them from noting. Things got even harder when actually coding. Refactoring existing code, building new functions, writing tests were all painfully difficlut and slow. Writing code completely un-aided had become something unfamilar. I was shocked and quite embarassed. I always prided myself on my ability to solve problems in code and produce nice, clean code. This seemed to be a skill that had dramatically diminished in the span of only a few months. It went beyond just being a little rusty. It felt like learning to code all over again. It genuinely took me a couple days of doing tasks purely by hand to start to feel more comfortable again.

I find the whole experience rather disconcerting. In the course of a few months, my brain had re-wired itself to expect the AI co-pilot to be there. It had embraced the easy way. I still reviewd everythign and made corrections by hand, but without realizing it I had offloaded an entire part of software engineering onto the AI. The part that can actually turn thoughts into code. I can’t say I really gained anything from this tradeoff. It still took a while to produce the code - I was just typing thoughts into the AI chat, discussing soltion options, then pushing the AI to generate code in the way I would like it to be written. While that often took less time than hand writing all the code, I lost something along the way. Actual experience writing code.

In spoken languages, there are two parts to learning a foreign language - comprehension and output. They are actually very different skills handled by different areas of the brain. There is this phenomenon that can happen when you learn a language simply by listening but never outputing the language. You can become fluent in comprehension, but unable to speak. You can hear and fully understand what someone says to you in the second language, but when it comes time to respond, only a few words come out. No sentences, no proper grammar - just sporadic words. It seems that a similar thing happens with coding. When you spend most of your time review code instead of thinking through the problem and producing code on your own, your ability to write code starts to decline. When you are forced to write code, you find you struggle more than you expect.

I have many fresh concerns after this experience. Firstly is the decline of existing tallent. The stats say, if you are a software engineer today, you are likely using generative AI in your work in some way. Thats a lot of pressure. If everyone around you is using generative AI in their work, and you are not - you risk being perceived as being outpaced and potentially replaced. Thats a lot of pressure to adopt AI in your workflow. How many skilled software engineers are unknowingly having their skills dulled by the way they are using generative AI in their workflows?

Secondly, I am concerned for the younger generation of coders. People who leared software engineering in the age of AI co-pilots are at risk of missing out on something I believe is vital to all software engineers - actual experience solving problems with code. When you are forced to think through problems and write code, have it reviewd by your peers, make mistakes, see your mistakes cause real problems first hand, etc. You build up a body of experience that you can’t really articulate directly, but is there guiding your decisions. Its your “gut” for lack of a better term. All software engineers build up these instincts via hands on software engineering. In a world with the AI is doing most of the code generation, are software engineers really going to build up that “gut”? With the experince of only building code with an AI co-pilot, will they be able to know when certain design patterns are going to work in the long run? The co-pilot doesnt know, its just giving you an average answer or presenting many options. Its up to the engineer to steer the AI in the direction that makes the most sense for the project as a whole.

I can’t really say for sure that either of these things are going to manifest into true problems. We may just be on the verge of a new era of software engineering. Its possible that developing that “gut” instinct or being skilled at producing code are going to become irrelevant in the future. But because we don’t yet know, perhapse we should move with a little caution. We should take it upon ourselves to prevent our existing coding skills frun dulling and actively try to sharpen them.. This means putting the AI co-pilot away now and again. And I dont mean, just fill in a single method without the AI, I mean do an entire task AI free. It also means being mindful of how we use AI in our workflows. AI co-pilots are a great tool, and like most tools they serve a specific purpose. We should be mindful of that addage “everything looks like a nail when the only tool you have is a hammer”. Let’s keep using AI in a way that makes us better, more efficient engineers, instead of delegating our jobs to the co-pilot.