I was thinking that I cannot be the only one who sees such parallels between AI and the story of The Emperor’s New Clothes, and I was correct. Two years ago, in The Emperor’s New AI, Tabrez Syed pointed out that one-fourth of organizations reported negative consequences due to AI inaccuracy. There is the MIT study that found 95% of AI project yielded zero return. On the flip side, a paper by a different group at MIT finds that AI does increase developer productivity.
LinkedIn is full of posts saying AI made me a 5x developer, and yet …
I’m not in the mood to call out any specific person, but you can search on 5x developer, “I made $10,000 in a week with vibe coding”, “AI is going to replace your job” or whatever hype phrase that strikes your fancy. Anyone who challenges this in the comments is told that they are doing it wrong, are going to be replaced by a developer who uses AI, just can’t accept that “anyone can now code.” It makes me tired to list the responses.
Like the emperor’s new clothes, many people are hesitant to call bullshit because they are afraid of looking stupid or not up on the latest tech.
Who are you going to believe, me or your own eyes?
I feel obligated to preface this with I do use Gemini Pro, Copilot and Adobe Firefly (which uses nano banana) fairly often. There are good use cases for AI, but that’s the topic of another post.
As someone who writes code on a near-daily basis and has been programming since punched cards, through FORTRAN 77, BASIC and now mostly JavaScript, I truly do not see any 5x improvement. What I do see on a daily basis are incorrect answers.
- Example: I wanted to clone a private repository from github. I asked Gemini Pro how to do this and it gave me a couple of Unix commands. I first had to look up my github password, which I had saved, then, I opened the terminal, pasted these commands in and it did not work. I pasted the error code back into Gemini and got the response “You’re right. Those commands will not work.”
- Example: I read the WebStorms documentation and used the Git menu in WebStorms to clone the repository but for some reason it did not show the latest changes. I asked Copilot and got several reasonable suggestions – had I forgotten to commit those changes? Had I committed and not pushed? It gave me several ways to verify that the local and remote branches were the same. In the end, none of it worked and, just for the hell of it, I did a pull request, even though I had just cloned the repository, and that worked.
These are just two, simple examples on a random day. I did not cherry-pick these. In this case, AI definitely took more time than just looking it up in the documentation or trying random ideas without AI suggesting those.
Some days, I use AI hardly at all because I’m very familiar with our code base and if I am using 7 Gen Blocks to create a new game, I rarely need to look anything up. Either the code just works, or, if there is an error, I can look in the console and see it says something like “Cannot find promise_resolve.js” and immediately guess that the drag.js file imports promise_resolve.js and I forgot to copy over that file.
I spent several months working on creating an AI agent to write code to make games and we ended up being in that over 40% of companies that scrapped their AI initiatives. For some of this project, I used Python, which is not a language I had used extensively and when I would try debugging, the time and repeated steps it took to get a correct answer was often laughably bad. There were three reasons we abandoned this project. First of all, if we had integrated Gemini and Vertex AI into our product, we’d be at the mercy of whatever Google decided to charge in the future, which didn’t seem a very robust business model. Second, we would not own 100% of the code. Third, the AI agent was not 100% accurate. It asked the user questions like, “A good target is one the player should click on to earn points. A bad target is a distractor they should not click on.What type of target is the first image, good or bad?” and maybe 85% of the time or more, it wrote the correct code. Sometimes, it didn’t.
We scrapped the project because we realized that we could write it ourselves without using AI, with a lower degree of complexity (easier to maintain), at a lower cost. The “negative” AI study from MIT, “The state of AI in business 2025” stated
(Myth 4):The biggest thing holding back AI is model quality, legal, data, risk → What’s really holding it back is that most AI tools don’t learn and don’t integrate well into workflows.”
- Challapally, et al. , 2025
We certainly found that to be true. We used retrieval augmented generation, writing and uploading extensive documentation on our codebase and that improved the AI-generated code initially, but no matter how much we used it, errors still popped up. We needed to keep specifying the context and adding to the instructions to the point it seemed it would be just as easy to code the platform ourselves. So, we did.
TL; DR; AI doesn’t always work for the simple tasks nor for the hard ones.
There is a vast gap between “AI works sometimes” and “AI made me a 5x developer”
There are specific tasks on which I have found Gemini to be a huge timesaver, like writing Google apps scripts – but that is something I might need every other week, not daily.
The biggest misconception I see in all of the AI hype is that software development is mostly typing code. When I read that “Developers will be doing less coding and more defining requirements, breaking tasks into small steps, writing tests and debugging”, I wonder how the hell those people think most developers are spending their time now? If you’re in a senior position, you’ve probably had plenty of experience with junior developers who ask you so many questions that you’d be just as fast doing the project yourself without them. However, as we all know, those junior developers learn from the tasks and eventually are much better.
My point, and you by now may have despaired of me having one, is that we are all seeing that AI is often like a junior developer that needs specific direction, gets answers wrong and always promising to get better but never does. And yet, we are supposed to pretend that we are all just “prompting it wrong” or “just not using the right AI.”
In the meantime, I’m waiting to see the record profits or lower product costs from all of these 5x and 10x developer companies.
I use Ada, the premier language for safety-critical software applications. There is a heavy focus on data precision. For example, we rarely use predefined data types such as int or float. Instead we design our own data types with exact upper and lower bounds.
We have many more language features to ensure run-time safety.
Safe software must be engineered to predictable performance and exacting specifications.
I don’t see any AI tools that are ready — that I would trust — for creating the avionics source code for fly-by-wire avionics yet.
I heard a very good comment at ASU-GSV last year. The presenter (president of The Learning Co) said something like “AI is acceptable in situations where some degree of inaccuracy is acceptable.” For example, I know many teachers who use AI to create distractors for multiple choice test questions. In that case, the teacher can look at the 25 questions generated, delete the three that are not appropriate or correct and select 10 from the remainder for the test. In the case of aircraft, a 12% failure rate is a catastrophe.