Trusting LLM-written code

There's no shortage of AI detractors out there on the internet these days.

I'm not one of them. Not when it comes to writing source code, at least.

AI has been good enough to write source code for quite some time, in my view. Everything above one of the ChatGPT4 minor versions (maybe 4.2? I forget) has been more than good enough for my purposes.

In terms of pure code generation, it's as good at (or better) than most developers I've ever worked with. I'm using it on a daily basis to handle all sorts of tasks that I'd either have to take a few hours to write myself, or hand off to a professional coder. A short sampling:

• Scripts for sorting out and regularizing metadata in Kicad libraries

• Django plugins for tracking and scheduling orders and shipments

• Bare metal firmware for interfacing with I2C light sensors (Did you know that AI has an intimate knowledge of ARM's CMSIS? It does, and I'm mostly convinced this will kill most vendor HALs.)

A well written prompt got many of these things working for me at a first pass. A well phrased use case, and some prompt coaching from the LLM, is usually enough to get to that. After the codegen is done, a few minutes of testing is usually enough to help me decide whether or not I need a second pass.

I'd estimate that, 80% of the time, I don't need a second pass. For the most part, the code just works. In the cases where it doesn't, it's usually trivial to get it to a state that meets my needs.

Does this mean it's time to replace software developers? Absolutely not. I've worked in my field long enough to know that developing software is far more than just pure code generation.

I am, however, starting to suspect that AI deniers in the software field are playing the game with one hand tied behind their back.