I hate to say it but, I have to agree. GPT4 is a significant improvement over GPT3. I needed to use a Python library for something that was meant to be a small, simple CLI app. It turned into something bigger and accumulated technical debt. Eventually, I was having problems that were niche and hard to trace, even with logging and all the other approaches.

I eventually said fuck it, and so I threw a shit tonne of my code into it, explaining what I was doing, how I was doing it, why I wasn't doing it another way, and what I expected vs the actual result. Sometimes it suggests something that is on the right path or is entirely spot on. Other times, it thinks it knows better than you, to which you tell yourself it isn't, because you tried all its suggestions, and then you realise something that would technically allow GPT to say, "I told you so", but out of spite you just close the tab until the next issue.

For practical tasks, GPT has come pretty far. For technical ones, it is hit or miss, but it can give you some sound advice in place of a solution, sometimes.

I had another issue involving Matplotlib, converting to and from coordinate systems, and having plots that had artifacts due to something not quite right. The atan2 function catches many people out, but I'm experienced enough to know better..... Well, normally. In this particular case, it was a complex situation and I could not reason why the result was distorted. Spending hours with GPT4 lead me in circles. Sometimes it would tell me to do things I just said I did, or that I said don't work. Then, I say to it, "what if we represent this system of parametric equations as a single complex-valued function, instead of dealing with Cartesian to polar conversations?". Then it would zip up a whole lot of math (related to my problem). The damn thing handed me a solution and a half. In theory, it was a great solution. In practice, my code is illiterate, so it doesn't care.

All in all, while it failed to help me solve my issue, it was able to reason and provide feedback to a wide range of challenges. Sometimes it needed prompting to change the trajectory it intends to follow, and this is the part you need to learn as a skill. Until these LLMs are more capable of thinking for themselves. Give it time.