AI Erodes Knowledge

If one has an LLM “do the work,” then one does not actually learn anything.  Taking notes improves recall, and so does trying to remember the answer before looking something up.  In any case, to become more proficient, something has to happen inside the mind. If we skip that part in our rush to produce volume of output, we are trading away our future skills.

“Not gaining proficiency” would be indictment enough, but worse, unused skills decay. There is a reason that C is not on my résumé anymore!  I can’t imagine what would become of my PHP if I didn’t do the coding, and I utterly dread the potential outcome of using an LLM to make changes for an entire month.

An LLM will never produce original work, since it is trying to “predict” based on a corpus of past, public work.  It’s great for writing code like fanboys on the internet write code.  It’s not so great at cost optimization.

Overall, I wonder about how institutional knowledge might be affected by heavier LLM usage.  If we’re not doing the thinking, will we be able to remember anything about our own history?

Early Thoughts on Coding with an LLM

I temporarily displaced my misgivings about the Plagiarism Machine That Also Destroys Earth.  Since I haven’t yet spent a full week working with it, I have some first impressions only:

1. All generators have the highest accuracy on the smallest tasks, where they are the least useful.
2. The tools all seem to prefer the same model, so there’s less differentiation than one might hope for. Nothing is worse… but nothing is better.
3. Writing a good prompt takes a lot of planning.
4. Every change must be tediously reviewed.  This includes every tab-completable inline suggestion in the editor.
5. Junie is not actually integrated very deeply into PHPStorm.
6. Agent mode is full-on Sorcerer’s Apprentice.  One must always be standing by on the “Emergency Stop – Never Use” button.

Due to homogenous model choices, UI is an important differentiator right now.  Using Control+Backslash to generate code in PHPStorm makes for results that are difficult for me to understand, because the diff is in character mode.  Cursor is much better at this, with line diffs.

As for Junie’s lack of integration, it failed to recognize «run the "foo api unit tests"» as an instruction to run the pre-existing test configuration named “foo api unit tests”.  I let it try running foo api unit tests in the terminal to see what it would do.  After getting a command-not-found error, it attempted to find any test that it could run, and try running that instead.  Fortunately, it needed permission to run further off the rails, so I denied it.

Summary

As for the overall experience…

The LLM removes the fun part of programming, the writing of the code, leaving the planning and debugging parts I am less fond of.  The incessant demands for attention from inline suggestions also fundamentally block entering a flow state. Meanwhile, hallucinations are always trying to stab me in the back; there is absolutely no meta-analysis of whether the prompt itself is misguided.

I don’t even see these tools as useful for exploring unfamiliar code.  My IDE already has a set of tools for that; like Find in Files, Go to Definition, and Find References/Usages.  Since these aren’t constrained to a sidebar, the results are also far more usable.

Even if it had no downsides, at first blush, the (paid) systems still rate a solid “meh.”

Podman Desktop Isn’t Great

Since buying a new computer, my primary desktop is no longer Linux, and containers are no longer native.  I decided that the path of least resistance would be to try Podman Desktop, but it leaves a lot to be desired.

My Experience with Switching from Psalm to PHPStan

Due to Psalm’s lack of support for being installed with PHP 8.4 or PHPUnit 11 at the time (January 15, 2025, prior to the Psalm 6.0 release), I finally gave PHPStan a try.

The big difference that has caused the most trouble is that PHPStan wants iterable/container types to explicitly document their contents.  Any time a method returns array, PHPStan wants to know, array of what? Psalm was happy to observe what the method put in the array for return, and use that de facto type as the developer’s intention.

Outside of the smallest library repositories, that rule got ignored.  It is responsible for maybe 75% of issue reports.  If I can take it from 1200 down to 275 with a single ignore, that is the difference between “there are too many things to deal with” and “I can make a dent in this today.”

The next obvious difference has been that PHPStan is much more interested in handling the potential false returned from preg_replace('/\\W+/', '', $str); calls.  Psalm expected that giving three string arguments to preg_replace() will always result in a string of some sort.

There’s also a class of issues reported by PHPStan due to a disagreement in the other direction. Psalm seemed to think that number_format() returned string|false, requiring an is_numeric() check on the variable.  PHPStan thinks that is redundant, i.e. that number_format() has already returned a numeric-string.

I don’t have a sense yet for how effective PHPStan is at finding problems overall.  In code that was previously checked with Psalm, many defects visible to static analysis have already been removed, leaving little fruit behind for PHPStan to pick.

As of early February, PHPStan can be considered a successful migration.  I haven’t touched PHPStan Pro, but I may try it if I ever want to fix those hundreds of issues with array types.