Yeah but they don’t use LLMs for this, they’ll use some other kind of machine learning mixed in a big pipeline of data processing. It makes it really hard to guess how much work it would take to fix. It might require retraining, might just require an easy patch of the rest of the pipeline.
My guess is that they’re just shitty jumpers and there’s nothing to fix anyway.
I just went looking for this. Here’s a link to the podcast for everyone else: https://pca.st/episode/b8388458-0062-47c5-a259-fae295a45305