Koboldcpp should allow you to run much larger models with a little bit of ram offloading. There’s a fork that supports rocm for AMD cards: https://github.com/YellowRoseCx/koboldcpp-rocm
Make sure to use quantized models for the best performace, q4k_M being the standard.
I’m not sure why it would be any different from how this is treated with search engines. Both scrape massive amounts of openly available data and make it available in some form. Any training data or information that a model could potentially spit out is already available through a search engine’s index.
In the case of Machine learning the term has sort of been morphed to mean “open weights” as well as open inference/training code. To me the OSI is just being elitist and gate keeping the term.
That isn’t neccesarily true, though for now there’s no way to tell since they’ve yet to release their code. If the timeline is anything like their last paper it will be out around a month after publication, which will be Nov 20th.
There have been similar papers for confusing image classification models, not sure how successful they’ve been IRL.
They could have added their own repos which is the concern here.
I’ve used the tplink ones that they’re using and they’ve been pretty solid. I can’t say how they’d fare in a 24/7 setup though since they’re not really intended for that.
No I’d say that it has more to do with improved usability and better design overall making them unable to fix issues when they do occur. There isn’t one specific company or system to blame. Nearly everything has, for better or for worse, been boiled down into a webapp where there is minimal potential for error.
It’s also not really fair to compare gen z to Millenials as Millennials have had nearly twice as much time to figure things out.
LLMs only predict the next token. Sometimes those predictions are correct, sometimes they’re incorrect. Larger models trained on a greater number of examples make better predictions, but they are always just predictions. This is why incorrect responses often sound plausable even if logically they don’t make sense.
Fixing hallucinations is more about decreasing inaccuracies rather than fixing an actual problem with the model itself.
Cohere’s command-r models are trained for exactly this type of task. The real struggle is finding a way to feed relevant sources into the model. There are plenty of projects that have attempted it but few can do more than pulling the first few search results.