• 1 Post
  • 17 Comments
Joined 1 year ago
cake
Cake day: August 8th, 2023

help-circle



  • There’s plenty of open source projects that distribute executables (i.e. all that use compiled languages). The projects just provide checksums, ensure their builds are reproducible, or provide some other method to verify.

    In practice, you’re going to wind up in dependency hell before pypi stops hosting the package. E.g. you need to use package A and package B, but package A depends on v1 of package C, and package B depends on v2 of package C.

    And you don’t need to use pypi or pip at all. You could just download the code and directly from tbe repo, import it into your project (possibly needing to build if it has binary components). However, if it was on pypi before, then the source repo likely had all the code pip needs to install it (i.e. contains setup.py and any related files).










  • IDK, looks like 48GB cloud pricing would be 0.35/hr => $255/month. Used 3090s go for $700. Two 3090s would give you 48GB of VRAM, and cost $1400 (I’m assuming you can do “model-parallel” will Llama; never tried running an LLM, but it should be possible and work well). So, the break-even point would be <6 months. Hmm, but if Severless works well, that could be pretty cheap. Would probably take a few minutes to process and load a ~48GB model every cold start though?





  • It’s also trained on data people reasonably expected would be private (private github repos, Adobe creative cloud, etc). Even if it was just public data, it can still be dangerous. I.e. It could be possible to give an LLM a prompt like, “give me a list of climate activists, their addresses, and their employers” if it was trained on this data or was good at “browsing” on its own. That’s currently not possible due to the guardrails on most models, and I’m guessing they try to avoid training on personal data that’s public, but a government agency could make an LLM without these guardrails. That data could be public, but would take a person quite a bit of work to track down compared to the ease and efficiency of just asking an LLM.