How exactly do you backup an AI model anyway? How big is an AI model? Is it just a big zip file? ????
this post was submitted on 23 Nov 2023
3 points (100.0% liked)
Data Hoarder
11 readers
1 users here now
We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.
founded 1 year ago
MODERATORS
-
You simply download the file and keep it. We're talking about stuff you can run on your own computer, not services like chatgpt or such.
-
Highly variable. Depends on model size and format. For LLMs, range would be from under 1GB up to a current maximum of ~240GB.
-
It's a big file of numbers ("weights" or "parameters") either integers or floating point depending on format.
Anywhere from 1 to several hundred GB. Quantized (compressed), the most popular models are 8-40gb each. LORAs are a lot smaller, but full models take up a lot of space.
Someone make a torrent and ill archive them for sure