405 is a lot of B's. What does it take to run or host that? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tpurves on July 23, 2024 \| parent \| context \| favorite \| on: Open source AI is the path forward 405 is a lot of B's. What does it take to run or host that?

danielmarkbruce on July 23, 2024 [–]

quantize to 0 bit. Run on a potato.

Jokes aside ~ 405b x 2 bytes of memory (FP16), so say 810 gigs, maybe 1000 gigs or so required in reality, need maybe 2 aws p5 instances?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact