AirLLM 70B inference with single 4GB GPU

AirLLM enables running large language models up to 70 billion parameters on a single 4GB GPU without traditional compression techniques. It supports even larger models like the 2.8 trillion parameter Kimi K3 on under 4GB by streaming sparse MoE experts individually.