Machine Learning Chips & Compute14 min reading time

Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio

Apple Research BlogPublished 11 days ago

MLX on Apple silicon Mac Studio enables local and distributed inference of large language models, including the 816B-parameter Muse Spark Open, achieving over 36 tokens per second on a four-Mac cluster using RDMA over Thunderbolt.

#apple-silicon #distributed-inference #large-language-models #mac-studio #mlx #muse-spark-open

Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio

More in Machine Learning

Jack Dorsey’s Block made an AI model that talks to you like an awkward new friend

AI21 Labs Releases LIFT-Bench to Curb Reward Hacking in RL Training

OpenAI Resumes Training AI, But Without Internet Access