Thinking Machines Lab launches Inkling 2 Mini with 1M-token context for local AI

Covered by 2 sources
Read onHugging Face· 16 min
Thinking Machines Lab introduced Inkling 2 Mini, a 26B-parameter multimodal transformer model with a looping mixture-of-experts architecture enabling a 1 million token context window and image-text input. The model is optimized for local use on consumer laptops with around 20GB of memory and is available on Hugging Face with various inference options.

What happened

Thinking Machines Lab has launched Inkling 2 Mini, a new multimodal transformer model designed for local use on consumer devices. The model features a 26B-parameter mixture-of-experts architecture with a unique looping layer design, allowing it to handle both image and text inputs and process up to a 1 million token context. Inkling 2 Mini is available with open weights and supports various local inference frameworks, including transformers, llama.cpp, and MLX. The release also includes a research variant that allows users to adjust the number of loop passes for faster inference at the cost of output quality.

Key details

  • Inkling 2 Mini has 26.4 billion parameters, with 3 billion active per token.
  • The model uses 24 decoder layers, with the middle 12 layers looping three times, simulating 48 logical layers.
  • It supports a 1 million token context window and accepts both images and text as input.
  • Optimized for local AI setups, it can run on laptops with about 20GB of memory.
  • Available on Hugging Face, it supports multiple local inference options and includes a DSpark drafter for speed-up.

Why it matters

Developers and researchers can now run a high-capacity, multimodal AI model directly on their local machines without relying on cloud services. This enables more private, customizable, and cost-effective AI applications for coding, research, and vision tasks. The open-weights release and compatibility with popular inference tools lower the barrier for individuals and small teams to experiment with advanced AI capabilities.

Also covered by 1 source


More in Machine Learning