Thinking Machines Lab launches Inkling 2 Mini with 1M-token context for local AI

What happened
Thinking Machines Lab has launched Inkling 2 Mini, a new multimodal transformer model designed for local use on consumer devices. The model features a 26B-parameter mixture-of-experts architecture with a unique looping layer design, allowing it to handle both image and text inputs and process up to a 1 million token context. Inkling 2 Mini is available with open weights and supports various local inference frameworks, including transformers, llama.cpp, and MLX. The release also includes a research variant that allows users to adjust the number of loop passes for faster inference at the cost of output quality.
Key details
- Inkling 2 Mini has 26.4 billion parameters, with 3 billion active per token.
- The model uses 24 decoder layers, with the middle 12 layers looping three times, simulating 48 logical layers.
- It supports a 1 million token context window and accepts both images and text as input.
- Optimized for local AI setups, it can run on laptops with about 20GB of memory.
- Available on Hugging Face, it supports multiple local inference options and includes a DSpark drafter for speed-up.
Why it matters
Developers and researchers can now run a high-capacity, multimodal AI model directly on their local machines without relying on cloud services. This enables more private, customizable, and cost-effective AI applications for coding, research, and vision tasks. The open-weights release and compatibility with popular inference tools lower the barrier for individuals and small teams to experiment with advanced AI capabilities.

