SiMa.ai’s Palette 3.0 boosts AI chip throughput by up to 3.5 times

Covered by 2 sources
Read onSiliconANGLE· 4 min
SiMa.ai introduced Palette 3.0, software that allows its Modalix AI chip to run multiple AI models simultaneously, increasing processing speed by as much as 3.5 times. This upgrade supports large language and vision-language models for autonomous machines such as drones and robots.

What happened

SiMa.ai released Palette 3.0, a new version of its software suite designed for its Modalix AI chips. The update enables the chips to run several AI models at once and increases processing speed by up to 3.5 times. This software is aimed at developers working on autonomous machines such as drones and robots, supporting both large language models and vision-language models. The new version also introduces open-source tools for generative AI applications.

Key details

  • Palette 3.0 allows concurrent execution of multiple AI models on a single Modalix MLSoC chip.
  • The software uses speculative decoding methods to achieve up to 3.5 times faster language model generation.
  • It supports large mixture-of-experts models, including GPT-OSS-20B, on the Modalix MLSoC 16GB system-on-module.
  • Developers can access open-source generative AI tools and adapt the code via GitHub.
  • The update maintains similar power consumption while increasing throughput.

Why it matters

Companies and developers building autonomous machines, such as drones and robots, can now process more complex AI tasks on-device without increasing power usage. This reduces the need for cloud-based processing, potentially lowering costs and improving real-time performance. The ability to run multiple models and large language models simultaneously on a single chip gives manufacturers more flexibility and capability in designing advanced autonomous systems.

Also covered by 1 source


More in Robotics