The next age of LLMs? Dev gets a small LLM running at 10 tokens a second locally on a $10 microcontroller

Developer slvDev created a 28.9M-parameter language model running offline at nearly 10 tokens/sec on a $10 ESP32-S3 microcontroller by storing most weights in flash memory and using 4-bit quantization. The model, trained on Microsoft's TinyStories dataset, operates within the chip's limited RAM and flash constraints, demonstrating efficient LLM deployment on minimal hardware.