PrismML says Bonsai 27B runs on iPhone

PrismML has released Bonsai 27B, a new model the startup says runs natively on iPhone, iPad, and Mac. The launch comes days after The Information reported that Apple had held meetings with the company about possible uses for its technology, and after CEO Babak Hassibi told CNBC that Apple is now evaluating PrismML’s models.

According to PrismML, Bonsai 27B reaches up to 163 tok/s in 1-bit and 134 tok/s in Ternary on an NVIDIA GeForce RTX 5090. On an M5 Max, it reaches up to 87 tok/s in 1-bit and 58 tok/s in Ternary.

The company argues that storage alone does not determine whether a model can run on a phone. It says a 12 GB iPhone exposes only about 6 GB to an app for on-device use, and that budget must also cover the model’s KV cache and activations. PrismML says that at about 4 GB, its 1-bit Bonsai 27B is the first conventional 27B-class model to fit with enough headroom to operate.

In its press release, PrismML said the model runs on Apple devices through MLX and on NVIDIA GPUs through CUDA, using custom low-bit kernels built for its hybrid-attention architecture. The model weights are available now under the Apache 2.0 License, and the company is also offering a free, limited-time developer preview API.

Hassibi told CNBC that Apple and other companies have been testing PrismML’s models for speed, energy efficiency, and on-device performance.

Recommended reading

Open models are overtaking AI’s front line

“They’re really evaluating our technology right now.”

He described the talks with Apple as very early and said it is still unclear where they may lead, adding that “things are progressing nicely.” Apple did not immediately respond to CNBC’s request for comment.