InfoQ reported on November 13, 2026, that Google had introduced LiteRT Tensor API, a lightweight C++ API for building, customizing and optimizing on-device machine-learning models.
The API lets developers assemble a computation graph operator by operator in C++. According to Google, the same graph can run through LiteRT on CPUs, GPUs and NPUs without being rewritten for each hardware target. The company also says the API supports loading weights directly from checkpoints, including Hugging Face .safetensors files, and sharing accelerator buffers across preprocessing, inference and postprocessing without copying them.
Google used a coding agent to build a multi-object segmentation and tracking demo. In the company’s tests, the same SAM 2 C++ code ran on a macOS laptop and an iOS phone using Metal, and an Android phone using OpenCL. Google reported that its MacBook Pro run was about 1.5 times faster than Apple MLX for the fp16 image encoder and mask decoder. That is a company-reported result, not a general performance guarantee.
Arm AI director Laurence Moroney advised developers to start with model conversion and consider the C++ API when they need graph-level control or want preprocessing and postprocessing to remain on the accelerator. Google also outlined planned improvements, including native ML Drift backend support, more NPU-friendly operations, automatic full-graph quantization and broader dynamic-shape support.