Browser-based machine learning has spent years running on a compromise. TensorFlow.js let developers run models client-side, but its JavaScript kernels never came close to the performance of native mobile runtimes.
Google introduced LiteRT.js on July 9, bringing its LiteRT on-device inference engine to the browser through WebAssembly. The JavaScript binding allows developers to run AI models locally in web apps with much higher performance than earlier browser-based approaches.
What Actually Changed Under the Hood
The core shift is architectural rather than incremental. Instead of running JavaScript-based kernels, LiteRT.js exposes Google’s native, cross-platform runtime through WebAssembly, giving developers building AI applications the same optimizations mobile developers have had for years.
It routes inference across three backends depending on available hardware: XNNPACK for CPU, ML Drift for GPU by way of WebGPU, and the still-experimental WebNN API for dedicated NPUs, currently limited to Chrome and Edge.
Model conversion is the other meaningful change. PyTorch models can now go directly into LiteRT’s .tflite format in a single step through LiteRT Torch, replacing the old path of converting PyTorch to ONNX, then to TensorFlow, then to TensorFlow.js, a chain that broke often enough to be a genuine adoption barrier.
For teams not ready to rip out an existing TensorFlow.js pipeline, an interop package lets LiteRT.js slot in as a drop-in replacement for just the inference step, accepting and returning the same TensorFlow.js tensors.
The Performance Numbers, and What’s Still Rough
Google’s own benchmarks claim up to 3x faster inference than existing web runtimes across classical computer vision and audio models on both CPU and GPU, with GPU or NPU acceleration delivering 5 to 60 times faster results than CPU-only execution for real-time tasks like object tracking or transcription.
Those figures come from a single test environment, a 2024 MacBook Pro running Apple’s M4 chip, which leaves an open question about how consistent the gains are across the far more fragmented hardware and driver landscape most web apps actually run on.
WebNN’s NPU support is also still experimental, meaning the biggest efficiency gains aren’t universally available yet.
Google has already integrated LiteRT export into Ultralytics’ YOLO26 object detection models. Its demos also showcase real-time monocular depth estimation with Depth Anything V2 and 4× image upscaling using Real-ESRGAN. Both examples run entirely on the client side.
The bigger unresolved question, one independent observers have already raised, is what happens to TensorFlow.js itself now that Google has effectively built its faster, more capable successor.
The Future of Edge AI in the Browser
LiteRT.js is more than a standalone JavaScript library. It extends Google’s edge AI strategy from mobile devices to the browser.
The same benefits of on-device AI, including better privacy, lower latency, and reduced costs, now apply to modern web applications.
Google has already flagged generative AI and LLM support, via a companion LiteRT-LM.js, as the next target for the same runtime.
Source: Official Google Developers Blog, "LiteRT.js: Google's High-Performance Web AI Inference Runtime"
