Google releases LiteRT.js for running AI models directly in the browser

What changed

Source summary

Google released LiteRT.js, a JavaScript binding of LiteRT that runs .tflite models (and PyTorch models converted in a single step) directly inside a web browser, with CPU acceleration via XNNPACK, GPU acceleration via WebGPU, and experimental NPU acceleration via WebNN in Chrome and Edge. Google reports up to 3x faster CPU/GPU inference than other web runtimes across classical computer vision and audio processing models. Its 5-60x GPU/NPU speedup over standard CPU execution comes from benchmarking popular models for real-time tasks such as object tracking, audio transcription and image manipulation, with WebNN tested via Apple CoreML on a 2024 MacBook Pro with M4 Apple Silicon in a controlled browser environment. It is available now as the @litertjs/core npm package.

Why it matters to your work

Practical interpretation

This is a technical release for web developers: it lets a website or web app run AI models (such as vision or audio models in the .tflite format, or PyTorch models converted to it) directly in the visitor's browser with GPU acceleration, instead of needing a server round-trip for every inference. It is mainly relevant if your organisation has developers building AI-powered features into a website or web app.

When this could help

Useful when a developer is adding AI-model inference to a website or web app and wants it to run client-side in the visitor's browser rather than round-tripping to a server for every call.

LiteRT.js is a JavaScript binding of LiteRT that runs .tflite models -- or PyTorch models converted in a single step -- directly inside the browser, using CPU acceleration via XNNPACK, GPU acceleration via WebGPU, and experimental NPU acceleration via WebNN in Chrome and Edge. Google's own benchmark reports it outperforming other web runtimes by up to 3x on CPU and GPU inference for classical computer vision and audio processing models, with a 5-60x speedup over standard CPU execution on GPU/NPU backends.

How to get started

1. Download @litertjs/core from npm for your web project.

   Provider guidance ↗

2. Import loadLiteRt, loadAndCompile and Tensor from '@litertjs/core'. Call loadLiteRt() with the path to the LiteRT.js Wasm files before loading and compiling a .tflite model, creating input Tensors and calling model.run(). The official snippet demonstrates GPU acceleration via WebGPU. PyTorch models must first be converted using the separate LiteRT Torch guide; consult the linked LiteRT.js documentation for other accelerator options.

   Provider guidance ↗

Build it into your routine

SCALA suggested workflow

1. Choosing a runtime for a new client-side AI feature on the web

   When a web feature needs low-latency, client-side model inference instead of a server round-trip, consider LiteRT.js for a .tflite or convertible PyTorch model before building a custom server API for it.

What to check

• The headline performance numbers (up to 3x faster than other web runtimes for classical computer vision and audio processing models; 5-60x speedup on GPU/NPU vs standard CPU execution) come from Google's own benchmark on a single machine -- a 2024 MacBook Pro with M4 Apple Silicon -- in a controlled browser environment. Results on other hardware, browsers or devices may differ. Source ↗
• NPU acceleration via WebNN is stated as 'currently experimental in Chrome and Edge' -- not confirmed production-ready, and not stated to be available in other browsers. Source ↗
• The announcement does not give an explicit install command, pricing, or a minimum browser/OS version beyond the WebNN Chrome/Edge caveat above -- treat anything else as unconfirmed. Source ↗