Software: Reimagining software for the AI era
We’re building a complete software stack for the AI era. Our unified ecosystem absorbs the complexity of the seams between software and hardware, delivering industry-leading scale, efficiency, and reliability.
Unlocking the potential of TPUs
Architecting TorchTPU for usability, portability, and performance
Run JAX and PyTorch natively on TPUs at Google scale, leveraging our custom silicon’s economics without changing a single line of PyTorch code. We’ve eliminated the "refactoring tax" by making TPUs a native destination for JAX and PyTorch.
Leading the way in cost-effective inference
Agentic workflows require coordinated full-stack orchestration. We manage the entire inference lifecycle. Hardware and software work in a single, seamless loop to minimize latency and cost, impacting how models are quantized, cached, and served globally.
Engine
We develop TPUs and Axion CPUs in tandem to ensure that every layer of the workload—from deep reasoning to general-purpose execution—is optimized for maximum performance and energy efficiency.
Orchestration
Through Google Kubernetes Engine (GKE) and our specialized ML compilers (XLA), we synchronize chips, storage, and software inference engines into a single, automated fabric.
Synchronization
This vertical integration allows you to scale agentic workflows with deterministic efficiency, ensuring predictable speed and low latency at planetary scale.
“True AI infrastructure innovation requires highly optimized, reusable software pipelines to translate raw silicon compute power into cost-effective, megascale AI training and ultra-fast inference—supported by structural openness and platform flexibility.”
Decoupling model intelligence from hardware constraints
Overcome brittleness in LLM prompt optimization
Type-Compliant Adaptation Cascades (TACs) guarantee structural correctness via typed workflows and use efficient LoRA adapters to free complex reasoning from heavy hardware.
Accelerate Holistic Latent Diffusion
Simultaneously compressing spatial, temporal, and architectural axes decouples high-fidelity generation from strict hardware limits.
Increase MLLM efficiency and accuracy
By filtering video streams at the edge, MetaSight decouples real-time reasoning from heavy cloud compute to enable wearable AI.
Optimizing critical storage bottlenecks
Advancing RAG serving
RAGSchema serves as a foundation for performance optimization and RAGO, an optimization framework, significantly improves throughput and reduces latency compared to existing systems.
Benchmarking autonomous TPU kernel optimization
JAXBench consists of 50 relevant JAX workloads designed to support autonomous GPU-to-TPU and TPU-specific kernel performance optimization and enhance kernel solving rates.
How 8th gen TPUs power the agentic era
Enabling agentic systems
Autonomous systems require the low-latency and high-reliability deterministic performance that can only be guaranteed when hardware and software are perfectly synced, from the silicon to the API.
Deterministic performance
With Google Kubernetes Engine (GKE) as the automated orchestration layer, we provide a managed path from the first line of code to a globally deployed agentic system.
-
Contextual agent security: A policy for every purpose
Conseca (Contextual Agent Security) is a framework designed to dynamically generate just-in-time, contextual, and human-verifiable security policies to account for complex, shifting contexts.
-
Learning skill curation for self-evolving agents
SkillOS is an experience-driven reinforcement learning training recipe that enables self-evolving LLM agents to continuously learn and optimize long-term skill curation policies from past interactions using an external repository.
-
Executing complex MLE tasks
The Modular Agent with Reflective Search framework achieves state-of-the-art performance optimizing budget-aware planning, modular construction, and comparative reflective memory.
-
Conceptualizing research report generation as a diffusion process
The Test-Time Diffusion Deep Researcher can achieve state-of-the-art results on a wide array of benchmarks that require intensive search and multi-hop reasoning, significantly outperforming existing deep research agents.
Agentic AI code review of the Linux Kernel
Sashiko, our open-source autonomous Linux kernel reviewer, successfully identified 53% of bugs missed by human maintainers with an 85% true positive rate.
Maximizing resource efficiency with intelligent orchestration
We focus on tightly coupled software-hardware co-design, where deep, microsecond-scale telemetry and intelligent runtime software continuously optimize networking, compute, memory, and scheduling resources.
Managing powerful hyper-scale clusters
Google's Borg cluster manager pioneered massive workload colocation and dynamic resource over-commitment, handling hardware heterogeneity and workload placement at planetary scale.
Maximizing AI training performance
Automated straggler detection passively monitors GPU clusters and automatically flags and mitigates lagging nodes to prevent training delays.
Efficiently allocating ML training resources
Quota Marketplace achieves intelligent orchestration by letting users express workload importance through virtual bidding, shifting flexible tasks to off-peak hours based on dynamic supply-and-demand pricing.
Deploying runtime solutions to leverage emerging hardware platforms
Free up critical host CPU resources with SmartNIC offloading
A novel framework designed to offload userspace system software to SmartNIC ARM cores has successfully recovered up to 16 host cores with minimal performance trade-off, maximizing efficiency for paying data center customers.
Program modern memory systems easily
We demonstrate the potential of Declarative Memory Services (DMS) to enable developers to simply specify desired properties while a specialized runtime manages the execution and optimizations automatically.
Powering megascale training with custom fabrics and protocols
Megascale fabric architecture designed specifically for modern AI hypercomputers.
Virgo Network is a scale-out fabric designed for the extreme requirements of modern AI workloads. Built on high-radix switches that reduce network layers by allowing more ports per switch, it employs a flat, two-layer non-blocking topology. This streamlined architecture delivers the massive bisection bandwidth and deterministic low latency necessary for both distributed training and serving workloads.
Advanced congestion control designed specifically for high-throughput data centers.
The CSIG protocol delivers microsecond-granularity network visibility by embedding precise, multi-bit bottleneck congestion signals into a streamlined Layer 2 Ethernet header. By using these real-time signals, CSIG's Fast Ramp-Up primitive significantly reduces latency and wasted bandwidth across commodity switches and NICs without requiring forklift hardware upgrades.
A hardware-driven transport layer delivering sub-microsecond latency and high reliability.
Google’s Falcon is a hardware transport protocol that delivers reliable, low-latency, and high-bandwidth communication on commodity Ethernet networks. By utilizing co-designed hardware and software (such as delay-based congestion control and multi-path routing), Falcon supports multiple upper-layer protocols (like RDMA and NVMe) over standard lossy Ethernet without requiring specialized, expensive network infrastructure upgrades.
Use our complete AI software stack
Get access to the same frontier-scale infrastructure that powers advanced models like Gemini and iconic products like Search and YouTube. Available to developers and enterprises through Google Cloud.