CentML

  Executive Briefing

CentML was a Toronto-based AI inference optimization company that built what its founders described as a deep software layer capable of extracting dramatically more performance from existing GPU hardware — without requiring customers to acquire additional scarce silicon. Co-founded in March 2022 by Gennady Pekhimenko (CEO), Shang Wang (CTO), Anand Jayarajan (Chief Architect), and Akbar Nurlybayev (COO), CentML grew out of Pekhimenko's academic lab at the University of Toronto's Department of Computer Science and the Vector Institute. Pekhimenko, an Associate Professor of CS and Vector Institute faculty member who had previously worked at Microsoft Research, had identified critical GPU underutilization problems in production ML workloads. His doctoral students Wang and Jayarajan, whose research focused on optimizing ML models on cost-effective hardware, formed the technical core of the company alongside Nurlybayev, a U of T CS alumnus with industry and entrepreneurial experience.¹

The company's founding thesis — that a software-first optimization layer could close the gap between raw GPU performance and what production LLM workloads actually achieved — proved commercially viable and technically defensible. CentML built Hidet, an open-source deep learning compiler, and CServe, a high-performance LLM inference engine layered atop vLLM, alongside the Hexcute domain-specific language for GPU kernel fusion. These tools delivered documented speedups across the Llama, Mistral, and DeepSeek model families, with internal benchmarks showing 2–11x improvements on DeepSeek R1's Mixture-of-Experts layers and a 1.78x throughput gain over vanilla vLLM via the company's Seesaw dynamic parallelism technique — work accepted at MLSys 2025.²

CentML raised approximately $30.9 million USD across a pre-seed and seed round before NVIDIA acquired the company in June 2025. The deal value was reported by The Logic as potentially exceeding $400 million USD including earn-outs, though NVIDIA never confirmed the figure.³ All four co-founders moved into senior roles at NVIDIA following the acquisition. CentML officially ceased operations on July 17, 2025, bringing an independent run of just over three years to a close but validating the inference-optimization thesis at the highest level of the AI hardware industry.

For the AI infrastructure landscape, CentML's arc is instructive: a 30-person compiler and inference startup, seeded by the leading AI-focused VCs (Gradient Ventures, Radical Ventures) and backed by NVIDIA itself, achieved an exit reportedly north of $300 million for investors by demonstrating that the GPU software layer — not just the hardware — was worth acquiring at strategic prices. Both NVIDIA and AMD were reported to be competing for Canadian AI efficiency talent at the time of the deal.⁴

  At a Glance

  Origins & Founding

CentML was officially co-founded in March 2022 in Toronto by Gennady Pekhimenko, Shang Wang, Anand Jayarajan, and Akbar Nurlybayev. The company emerged directly from Pekhimenko's academic research group at the University of Toronto's Department of Computer Science, where he served as an Associate Professor and held a concurrent faculty position at the Vector Institute — Canada's national centre for AI research.⁵

Pekhimenko had previously spent time at Microsoft Research, where he observed that production machine-learning workloads routinely left large fractions of GPU capacity underutilized. The gap between theoretical GPU peak performance and what practitioners actually extracted was, in his view, a tractable software problem rather than a hardware constraint. He assembled his closest doctoral researchers: Wang (who received his B.A.Sc. in 2018 and M.S. in 2020 from U of T) and Jayarajan, both of whom had spent their PhD research on optimizing ML models for cost-effective hardware. Nurlybayev, a U of T Computer Science alumnus (B.S. 2007) with both industry and entrepreneurial experience, completed the founding team.

The four co-founders reconnected partly through their shared background in U of T's computer science community and, by some accounts, its Russian-speaking research network.¹ The founding thesis was clear: a deep software optimization layer sitting between model weights and GPU hardware could extract dramatically more performance from existing silicon — a proposition with immediate relevance as the AI chip shortage made adding new GPU capacity difficult and expensive for most enterprises. The company incorporated and began work in March 2022, securing its first outside capital within months.⁶

  History & Timeline

    2022: Founding and pre-seed (March–December 2022)

CentML incorporated in March 2022 and remained in stealth through the year. In June 2022, Radical Ventures — a Toronto-based AI-focused venture fund — led a $4.8 million CAD (approximately $3.5 million USD) pre-seed round.⁶ The funding supported early team-building and continued development of the Hidet compiler, which began as a research-grade tool in Pekhimenko's lab. The team of roughly four co-founders and early hires worked on validating the core thesis: that compiler-level optimizations could close the GPU utilization gap at production scale.

    2023: Emergence from stealth and seed round

CentML emerged from stealth in June 2023, publicly disclosing its compiler-and-inference-optimization approach for the first time.⁵ The reception validated the team's confidence: the company moved quickly to close a significantly larger financing round. The seed round closed in early September 2023 and was publicly announced on October 25, 2023: $37 million CAD (approximately $27 million USD), led by Gradient Ventures, Google's AI-focused early-stage fund. Co-investors included NVIDIA, Deloitte Ventures, Thomson Reuters Ventures, and returning investor Radical Ventures. Misha Bilenko, VP of Microsoft Azure AI, invested personally.⁷

The strategic composition of the seed round drew significant attention. NVIDIA's participation as a backer was notable given the company's position as CentML's eventual acquirer; Gradient Ventures' involvement signalled Google's interest in the inference-optimization layer. At the time of the announcement, Amazon and Snowflake were named as early enterprise customers.⁷ CentML had approximately 32 employees at close and planned to roughly double to around 64 by end of 2024.

    2024: Platform launch and US expansion

In March 2024 CentML opened a US office in Palo Alto, California, signalling a shift toward direct enterprise sales in the largest AI market. The year's headline product moment came in November 2024 with the official launch of the CentML Platform: a suite combining a serverless inference API, managed GPU clusters (CCluster), fine-tuning, and the CentML Planner for hardware configuration advice.⁸ At launch, pricing for serverless access to Llama 3.1 405B was set at $2.50 per million tokens — competitive with prevailing market rates. Also in November 2024, CentML joined NVIDIA's accelerator program, receiving hardware discounts and technical resources.

    2025: MLSys recognition, acquisition, and shutdown

Early 2025 brought academic validation for the team's Seesaw dynamic parallelism technique, with the paper accepted at MLSys 2025 — the field's leading systems conference for machine learning.² In May 2025, The Register published a substantial feature on CentML's open-systems performance strategy, noting the resilience of the open-compiler approach in surfacing edge cases.⁴

The company's trajectory changed decisively in June 2025. On June 6, CentML discontinued its federal Canadian corporate registration and re-incorporated in British Columbia — a procedural step that preceded the acquisition. On June 27, 2025, NVIDIA announced it had acquired CentML. All four co-founders moved into senior roles: Pekhimenko as Senior Director of AI Software, Nurlybayev as Senior Manager of AI Software, Wang as Manager of AI Systems Software, and Jayarajan into an engineering role. At least 15–18 technical staff transferred to NVIDIA alongside the founders. On July 17, 2025, CentML officially ceased operations as an independent entity and shut down all services.

  What They Offer — Products & Platform

CentML's platform, as launched in November 2024, comprised four integrated offerings targeting the full lifecycle of enterprise LLM deployment. The common thread was the company's proprietary compiler and kernel optimization stack applied at each layer.

The Serverless Inference API provided OpenAI-compatible endpoints for instant deployment of open-source models, with automatic autoscaling and per-token pricing. Developers could stand up inference for major open-weight models in seconds without managing infrastructure. The API was positioned as accessible to developers and mid-sized teams who could not afford or justify dedicated GPU fleets.

CCluster (Dedicated GPU Deployment) served enterprises requiring workload isolation, higher sustained throughput, or the ability to deploy custom model variants. The managed clusters included job scheduling, traffic control, auto-scaling, and real-time monitoring dashboards. This tier was the primary vector for CentML's B2B enterprise relationships with customers such as Amazon and Snowflake.

Fine-Tuning allowed customers to customize pre-trained models on proprietary data within the CentML platform, with security guarantees designed for enterprise workloads. This offering competed with hosted fine-tuning services from larger providers while delivering CentML's optimization layer on the resulting custom checkpoints.

CentML Planner was an automated recommendation tool that analyzed a model's computational requirements and proposed optimal hardware configurations and deployment strategies, reducing the specialist knowledge required for efficient LLM deployment.

Underpinning all four offerings were the company's open-source tools — DeepView, a ML profiler and predictor integrated with PyTorch for forecasting model performance across hardware configurations before deployment, and Hidet, the deep learning compiler available on GitHub. The open-source layer served both community-building and talent-acquisition purposes, generating academic and developer credibility that supported enterprise sales.

  Technology & Infrastructure

CentML's technical differentiation resided in its compiler and inference stack rather than in proprietary hardware. The company ran primarily on NVIDIA H100s for its inference fleet, with support extending to AMD MI300X GPUs, Google TPUs, NVIDIA L4 GPUs (for cost-optimized serving), and NVIDIA A10 GPUs (used in early deployments).

Hidet is an open-source deep learning compiler written in Python that compiles DNN models from PyTorch and ONNX to optimized CUDA kernels. Accompanying Hidet is Hexcute, a domain-specific language for GPU kernel fusion that enables the compiler to collapse multi-step operations into single highly optimized kernels. On DeepSeek R1's Mixture-of-Experts layers, Hidet achieved 2–11x speedups through fully-fused GPU kernels, outperforming equivalent Triton-compiled implementations.⁴ This was among the most externally verifiable performance claims CentML published.

CServe is CentML's LLM inference engine, built atop vLLM and augmented with Hidet's proprietary kernel optimizations. CServe's performance envelope included several specific techniques:

• EAGLE speculative decoding — delivering 1.5–2x inference speedup by using a smaller draft model to propose token sequences that the main model verifies in parallel.
• Flash attention — reducing attention computation memory bandwidth requirements.
• Seesaw dynamic parallelism — a technique accepted at MLSys 2025 that dynamically switches between tensor parallelism (optimal for prefill) and pipeline parallelism (optimal for decode) based on the phase of inference. Seesaw demonstrated a 1.78x peak throughput improvement (1.36x average) over vanilla vLLM in published benchmarks.²
• Pipeline parallelism — distributing model layers across multiple GPU devices to enable serving of models too large for a single card.

On the training side, HFTA (Horizontally Fused Training Array) fused multiple model training tasks running in parallel, demonstrating a 15.1x increase in training throughput in benchmarks. HFTA was primarily a research contribution rather than a core commercial product.

Data center infrastructure was cloud-hosted, with the principal partnership being Oracle Cloud Infrastructure (OCI), through which CentML offered joint LLM optimization deployments including benchmarks on LLaMA v2 and Falcon 40B. On OCI, CentML reported a 48% improvement in LLaMA v2 inference serving over competitor baselines, and a 36% additional optimization gain relative to other cloud providers for equivalent workloads.⁸ The company operated no disclosed proprietary data centers.

  Model Catalog & Performance

CentML's serverless API served the leading open-weight model families, with optimization applied at the compiler and inference-engine level for each. Key served models at platform launch included the Meta Llama 3.1 family (8B, 70B, and 405B), Meta Llama 2 (7B, 13B, 70B), Mistral 7B and Mixtral variants, DeepSeek R1 with MoE-specific kernel optimization, and Falcon 40B via the OCI partnership. The full model catalog at platform launch was not exhaustively published; the selection above reflects confirmed deployments from public materials.

Performance positioning emphasized throughput and latency improvements rather than model breadth. Internal benchmarks showed CentML's CServe stack outperforming Together AI endpoints on token throughput and latency, though these comparisons were not independently audited.⁴ The A10 GPU deployment demonstrated 1.2x improvement over NVIDIA's own baseline serving configuration for the same hardware — a meaningful claim given that NVIDIA wrote the baseline. For Llama 2 specifically, CentML reported 3x speedup versus unoptimized serving on equivalent hardware.

  Pricing & Performance Position

At the November 2024 platform launch, CentML's publicly confirmed serverless pricing was $2.50 per million tokens for Llama 3.1 405B — the largest and most expensive model in the catalog. Pricing for smaller models (8B, 70B) was not independently confirmed in public sources. The 405B rate was competitive with prevailing market pricing from peers such as Together AI and Fireworks AI at the time, and the positioning emphasized that CentML's optimization layer would deliver equivalent or better throughput at that price point rather than competing solely on headline cost.⁷⁸

The company's performance-differentiation narrative rested on a stack of specific, published claims: the Seesaw 1.78x throughput improvement over vLLM, EAGLE speculative decoding at 1.5–2x speedup, the 2–11x MoE layer gains on DeepSeek R1, and the 48% LLaMA v2 improvement on OCI. Taken together, these positioned CentML as a provider where the software optimization layer — rather than hardware scale or model breadth — was the purchasing rationale. Business model was usage-based B2B SaaS, with per-token serverless fees and subscription or on-demand managed GPU cluster pricing for enterprise tiers.

  People & Leadership

The four co-founders constituted CentML's entire senior leadership team through the company's independent life.

Gennady Pekhimenko (CEO) is an Associate Professor in the Department of Computer Science at the University of Toronto and a faculty member at the Vector Institute. His research at U of T focused on ML systems, computer architecture, and the intersection of the two. Prior to founding CentML, he held a position at Microsoft Research where his exposure to production ML workloads informed the company's core thesis.⁵ Post-acquisition, he joined NVIDIA as Senior Director of AI Software.

Shang Wang (CTO) received his B.A.Sc. in Engineering Science in 2018 and his M.S. in Computer Science in 2020, both from the University of Toronto, before pursuing doctoral research in Pekhimenko's lab. His technical focus was on efficient ML model execution on cost-effective hardware. Post-acquisition, he became Manager of AI Systems Software at NVIDIA.

Anand Jayarajan (Chief Architect) was a PhD student in Pekhimenko's lab at U of T whose dissertation research contributed directly to CentML's compiler and kernel optimization work. Post-acquisition, he moved into an engineering role at NVIDIA.

Akbar Nurlybayev (COO) holds a B.S. in Computer Science from U of T's New College (2007). He brought industry operations and entrepreneurial experience to a technically-oriented founding team, handling go-to-market, business development, and operational functions. Post-acquisition, he became Senior Manager of AI Software at NVIDIA.

Notable among CentML's investors was Misha Bilenko, VP of Microsoft Azure AI, who invested personally in the seed round — an early signal that senior ML infrastructure practitioners viewed the company's technical approach as credible.⁷ At the time of the seed round CentML had approximately 32 employees; the company planned to roughly double by end of 2024, though the exact headcount at acquisition is not confirmed. At least 15–18 engineers are reported to have transferred to NVIDIA alongside the four co-founders.⁹

  Funding, Ownership & Business

CentML raised two rounds of outside financing before its acquisition.

The pre-seed closed in June 2022 at $4.8 million CAD (approximately $3.5 million USD), led by Radical Ventures, a Toronto-based AI-focused fund with a concentrated portfolio of early-stage AI companies. This initial capital funded team formation and early compiler development.⁶

The seed round closed in September 2023 and was publicly announced on October 25, 2023 at $37 million CAD (approximately $27 million USD). The round was structured as a mix of equity and SAFE notes and was led by Gradient Ventures, Google's AI-focused venture arm. Co-investors were NVIDIA, Deloitte Ventures, Thomson Reuters Ventures, and returning investor Radical Ventures. Misha Bilenko (VP, Microsoft Azure AI) invested personally. Total raised pre-acquisition was approximately $30.9 million USD.⁷

The acquisition by NVIDIA was announced on June 27, 2025. The Logic reported the deal value as potentially exceeding $400 million USD including earn-outs tied to undisclosed performance targets; the transaction was reported to have exceeded $300 million for investors and shareholders.³ NVIDIA declined to confirm the deal value. No valuation was disclosed during either funding round. The acquisition represented a return of roughly 10x on invested capital at the reported floor, in approximately two years from seed close.

  Customers & Partnerships

CentML's disclosed enterprise customer base was deliberately limited in public materials, reflecting the company's enterprise sales focus and NDA-governed relationships. At the time of the seed round announcement in October 2023, Amazon and Snowflake were named as early customers — a notable validation given both companies' scale of LLM infrastructure consumption.⁷

EquoAI was identified in 2024 platform launch materials as a customer using CentML for LLM-based legal document summarization. EquoAI reportedly achieved cost savings of up to $250,000 per year by replacing higher-cost inference providers with CentML's optimized endpoints.⁸

The most significant partnership was with Oracle Cloud Infrastructure (OCI). The two companies collaborated on joint LLM optimization benchmarking and customer deployments, with CentML's stack running on OCI hardware and the results published as part of both companies' go-to-market materials. Benchmarks covered LLaMA v2 and Falcon 40B serving, with CentML demonstrating substantial improvements over non-optimized OCI baselines.⁸

In November 2024, CentML joined NVIDIA's accelerator program, receiving hardware discounts and technical resources — an arrangement that further deepened the relationship with its eventual acquirer.

  Competitive Position

CentML positioned itself as a software-first inference optimization layer, differentiating from both directions of the competitive landscape: hyperscalers (too expensive, hardware-locked, and slow to adopt new models) and raw GPU cloud providers (no optimization layer, requiring customers to manage their own inference stack). The company's principal competitive differentiation was its proprietary compiler stack — Hidet and the Hexcute DSL — which was open-source enough to build community credibility but technically deep enough to resist easy replication.⁴

Direct competitors at the time of CentML's operation included Fireworks AI, Together AI, Modal, Replicate, OctoML (acquired by Apple), and MosaicML (acquired by Databricks for a reported $1.3 billion). Among these, CentML's open-compiler strategy was most directly analogous to OctoML's, though OctoML was also acquired before CentML reached full commercial scale. CB Insights categorized CentML alongside Nebius Group in the AI cloud with GPU accelerator space.

The Register's May 2025 coverage framed CentML's open-systems approach as structurally resilient: community contributions to Hidet and related tooling surface edge cases that a purely proprietary stack would miss, compounding the optimization advantage over time.⁴ Internal benchmarks showed CentML's stack outperforming Together AI endpoints on token throughput and latency, though these comparisons were not independently audited. NVIDIA's acquisition at a reported premium to the company's modest capital raise validated the core thesis: that software-layer inference optimization was worth acquiring at strategic prices rather than building internally from scratch.

  Notable Events

October 2023 seed round — The announcement of NVIDIA as a strategic co-investor in a Canadian inference startup was itself notable; it represented NVIDIA placing a bet on software-layer optimization to complement its hardware business and signalled the inference-optimization category's strategic importance.⁷

Seesaw at MLSys 2025 — The acceptance of CentML's Seesaw paper at the Machine Learning Systems conference, one of the most competitive venues for ML infrastructure research, was the company's most significant academic validation. The 1.78x throughput improvement over vLLM represented a concrete, conference-reviewed performance benchmark.²

NVIDIA acquisition, June 2025 — The acquisition brought an independent run of three years to a close and gave NVIDIA direct access to Hidet, Hexcute, CServe, and the engineering team behind them. The reported deal value — potentially exceeding $400M USD — made it one of the larger acqui-hires in Canadian AI history. AMD was reported to be competing for similar Canadian AI efficiency talent at the same time, suggesting the chip vendors' race to internalize inference optimization software was active and contested.⁹

Shutdown, July 2025 — CentML officially ceased all operations on July 17, 2025, approximately three weeks after the acquisition announcement. Customers on the serverless API and CCluster managed clusters had to migrate to alternative providers.

  Outlook & Roadmap

CentML no longer operates as an independent company. As of July 17, 2025, all services have been discontinued and the company has been absorbed into NVIDIA. The co-founders are now senior members of NVIDIA's AI software organization: Pekhimenko as Senior Director of AI Software, Nurlybayev as Senior Manager of AI Software, and Wang as Manager of AI Systems Software.

The strategic direction for CentML's technology inside NVIDIA is not publicly confirmed, but the logical integration targets are NVIDIA's existing software platform — TensorRT, CUDA, cuDNN, and the broader NVIDIA AI software stack. The Hidet compiler and Hexcute DSL are natural complements to TensorRT's existing kernel optimization approach; CServe's Seesaw dynamic parallelism technique could inform future vLLM contributions or NVIDIA-native inference servers. Whether the open-source Hidet repository on GitHub continues to receive active updates under NVIDIA's stewardship is not confirmed as of this writing.

The broader implication of CentML's arc is that the software optimization layer for LLM inference was sufficiently valuable — and sufficiently difficult — that NVIDIA preferred acquisition to organic development. As inference workloads grow faster than hardware supply, the efficiency gap that CentML was built to close will remain a structural feature of the AI infrastructure market.

---

  References

1. CentML founding story — University of Toronto CS
2. CentML seed round — BetaKit
3. CentML seed round — TechCrunch
4. NVIDIA acquisition deal value — The Logic
5. CentML open-systems performance — The Register
6. NVIDIA acquires CentML — BetaKit
7. Seesaw paper — MLSys 2025
8. Pekhimenko at U of T — University of Toronto CS
9. Globe and Mail — NVIDIA acquires CentML

  References

1. CentML founding story — University of Toronto CS. ↩ ↩²

2. Seesaw dynamic parallelism — accepted at MLSys 2025 — MLSys proceedings. ↩ ↩² ↩³ ↩⁴

3. Acquisition deal value reported; not confirmed by NVIDIA — The Logic. ↩ ↩² ↩³

4. CentML's open-systems compiler strategy — The Register, May 2025. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

5. Pekhimenko background and Vector Institute affiliation — University of Toronto CS. ↩ ↩² ↩³

6. Pre-seed round and BetaKit coverage — BetaKit. ↩ ↩² ↩³

7. Seed round announcement — TechCrunch, October 2023. ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷

8. CentML Platform launch and OCI partnership — Globe and Mail. ↩ ↩² ↩³ ↩⁴ ↩⁵

9. NVIDIA acquisition announcement and staff transfer — BetaKit. ↩ ↩²