Developer docs
API playgroundTry for free, no card

Search company profiles

MLCommons

Full company profile

uuid0004jw5

Namestring
MLCommons
Legal namestring
MLCommons Association
Websiteurl
mlcommons.org
Company typeenum
Private
Founded yearint
2018
Descriptiontext

MLCommons Association is a US-based non-profit consortium founded in 2018 that develops open, industry-standard benchmarks and data tooling for measuring AI quality, performance, and safety. Its flagship offerings span the MLPerf benchmark family — Training, Inference (Datacenter, Edge, Mobile, Tiny), Storage, Client, Automotive, and Endpoints — plus the AILuminate safety suite (twelve hazard categories, 24,000+ test prompts per language), AlgoPerf training-algorithm benchmarks, the Croissant metadata standard (700k+ datasets), MLCube containerization, and Chakra execution traces.

The organization's technical proposition is that of a neutral, peer-reviewed measurement infrastructure: MLPerf submissions are bi-annual and results are publicly published, with hardware vendors (NVIDIA Blackwell Ultra, AMD GPUs, Intel Xeon 6, Qualcomm Snapdragon), hyperscalers (Microsoft Azure, Oracle, Google), and AI labs (Meta, Shopify-side collaborations) competing on leaderboards. AILuminate and the AI Risk & Reliability working group extend the measurement layer into agentic, multimodal/multilingual, and security benchmarks, increasingly aligned with emerging regulation (EU AI Act, Colorado AI Act) and government-led third-party evaluations (UK AI Security Institute, U.S. NSF NAIRR pilot).

MLCommons operates a membership-based revenue model with 125+ member organizations contributing membership fees (undisclosed publicly); membership is required for participation in most working groups, while non-member submissions are permitted in select public groups under a Non-member Test Agreement. Distribution is primarily digital — GitHub repositories, a public results dashboard at mlcommons.org/visualizer, mobile apps via Apple App Store and Google Play, and benchmark demonstrations at industry venues such as GTC 2026. Leadership is anchored by President Peter Mattson (former Google/NVIDIA research) with working-group chairs drawn from Google DeepMind, Qualcomm, Microsoft CoreAI, AMD, NVIDIA, Intel, and leading academic institutions.

Short descriptiontext

MLCommons Association is a non-profit consortium developing open MLPerf benchmarks and AILuminate AI safety benchmarks, serving 125+ member organizations across chip vendors, hyperscalers, AI labs, and academia.

Operating statusenum
Operating
Ownership categoryenum
Headcount rangeband
51–100
akta.pro rankint
HeadquartersSan Francisco, United States
HQ citystring
San Francisco
HQ countrystring
United States
HQ regionstring
North America
Markets served

Serves global market

Keyword5 values
AI benchmark suites, machine learning standards, AI performance measurement, MLPerf benchmarks, AI safety evaluation
Industry3 codes
PRIMARYHDAAABAA
End-to-End MLOps & ML Platform Suites
Hardware, Technology & Telecom/Artificial Intelligence & Machine Learning/ML Platforms, MLOps & Model Lifecycle Management
SECONDARYHDAAABAH
Model Testing, Validation & Quality Assurance
Hardware, Technology & Telecom/Artificial Intelligence & Machine Learning/ML Platforms, MLOps & Model Lifecycle Management
SECONDARYHDAAABAK
ML Lifecycle Collaboration & Workspace Management
Hardware, Technology & Telecom/Artificial Intelligence & Machine Learning/ML Platforms, MLOps & Model Lifecycle Management
NAICS code1 code
  • Professional Organizations813920
SIC code1 code
  • Services-Testing Laboratories8734
Product category
AI Benchmarking and Standards
GTM motion1 record

Each record includes

Type, Description, Source

Revenue model1 record
1Membership Fees
TypeSubscription Recurring
Description

MLCommons operates as a membership-based consortium with over 125 members including startups, leading companies, academics, and non-profits. Membership is required for participation in most benchmark working groups and provides access to collaboration resources and decision-making processes.

mlcommons.org
Marketing channels8 records

Each record includes

Title, Type, Stage, Description, Source

Distribution channels5 records

Each record includes

Title, Type, Scope, Target buyer, Description, Source

Cost components4 values
Personnel, Technology or R&D, Operations, Others
GTM typeB2B
B2B
Offering typeSoftware
Software
Brand1 of 10 records shown
1AILuminate
Description

AI safety and security benchmarks assessing genAI across 12 hazard categories.

mlcommons.org
+9 more records
Core offering1 text field

MLCommons is a non-profit consortium that develops open industry-standard benchmarks and data tooling for measuring AI system quality, performance, and risk. Its core offerings include the MLPerf benchmark suites (Training, Inference Datacenter/Edge/Mobile/Tiny, Storage, Client, Automotive, Endpoints), AILuminate AI safety benchmarks covering 12 hazard categories, AlgoPerf training algorithm benchmarks, and the Croissant metadata standard used by 700k+ datasets.

Differentiator
Functional benefit
Problem solved
Quantifiable outcome1 of 4 values shown
  • 89.7k+ MLPerf Performance Results to-date
+3 more records
Product overview1 text field

MLCommons is a nonprofit organization operating as a benchmarking and standards body rather than a traditional product company. Its portfolio consists of industry-standard benchmark suites, data standards, and research tools. The core MLPerf family includes MLPerf Training (measuring model training speed), MLPerf Inference variants for Datacenter, Edge, Mobile, and Tiny environments, plus specialized suites for Automotive and Client systems. AILuminate provides AI safety benchmarking across twelve hazard categories. AlgoPerf measures training algorithm optimization speedups. Supporting infrastructure includes MLCube containers, the Croissant metadata standard (used by 700,000+ datasets), and Chakra execution traces. The organization publishes results from 89.7K+ benchmark submissions and maintains over 125 member organizations across industry and academia.

Product and service13 records
1MLPerf Training
CategoryAI Benchmark Suite
Description

Industry-standard benchmark measuring how fast systems can train AI models to a target quality metric, covering LLMs, image generation, recommendation, GNNs, BERT, object detection, and speech recognition.

2MLPerf Inference: Datacenter
CategoryAI Benchmark Suite
Description

Benchmark measuring how fast datacenter systems process inputs and produce results using trained models across Server, Offline, Single Stream, and Multiple Stream scenarios with latency constraints.

3MLPerf Inference: Edge
CategoryAI Benchmark Suite
Description

Benchmark measuring AI inference performance on edge systems, testing how fast systems can process inputs and produce results using trained models.

4MLPerf Inference: Mobile
CategoryAI Benchmark Suite
Description

Benchmark measuring AI performance on consumer mobile devices (smartphones, tablets, notebooks), testing LLMs, image classification, object detection, segmentation, and language processing.

5MLPerf Inference: Tiny
CategoryAI Benchmark Suite
Description

Benchmark measuring AI inference performance on ultra-low-power systems including image classification, keyword spotting, person detection, and anomaly detection.

6MLPerf Storage
CategoryAI Benchmark Suite
Description

Benchmark measuring storage system performance for AI training workloads, testing how fast storage can supply training data to keep GPUs/accelerators fed.

7MLPerf Client
CategoryAI Benchmark Suite
Description

Benchmark evaluating AI performance on personal computers (laptops, desktops, workstations), testing LLMs such as Llama 3.1 8B and Phi models for code analysis, content generation, and summarization tasks.

8MLPerf Automotive
CategoryAI Benchmark Suite
Description

Benchmark suite measuring AI performance for automotive ADAS/AD perception and In-Vehicle Infotainment systems, testing 2D/3D object detection and semantic segmentation on Cognata and nuScenes datasets.

9MLPerf Endpoints
CategoryAI Benchmark Suite
Description

Benchmark for evaluating GenAI API endpoint performance using an OpenAI-compatible API interface, with Pareto curves measuring throughput vs interactivity tradeoffs across the full operating range.

10AILuminate
CategoryAI Safety Benchmark
Description

AI safety and security benchmark suite assessing generative AI systems across twelve hazard categories, including Safety v1.0 for safety response evaluation and Jailbreak v0.5 for adversarial prompt resistance testing.

11AlgoPerf: Training Algorithms
CategoryAI Benchmark Suite
Description

Benchmark measuring neural network training speedups due to algorithmic improvements (optimizer, hyperparameters), using eight fixed workloads on standardized 8x V100 GPU hardware.

12Croissant Metadata Standard
CategoryData Standard
Description

Metadata format standardizing how ML datasets are described, making ML work easier to reproduce and replicate, adopted by over 700,000 datasets.

13MLCube
CategoryContainerization Standard
Description

Containerization standard for packaging and running ML models consistently, improving AI ease-of-use and enabling AI to scale to more users.

Scale indicator6 records

Each record includes

Type, Value, Description, Source

Partnership18 partners
Strategic tierMinorTypeStrategic or Co-development PartnerAnnounced on2026-05-12
Description

Co-hosted the 3rd annual 'New Wave of AI in Healthcare' symposium with Icahn School of Medicine at Mount Sinai, discussing AI's expanding role in healthcare delivery, diagnostics, and drug discovery.

Strategic tierMinorTypeStrategic or Co-development PartnerAnnounced on2026-05-12
Description

Co-hosted the 3rd annual 'New Wave of AI in Healthcare' symposium with the New York Academy of Sciences, bringing together researchers, clinicians, and industry leaders to discuss AI's role in healthcare.

Strategic tierCoreTypeStrategic or Co-development Partner
Description

MLCommons partners with AVCC on the MLPerf Automotive benchmark suite. AVCC provides technical reports (TR003, TR004, TR007) and MLCommons provides the benchmarking infrastructure. The collaboration focuses on developing industry-standard ML benchmarks for automotive computing platforms used in ADAS/AD and IVI systems.

Strategic tierCoreTypeTechnology or Integration
Description

AMD actively submits results to MLPerf benchmark suites, including Inference v6.0 and Training v6.0. AMD showcases its GPU performance through MLCommons benchmarks as part of its AI hardware validation.

Strategic tierCoreTypeTechnology or Integration
Description

NVIDIA is a key benchmark submitter to MLPerf suites. NVIDIA's Blackwell Ultra architecture topped the MLPerf Inference v5.1 leaderboard, achieving up to 5,842 tokens per second per GPU offline.

Strategic tierCoreTypeTechnology or Integration
Description

Google submits benchmark results and contributes to working groups. Google DeepMind team members lead multimodal safety benchmarking workstream within the AI Risk & Reliability working group.

7Microsoft Azure
Strategic tierMajorTypeTechnology or Integration
Description

Microsoft Azure submits cloud-based AI training benchmark results and contributes to MLPerf Training working groups. Microsoft CoreAI team works on AI evaluation tooling.

hpcwire.com
Strategic tierMajorTypeTechnology or Integration
Description

Oracle participated in MLPerf Training v6.0 benchmark submissions as one of 24 organizations submitting results.

Strategic tierCoreTypeTechnology or Integration
Description

Meta collaborates with MLCommons on MLPerf Inference benchmarks and is listed as a key collaboration partner for increased adoption of large-scale multi-node systems.

Strategic tierMajorTypeTechnology or Integration
Description

Shopify collaborates with MLCommons on MLPerf Inference benchmarks, contributing to the diverse set of industry participants.

Strategic tierMajorTypeTechnology or Integration
Description

Ultralytics collaborates on MLPerf Inference benchmarks as part of the expanding ecosystem of AI system developers.

12IBM
Strategic tierMajorTypeTechnology or Integration
Description

IBM Storage Scale participates in MLPerf Storage v2.0 benchmarks, achieving 656.7 GiB/s read bandwidth for 1T parameter model training.

13Singapore Ministry of Digital Development and Information
Strategic tierMajorTypeStrategic or Co-development Partner
Description

Google and Singapore's Ministry expanded their National AI Partnership, building on a 2022 agreement to deploy AI across public services. This partnership will serve as a model for national AI programs.

itbrief.asia
14U.S. National Science Foundation (NSF)
Strategic tierMajorTypeStrategic or Co-development Partner
Description

NSF launched the National AI Research Resource (NAIRR) pilot, partnering with MLCommons and 25 other organizations to democratize AI research access. Initial focus on safe, secure AI and healthcare applications.

nsf.gov
Strategic tierCoreTypeStrategic or Co-development Partner
Description

The layered assurance ecosystem combines lab self-attestation with government institute validation from UK AI Security Institute and independent third-party evaluation from organizations including MLCommons.

16Oxford Internet Institute
Strategic tierMinorTypeStrategic or Co-development Partner
Description

Researchers from Oxford Internet Institute and Nuffield Department of Primary Care Health Sciences at the University of Oxford collaborated with MLCommons on a medical advice study comparing AI chatbots against traditional methods.

computerworld.com
17Nuffield Department of Primary Care Health Sciences, University of Oxford
Strategic tierMinorTypeStrategic or Co-development Partner
Description

Collaborated with MLCommons on a study involving 1,298 people in the UK comparing effectiveness of AI chatbots (GPT-4o, Llama 3, Command R) against traditional methods for assessing medical symptoms.

computerworld.com
Strategic tierMinorTypeStrategic or Co-development Partner
Description

Cognata provides the MLCommons Cognata Dataset used in MLPerf Automotive benchmarks for 2D object detection and semantic segmentation testing.

Recent move6 records

Each record includes

Date, Type, Title, Description, Source

Expansion highlight7 records

Each record includes

Type, Description

Peers10 records
1HELM (Stanford CRFM)
TypeDirect peer
Description

Stanford's Holistic Evaluation of Language Models is a leading academic benchmark for evaluating large language models across accuracy, calibration, robustness, fairness, and efficiency. Directly comparable to MLCommons as a primary AI benchmarking initiative targeting transparency and reproducibility.

2LMSYS Chatbot Arena
TypeDirect peer
Description

Large Model Systems Organization's crowdsourced LLM benchmark using pairwise human preference votes. Comparable to MLCommons as a widely-cited third-party AI evaluation platform with community-driven methodology.

TypeDirect peer
Description

Hugging Face's benchmark for open-source LLMs evaluating reasoning, knowledge, and truthfulness. Comparable to MLCommons' Inference benchmarks as a public-facing AI evaluation reference for the open-source community.

TypeBroad incumbent
Description

Open-source software foundation operating as a membership-driven consortium with vendor-neutral governance. Comparable to MLCommons as a precedent for industry consortium structures that develop shared technical standards with member-funded operations.

TypeBroad incumbent
Description

Largest open-source foundation supporting Linux, Kubernetes, and other critical infrastructure projects. Comparable to MLCommons as a mature model of vendor-neutral consortium governance and standards stewardship that MLCommons aspires to for AI benchmarking infrastructure.

TypeDirect peer
Description

Multi-stakeholder nonprofit organization developing best practices on AI safety, fairness, and governance. Directly comparable to MLCommons as a similar consortium model with industry, academia, and civil society members focused on responsible AI development.

TypeBroad incumbent
Description

Major international standards body developing technical standards across industries including AI. Comparable to MLCommons as a precedent for member-driven technology standardization, though much larger and more established.

TypeEmerging player
Description

OpenAI's open-source evaluation framework for testing LLM capabilities. Comparable as an emerging benchmark initiative but single-vendor-driven, contrasting with MLCommons' multi-stakeholder approach.

TypeEmerging player
Description

Open-source framework for evaluating autoregressive language models across hundreds of standardized tasks. Comparable to MLCommons as a community-driven AI evaluation tool, though focused primarily on academic and open-model use cases.

TypeOthers
Description

Listed here for completeness only — MLPerf is MLCommons' flagship product and trademark, not a separate peer.

Market position
Strengths5 records

Each record includes

Headline, Details, Source

Weaknesses5 records

Each record includes

Headline, Details, Source

Competitive moat5 records

Each record includes

Type, Details

Key risks6 records

Each record includes

Headline, Details, Source

Key highlights6 records

Each record includes

Headline, Details, Source

Customer concentration

Classification, Details

Named customers11 records

Each record includes

Name, Industry, Type, Use case, Source, UUID

Segment5 records

Each record includes

Title, Type, Primary, Description, Pain point addressed, Use case, Source

Ideal customer profile4 records

Each record includes

Profile, Firmographic size, Sales motion, Sales cycle length, Buying structure, Purchase trigger, Buyer persona, Geography, Industry vertical, Primary use case, Description, Pain points, Evidence proof points, Target buyer

Technology focused
Yes
API detail
Has APIbool
No

Docs URL, Description

AI capability15 records

Each record includes

Type, Description, Source

AI maturity
App detail

Has app

Feature8 records

Each record includes

Title, Differentiator, Description, Source

Core technology
Revenue estimate
Valuation estimate
Number of profiles
Profiles8 records

Each record includes

Name, Designation, Designation category, Overview, Profile commentary, Source

No data
No data
Funding overview

Funding stage, Last funding date, Total funding USD

Funding rounds

Each record includes

Round, Amount USD, Date, Pre money valuation, Total investors, Investors, News

Investors

Each record includes

Name, Type, Date of entry, Rounds participated, Website

Funding detail is available on the Subscription and Enterprise plan.Contact sales →

M&A

Each record includes

Name, Acquisition type, Announced date, Completed date, Status, Website, News

Investment

Each record includes

Name, Round, Announced date, Lead investor, Website, News

M&A and investment is available on the Subscription and Enterprise plan.Contact sales →

MLCommons

AI Benchmarking and Standardsmlcommons.org

MLCommons Association is a non-profit consortium developing open MLPerf benchmarks and AILuminate AI safety benchmarks, serving 125+ member organizations across chip vendors, hyperscalers, AI labs, and academia.

What MLCommons does

MLCommons Association is a US-based non-profit consortium founded in 2018 that develops open, industry-standard benchmarks and data tooling for measuring AI quality, performance, and safety. Its flagship offerings span the MLPerf benchmark family — Training, Inference (Datacenter, Edge, Mobile, Tiny), Storage, Client, Automotive, and Endpoints — plus the AILuminate safety suite (twelve hazard categories, 24,000+ test prompts per language), AlgoPerf training-algorithm benchmarks, the Croissant metadata standard (700k+ datasets), MLCube containerization, and Chakra execution traces.

The organization's technical proposition is that of a neutral, peer-reviewed measurement infrastructure: MLPerf submissions are bi-annual and results are publicly published, with hardware vendors (NVIDIA Blackwell Ultra, AMD GPUs, Intel Xeon 6, Qualcomm Snapdragon), hyperscalers (Microsoft Azure, Oracle, Google), and AI labs (Meta, Shopify-side collaborations) competing on leaderboards. AILuminate and the AI Risk & Reliability working group extend the measurement layer into agentic, multimodal/multilingual, and security benchmarks, increasingly aligned with emerging regulation (EU AI Act, Colorado AI Act) and government-led third-party evaluations (UK AI Security Institute, U.S. NSF NAIRR pilot).

MLCommons operates a membership-based revenue model with 125+ member organizations contributing membership fees (undisclosed publicly); membership is required for participation in most working groups, while non-member submissions are permitted in select public groups under a Non-member Test Agreement. Distribution is primarily digital — GitHub repositories, a public results dashboard at mlcommons.org/visualizer, mobile apps via Apple App Store and Google Play, and benchmark demonstrations at industry venues such as GTC 2026. Leadership is anchored by President Peter Mattson (former Google/NVIDIA research) with working-group chairs drawn from Google DeepMind, Qualcomm, Microsoft CoreAI, AMD, NVIDIA, Intel, and leading academic institutions.

MLCommons firmographics

Firmographics
Name
MLCommons
Legal name
MLCommons Association
Website
https://mlcommons.org
Company type
Private
Founded year
2018
Operating status
Operating
Headcount range
51–100 employees
Short description
MLCommons Association is a non-profit consortium developing open MLPerf benchmarks and AILuminate AI safety benchmarks, serving 125+ member organizations across chip vendors, hyperscalers, AI labs, and academia.
Ownership category
akta.pro rank

MLCommons industry classification

Industry
Product category
AI Benchmarking and Standards
NAICS
Professional Organizations (813920)
SIC
Services-Testing Laboratories (8734)
akta.pro industry
PRIMARYHDAAABAA
End-to-End MLOps & ML Platform Suites
Hardware, Technology & Telecom/Artificial Intelligence & Machine Learning/ML Platforms, MLOps & Model Lifecycle Management
SECONDARYHDAAABAH
Model Testing, Validation & Quality Assurance
Hardware, Technology & Telecom/Artificial Intelligence & Machine Learning/ML Platforms, MLOps & Model Lifecycle Management
SECONDARYHDAAABAK
ML Lifecycle Collaboration & Workspace Management
Hardware, Technology & Telecom/Artificial Intelligence & Machine Learning/ML Platforms, MLOps & Model Lifecycle Management

Keywords

  • AI benchmark suites
  • Machine learning standards
  • AI performance measurement
  • MLPerf benchmarks
  • AI safety evaluation

Where MLCommons is headquartered

Location

Headquarters

HQ city
San Francisco
HQ country
United States
HQ region
North America

Markets served

MLCommons business model

Business model
GTM type
B2B
Offering type
Software
Cost components
Personnel, Technology or R&D, Operations, Others

Revenue model

  1. Membership Fees: MLCommons operates as a membership-based consortium with over 125 members including startups, leading companies, academics, and non-profits. Membership is required for participation in most benchmark working groups and provides access to collaboration resources and decision-making processes.

Go-to-market motion1 record

Distribution channels5 records

Marketing channels8 records

MLCommons product offering

Product offering

Core offering

MLCommons is a non-profit consortium that develops open industry-standard benchmarks and data tooling for measuring AI system quality, performance, and risk. Its core offerings include the MLPerf benchmark suites (Training, Inference Datacenter/Edge/Mobile/Tiny, Storage, Client, Automotive, Endpoints), AILuminate AI safety benchmarks covering 12 hazard categories, AlgoPerf training algorithm benchmarks, and the Croissant metadata standard used by 700k+ datasets.

Product overview

MLCommons is a nonprofit organization operating as a benchmarking and standards body rather than a traditional product company. Its portfolio consists of industry-standard benchmark suites, data standards, and research tools. The core MLPerf family includes MLPerf Training (measuring model training speed), MLPerf Inference variants for Datacenter, Edge, Mobile, and Tiny environments, plus specialized suites for Automotive and Client systems. AILuminate provides AI safety benchmarking across twelve hazard categories. AlgoPerf measures training algorithm optimization speedups. Supporting infrastructure includes MLCube containers, the Croissant metadata standard (used by 700,000+ datasets), and Chakra execution traces. The organization publishes results from 89.7K+ benchmark submissions and maintains over 125 member organizations across industry and academia.

Differentiator

Problem solved

Functional benefit

Brands

  • AILuminate: AI safety and security benchmarks assessing genAI across 12 hazard categories.
  • AlgoPerf
  • MLPerf Training
  • MLPerf Inference
  • MLPerf Mobile
  • MLPerf Client
  • MLPerf Automotive
  • MLPerf Inference Tiny
  • MLPerf Storage
  • AILuminate Jailbreak

Products and services

  • MLPerf Training Industry-standard benchmark measuring how fast systems can train AI models to a target quality metric, covering LLMs, image generation, recommendation, GNNs, BERT, object detection, and speech recognition.
  • MLPerf Inference: Datacenter Benchmark measuring how fast datacenter systems process inputs and produce results using trained models across Server, Offline, Single Stream, and Multiple Stream scenarios with latency constraints.
  • MLPerf Inference: Edge Benchmark measuring AI inference performance on edge systems, testing how fast systems can process inputs and produce results using trained models.
  • MLPerf Inference: Mobile Benchmark measuring AI performance on consumer mobile devices (smartphones, tablets, notebooks), testing LLMs, image classification, object detection, segmentation, and language processing.
  • MLPerf Inference: Tiny Benchmark measuring AI inference performance on ultra-low-power systems including image classification, keyword spotting, person detection, and anomaly detection.
  • MLPerf Storage Benchmark measuring storage system performance for AI training workloads, testing how fast storage can supply training data to keep GPUs/accelerators fed.
  • MLPerf Client Benchmark evaluating AI performance on personal computers (laptops, desktops, workstations), testing LLMs such as Llama 3.1 8B and Phi models for code analysis, content generation, and summarization tasks.
  • MLPerf Automotive Benchmark suite measuring AI performance for automotive ADAS/AD perception and In-Vehicle Infotainment systems, testing 2D/3D object detection and semantic segmentation on Cognata and nuScenes datasets.
  • MLPerf Endpoints Benchmark for evaluating GenAI API endpoint performance using an OpenAI-compatible API interface, with Pareto curves measuring throughput vs interactivity tradeoffs across the full operating range.
  • AILuminate AI safety and security benchmark suite assessing generative AI systems across twelve hazard categories, including Safety v1.0 for safety response evaluation and Jailbreak v0.5 for adversarial prompt resistance testing.
  • AlgoPerf: Training Algorithms Benchmark measuring neural network training speedups due to algorithmic improvements (optimizer, hyperparameters), using eight fixed workloads on standardized 8x V100 GPU hardware.
  • Croissant Metadata Standard Metadata format standardizing how ML datasets are described, making ML work easier to reproduce and replicate, adopted by over 700,000 datasets.
  • MLCube Containerization standard for packaging and running ML models consistently, improving AI ease-of-use and enabling AI to scale to more users.

Quantifiable outcome

  • 89.7k+ MLPerf Performance Results to-date
  • +3 more outcomes

Companies that use MLCommons

Customer profile

Named customers11 records

Segments5 records

Ideal customer profiles4 records

MLCommons technology and API

Technology

Technology focussed Yes

API detail

Has API
No
API docs
API detail

Core technology

AI maturity

App detail

AI capability15 records

Feature8 records

MLCommons partnerships and signals

Strategic signal

Partnerships

18 partnerships are on record, tiered minor, core and major.

  • New York Academy of SciencesminorStrategic or Co-development Partner · 12 May 2026Co-hosted the 3rd annual 'New Wave of AI in Healthcare' symposium with Icahn School of Medicine at Mount Sinai, discussing AI's expanding role in healthcare delivery, diagnostics, and drug discovery.
  • Icahn School of Medicine at Mount SinaiminorStrategic or Co-development Partner · 12 May 2026Co-hosted the 3rd annual 'New Wave of AI in Healthcare' symposium with the New York Academy of Sciences, bringing together researchers, clinicians, and industry leaders to discuss AI's role in healthcare.
  • Autonomous Vehicle Compute Consortium (AVCC)coreStrategic or Co-development PartnerMLCommons partners with AVCC on the MLPerf Automotive benchmark suite. AVCC provides technical reports (TR003, TR004, TR007) and MLCommons provides the benchmarking infrastructure. The collaboration focuses on developing industry-standard ML benchmarks for automotive computing platforms used in ADAS/AD and IVI systems.
  • AMDcoreTechnology or IntegrationAMD actively submits results to MLPerf benchmark suites, including Inference v6.0 and Training v6.0. AMD showcases its GPU performance through MLCommons benchmarks as part of its AI hardware validation.
  • NVIDIAcoreTechnology or IntegrationNVIDIA is a key benchmark submitter to MLPerf suites. NVIDIA's Blackwell Ultra architecture topped the MLPerf Inference v5.1 leaderboard, achieving up to 5,842 tokens per second per GPU offline.
  • GooglecoreTechnology or IntegrationGoogle submits benchmark results and contributes to working groups. Google DeepMind team members lead multimodal safety benchmarking workstream within the AI Risk & Reliability working group.
  • Microsoft AzuremajorTechnology or IntegrationMicrosoft Azure submits cloud-based AI training benchmark results and contributes to MLPerf Training working groups. Microsoft CoreAI team works on AI evaluation tooling.
  • OraclemajorTechnology or IntegrationOracle participated in MLPerf Training v6.0 benchmark submissions as one of 24 organizations submitting results.
  • MetacoreTechnology or IntegrationMeta collaborates with MLCommons on MLPerf Inference benchmarks and is listed as a key collaboration partner for increased adoption of large-scale multi-node systems.
  • ShopifymajorTechnology or IntegrationShopify collaborates with MLCommons on MLPerf Inference benchmarks, contributing to the diverse set of industry participants.
  • UltralyticsmajorTechnology or IntegrationUltralytics collaborates on MLPerf Inference benchmarks as part of the expanding ecosystem of AI system developers.
  • IBMmajorTechnology or IntegrationIBM Storage Scale participates in MLPerf Storage v2.0 benchmarks, achieving 656.7 GiB/s read bandwidth for 1T parameter model training.
  • Singapore Ministry of Digital Development and InformationmajorStrategic or Co-development PartnerGoogle and Singapore's Ministry expanded their National AI Partnership, building on a 2022 agreement to deploy AI across public services. This partnership will serve as a model for national AI programs.
  • U.S. National Science Foundation (NSF)majorStrategic or Co-development PartnerNSF launched the National AI Research Resource (NAIRR) pilot, partnering with MLCommons and 25 other organizations to democratize AI research access. Initial focus on safe, secure AI and healthcare applications.
  • UK AI Security InstitutecoreStrategic or Co-development PartnerThe layered assurance ecosystem combines lab self-attestation with government institute validation from UK AI Security Institute and independent third-party evaluation from organizations including MLCommons.
  • Oxford Internet InstituteminorStrategic or Co-development PartnerResearchers from Oxford Internet Institute and Nuffield Department of Primary Care Health Sciences at the University of Oxford collaborated with MLCommons on a medical advice study comparing AI chatbots against traditional methods.
  • Nuffield Department of Primary Care Health Sciences, University of OxfordminorStrategic or Co-development PartnerCollaborated with MLCommons on a study involving 1,298 people in the UK comparing effectiveness of AI chatbots (GPT-4o, Llama 3, Command R) against traditional methods for assessing medical symptoms.
  • CognataminorStrategic or Co-development PartnerCognata provides the MLCommons Cognata Dataset used in MLPerf Automotive benchmarks for 2D object detection and semantic segmentation testing.

Scale indicators6 records

Recent moves6 records

Expansion highlights7 records

MLCommons competitors and assessment

Company assessment

Direct peers

  • HELM (Stanford CRFM): Stanford's Holistic Evaluation of Language Models is a leading academic benchmark for evaluating large language models across accuracy, calibration, robustness, fairness, and efficiency. Directly comparable to MLCommons as a primary AI benchmarking initiative targeting transparency and reproducibility.
  • LMSYS Chatbot Arena: Large Model Systems Organization's crowdsourced LLM benchmark using pairwise human preference votes. Comparable to MLCommons as a widely-cited third-party AI evaluation platform with community-driven methodology.
  • Hugging Face Open LLM Leaderboard: Hugging Face's benchmark for open-source LLMs evaluating reasoning, knowledge, and truthfulness. Comparable to MLCommons' Inference benchmarks as a public-facing AI evaluation reference for the open-source community.
  • Partnership on AI: Multi-stakeholder nonprofit organization developing best practices on AI safety, fairness, and governance. Directly comparable to MLCommons as a similar consortium model with industry, academia, and civil society members focused on responsible AI development.

Broad incumbents

  • Apache Software Foundation: Open-source software foundation operating as a membership-driven consortium with vendor-neutral governance. Comparable to MLCommons as a precedent for industry consortium structures that develop shared technical standards with member-funded operations.
  • Linux Foundation: Largest open-source foundation supporting Linux, Kubernetes, and other critical infrastructure projects. Comparable to MLCommons as a mature model of vendor-neutral consortium governance and standards stewardship that MLCommons aspires to for AI benchmarking infrastructure.
  • IEEE Standards Association: Major international standards body developing technical standards across industries including AI. Comparable to MLCommons as a precedent for member-driven technology standardization, though much larger and more established.

Emerging players

  • OpenAI Evals: OpenAI's open-source evaluation framework for testing LLM capabilities. Comparable as an emerging benchmark initiative but single-vendor-driven, contrasting with MLCommons' multi-stakeholder approach.
  • EleutherAI Evaluation Harness: Open-source framework for evaluating autoregressive language models across hundreds of standardized tasks. Comparable to MLCommons as a community-driven AI evaluation tool, though focused primarily on academic and open-model use cases.

Others

Market position

Strengths5 records

Weaknesses5 records

Competitive moat5 records

Key risks6 records

Key highlights6 records

Customer concentration

MLCommons social profiles

Digital presence

MLCommons financial estimates

Financial estimate

Revenue estimate

Valuation estimate

MLCommons leadership team

Management profile

Number of profiles

Profiles8 records

MLCommons funding detail

Funding detail

Funding overview

Funding rounds

Investors

Funding detail is available on the Subscription and Enterprise plan.Contact sales →

MLCommons M&A and investment

M&A and investment

M&A

Investments

M&A and investment is available on the Subscription and Enterprise plan.Contact sales →

Frequently asked questions about MLCommons

What does MLCommons do?

MLCommons is a non-profit consortium that develops open industry-standard benchmarks and data tooling for measuring AI system quality, performance, and risk. Its core offerings include the MLPerf benchmark suites (Training, Inference Datacenter/Edge/Mobile/Tiny, Storage, Client, Automotive, Endpoints), AILuminate AI safety benchmarks covering 12 hazard categories, AlgoPerf training algorithm benchmarks, and the Croissant metadata standard used by 700k+ datasets.

Is MLCommons a public or private company?

MLCommons is a private company. It is classified as nonprofit foundation owned and is currently operating.

When was MLCommons founded?

MLCommons was founded in 2018. It employs 51 to 100 people.

Where is MLCommons based?

MLCommons is headquartered in San Francisco, United States, in the North America region.

How does MLCommons make money?

One revenue line is on record: membership Fees.

Who are MLCommons's main competitors?

Direct peers on record are HELM (Stanford CRFM), LMSYS Chatbot Arena, Hugging Face Open LLM Leaderboard and Partnership on AI. Broad incumbents are Apache Software Foundation, Linux Foundation and IEEE Standards Association. Emerging players are OpenAI Evals and EleutherAI Evaluation Harness. MLPerf (benchmark suite brand) is listed as an others.

Does MLCommons have an API?

No public API is recorded for MLCommons.

What industry is MLCommons in?

MLCommons's product category is AI Benchmarking and Standards. Its primary akta.pro industry code is HDAAABAA, End-to-End MLOps & ML Platform Suites, with a secondary code of HDAAABAH, Model Testing, Validation & Quality Assurance. Its NAICS code is 813920 and its SIC code is 8734.

Unlock the full company data

50 free credits on sign-up, no credit card required.

Contact sales
Live signals
HpcwireMLCommons Releases MLPerf Training v6.1 Results, Spotlighting Recent Advances in AI Training - HPCwireMLCommons released MLPerf Training v6.1 results, adding two new benchmarks: a recommender system and a large-scale LLM post-training workload. The round saw 153 systems and 250 results, with 37% cloud-based and 12 systems using 1,000+ accelerators. The results highlight diverse hardware and increased participation.Naver피지컬AI 주도권 경쟁…정부, 표준·특허 선점 나선다South Korea's Ministry of Science and ICT and the Intellectual Property Office will hold the 10th GISC conference from Feb 2-4 in Seoul, focusing on physical AI and agent AI standards and patents. The event will feature MLCommons, IEEE, and ETSI, and the government aims to secure international standards and patents in AI.Nate과기정통부, 지식재산처와 '글로벌 ICT 표준 컨퍼런스' 개최 : 네이트 뉴스The Ministry of Science and ICT and the Intellectual Property Office will hold the 2026 Global ICT Standards Conference in Seoul on April 4. About 1,000 experts will discuss AI, quantum, and satellite communication standards, with MLCommons, IEEE, and ETSI participating. The event includes keynote speeches, token conferences, and awards for 27 institutions and companies.HpcwireSince 1987 – Covering the Fastest Computers in the World and the People Who Run ThemMLCommons released MLPerf Inference v6.1 benchmark results, introducing End-to-End RAG and Edge Agentic Inference tests. The benchmark saw a record 30 submitters and up to 5.7X performance gains over a year. It also added support for speculative decoding and API-centric harness.HpcwireSince 1987 – Covering the Fastest Computers in the World and the People Who Run ThemMLCommons added KV cache, vector database, and S3 support to its MLPerf Storage benchmark. The new metrics target AI inference workloads, with 20 submissions for KV cache and 19 for vector DB. The group plans to expand support for Azure, Google, and OCI protocols.HpcwireSince 1987 – Covering the Fastest Computers in the World and the People Who Run ThemMLCommons released MLPerf Storage v3.0 benchmark results, adding KV cache and vector database tests and S3 object storage support. Nineteen organizations submitted, including eleven first-time submitters, with on-premises checkpointing write test achieving a median of 14 GB/second per watt.FinancialContent Business PageMLCommons Releases New MLPerf Storage v3.0 Benchmark ResultsMLCommons released MLPerf Storage v3.0 benchmark results, adding KV cache and vector database tests and S3 object storage support. Nineteen organizations submitted, including 11 first-time submitters, with on-premises checkpointing write test achieving a median of 14 GB/second per watt.Blockchain NewsDeepMind Launches First Double-Blind AI Model EvaluationGoogle DeepMind announced on August 27, 2026 a double-blind evaluation of its Gemini Flash Lite model, using cryptographically secure environments to prevent evaluators and model creators from biasing tests. The pilot, with partners including the Singapore AI Safety Institute, OpenMined and MLCommons, aims to curb benchmark contamination and inflated results. DeepMind said wider adoption could shape AI policy and investment.Google DeepMindPiloting the world's first double-blind AI evaluationsGoogle announced a double-blind evaluation of its Gemini Flash Lite model, conducted in a cryptographic "box" that prevents external evaluators from using test prompts to optimize performance ahead of testing. The evaluation partners include the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons, using confidential benchmarks. Google said this technical safeguard complements existing zero-logging protocols and contractual safeguards.GlobeNewswireMLPerf Client v2.0 Expands AI PC Benchmarking with Image Generation and Agentic AIMLCommons released MLPerf Client v2.0, an AI benchmark for personal computers, adding Agentic AI and Image Generation categories. The update includes Flux.2 klein 4B and Qwen 3 8B as experimental tests, and reports end-to-end performance for agentic AI scenarios. The benchmark is freely available for download.