ML Perf
MLCommons Association (MLPerf) is a non-profit membership consortium that operates the MLPerf benchmark suite, providing open, standardized performance, storage, and AI safety benchmarks for AI hardware vendors, cloud platforms, researchers, and enterprise AI buyers across 12+ benchmark products.
- Company typePrivate
- Founded2018
- HeadquartersMountain View, United States
- Headcount501–1,000
- GTM typeB2B
- OfferingSoftware
What ML Perf does
MLCommons Association (doing business as MLPerf) is a Mountain View-based non-profit association founded in 2018 that operates the MLPerf benchmark suite, the most widely referenced open benchmark framework for evaluating AI/ML system performance. The organization serves three primary segments: commercial AI and hardware companies that need reproducible benchmarks to compare competing chips, systems, and cloud platforms; research institutions and academia that need standardized benchmarks for evaluating algorithms and training methods; and enterprise AI buyers that use benchmark results to inform procurement decisions.
MLCommons' product portfolio spans 12+ benchmarks organized into the 'AI model tripod' of model, dataset, and hardware measurement. The MLPerf Inference suite covers Datacenter, Edge, Mobile, Tiny, and Endpoints; MLPerf Training covers both standard and HPC-scale training; MLPerf Client evaluates LLM and AI workloads on PCs; MLPerf Automotive covers ADAS/AD and IVI systems; MLPerf Storage measures how fast storage feeds training data; AILuminate evaluates chatbot generative AI safety across Safety, Jailbreak, Agentic, and Multimodal dimensions; and AlgoPerf isolates gains from training algorithm optimization. Benchmarks are governed by expert working groups that enforce standardized rules around models, datasets, and measurement methodology.
The business model is a membership consortium funded by recurring member dues paid by technology companies, research institutions, and other stakeholders. Most working groups require membership; some public working groups allow non-member submissions via a Non-member Test Agreement. The organization also offers trademark licensing agreements. Go-to-market is community-led, relying on open-source repositories, GitHub, Discord, X, LinkedIn, YouTube, working group meetings, and industry events for member acquisition and engagement.
ML Perf firmographics
Firmographics- Name
- ML Perf
- Legal name
- MLCommons Association
- Website
- https://mlperf.org
- Company type
- Private
- Founded year
- 2018
- Operating status
- Operating
- Headcount range
- 501–1,000 employees
- Short description
- MLCommons Association (MLPerf) is a non-profit membership consortium that operates the MLPerf benchmark suite, providing open, standardized performance, storage, and AI safety benchmarks for AI hardware vendors, cloud platforms, researchers, and enterprise AI buyers across 12+ benchmark products.
- Ownership category
- akta.pro rank
ML Perf industry classification
Industry- Product category
- AI/ML Performance Benchmarking
- NAICS
- Software Publishers (5132), Software Publishers (51321)
- SIC
- Services-Prepackaged Software (7372), Services-Computer Programming, Data Processing, Etc. (7370)
- akta.pro primary industry
- Model Testing, Validation & Quality Assurance (HDAAABAH)
- akta.pro secondary industries
- End-to-End MLOps & ML Platform Suites (HDAAABAA), Confidential AI & Privacy-Preserving ML (federated learning, MPC, HE, TEEs) (HDAAAKAI), MLOps/LLMOps & Model Lifecycle Management Services (BPAEAHAH)
Keywords
Where ML Perf is headquartered
LocationHeadquarters
- HQ city
- Mountain View
- HQ country
- United States
- HQ region
- North America
Markets served
ML Perf business model
Business model- GTM type
- B2B
- Offering type
- Software
- Cost components
- Personnel, Technology or R&D, Operations, Marketing or Sales, Others
Revenue model
- Membership Fees: MLCommons operates as a membership-based organization where technology companies, researchers, and other stakeholders pay membership fees to participate in benchmark development and working groups.
Go-to-market motion1 record
Distribution channels1 record
Marketing channels5 records
ML Perf product offering
Product offeringCore offering
ML Perf (MLCommons Association) develops and maintains MLPerf, an open benchmarking suite that evaluates the performance of AI/ML software frameworks, hardware accelerators, and cloud platforms across training speed, inference speed, storage, and AI safety. The suite covers datacenter, edge, mobile, tiny, client, automotive, and HPC-scale environments, and is delivered through expert working groups and standardized rules that enable fair, reproducible comparison of competing AI systems. The organization operates as a community-led non-profit consortium funded primarily through membership fees.
Product overview
MLPerf is a comprehensive benchmark suite for AI/ML performance measurement, delivered through MLCommons as an open-source benchmarking framework. The portfolio includes MLPerf Inference benchmarks (Datacenter, Edge, Mobile, Tiny, and Endpoints) that measure how fast systems process inputs using trained models, MLPerf Training benchmarks (standard and HPC) that measure training speed to target quality, specialized MLPerf Client for PC workloads, MLPerf Automotive for ADAS/AD and IVI systems, and MLPerf Storage for data supply chain evaluation. Additional offerings include AILuminate for AI safety assessment of chatbot systems and AlgoPerf for training algorithm optimization research. These benchmarks work within a defined AI model tripod framework—evaluating the AI model itself, the training dataset, and hardware performance—to provide fair, reproducible, and comprehensive performance measurement across commercial and research communities.
Differentiator
Problem solved
Functional benefit
Brands
- MLPerf: A registered trademark and benchmark suite that delivers representative benchmarks for AI/ML systems performance evaluation.
Products and services
- MLPerf Client Evaluates the performance of large language models (LLMs) and other AI workloads on personal computers, from laptops and desktops to workstations.
- MLPerf Inference: Datacenter Measures how fast systems can process inputs and produce results using a trained model in datacenter environments.
- MLPerf Inference: Edge Measures how fast systems can process inputs and produce results using a trained model on edge systems with constrained resources.
- MLPerf Inference: Mobile Measures how fast consumer mobile devices with different AI chips and software stacks can process inputs and produce results using a trained model.
- MLPerf Inference: Tiny Measures how fast systems can process inputs and produce results using a trained model on ultra-low-power embedded systems.
- MLPerf Training Measures how fast systems can train neural network models to a target quality metric.
- MLPerf Training: HPC Measures how fast systems can train models to a target quality metric running on large-scale supercomputers.
- MLPerf Automotive Measures the performance of computers intended for automotive use, including Advanced Driving Assistance System/Autonomous Driving (ADAS/AD) systems and In-Vehicle Infotainment (IVI) embedded systems.
- MLPerf Endpoints Measures how fast systems can process inputs and produce results using a trained model through API endpoints.
- MLPerf Storage Measures how fast storage systems can supply training data when a neural network model is being trained.
- AILuminate The MLCommons AILuminate benchmark assesses the safety of general chatbot generative AI systems to help guide development, inform purchasers and consumers, and support standards bodies and policymakers.
- AlgoPerf: Training Algorithms Measures how much faster neural network models can be trained to a given target performance by changing the underlying training algorithm rather than hardware.
Quantifiable outcome
- Standardized benchmarks enable reproducible and comparable AI performance measurements across competing systems
Companies that use ML Perf
Customer profileSegments3 records
Ideal customer profiles3 records
ML Perf technology and API
TechnologyTechnology focussed Yes
API detail
- Has API
- No
- API docs
- API detail
Core technology
AI maturity
App detail
AI capability11 records
Feature5 records
ML Perf partnerships and signals
Strategic signalPartnerships
One partnership is on record.
- MLCommons MemberscoreMLCommons operates as a membership consortium with technology companies, research institutions, and other stakeholders as members. Members contribute to benchmark development, participate in working groups, and help define fair benchmarks for AI systems. The organization has a Rising Stars Program for emerging talent.
Scale indicators2 records
Recent moves6 records
Expansion highlights7 records
ML Perf competitors and assessment
Company assessmentBroad incumbents
- UL Solutions (3DMark / PCMark): Major commercial benchmarking brand (3DMark, PCMark) used for graphics and system performance. Comparable as a widely-recognized, paid benchmarking suite informing hardware purchasing, though operated commercially rather than as a non-profit consortium.
- Geekbench (Primate Labs): Long-standing cross-platform CPU/GPU benchmarking suite widely cited by hardware reviewers and chip vendors. Comparable as a standardized performance benchmark consumed in AI-adjacent hardware purchasing decisions, though broader in scope (general compute, not ML-specific).
- PassMark Software: Independent software vendor producing system-performance benchmarks (CPU, GPU, memory) used by enterprise buyers. Comparable as a third-party performance benchmark feeding hardware procurement decisions, though commercial rather than consortium-governed.
- NIST AI Safety Institute: US government body developing AI evaluations and safety standards, including for generative AI. AILuminate's stated mission to support 'standards bodies and policymakers' positions MLCommons as a potential private-sector partner/peer to NIST's AI evaluation work.
Emerging players
- Hugging Face Open LLM Leaderboard: Community-maintained leaderboard benchmarking open large language models on standardized tasks. Directly comparable as an open, community-driven ML benchmark effort — though Hugging Face is also a broader model hub, not a pure-play standards body.
- MLPerf Training alternative: DAWNBench (Stanford CRFM): Stanford-hosted ML training/inference benchmark that preceded and inspired parts of MLPerf. Comparable as a community-led ML benchmarking initiative aimed at measuring end-to-end training cost and time, though academic and lower-scale than MLCommons.
- Epoch AI: Research organization tracking AI capabilities, compute trends, and model performance benchmarks. Overlaps with MLCommons on AI capability benchmarking and data, though Epoch AI leans more research/analytics than standards-setting.
Direct peers
- Transaction Processing Performance Council (TPC): Non-profit consortium publishing standardized database and transaction-processing benchmarks. Shares MLCommons' membership-funded, community-governed structure for producing reproducible performance benchmarks across competing vendor systems.
- SPEC (Standard Performance Evaluation Corporation): Industry consortium that defines standardized benchmarks for compute and server performance. Highly analogous governance and revenue model (membership-funded, benchmark-driven), and the closest historical analogue to MLPerf's role as a neutral AI performance yardstick.
Others
- Apache Software Foundation (benchmarking projects): Non-profit foundation stewarding open-source projects including ML-relevant tooling. Comparable governance pattern (vendor-funded, community-governed, foundation-managed) though broader in scope than ML performance benchmarking.
Market position
Strengths4 records
Weaknesses5 records
Competitive moat5 records
Key risks5 records
Key highlights7 records
Customer concentration
ML Perf social profiles
Digital presenceML Perf financial estimates
Financial estimateRevenue estimate
Valuation estimate
ML Perf leadership team
Management profileNumber of profiles
Profiles2 records
ML Perf funding detail
Funding detailFunding overview
Funding rounds
Investors
Funding detail is available on the Subscription and Enterprise plan.Contact sales →
ML Perf M&A and investment
M&A and investmentM&A
Investments
M&A and investment is available on the Subscription and Enterprise plan.Contact sales →
Frequently asked questions about ML Perf
What does ML Perf do?
ML Perf (MLCommons Association) develops and maintains MLPerf, an open benchmarking suite that evaluates the performance of AI/ML software frameworks, hardware accelerators, and cloud platforms across training speed, inference speed, storage, and AI safety. The suite covers datacenter, edge, mobile, tiny, client, automotive, and HPC-scale environments, and is delivered through expert working groups and standardized rules that enable fair, reproducible comparison of competing AI systems. The organization operates as a community-led non-profit consortium funded primarily through membership fees.
Is ML Perf a public or private company?
ML Perf is a private company. It is classified as nonprofit foundation owned and is currently operating.
When was ML Perf founded?
ML Perf was founded in 2018. It employs 501 to 1,000 people.
Where is ML Perf based?
ML Perf is headquartered in Mountain View, United States, in the North America region.
How does ML Perf make money?
One revenue line is on record: membership Fees.
Who are ML Perf's main competitors?
Broad incumbents on record are UL Solutions (3DMark / PCMark), Geekbench (Primate Labs), PassMark Software and NIST AI Safety Institute. Emerging players are Hugging Face Open LLM Leaderboard, MLPerf Training alternative: DAWNBench (Stanford CRFM) and Epoch AI. Direct peers are Transaction Processing Performance Council (TPC) and SPEC (Standard Performance Evaluation Corporation). Apache Software Foundation (benchmarking projects) is listed as an others.
Does ML Perf have an API?
No public API is recorded for ML Perf.
What industry is ML Perf in?
ML Perf's product category is AI/ML Performance Benchmarking. Its primary akta.pro industry code is HDAAABAH, Model Testing, Validation & Quality Assurance, with a secondary code of HDAAABAA, End-to-End MLOps & ML Platform Suites. Its NAICS code is 5132 and its SIC code is 7372.