Developer docs
API playgroundTry for free, no card

Search company profiles

Quesma

Full company profile

uuid0002dn7

Namestring
Quesma
Legal namestring
Quesma Poland Sp. z o.o.
Websiteurl
quesma.com
Company typeenum
Private
Founded yearint
2023
Operating statusenum
Operating
Ownership categoryenum
Headcount rangeband
11–50
akta.pro rankint
HeadquartersWarsaw, Poland
HQ citystring
Warsaw
HQ countrystring
Poland
HQ regionstring
Europe
Markets served

Serves global market

Offices1 record

Each record includes

City, Country, Type, Description, Source

Keyword5 values
AI agent evaluation, LLM benchmarking, open-source AI benchmarks, AI code generation testing, AI training rewards
Industry3 codes
PRIMARYHDAEANAA
End-to-End Enterprise AI Platforms (MLOps & Model Lifecycle Management)
Hardware, Technology & Telecom/Enterprise Platforms & Databases/Artificial Intelligence Platforms for Enterprise
SECONDARYHDAAAKAC
Model Security Testing & Red Teaming (adversarial ML, jailbreaks)
SECONDARYHDAEANAB
Model Development & Training Platforms (AutoML, Notebooks, Feature Stores)
Hardware, Technology & Telecom/Enterprise Platforms & Databases/Artificial Intelligence Platforms for Enterprise
NAICS code3 codes
  • Software Publishers5132
  • Custom Computer Programming Services541511
  • Other Computer Related Services541519
SIC code2 codes
  • Services-Prepackaged Software7372
  • Services-Computer Programming, Data Processing, Etc.7370
Product category
AI Agent Evaluation and Benchmarking
GTM motion1 record

Each record includes

Type, Description, Source

Revenue model2 records
1AI Agent Evaluation Services
TypeSubscription Recurring
Description

Provides independent verification and benchmarking services for AI labs and enterprise buyers seeking to evaluate AI agent capabilities. Likely subscription or project-based pricing.

quesma.com
2Quesma Charts (B2C Product)
TypeFreemium
Description

AI-powered chart generation tool using ggplot2 for professional data visualization. Alpha version launched for individual users.

charts.quesma.com
Marketing channels5 records

Each record includes

Title, Type, Stage, Description, Source

Distribution channels2 records

Each record includes

Title, Type, Scope, Target buyer, Description, Source

Cost components4 values
Personnel, Technology or R&D, Marketing or Sales, Infrastructure
GTM typeB2B
B2B
Offering typeSoftware
Software
Brand1 record
1Quesma Charts
Description

AI-powered data visualization tool that turns CSV, Excel, or SQL data into clean, accurate, and professional visuals using prompts.

charts.quesma.com
Core offering1 text field

Quesma provides independent evaluation and training services for the AI agent ecosystem, built on open-source benchmarks such as BinaryAudit, OTelBench, CompileBench, Tau², and SWE-Bench Pro. Its offering combines realistic multi-hour simulation environments with cheat-proof reward functions for reinforcement learning training of AI agents. The company also operates Quesma Charts, an open-source AI-powered data visualization tool.

Differentiator
Functional benefit
Problem solved
Quantifiable outcome1 of 3 values shown
  • Claude Opus 4.6 achieved 49% detection rate on BinaryAudit (finding backdoors in binaries)
+2 more records
Product overview1 text field

Quesma is a technology company that develops a suite of open-source AI benchmarking and testing tools. The portfolio includes OTelBench for evaluating LLMs on OpenTelemetry instrumentation, CompileBench for testing AI on compilation tasks, BinaryAudit for security analysis of binaries, Quesma Charts for AI-powered data visualization, Tau² for AI agent evaluation, and SWE-Bench Pro for testing AI on real GitHub issues. The company focuses on creating evaluation frameworks to assess AI model capabilities across software engineering, security, and instrumentation tasks.

Product and service4 records
1OTelBench
CategoryAI benchmarking tool
Description

Open-source benchmark that evaluates large language models on OpenTelemetry instrumentation tasks, testing 14 state-of-the-art models across 23 real-world tasks in 11 programming languages. Used by AI app developers and frontier labs to measure model capability on observability code generation.

2BinaryAudit
CategorySecurity analysis tool
Description

Open-source benchmark testing AI agents on detecting hidden backdoors in compiled binaries of real open-source servers, proxies, and network infrastructure. Uses Ghidra and Radare2 for reverse engineering and static analysis. Aimed at AI security researchers and frontier labs.

3CompileBench
CategoryAI benchmarking tool
Description

Open-source benchmark testing LLMs on real-world compilation tasks such as dependency resolution, Makefile analysis, legacy toolchain support, cross-compilation, and resurrecting old code. Used to evaluate AI agent capability on software engineering build challenges.

4Tau²
Scale indicator1 record

Each record includes

Type, Value, Description, Source

Recent move6 records

Each record includes

Date, Type, Title, Description, Source

Expansion highlight5 records

Each record includes

Type, Description

Peers10 records
TypeBroad incumbent
Description

Scale AI runs SWE-Bench Pro (which Quesma uses) and offers evaluation services for frontier AI models. Both companies provide AI benchmarking and enterprise verification, but Scale AI is far larger and operates as a broad data/eval infrastructure incumbent.

TypeBroad incumbent
Description

Hugging Face operates the Open LLM Leaderboard and extensive model evaluation infrastructure. Both companies publish open-source AI benchmarks and serve the developer community, though Hugging Face is a much broader model-and-data platform.

TypeDirect peer
Description

Patronus AI provides automated evaluation and testing for LLM applications, including agent evaluation. Both companies offer independent verification of AI capabilities for enterprise buyers and AI developers building production agents.

TypeDirect peer
Description

Galileo provides AI evaluation and observability tools for LLM-powered applications and agents. Both Quesma and Galileo target AI app developers and enterprise buyers needing to measure and improve AI agent quality.

TypeDirect peer
Description

Arize AI offers evaluation, observability, and monitoring for LLM applications. Both companies serve AI developers and enterprise teams deploying AI agents, providing independent quality measurement and performance tracking.

TypeDirect peer
Description

Braintrust provides an AI evaluation platform for LLM applications with scoring, tracing, and benchmarking. Both companies enable AI app developers to measure quality and pick optimal models in a fast-changing landscape.

TypeEmerging player
Description

Langfuse offers open-source LLM observability and evaluation tooling. Both Quesma and Langfuse target developers building AI agents with open-source, developer-led distribution models.

TypeBroad incumbent
Description

Weights & Biases provides MLOps and model evaluation tooling as part of its broader platform. Both companies support AI developers in tracking, benchmarking, and improving AI model performance.

TypeEmerging player
Description

WhyLabs provides AI observability and monitoring for LLM applications. Both companies focus on independent measurement of AI system behavior and serve enterprise buyers deploying AI in production.

TypeDirect peer
Description

Vellum offers LLM evaluation, prompt testing, and benchmarking tooling for AI developers and enterprises. Both companies provide AI quality measurement and model selection capabilities.

Market position
Strengths4 records

Each record includes

Headline, Details, Source

Weaknesses5 records

Each record includes

Headline, Details, Source

Competitive moat4 records

Each record includes

Type, Details

Key risks6 records

Each record includes

Headline, Details, Source

Key highlights6 records

Each record includes

Headline, Details, Source

Customer concentration

Classification, Details

Segment3 records

Each record includes

Title, Type, Primary, Description, Pain point addressed, Use case, Source

Ideal customer profile3 records

Each record includes

Profile, Firmographic size, Sales motion, Sales cycle length, Buying structure, Purchase trigger, Buyer persona, Geography, Industry vertical, Primary use case, Description, Pain points, Evidence proof points, Target buyer

Technology focused
Yes
API detail
Has APIbool
No

Docs URL, Description

AI capability9 records

Each record includes

Type, Description, Source

AI maturity
App detail

Has app

Feature4 records

Each record includes

Title, Differentiator, Description, Source

Core technology
Revenue estimate
Valuation estimate
Number of profiles
Profiles6 records

Each record includes

Name, Designation, Designation category, Overview, Profile commentary, Source

No data
No data
Funding overview

Funding stage, Last funding date, Total funding USD

Funding rounds1 record

Each record includes

Round, Amount USD, Date, Pre money valuation, Total investors, Investors, News

Investors2 records

Each record includes

Name, Type, Date of entry, Rounds participated, Website

Funding detail is available on the Subscription and Enterprise plan.Contact sales →

M&A

Each record includes

Name, Acquisition type, Announced date, Completed date, Status, Website, News

Investment

Each record includes

Name, Round, Announced date, Lead investor, Website, News

M&A and investment is available on the Subscription and Enterprise plan.Contact sales →

Quesma

AI Agent Evaluation and Benchmarkingquesma.com

Quesma firmographics

Firmographics
Name
Quesma
Legal name
Quesma Poland Sp. z o.o.
Website
https://quesma.com
Company type
Private
Founded year
2023
Operating status
Operating
Headcount range
11–50 employees
Ownership category
akta.pro rank

Quesma industry classification

Industry
Product category
AI Agent Evaluation and Benchmarking
NAICS
Software Publishers (5132), Custom Computer Programming Services (541511), Other Computer Related Services (541519)
SIC
Services-Prepackaged Software (7372), Services-Computer Programming, Data Processing, Etc. (7370)
akta.pro industry
PRIMARYHDAEANAA
End-to-End Enterprise AI Platforms (MLOps & Model Lifecycle Management)
Hardware, Technology & Telecom/Enterprise Platforms & Databases/Artificial Intelligence Platforms for Enterprise
SECONDARYHDAAAKAC
Model Security Testing & Red Teaming (adversarial ML, jailbreaks)
SECONDARYHDAEANAB
Model Development & Training Platforms (AutoML, Notebooks, Feature Stores)
Hardware, Technology & Telecom/Enterprise Platforms & Databases/Artificial Intelligence Platforms for Enterprise

Keywords

  • AI agent evaluation
  • LLM benchmarking
  • Open-source AI benchmarks
  • AI code generation testing
  • AI training rewards

Where Quesma is headquartered

Location

Headquarters

HQ city
Warsaw
HQ country
Poland
HQ region
Europe

Offices1 record

Markets served

Quesma business model

Business model
GTM type
B2B
Offering type
Software
Cost components
Personnel, Technology or R&D, Marketing or Sales, Infrastructure

Revenue model

  1. AI Agent Evaluation Services: Provides independent verification and benchmarking services for AI labs and enterprise buyers seeking to evaluate AI agent capabilities. Likely subscription or project-based pricing.
  2. Quesma Charts (B2C Product): AI-powered chart generation tool using ggplot2 for professional data visualization. Alpha version launched for individual users.

Go-to-market motion1 record

Distribution channels2 records

Marketing channels5 records

Quesma product offering

Product offering

Core offering

Quesma provides independent evaluation and training services for the AI agent ecosystem, built on open-source benchmarks such as BinaryAudit, OTelBench, CompileBench, Tau², and SWE-Bench Pro. Its offering combines realistic multi-hour simulation environments with cheat-proof reward functions for reinforcement learning training of AI agents. The company also operates Quesma Charts, an open-source AI-powered data visualization tool.

Product overview

Quesma is a technology company that develops a suite of open-source AI benchmarking and testing tools. The portfolio includes OTelBench for evaluating LLMs on OpenTelemetry instrumentation, CompileBench for testing AI on compilation tasks, BinaryAudit for security analysis of binaries, Quesma Charts for AI-powered data visualization, Tau² for AI agent evaluation, and SWE-Bench Pro for testing AI on real GitHub issues. The company focuses on creating evaluation frameworks to assess AI model capabilities across software engineering, security, and instrumentation tasks.

Differentiator

Problem solved

Functional benefit

Brands

  • Quesma Charts: AI-powered data visualization tool that turns CSV, Excel, or SQL data into clean, accurate, and professional visuals using prompts.

Products and services

  • OTelBench Open-source benchmark that evaluates large language models on OpenTelemetry instrumentation tasks, testing 14 state-of-the-art models across 23 real-world tasks in 11 programming languages. Used by AI app developers and frontier labs to measure model capability on observability code generation.
  • BinaryAudit Open-source benchmark testing AI agents on detecting hidden backdoors in compiled binaries of real open-source servers, proxies, and network infrastructure. Uses Ghidra and Radare2 for reverse engineering and static analysis. Aimed at AI security researchers and frontier labs.
  • CompileBench Open-source benchmark testing LLMs on real-world compilation tasks such as dependency resolution, Makefile analysis, legacy toolchain support, cross-compilation, and resurrecting old code. Used to evaluate AI agent capability on software engineering build challenges.
  • Tau²

Quantifiable outcome

  • Claude Opus 4.6 achieved 49% detection rate on BinaryAudit (finding backdoors in binaries)
  • +2 more outcomes

Companies that use Quesma

Customer profile

Segments3 records

Ideal customer profiles3 records

Quesma technology and API

Technology

Technology focussed Yes

API detail

Has API
No
API docs
API detail

Core technology

AI maturity

App detail

AI capability9 records

Feature4 records

Quesma partnerships and signals

Strategic signal

Scale indicators1 record

Recent moves6 records

Expansion highlights5 records

Quesma competitors and assessment

Company assessment

Broad incumbents

  • Scale AI: Scale AI runs SWE-Bench Pro (which Quesma uses) and offers evaluation services for frontier AI models. Both companies provide AI benchmarking and enterprise verification, but Scale AI is far larger and operates as a broad data/eval infrastructure incumbent.
  • Hugging Face: Hugging Face operates the Open LLM Leaderboard and extensive model evaluation infrastructure. Both companies publish open-source AI benchmarks and serve the developer community, though Hugging Face is a much broader model-and-data platform.
  • Weights & Biases: Weights & Biases provides MLOps and model evaluation tooling as part of its broader platform. Both companies support AI developers in tracking, benchmarking, and improving AI model performance.

Direct peers

  • Patronus AI: Patronus AI provides automated evaluation and testing for LLM applications, including agent evaluation. Both companies offer independent verification of AI capabilities for enterprise buyers and AI developers building production agents.
  • Galileo (Galileo AI): Galileo provides AI evaluation and observability tools for LLM-powered applications and agents. Both Quesma and Galileo target AI app developers and enterprise buyers needing to measure and improve AI agent quality.
  • Arize AI: Arize AI offers evaluation, observability, and monitoring for LLM applications. Both companies serve AI developers and enterprise teams deploying AI agents, providing independent quality measurement and performance tracking.
  • Braintrust: Braintrust provides an AI evaluation platform for LLM applications with scoring, tracing, and benchmarking. Both companies enable AI app developers to measure quality and pick optimal models in a fast-changing landscape.
  • Vellum AI: Vellum offers LLM evaluation, prompt testing, and benchmarking tooling for AI developers and enterprises. Both companies provide AI quality measurement and model selection capabilities.

Emerging players

  • Langfuse: Langfuse offers open-source LLM observability and evaluation tooling. Both Quesma and Langfuse target developers building AI agents with open-source, developer-led distribution models.
  • WhyLabs: WhyLabs provides AI observability and monitoring for LLM applications. Both companies focus on independent measurement of AI system behavior and serve enterprise buyers deploying AI in production.

Market position

Strengths4 records

Weaknesses5 records

Competitive moat4 records

Key risks6 records

Key highlights6 records

Customer concentration

Quesma social profiles

Digital presence

Quesma financial estimates

Financial estimate

Revenue estimate

Valuation estimate

Quesma leadership team

Management profile

Number of profiles

Profiles6 records

Quesma funding detail

Funding detail

Funding overview

Funding rounds1 record

Investors2 records

Funding detail is available on the Subscription and Enterprise plan.Contact sales →

Quesma M&A and investment

M&A and investment

M&A

Investments

M&A and investment is available on the Subscription and Enterprise plan.Contact sales →

Frequently asked questions about Quesma

What does Quesma do?

Quesma provides independent evaluation and training services for the AI agent ecosystem, built on open-source benchmarks such as BinaryAudit, OTelBench, CompileBench, Tau², and SWE-Bench Pro. Its offering combines realistic multi-hour simulation environments with cheat-proof reward functions for reinforcement learning training of AI agents. The company also operates Quesma Charts, an open-source AI-powered data visualization tool.

Is Quesma a public or private company?

Quesma is a private company. It is classified as venture growth investor backed and is currently operating.

When was Quesma founded?

Quesma was founded in 2023. It employs 11 to 50 people.

Where is Quesma based?

Quesma is headquartered in Warsaw, Poland, in the Europe region.

How does Quesma make money?

Two revenue lines are on record. AI Agent Evaluation Services are the primary driver. The others are quesma Charts (B2C Product).

Who are Quesma's main competitors?

Broad incumbents on record are Scale AI, Hugging Face and Weights & Biases. Direct peers are Patronus AI, Galileo (Galileo AI), Arize AI, Braintrust and Vellum AI. Emerging players are Langfuse and WhyLabs.

Does Quesma have an API?

No public API is recorded for Quesma.

What industry is Quesma in?

Quesma's product category is AI Agent Evaluation and Benchmarking. Its primary akta.pro industry code is HDAEANAA, End-to-End Enterprise AI Platforms (MLOps & Model Lifecycle Management), with a secondary code of HDAAAKAC, Model Security Testing & Red Teaming (adversarial ML, jailbreaks). Its NAICS code is 5132 and its SIC code is 7372.

Unlock the full company data

50 free credits on sign-up, no credit card required.

Contact sales
Live signals
VestbeeEuropean startup pivots: why some succeed and some failFour European startups—LAM'ON, Synthesia, n8n, and Quesma—illustrate how pivots can succeed or fail. In H1 2026, growth-stage companies captured 72% of funding while early-stage deals fell to 28% of capital. The cases show that pivots are often incremental responses to market signals, not single decisions.StartupHub.aiAI Struggles to Secure Software Supply ChainsQuesma, an AI model evaluation company, has released a new benchmark called BinaryAudit testing AI's ability to detect hidden threats in software binaries. The top-performing model, Claude Opus 4.6, achieved only a 49% success rate while also frequently flagging legitimate software as dangerous, highlighting the nascent stage of AI in supply chain security. Quesma CEO Jacek Migdał stated that at current performance levels, AI functions as "an assistant, not a solution," though he expressed hope that future models could make binary analysis mainstream.SalesTech StarQuesma Explores Novel AI’s Security Capabilities Against Supply-Chain AttacksQuesma announced BinaryAudit, an independent benchmark testing whether AI models can detect hidden threats in software binaries as a proactive supply-chain security measure. The benchmark found that even the best-performing model, Claude Opus 4.6, succeeded only 49% of the time while frequently flagging safe software as dangerous, revealing significant limitations in current AI capabilities for binary analysis. The company believes AI binary analysis could become a preventive security layer, though the CEO noted that at current performance levels, it serves as an assistant rather than a solution.Business Wire BlogQuesma Explores Novel AI's Security Capabilities Against Supply-Chain AttacksQuesma, Inc. announced BinaryAudit, an independent benchmark testing whether AI can detect hidden threats in software binaries before deployment. The results show that even the best-performing model, Claude Opus 4.6, succeeded only 49% of the time and frequently flagged safe software as dangerous. The company suggests AI binary analysis could become a proactive layer in supply-chain security, with new AI models potentially making the technology mainstream within 1-2 years.Lansing State JournalQuesma Releases OTelBench: Independent Benchmark Reveals Frontier LLMs Struggle with Real-World SRE TasksQuesma, Inc. announced the release of OTelBench, the first comprehensive open-source benchmark for evaluating LLMs on OpenTelemetry instrumentation tasks, tested across 14 state-of-the-art models and 23 real-world tasks in 11 programming languages. The benchmark reveals that the best-performing model, Claude Opus 4.5, achieved only a 29% pass rate on OpenTelemetry tasks versus 80.9% in SWE-Bench, exposing critical limitations in AI's cross-cutting reasoning required for production SRE work. The benchmark is available as open-source to enable researchers and practitioners to track progress in AI-assisted observability.PR NewswireHydrolix Adds Quesma Gateway Proxy IP to Its Portfolio, Assuring Continued Kibana SupportHydrolix has purchased intellectual property from Quesma to continue supporting Kibana interoperability for its customers, after Quesma notified the company of its intent to cease support for the Kibana gateway proxy product. The acquired technology, integrated into Hydrolix since 2024, allows organizations to retain existing Kibana workflows while using Hydrolix's high-performance, cost-efficient data lake platform. Hydrolix plans to extend Kibana support as needed to meet customer requirements and maintain its broader ecosystem approach alongside integrations with Splunk, Databricks, and other tools.