Humans in the Loop
Humans in the Loop is a social enterprise founded in 2017 that provides ethical AI data annotation and real-time model validation services to enterprise AI developers, sourcing its distributed workforce from conflict-affected communities across 12+ countries and operating a 'Doctors in the Loop' sub-brand for certified medical AI annotation.
- Company typePrivate
- Founded2017
- HeadquartersSofia, Bulgaria
- Headcount51–100
- GTM typeB2B
- OfferingServices
What Humans in the Loop does
Humans in the Loop is a privately held social enterprise founded in 2017 and headquartered in Sofia, Bulgaria, that provides ethical data annotation and real-time AI model validation services to enterprise AI developers across the computer vision industry. The company sources its workforce from conflict-affected and displaced communities across 12+ countries, including Bulgaria, Turkey, Syria, Iraq, Afghanistan, Lebanon, Yemen, Portugal, Ukraine, Moldova, Kenya, and the DRC, and maintains operational offices in Sofia, Lebanon (Beirut, Tripoli, West Bekaa), Kabul, and Syria (Damascus, Idlib). Its service portfolio spans core annotation types (bounding boxes, polygons, keypoints, semantic segmentation, 3D annotation, video annotation) plus higher-value solutions including custom dataset collection, human-in-the-loop active learning for continuous model maintenance, 24/7 real-time edge case handling via API, and reinforcement learning with human feedback (RLHF) for generative AI developers. In December 2022, the company launched 'Doctors in the Loop' as a wholly-owned subsidiary staffed by 60+ certified medical specialists across 4 locations, serving regulated healthcare AI use cases in radiology, pathology, ultrasound, robotic surgery, and clinical NLP.
The business operates on multiple revenue mechanics simultaneously: managed annotation services sold as dedicated expert teams (managed services), custom dataset collection (professional services), real-time edge case handling (subscription/recurring), and RLHF work including red teaming (professional services). Pricing is quote-based with a freemium pilot funnel ('Start a Free Pilot', 'Book a Call') supporting a sales-led enterprise GTM motion. Distribution is primarily direct enterprise sales targeting AI developers in medical, automotive, geospatial, industrial, retail, and agricultural verticals, complemented by thought-leadership content marketing, webinars, and case studies with named customers including Milestone Systems, Auto1, Picvisa, Restor, TrialHub, and INRAe.
The company's competitive position rests on a talent moat built from its ethically sourced, globally distributed workforce, complemented by intangible brand assets reinforced by Forbes 30 Under 30, MIT Solve, and Dubai Expo Live awards, and a data advantage in certified medical annotation capacity through the Doctors in the Loop subsidiary. Strategic local implementation partnerships with Beetroot (Ukraine), WorkWell (Iraq), and Roia (Syria) extend workforce reach, while a research partnership with Imperial College London anchors credibility in medical AI safety work. Cumulative disclosed funding totals approximately $274K across three grant rounds (2019–2021), with no equity financing disclosed, indicating the company has scaled through non-dilutive, mission-aligned capital rather than venture investment.
Humans in the Loop firmographics
Firmographics- Name
- Humans in the Loop
- Legal name
- Humans in the Loop
- Website
- https://humansintheloop.org
- Company type
- Private
- Founded year
- 2017
- Operating status
- Operating
- Headcount range
- 51–100 employees
- Short description
- Humans in the Loop is a social enterprise founded in 2017 that provides ethical AI data annotation and real-time model validation services to enterprise AI developers, sourcing its distributed workforce from conflict-affected communities across 12+ countries and operating a 'Doctors in the Loop' sub-brand for certified medical AI annotation.
- Ownership category
- akta.pro rank
Humans in the Loop industry classification
Industry- Product category
- AI Data Annotation Services
- NAICS
- Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services (518210)
- SIC
- Services-Computer Programming, Data Processing, Etc. (7370), Services-Computer Programming Services (7371)
- akta.pro primary industry
- Data Labeling & Annotation Services (HDAAALAB)
- akta.pro secondary industries
- RLHF, Human Feedback & Evaluation Data Tools (HDAAALAJ), Domain-Specific Training Data Solutions (e.g., medical, AV, geospatial) (HDAAALAL), Data Augmentation & Programmatic Labeling (HDAAALAH), Bias, Fairness & Representativeness Testing for Datasets (HDAAALAK)
Keywords
Where Humans in the Loop is headquartered
LocationHeadquarters
- HQ city
- Sofia
- HQ country
- Bulgaria
- HQ region
- Europe
Offices4 records
Markets served
Humans in the Loop business model
Business model- GTM type
- B2B
- Offering type
- Services
- Cost components
- Personnel, Operations, Technology or R&D, Marketing or Sales
Revenue model
- Managed Data Annotation Services: Dedicated expert teams provide high-quality ground truth data annotation across multiple domains including medical AI, automotive, geospatial, industrial, retail, and agricultural applications. Teams are assembled based on project requirements and become dedicated to specific client projects.
- Custom Dataset Collection: On-demand dataset collection with custom parameters, sourcing locations and distribution according to client requirements. Data is fully annotated and remains entirely owned by the client. Uses global workforce to source data from specific geographies and demographics.
- Human-in-the-Loop for Active Learning: Continuous model maintenance through human review of model predictions to avoid data drift and close model retraining loop. Humans label new data for ground truth comparison and identify false predictions.
- Real-time Edge Case Handling: 24/7 monitoring of live AI systems with human operators handling edge cases and model alerts. Available for wildfire detection, intrusion detection, and other real-time AI monitoring applications.
- Reinforcement Learning with Human Feedback: Human trainers interact with model outputs, evaluate them, rank model outputs across attributes (relevance, safety, accuracy), perform A/B testing, and provide red teaming services to identify model weak spots and biases.
Pricing tiers
| Model | Billing | Price |
|---|---|---|
| Freemium | Pay-as-you-go | Free Pilot Program |
Go-to-market motion2 records
Distribution channels3 records
Marketing channels7 records
Humans in the Loop product offering
Product offeringCore offering
Humans in the Loop is a social enterprise that provides managed data annotation and real-time AI model validation services for the computer vision industry. Dedicated expert teams deliver 95% accurate ground-truth annotation across multiple formats (bounding boxes, polygons, keypoints, semantic segmentation, 3D, video, medical imaging), with the Doctors in the Loop sub-brand providing certified medical-professional annotation for healthcare AI. The company also offers dataset collection, active-learning human-in-the-loop services, real-time edge case handling, and reinforcement learning with human feedback (RLHF).
Product overview
Humans in the Loop是一家社会企业，通过合乎道德规范的多元化员工队伍提供高质量人工智能数据标注服务，实现95%数据准确率。公司采用平台加服务架构，核心服务包括多种标注类型（边界框、多边形、关键点、语义分割、医学标注、视频标注、3D标注）以及四大解决方案服务：数据集收集、数据集标注、主动学习人类反馈回路、实时边缘案例处理、基于人类反馈的强化学习。此外还运营专业医疗标注子品牌"Doctors in the Loop"，由认证医疗专业人员处理医学影像和临床NLP任务。公司业务覆盖医学、实时人工智能、自动驾驶、零售、地理空间、农业和工业等多个行业领域，提供24/7全球覆盖的API集成服务，支持英语、西班牙语、法语、葡萄牙语、阿拉伯语和波斯语等多语言标注。
Differentiator
Problem solved
Functional benefit
Brands
- Doctors in the Loop: An annotation company specializing in medical AI projects, part of the social enterprise group Humans in the Loop. Mission is to provide employment to conflict-affected medical professionals. Includes 60 medical specialists across 4 locations with 8 medical projects.
Products and services
- Bounding Box Annotation Bounding box annotation service used to localize and frame target objects in images; the foundational annotation type for object detection and object recognition training data.
- Polygon Annotation Polygon annotation service for precise outlining of irregularly shaped object boundaries, used where exact contours are required.
- Keypoint Annotation Keypoint annotation service for identifying human poses, facial landmarks, and joint positions, used for motion recognition and behavior analysis.
- Semantic Segmentation Semantic segmentation service that classifies every pixel in an image to enable pixel-level precise recognition.
- Medical Annotation Medical image annotation service delivered by certified medical professionals, covering precise labeling of radiology, pathology, surgical video, ultrasound and other medical imagery.
- Video Annotation Video annotation service supporting frame-level and object-level labeling, including frame interpolation and cross-frame tracking, used for autonomous driving, surveillance video, and medical video analysis.
- 3D Annotation 3D annotation service for processing point cloud and 3D reconstruction data, supporting LiDAR and sensor fusion use cases.
- Dataset Collection Custom dataset collection service that gathers data with client-specified parameters, supports sourcing from specific geographies and demographic distributions to ensure data diversity and representativeness.
- Dataset Annotation Professional team-based image and video annotation service supporting bounding boxes, polygons, keypoints, 3D annotation and other types, using custom taxonomies iterated quickly to deliver top quality.
- Human-in-the-Loop for Active Learning Active learning service that monitors data instances near decision boundaries or out of standard distribution, periodically routes them to human reviewers, generates new ground-truth labels for model prediction comparison, and audits model predictions post-deployment to surface bias and failure modes.
- Real-time Edge Case Handling Real-time edge case handling service that integrates via API endpoints with production AI models, providing 24/7 global coverage to verify model alerts, confirm low-confidence predictions, and handle complex steps to ensure reliable model output.
- Reinforcement Learning with Human Feedback
- Doctors in the Loop
Quantifiable outcome
- 95% data accuracy claimed for annotation services
- +4 more outcomes
Companies that use Humans in the Loop
Customer profileNamed customers6 records
Segments6 records
Ideal customer profiles5 records
Humans in the Loop technology and API
TechnologyTechnology focussed No
API detail
- Has API
- Yes
- API docs
- API detail
Core technology
AI maturity
App detail
AI capability10 records
Feature5 records
Humans in the Loop partnerships and signals
Strategic signalPartnerships
Four partnerships are on record, tiered core and notable.
- BeetrootcorePartnership in Ukraine providing digital work opportunities to conflict-affected communities. Beetroot connects Humans in the Loop with Ukrainian professionals for annotation work, extending the social enterprise's workforce reach.
- WorkWellcorePartnership in Iraq extending Humans in the Loop's operational reach to conflict-affected communities in the region. WorkWell serves as a local partner for workforce development and digital job creation.
- RoiacorePartnership in Syria with Roia organization to provide digital work opportunities in conflict-affected regions. Roia serves as the local implementation partner for Syrian operations.
- Imperial College LondonnotablePartnership for robotic surgery safety project involving polygon annotation of surgical tool images to enable 3D instrument localization during live procedures. Annotation output fed directly into network training for surgical safety improvement.
Scale indicators6 records
Recent moves7 records
Expansion highlights3 records
Humans in the Loop competitors and assessment
Company assessmentDirect peers
- Scale AI: Scale AI is the largest venture-backed data annotation platform serving enterprise AI customers across computer vision, NLP, and RLHF. Direct competitor in managed annotation, RLHF, and active learning services for the same enterprise buyer HITL targets.
- Sama: Sama (formerly Samasource) is a social enterprise providing AI data annotation with employment for underserved communities. Closest comparable in dual social-impact and commercial model, serving enterprise computer vision customers.
- iMerit: iMerit is a data annotation and enrichment company providing skilled workforce services to enterprise AI customers in healthcare, automotive, and geospatial. Comparable in managed annotation services and focus on quality outcomes for regulated industries.
- CloudFactory: CloudFactory provides managed AI data annotation services with a workforce from developing countries. Comparable in workforce model and enterprise-focused managed annotation across computer vision and NLP tasks.
- Surge AI: Surge AI is a venture-backed data labeling platform specializing in high-quality NLP and RLHF datasets for foundation model developers. Direct competitor in the RLHF and human-feedback services HITL offers.
- Defined.ai: Defined.ai (formerly DefinedCrowd) provides training data and annotation services for enterprise AI applications across multiple verticals. Direct competitor in managed annotation and RLHF/dataset services.
Broad incumbents
- Appen: Appen is a publicly listed incumbent providing large-scale data annotation and dataset services for AI. Competitor in enterprise computer vision and NLP annotation with broader geographic and customer footprint but less specialization in medical and conflict-affected sourcing.
- Toloka AI: Toloka is a crowdsourced data annotation platform serving AI developers globally. Competes in computer vision and NLP annotation but with a crowdsourced (rather than managed dedicated teams) delivery model.
Emerging players
- Labelbox: Labelbox provides an annotation platform with managed labeling services for enterprise AI teams. Partial overlap with HITL in enterprise computer vision annotation, with stronger platform/tool focus and less services-led differentiation.
- SuperAnnotate: SuperAnnotate provides an AI annotation platform with managed services for computer vision. Partial overlap with HITL, with stronger tooling focus and growing services offering for enterprise CV customers.
Market position
Strengths5 records
Weaknesses5 records
Competitive moat5 records
Key risks6 records
Key highlights7 records
Customer concentration
Humans in the Loop social profiles
Digital presenceHumans in the Loop financial estimates
Financial estimateRevenue estimate
Valuation estimate
Humans in the Loop leadership team
Management profileNumber of profiles
Profiles5 records
Humans in the Loop subsidiaries and ownership
Company hierarchySubsidiaries1 record
Humans in the Loop funding detail
Funding detailFunding overview
Funding rounds4 records
Investors4 records
Funding detail is available on the Subscription and Enterprise plan.Contact sales →
Humans in the Loop M&A and investment
M&A and investmentM&A
Investments
M&A and investment is available on the Subscription and Enterprise plan.Contact sales →
Frequently asked questions about Humans in the Loop
What does Humans in the Loop do?
Humans in the Loop is a social enterprise that provides managed data annotation and real-time AI model validation services for the computer vision industry. Dedicated expert teams deliver 95% accurate ground-truth annotation across multiple formats (bounding boxes, polygons, keypoints, semantic segmentation, 3D, video, medical imaging), with the Doctors in the Loop sub-brand providing certified medical-professional annotation for healthcare AI. The company also offers dataset collection, active-learning human-in-the-loop services, real-time edge case handling, and reinforcement learning with human feedback (RLHF).
Is Humans in the Loop a public or private company?
Humans in the Loop is a private company. It is classified as founder individual operated bootstrapped and is currently operating.
When was Humans in the Loop founded?
Humans in the Loop was founded in 2017. It employs 51 to 100 people.
Where is Humans in the Loop based?
Humans in the Loop is headquartered in Sofia, Bulgaria, in the Europe region.
How does Humans in the Loop make money?
Five revenue lines are on record. Managed Data Annotation Services are the primary driver. The others are custom Dataset Collection, human-in-the-Loop for Active Learning, real-time Edge Case Handling and reinforcement Learning with Human Feedback.
Who are Humans in the Loop's main competitors?
Direct peers on record are Scale AI, Sama, iMerit, CloudFactory, Surge AI and Defined.ai. Broad incumbents are Appen and Toloka AI. Emerging players are Labelbox and SuperAnnotate.
Does Humans in the Loop have an API?
Yes. Provides API endpoints for clients to create jobs for the workforce and define callback URLs to receive results. Enables real-time 24/7 monitoring integration. The API allows sending model predictions, certainty scores, and model outputs for human review, with responses sent back within as little as one minute. Supports sending model outputs, A/B tests, and data generation requests. The workforce can document failure modes and biases through the API.
What industry is Humans in the Loop in?
Humans in the Loop's product category is AI Data Annotation Services. Its primary akta.pro industry code is HDAAALAB, Data Labeling & Annotation Services, with a secondary code of HDAAALAJ, RLHF, Human Feedback & Evaluation Data Tools. Its NAICS code is 518210 and its SIC code is 7370.