What is Ground Truth? The Foundation of AI, Machine Learning, and Data-Driven Decisions
- Muiz As-Siddeeqi
- Dec 2, 2025
- 28 min read
Every day, AI systems make millions of decisions that shape your life—from flagging spam emails to diagnosing diseases in medical scans. Behind every accurate prediction lies a hidden hero: ground truth. Without this verified, factual data, AI would be guessing blindly. Medical diagnoses could be wrong. Self-driving cars could misidentify pedestrians. Financial fraud detection could fail. Ground truth transforms raw information into reliable intelligence. It's the benchmark that separates accurate AI from dangerous guesswork.
Don’t Just Read About AI — Own It. Right Here
TL;DR
Ground truth is verified, factual data used as the definitive benchmark for training and evaluating AI models
The global data annotation market (which produces ground truth) reached $2.11 billion in 2024 and will grow to $12.45 billion by 2033 (IMARC Group, 2024)
Ground truth spans machine learning, remote sensing, GIS, medical imaging, and autonomous vehicles
Creating accurate ground truth costs time and money—expert annotation averages $15-50 per hour, depending on complexity
Inter-annotator disagreement remains a persistent challenge, with typical agreement rates ranging from 70-95% depending on task complexity
Future trends include automated labeling, synthetic data generation, and human-in-the-loop AI systems
Ground truth refers to verified, accurate data that serves as the definitive reference point for training, testing, and validating AI and machine learning models. It represents the "correct answer" against which model predictions are measured. In machine learning, ground truth consists of carefully labeled datasets where human experts or direct observations have confirmed the accuracy of each data point.
Table of Contents
What Ground Truth Actually Means
Ground truth is information known to be real or true, provided by direct observation and measurement rather than inference or estimation.
The term originated in remote sensing and meteorology during the 1960s and 1970s. NASA described it as early as 1972 as essential "data about materials on the earth's surface" used to calibrate measurements from satellites and aerial imagery (NASA, 1972). The concept was later adopted by statistical modeling and machine learning communities (Wikipedia, 2025).
In machine learning and AI, ground truth refers to verified, labeled data used to train algorithms and evaluate their performance. Think of it as the answer key to a test. The AI model is the student making predictions. Ground truth is the set of correct answers used to grade the student's work and identify where the model needs improvement (IBM, 2024).
Without ground truth, you cannot know if an AI model has learned anything useful. A self-driving car might think a stop sign is a speed limit sign. A medical AI could mistake a benign tumor for cancer. Ground truth prevents these catastrophic errors by providing a factual baseline.
The Oxford English Dictionary traces the word "Groundtruth" (as one word) to Henry Ellison's 1833 poem "The Siberian Exile's Tale," where it meant "fundamental truth" (Wikipedia, 2025). Today, it functions as a noun, adjective, and verb across multiple technical disciplines.
The Origins and Evolution of Ground Truth
Ground truth emerged from necessity. When satellites first began capturing images of Earth in the 1960s, scientists needed a way to verify what those blurry pixels actually represented on the ground. Was that dark patch a lake, a shadow, or a parking lot? The only way to know was to physically visit the location and check.
This practice—called "ground truthing"—involved field teams taking measurements, recording GPS coordinates, and documenting features they observed directly. These on-site observations became the reference standard for calibrating remote sensing instruments and validating image classifications (GIS Geography, 2025).
By the 1990s, as machine learning gained traction, researchers borrowed the term. Supervised learning algorithms required labeled datasets where the correct classification was known for each training example. This labeled data became the machine learning equivalent of ground truth.
The concept evolved further in the 2000s and 2010s with the explosion of big data and deep learning. Companies like Google, Facebook, and Amazon needed massive labeled datasets to train neural networks for image recognition, natural language processing, and recommendation systems. This demand spawned an entire industry dedicated to creating ground truth through data annotation.
Today, ground truth has expanded beyond its original technical meaning. In business, "ground truth" sometimes refers to on-the-ground customer insights versus abstract market research. In journalism, it can mean verified facts versus hearsay or speculation.
Ground Truth Across Different Fields
Machine Learning and AI
In machine learning, ground truth is the labeled dataset used to train supervised learning models. Each data point has a label assigned by human annotators or obtained through direct observation.
For example, an image recognition system learns to identify cats by training on thousands of images where humans have labeled every picture as "cat" or "not cat." These human-provided labels are the ground truth (Domino Data Lab, 2025).
Ground truth enables supervised learning—the dominant paradigm in AI. Models learn by comparing their predictions to ground truth labels, calculating the error, and adjusting their internal parameters to reduce that error over time. Without ground truth, this feedback loop cannot exist.
According to IBM (2024), ground truth data is especially critical for:
Image and object recognition
Predictive analytics
Customer sentiment analysis
Spam detection
Natural language processing
The quality of ground truth directly determines model performance. Inaccurate labels teach models incorrect patterns. Biased labels perpetuate discrimination. Missing labels leave gaps in model knowledge.
Remote Sensing and GIS
In geographic information systems and remote sensing, ground truth refers to data collected on location to verify remotely sensed imagery. Field teams visit sites, take measurements, record GPS coordinates, and document what they observe. This ground data validates classifications made from satellite or aerial imagery (GIS Geography, 2025).
Ground truthing ensures accuracy in:
Land cover mapping
Agricultural monitoring
Forest health assessment
Urban planning
Environmental change detection
For instance, if a satellite image classifies a pixel as "deciduous forest," ground truthing verifies this by visiting the location and confirming that deciduous trees are actually present. GPS coordinates link the ground observations to specific pixels in the image (Wikipedia, 2025).
According to Esri's GIS Dictionary (2025), ground truth denotes "the accuracy of remotely sensed or mathematically calculated data based on data actually measured in the field."
Medical Imaging
In medical imaging and radiology, ground truth represents the definitive diagnosis used to train and evaluate AI diagnostic systems. This might come from biopsies, pathology reports, expert radiologist annotations, or patient outcomes over time.
The Checklist for Artificial Intelligence in Medical Imaging (CLAIM) 2024 Update recommends using the term "reference standard" instead of "ground truth" in medical contexts. The CLAIM Steering Committee explains: "Ground truth suggests that one knows the absolute truth, whereas in biomedicine, the label assigned as ground truth may be uncertain" (Radiology: Artificial Intelligence, 2024).
Medical ground truth faces unique challenges. Two expert radiologists might disagree on whether a lung nodule appears suspicious. A pathology report might be inconclusive. Patient outcomes might take years to determine. Despite these uncertainties, medical AI still requires reference standards to function.
A 2024 study in Radiology found that AI-assisted chest radiography interpretation increased sensitivity by 6-26% across all readers (thoracic radiologists, general radiologists, and residents) when compared to ground truth established by CT imaging. Mean reading time decreased from 81 seconds without AI to 56 seconds with AI—a 31% reduction (RSNA, 2024).
Autonomous Vehicles
Self-driving cars depend on ground truth to learn safe navigation. Training data must accurately label pedestrians, vehicles, traffic signs, lane markings, and obstacles. Ground truth for autonomous vehicles comes from:
LiDAR and radar sensor data
Human-annotated video footage
Detailed 3D maps with precise GPS coordinates
Real-world driving scenarios captured by test fleets
Waymo's open dataset contains 12 million LiDAR and 9.9 million camera annotations curated by trained labelers (Mordor Intelligence, 2025). Tesla processes thousands of video clips daily from its fleet to train Full Self-Driving software (Think Autonomous, 2025).
Ground truth quality directly impacts safety. A mislabeled pedestrian could teach the vehicle to ignore real people. An incorrectly mapped intersection could cause collisions. The stakes are literally life and death.
Why Ground Truth Matters So Much
Ground truth serves three critical functions:
1. Training Foundation
Supervised machine learning models learn by example. They need thousands or millions of correctly labeled examples to recognize patterns. Without accurate ground truth, models learn incorrect associations.
If you train a spam filter on emails mislabeled as "spam" when they're actually legitimate messages, the filter will incorrectly block important emails. The model is only as good as its training data.
2. Performance Evaluation
Ground truth provides the benchmark for measuring AI accuracy. Common metrics like precision, recall, F1-score, and accuracy all compare model predictions to ground truth labels.
A medical diagnostic AI might claim 95% accuracy, but that number is meaningless without verified ground truth diagnoses to test against. Ground truth is the reality check that keeps AI honest.
3. Model Improvement
By identifying where predictions differ from ground truth, developers can diagnose model weaknesses. Maybe the model struggles with edge cases. Perhaps it's biased toward common classes. Ground truth reveals these problems so they can be fixed.
C3 AI notes that most enterprise business problems require data from five or six disparate IT systems to establish ground truth, creating significant complexity (C3 AI Glossary, 2024). Organizations must invest heavily in data infrastructure to produce reliable ground truth at scale.
Without ground truth, AI becomes speculation. Lyzr.ai (2024) explains: "Without a source of truth, an AI is just guessing. Ground truth is the accurate, verified information that serves as a definitive reference point against which AI predictions are measured and evaluated."
How Ground Truth Data Is Created
Creating ground truth involves multiple methods, each with trade-offs:
Manual Annotation
Human experts label data by hand. For images, this means drawing bounding boxes around objects, segmenting regions, or classifying entire images. For text, it involves tagging entities, labeling sentiment, or categorizing documents.
Process:
Define clear annotation guidelines
Train annotators on the task
Distribute data to multiple annotators
Collect annotations
Resolve disagreements through consensus or expert review
Validate final labels
Advantages:
High accuracy for complex tasks
Captures nuanced judgments
Adapts to ambiguous cases
Disadvantages:
Time-consuming (average 30-300 seconds per image depending on complexity)
Expensive ($15-50+ per hour for skilled annotators)
Subject to human error and bias
Difficult to scale
According to data from Innovatiana (2025), inter-annotator agreement (IAA) measures the consistency between multiple annotators labeling the same data. Typical IAA scores using Cohen's Kappa or Fleiss' Kappa range from 0.6 to 0.9, with higher scores indicating better agreement.
Field Surveys and Direct Measurement
For remote sensing and GIS applications, field teams visit locations to collect ground truth data. They take photographs, record GPS coordinates, measure vegetation height, document land cover types, and note any relevant features.
Best practices:
Visit sites during the same season as imagery capture
Use high-accuracy GPS equipment (sub-meter precision)
Document weather conditions and visibility
Take multiple measurements for verification
Photograph sites from multiple angles
Malvern Panalytical offers spectroradiometers specifically for ground truthing field measurements, generating high-quality field spectra with illumination and viewing geometry equivalent to satellite sensors (Malvern Panalytical, 2025).
Automated and Semi-Automated Methods
As AI improves, automated labeling tools can pre-label data for human review. This speeds up annotation while maintaining quality control.
Techniques include:
Pre-trained models generating initial labels
Active learning selecting the most informative examples for human review
Weak supervision using multiple noisy labeling sources
Transfer learning from related tasks
Scale AI reported revenue of $870 million in 2024, tracking toward $2 billion in 2025, demonstrating how demand for massive multimodal datasets is reshaping the data annotation industry (Mordor Intelligence, 2025).
Crowdsourcing
Platforms like Amazon Mechanical Turk distribute annotation tasks to large crowds of workers. Multiple workers label each item, and majority vote determines the final label.
Quality control measures:
Gold standard test questions with known answers
Multiple annotations per item (typically 3-5)
Statistical outlier detection
Worker qualification requirements
Ongoing performance monitoring
Research from 2024 shows that using 10 or more annotators enhances agreement, consistency, and reduces bias, though even three high-quality annotators can achieve strong results for well-defined tasks (ResearchGate, 2024).
Real-World Case Studies
Case Study 1: Waymo's Autonomous Driving Ground Truth
Company: Waymo (Alphabet subsidiary)
Date: 2009-2025
Application: Self-driving vehicle training
Waymo has created one of the most comprehensive ground truth datasets for autonomous driving. As of 2024, the company's vehicles have driven over 7 million fully autonomous miles (IPG Media Lab, 2024).
Ground Truth Methods:
Combination of cameras, radar, and LiDAR sensors creating 3D environment maps
Trained human annotators labeling video footage frame-by-frame
Pre-mapped cities with detailed 3D data down to the centimeter level
Real-world driving data from thousands of hours of operation
Results:
Waymo served over 4 million fully autonomous rides in 2024 alone, bringing total rides to over 5 million (MishTalk, 2025)
December 2024 Swiss Re study showed Waymo had an 88% reduction in property damage claims and 92% reduction in bodily injury claims compared to human-driven vehicles (Contrary Research, 2025)
Safety analysis of 7 million driverless miles showed 85% fewer injury crashes and 57% fewer police-reported crashes per mile than human drivers (IPG Media Lab, 2024)
Key Insight: Waymo's methodical approach to ground truth collection—combining multiple sensor types and extensive pre-mapping—enabled higher safety levels than Tesla's camera-only approach. However, this comes at significant cost: Waymo's fifth-generation sensor suite costs approximately $12,700 per vehicle versus $400 for Tesla's camera setup (Contrary Research, 2025).
Case Study 2: Medical AI for Chest X-Ray Analysis
Institution: Multi-center radiology study
Date: 2024
Application: AI-assisted chest radiography interpretation
Researchers conducted a retrospective study of 500 patients who underwent both chest radiography and thoracic CT scans. CT scans served as the ground truth for validating chest X-ray interpretations (RSNA Radiology, 2024).
Ground Truth Methods:
CT imaging provided detailed cross-sectional views confirming abnormalities
Expert thoracic radiologists reviewed both X-rays and CT scans
Abnormalities cataloged included nodules, masses, pneumothorax, and other pathologies
Results:
AI assistance increased detection sensitivity by 6-26% (p < .001) for all reader groups
Mean reading time decreased from 81 seconds to 56 seconds (31% reduction, p < .001)
Time reduction was 17% for abnormal radiographs versus 38% for normal radiographs
All reader groups (thoracic radiologists, general radiologists, residents) showed improved accuracy with AI assistance
Challenges:
Inter-observer disagreement between radiologists on subtle findings
Some abnormalities visible on CT were nearly invisible on X-ray
AI occasionally flagged false positives that increased review burden
Key Insight: High-quality ground truth (CT scans) enabled accurate evaluation of AI performance. Without definitive ground truth, measuring the 6-26% improvement would have been impossible.
Case Study 3: ImageNet and Computer Vision Revolution
Organization: Stanford University researchers
Date: 2009-present
Application: Large-scale image classification
ImageNet is a dataset of over 14 million hand-annotated images organized into thousands of categories. It became the benchmark for computer vision, launching the deep learning revolution.
Ground Truth Methods:
Human annotators used Amazon Mechanical Turk to label images
Multiple annotators per image with majority vote consensus
Hierarchical category structure based on WordNet taxonomy
Ongoing quality control and error correction
Impact:
Annual ImageNet Large Scale Visual Recognition Challenge (2010-2017) drove rapid accuracy improvements
Top-5 error rate dropped from 28% in 2010 to 2.3% in 2017
Techniques developed for ImageNet transferred to medical imaging, autonomous vehicles, and countless other applications
Demonstrated the power of large-scale ground truth datasets for training neural networks
Lessons Learned: A 2022 study found that even ImageNet contains noisy labels, with some images mislabeled or ambiguous (Manufacturing Intelligence, 2024). This highlights that "ground truth" is not always perfectly accurate—it represents the best available reference, not absolute certainty.
The Ground Truth Market and Economics
The data annotation market—which produces ground truth data—has exploded alongside AI adoption:
Market Size and Growth
Multiple market research firms report similar explosive growth:
IMARC Group (2024): Market valued at $2.11 billion in 2024, projected to reach $12.45 billion by 2033 at 20.71% CAGR
Global Growth Insights (2024): Market at $2.24 billion in 2024, expected to grow to $19.92 billion by 2033 at 27.47% CAGR
Grand View Research (2023): Market was $1.02 billion in 2023, projected to reach $5.33 billion by 2030 at 26.5% CAGR
Expert Market Research (2024): Market at $836.24 million in 2024, projected to reach $9.13 billion by 2034 at 27% CAGR
While absolute numbers vary by methodology, all sources agree on rapid double-digit growth driven by AI adoption.
Market Segmentation
By Data Type (2024):
Text annotation: 36-38% market share (natural language processing applications)
Image annotation: 35% market share (computer vision, autonomous vehicles, medical imaging)
Video annotation: Growing rapidly for surveillance, autonomous driving, video analysis
Audio annotation: Speech recognition, voice assistants, audio classification
By Industry:
IT and technology: Largest segment
Healthcare: Growing at 22.9% CAGR for specialized medical data annotation (IMARC Group, 2024)
Automotive: Driven by autonomous vehicle development
Retail: Product categorization, visual search, recommendation systems
Financial services: Fraud detection, document processing, sentiment analysis
By Geography:
North America: 36-37% market share (strong AI adoption, major tech companies)
Asia Pacific: Fastest growing region (China's directive government support, manufacturing digitalization)
Europe: Robust growth in automotive and retail sectors
Cost Structure
Ground truth creation costs vary widely by complexity:
Simple Tasks:
Image classification (single label): $0.01-0.10 per image
Text categorization: $0.01-0.05 per document
Audio transcription: $0.10-1.00 per minute
Complex Tasks:
Medical image segmentation: $5-50 per image (requires medical expertise)
3D bounding boxes for autonomous vehicles: $1-10 per frame
Named entity recognition in specialized domains: $0.50-5.00 per document
Video action recognition: $2-20 per clip
Labor Costs:
Crowdworkers: $3-12 per hour
Trained annotators: $15-25 per hour
Domain experts (medical, legal): $30-100+ per hour
According to Mordor Intelligence (2025), in machine learning projects globally, more than 80% of engineering labor is devoted to data preparation and labeling. This explains why the third-party data annotation market is expected to triple by 2024.
Major Market Players
Leading companies providing ground truth annotation services include:
Scale AI: Revenue climbed to $870 million in 2024, tracking $2 billion in 2025
Appen Limited: Global workforce of annotators, serves major tech companies
Labelbox: Enterprise annotation platform with automation features
Amazon SageMaker Ground Truth: Integrated with AWS machine learning services
Sama: Launched Sama Multimodal in June 2025, achieving 35% accuracy improvement and 10% reduction in product returns for early adopters (IMARC Group, 2024)
Companies increasingly focus on automated annotation features, with cloud-based tools accounting for 63.5% of 2024 revenue and advancing at 22.6% CAGR (Mordor Intelligence, 2025).
Major Challenges and Limitations
Inter-Annotator Disagreement
Human annotators often disagree, especially on subjective or ambiguous tasks. When multiple people label the same data, they produce different answers.
Causes:
Ambiguous examples (Is that a dog or a wolf?)
Subjective judgments (Is this comment toxic or just critical?)
Unclear guidelines
Annotator fatigue or carelessness
Genuine differences in expert opinion
Measurement: Inter-annotator agreement is typically measured using Cohen's Kappa (two annotators) or Fleiss' Kappa (multiple annotators). Scores range from 0 to 1:
0.0-0.20: Slight agreement
0.21-0.40: Fair agreement
0.41-0.60: Moderate agreement
0.61-0.80: Substantial agreement
0.81-1.00: Almost perfect agreement
A 2024 study found that inter-annotator agreement varies significantly by task. Medical image segmentation often achieves 0.70-0.85 Kappa scores, while more subjective tasks like sentiment analysis might see 0.50-0.70 (PMC, 2023).
Solutions:
Multiple annotators per item with majority vote
Expert review for difficult cases
Clearer guidelines with detailed examples
Ongoing annotator training and feedback
Modeling disagreement as inherent data uncertainty rather than error
Cost and Scalability
Creating high-quality ground truth is expensive and slow. A single autonomous vehicle might generate terabytes of data daily, each frame requiring annotation. Medical datasets need expert annotators with years of training.
The global volume of data is expected to reach 175 zettabytes by 2025, driving massive need for scalable annotation tools (Expert Market Research, 2024). Current manual methods cannot keep pace.
Subjectivity and Bias
Ground truth reflects human judgment, which carries bias. If annotators are predominantly from one demographic group, their labels might not represent diverse perspectives.
Example biases include:
Facial recognition trained on datasets skewed toward certain ethnicities
Hiring AI trained on historical decisions reflecting past discrimination
Medical AI trained primarily on data from one population group
Even with clear guidelines, annotators bring their own interpretations, experiences, and blind spots.
Ground Truth Decay
Ground truth becomes outdated. A spam filter trained in 2018 fails against 2024 phishing attacks because the patterns have changed. The "truth" is not static.
As Lyzr.ai (2024) notes: "In 2024, a completely new type of sophisticated phishing attack emerges. How will the model perform, and why is its original ground truth now a liability? The model will fail. It will likely classify these new phishing emails as 'not spam' because they don't match the patterns it learned from the 2018 data."
Models need continuous updating with fresh ground truth to remain accurate.
The "Ground Truth" Misnomer
Philosopher Giles M. Foody (2024) argues that reference datasets are rarely perfect, and their errors can significantly bias accuracy assessments. Ground truth implies absolute certainty, but in reality, labels are:
Sometimes incorrect
Often uncertain (especially in medical diagnosis)
Influenced by the annotator's expertise level
Limited by the information available at labeling time
The CLAIM 2024 medical imaging guidelines now recommend "reference standard" instead of "ground truth" to acknowledge this uncertainty (RSNA, 2024).
Quality Control Complexity
Ensuring consistent quality across thousands or millions of annotations is difficult. Some annotators rush. Others misunderstand guidelines. Automated quality checks catch obvious errors but miss subtle mistakes.
A 2024 study on minority reports in annotation found that worker disagreement rates follow a geometric distribution, and workers experience exhaustion effects where their accuracy varies based on how long they've been working (arXiv, 2024).
Best Practices for High-Quality Ground Truth
1. Define Clear Guidelines
Provide detailed annotation instructions with examples of edge cases. Show both positive and negative examples. Explain how to handle ambiguous situations.
Include:
Decision trees for complex judgments
Visual examples with annotations
Common mistakes to avoid
When to escalate unclear cases
2. Use Multiple Annotators
Assign each item to 3-5 annotators. Use majority vote for final labels. This reduces individual error and bias.
Research shows 10+ annotators enhance agreement and reduce bias, though 3-5 skilled annotators often suffice for well-defined tasks (ResearchGate, 2024).
3. Implement Quality Control
Methods:
Gold standard examples with known correct answers mixed into workflow
Regular performance reviews with feedback
Statistical outlier detection flagging unusual annotation patterns
Expert review of difficult cases
Inter-annotator agreement monitoring
4. Train Annotators Thoroughly
Provide comprehensive onboarding:
Overview of the task and its purpose
Detailed guideline walkthrough
Practice annotations with immediate feedback
Qualification tests before production work
Ongoing training for edge cases
5. Stratified Sampling
Ensure training data represents the full distribution of real-world examples. Include:
Common cases (to build strong baseline performance)
Rare cases (to handle edge situations)
Balanced representation across categories
Geographic, demographic, and temporal diversity
6. Version Control and Documentation
Track every aspect of ground truth creation:
Annotation guidelines (version-dated)
Annotator identities and qualifications
Timestamps for all labels
Inter-annotator agreement scores
Quality control results
Dataset splits (training/validation/test)
7. Consider Active Learning
Intelligently select which data to annotate. Use model uncertainty to identify the most informative examples requiring human labels. This maximizes value per annotation dollar.
8. Validate Against External Standards
When possible, validate ground truth against objective external benchmarks:
Medical imaging: Biopsy results, patient outcomes
Remote sensing: Field surveys, higher resolution imagery
Financial fraud: Confirmed fraud cases from investigations
9. Embrace Uncertainty
For genuinely ambiguous cases, consider probabilistic labels rather than forcing binary choices. Some items might be 70% category A and 30% category B.
Recent research suggests multi-labeling that captures inherent ambiguity produces more robust AI than forced consensus (arXiv, 2025).
Tools and Platforms
Open-Source Annotation Tools
CVAT (Computer Vision Annotation Tool):
Free, open-source
Image and video annotation
Bounding boxes, polygons, keypoints
Integration with popular ML frameworks
Label Studio:
Open-source data labeling platform
Supports images, audio, text, time series
Machine learning-assisted labeling
Cloud or self-hosted deployment
VGG Image Annotator (VIA):
Lightweight, runs in browser
No installation required
Image region annotation
Used in research and education
Commercial Platforms
Labelbox:
Enterprise-grade annotation platform
Automated labeling features
Quality control workflows
Integrations with major cloud providers
Pricing: Custom enterprise pricing
Scale AI:
End-to-end data annotation services
Combines human annotators with AI assistance
Specializes in autonomous vehicle data
Revenue: $870 million in 2024
Amazon SageMaker Ground Truth:
Integrated with AWS ecosystem
Active learning reduces labeling costs up to 70%
Built-in workflows for common tasks
Pay-per-use pricing model
Appen:
Global crowdsourced annotation workforce
1 million+ contributors in 235 countries
Quality control through multi-level review
Enterprise and custom solutions
SuperAnnotate:
AI-powered annotation platform
Raised $36 million Series B in November 2024
Multimodal dataset tooling
Computer vision and NLP support
Specialized Medical Tools
iMerit ANCOR:
AI-driven Annotation Copilot for Radiology
Launched December 2024
38% better accuracy and 2x output speed
Supports mammography and cardiology workflows (IMARC Group, 2024)
Myths vs Facts
Myth 1: Ground Truth Is Always 100% Accurate
Reality: Ground truth represents the best available reference, not absolute perfection. Human annotators make mistakes. Medical diagnoses can be uncertain. Even with careful processes, some label errors persist. Studies show that even benchmark datasets like ImageNet contain noisy labels.
Myth 2: More Annotators Always Means Better Quality
Reality: While multiple annotators help, there are diminishing returns. Three qualified annotators often produce similar results to ten. Quality matters more than quantity. Well-trained annotators following clear guidelines outperform large crowds of untrained workers.
Myth 3: AI Can Replace Human Annotators
Reality: AI-assisted annotation speeds up the process, but humans remain essential for quality control, handling edge cases, and making nuanced judgments. Fully automated labeling works for simple tasks but fails on complex, ambiguous, or novel situations.
Myth 4: Ground Truth Never Changes
Reality: Ground truth must be updated as the world changes. New types of spam emerge. Medical diagnostic criteria evolve. Product categories expand. Static ground truth becomes outdated, causing model performance to decay over time.
Myth 5: High Inter-Annotator Agreement Guarantees Quality
Reality: Annotators can consistently agree on incorrect labels if they share the same misconception or bias. A 2024 study found that high agreement sometimes masks low-quality judgments when annotators lack expertise (arXiv, 2025).
Myth 6: Unsupervised Learning Doesn't Need Ground Truth
Reality: While unsupervised learning doesn't use labeled training data, ground truth is still needed to evaluate results. How do you know if your clustering algorithm works without ground truth labels to validate the clusters?
Future of Ground Truth
Automated Labeling and Self-Supervision
AI systems increasingly label their own training data through self-supervised learning. Models learn from the structure of data itself without explicit labels. Examples include:
BERT learning language by predicting masked words
SimCLR learning visual representations by augmenting images
Contrastive learning identifying similar and dissimilar examples
While promising, these methods still require ground truth for final validation and fine-tuning.
Synthetic Data Generation
Creating artificial data that mimics real-world patterns can supplement expensive ground truth collection. Synthetic data works well for:
Rare events (like accidents in autonomous driving simulation)
Privacy-sensitive information (synthetic medical records)
Augmenting limited real-world datasets
However, synthetic data must be validated against real ground truth to ensure it captures authentic patterns.
Human-in-the-Loop AI
Future systems will combine AI efficiency with human judgment. AI does initial labeling; humans review and correct errors. This hybrid approach balances cost, speed, and accuracy.
iMerit's ANCOR system demonstrates this trend, using AI to automate repetitive tasks while humans handle complex cases, achieving 38% better accuracy than fully manual processes (IMARC Group, 2024).
Foundation Models and Transfer Learning
Large language models and vision transformers trained on massive datasets can transfer knowledge to specialized tasks with minimal ground truth. Few-shot learning allows models to adapt to new categories with only 5-10 labeled examples per class.
This reduces but doesn't eliminate ground truth requirements. You still need high-quality examples for those few shots.
Federated Learning and Privacy-Preserving Ground Truth
As privacy regulations tighten (GDPR, CCPA), techniques for creating ground truth without centralizing sensitive data are emerging. Federated learning trains models on distributed data without moving it to central servers.
Multimodal Ground Truth
Future AI will process multiple data types simultaneously—text, images, audio, video, sensor data. Ground truth must capture relationships across modalities. For example, labeling not just "what is in this image" but "how does it relate to this caption" and "what sounds are present in this video clip."
Sama's Multimodal platform launched in June 2025 addresses this need, showing 35% accuracy improvement when combining multiple data types with human validation (IMARC Group, 2024).
Continuous Learning and Ground Truth Streams
Rather than static datasets, future systems will continuously update ground truth as new data arrives and real-world patterns shift. This requires infrastructure for ongoing annotation, quality control, and model updating.
Regulatory and Standardization
As AI becomes mission-critical, expect increased regulation around ground truth quality, especially in high-stakes domains like healthcare, finance, and autonomous vehicles.
China's NDRC issued national guidelines in January 2025 targeting 20% compound growth for the labeling sector by 2027 and creating standardized AI-training roles (Mordor Intelligence, 2025). Similar initiatives will likely emerge globally.
FAQ
What is ground truth in simple terms?
Ground truth is the "correct answer" used to train and test AI models. It's verified, factual data that serves as the benchmark for measuring whether an AI's predictions are accurate. Think of it as the answer key to a test—without it, you can't know if the AI has learned correctly.
Why is it called ground truth?
The term originated in remote sensing in the 1960s-70s, when NASA needed to verify satellite imagery by collecting "data about materials on the earth's surface" (the ground). Scientists would physically visit locations to confirm what satellite pixels represented, establishing the "truth" on the ground. The concept was later adopted by machine learning and statistics.
Is ground truth always 100% accurate?
No. Ground truth represents the best available reference, not absolute perfection. Human annotators make errors. Medical diagnoses can be uncertain. Even expert-labeled data contains noise. The CLAIM 2024 medical imaging guidelines recommend "reference standard" instead of "ground truth" to acknowledge this uncertainty.
How much does ground truth data cost?
Costs vary widely by complexity. Simple image classification might cost $0.01-0.10 per image. Medical image segmentation requiring expert radiologists costs $5-50 per image. Expert annotators earn $30-100+ per hour. The global data annotation market reached $2.11 billion in 2024 and is projected to hit $12.45 billion by 2033 (IMARC Group, 2024).
What is inter-annotator agreement?
Inter-annotator agreement (IAA) measures how consistently multiple annotators label the same data. It's typically measured using Cohen's Kappa (two annotators) or Fleiss' Kappa (multiple annotators), with scores from 0 to 1. Higher scores indicate better agreement. Typical IAA scores range from 0.6-0.9 depending on task complexity.
Can AI create its own ground truth?
Partially. Techniques like self-supervised learning and automated labeling can reduce human annotation needs. However, human-verified ground truth is still essential for validation, handling edge cases, and ensuring accuracy. AI-assisted annotation combines efficiency of automation with quality control of human judgment.
How is ground truth different from training data?
Ground truth is the correct labels or answers within training data. Training data is the full dataset used to teach a model—it includes both the inputs (images, text) and the ground truth labels (cats, spam, diagnosis). Without ground truth labels, training data is just raw, unlabeled information.
What happens if ground truth is wrong?
Models learn incorrect patterns. A spam filter trained on mislabeled emails will block legitimate messages. A medical AI trained on incorrect diagnoses will misdiagnose patients. Garbage in, garbage out—poor ground truth creates unreliable AI. This is why quality control and multiple annotators are critical.
How often should ground truth be updated?
It depends on how quickly your domain changes. Spam and fraud patterns evolve rapidly—update monthly or quarterly. Medical diagnostic criteria change more slowly—update annually. Monitor model performance; declining accuracy signals outdated ground truth. For safety-critical applications like autonomous vehicles, continuous updating is essential.
What is the difference between ground truth and gold standard?
These terms are often used interchangeably, but "ground truth" originated in remote sensing (verified ground observations), while "gold standard" originated in medicine (definitive diagnostic test). The CLAIM 2024 guidelines for medical imaging recommend "reference standard" instead, as neither term accurately captures the uncertainty inherent in medical labeling.
How many annotators do I need per item?
For most tasks, 3-5 annotators provide a good balance of quality and cost. Use majority vote for final labels. Research shows 10+ annotators enhance agreement but offer diminishing returns. For subjective tasks or where disagreement is informative, consider using all annotations rather than forcing consensus.
Can ground truth be automated?
Partially. Pre-trained models can generate initial labels for human review. Active learning identifies which examples most need human annotation. For simple, well-defined tasks, automation may suffice. For complex, ambiguous, or safety-critical applications, human expertise remains essential.
What is ground truthing in GIS?
Ground truthing in GIS means visiting physical locations to verify remote sensing data. Field teams take GPS coordinates, measurements, and photographs to confirm what satellite or aerial imagery shows. This validates land cover classifications, forest inventories, agricultural monitoring, and environmental change detection.
How do you measure ground truth quality?
Methods include:
Inter-annotator agreement (Cohen's Kappa, Fleiss' Kappa)
Expert review of sample labels
Validation against external objective standards (medical outcomes, confirmed fraud cases)
Model performance on held-out test sets
Audits for bias and representation gaps
What is weak supervision?
Weak supervision uses noisy, incomplete, or imprecise labels instead of carefully hand-labeled ground truth. It trades some accuracy for much lower cost and faster labeling. Techniques include distant supervision (using knowledge bases to automatically label text) and programmatic labeling (using rules to generate labels).
How does ground truth relate to data annotation?
Data annotation is the process of creating ground truth. Annotators label raw data (images, text, audio) to produce the verified, correct answers that become ground truth. The annotation market—which creates ground truth—reached $2.11 billion in 2024 (IMARC Group, 2024).
What are common ground truth errors?
Common errors include:
Mislabeling due to annotator fatigue or misunderstanding
Inconsistent application of guidelines
Subjective disagreement between annotators
Missing or incomplete labels
Systematic bias toward certain classes
Temporal inconsistency as guidelines or annotators change
How do autonomous vehicles use ground truth?
Autonomous vehicles train on massive datasets of annotated driving footage. Human annotators label pedestrians, vehicles, traffic signs, lane markings, and obstacles frame by frame. LiDAR and radar provide 3D ground truth about object locations and distances. Waymo's dataset contains 12 million LiDAR and 9.9 million camera annotations (Mordor Intelligence, 2025).
What is synthetic ground truth?
Synthetic ground truth is artificially generated labeled data. For example, computer graphics can render photorealistic images of cars with perfect labels for every pixel. Simulations can generate rare events (like accidents) that are difficult to capture in real data. While useful for augmentation, synthetic data must be validated against real ground truth.
How is ground truth used in medical imaging?
Medical AI systems learn to detect diseases by training on thousands of images labeled by expert radiologists. Ground truth might come from:
Radiologist annotations of abnormalities
Biopsy and pathology results confirming diagnoses
Patient outcomes over time
Consensus from multiple expert reviewers The CLAIM 2024 guidelines recommend "reference standard" terminology in medical contexts.
Key Takeaways
Ground truth is verified, factual data that serves as the definitive benchmark for training, testing, and validating AI models—it's the "correct answer" against which predictions are measured
The global data annotation market (which produces ground truth) reached $2.11 billion in 2024 and will grow to $12.45 billion by 2033, reflecting massive demand driven by AI adoption across industries
Ground truth spans multiple fields: machine learning and AI (labeled training data), remote sensing and GIS (field-verified observations), medical imaging (expert-confirmed diagnoses), and autonomous vehicles (sensor-labeled driving scenarios)
Creating high-quality ground truth is expensive and time-consuming—simple image classification costs $0.01-0.10 per image, while expert medical annotation costs $5-50 per image, and expert annotators earn $30-100+ per hour
Inter-annotator disagreement is a persistent challenge, with typical agreement rates ranging from 70-95% depending on task complexity—multiple annotators and majority voting help improve consistency
Ground truth is not perfectly accurate—human annotators make errors, medical diagnoses can be uncertain, and labels reflect subjective judgments and biases
Waymo's autonomous vehicles demonstrate the value of comprehensive ground truth: 7 million driverless miles with 88% reduction in property damage claims and 92% reduction in bodily injury claims compared to human drivers (Swiss Re, 2024)
Ground truth becomes outdated as patterns change—spam filters trained in 2018 fail against 2024 phishing attacks, requiring continuous updating to maintain model accuracy
Future trends include automated labeling (AI-assisted annotation), synthetic data generation, human-in-the-loop systems, multimodal ground truth (combining text, images, audio), and continuous learning frameworks
Best practices include using 3-5 annotators per item, implementing rigorous quality control, providing clear guidelines with examples, training annotators thoroughly, and tracking inter-annotator agreement metrics
Actionable Next Steps
Assess Your Ground Truth Needs: Evaluate whether your AI project requires expert annotators (medical, legal) or general annotators. Estimate volumes and budget $15-50 per hour for skilled labor or $0.01-10 per item depending on complexity.
Choose Annotation Tools: For images, explore CVAT (open-source) or Labelbox (enterprise). For text, try Label Studio or Prodigy. For large-scale needs, consider platforms like Scale AI or Amazon SageMaker Ground Truth.
Develop Clear Guidelines: Write detailed annotation instructions with visual examples, edge case handling, and decision trees. Include both positive and negative examples.
Implement Quality Control: Use 3-5 annotators per item with majority voting. Mix in gold standard test questions with known answers. Monitor inter-annotator agreement using Cohen's Kappa or Fleiss' Kappa.
Start Small and Iterate: Annotate a pilot batch of 100-500 items. Measure inter-annotator agreement, refine guidelines, provide annotator feedback, then scale up.
Track Everything: Version control your annotation guidelines. Log annotator IDs, timestamps, agreement scores, and quality metrics. Document any changes to labeling criteria.
Consider Active Learning: Use model uncertainty to identify the most informative examples needing annotation. This can reduce labeling costs by 40-70% while maintaining accuracy.
Plan for Updates: Establish a schedule for refreshing ground truth. Monitor model performance; declining accuracy signals outdated labels. For rapidly changing domains, budget for quarterly updates.
Validate Externally: When possible, test ground truth against objective external benchmarks—medical outcomes, confirmed fraud cases, field surveys for remote sensing data.
Explore Hybrid Approaches: Combine AI pre-labeling with human review. Tools like iMerit ANCOR achieve 38% better accuracy and 2x speed by automating routine tasks while humans handle complex cases.
Glossary
Active Learning: A machine learning approach where the algorithm intelligently selects which examples to label next, focusing on the most informative data points to minimize annotation costs.
Annotation: The process of labeling data (images, text, audio) with ground truth information that machines can learn from.
Bounding Box: A rectangular box drawn around an object in an image to indicate its location and extent, commonly used in object detection tasks.
Cohen's Kappa: A statistical measure of inter-annotator agreement for two raters, ranging from 0 (no agreement beyond chance) to 1 (perfect agreement).
Consensus: The agreed-upon label determined when multiple annotators label the same item, typically using majority vote.
Crowdsourcing: Distributing annotation tasks to large groups of workers, typically through platforms like Amazon Mechanical Turk.
Data Augmentation: Creating additional training examples by applying transformations (rotation, scaling, color adjustment) to existing labeled data.
Fleiss' Kappa: A statistical measure of inter-annotator agreement for three or more raters, accounting for agreement by chance.
Gold Standard: (See Ground Truth) A reference standard considered definitive, though the CLAIM 2024 guidelines recommend "reference standard" instead to acknowledge uncertainty.
Ground Truthing: The process of collecting ground truth data, particularly in remote sensing where field teams verify satellite imagery.
Inter-Annotator Agreement (IAA): A measure of how consistently multiple annotators label the same data, typically measured using Cohen's Kappa or Fleiss' Kappa.
Label Fusion: Combining annotations from multiple annotators into a single consensus label, using techniques like majority voting or probabilistic models like STAPLE.
Labeled Data: Data that has been annotated with ground truth information indicating the correct classification or answer.
LiDAR: Light Detection and Ranging, a remote sensing method using laser pulses to measure distances and create 3D maps, used extensively in autonomous vehicles.
Reference Standard: The preferred term in medical imaging for what is commonly called ground truth, acknowledging that the "truth" may be uncertain.
Segmentation: The process of partitioning an image into multiple segments or regions, typically by labeling each pixel with a category.
Self-Supervised Learning: A machine learning approach where models generate their own training signals from the structure of unlabeled data, reducing reliance on human-labeled ground truth.
STAPLE: Simultaneous Truth and Performance Level Estimation, an algorithm that combines multiple annotations to estimate ground truth and annotator performance levels.
Supervised Learning: A machine learning approach where models learn from labeled training data with known ground truth.
Synthetic Data: Artificially generated data created through simulation or computer graphics, used to supplement real-world ground truth.
Training Data: The dataset used to teach a machine learning model, consisting of inputs and their corresponding ground truth labels.
Transfer Learning: Using knowledge from a model trained on one task to improve performance on a related task, reducing the amount of ground truth needed for the new task.
Unsupervised Learning: Machine learning that finds patterns in unlabeled data without explicit ground truth, though ground truth is still needed for evaluation.
Weak Supervision: Using noisy, incomplete, or imprecise labels instead of carefully curated ground truth, trading some accuracy for lower cost and faster labeling.
Sources & References
IMARC Group. (2024). Data Annotation Tools Market Size and Forecast to 2033. Retrieved from https://www.imarcgroup.com/data-annotation-tools-market
IBM. (2024, November). What Is Ground Truth in Machine Learning? IBM Think Topics. Retrieved from https://www.ibm.com/think/topics/ground-truth
C3 AI. (2024, June 11). What is Ground Truth? Machine Learning Glossary Definition. Retrieved from https://c3.ai/glossary/machine-learning/ground-truth/
Domino Data Lab. (2025, June 5). What is Ground Truth in Machine Learning? Data Science Dictionary. Retrieved from https://domino.ai/data-science-dictionary/ground-truth
Wikipedia. (2025, August 21). Ground truth. Retrieved from https://en.wikipedia.org/wiki/Ground_truth
Lyzr.ai. (2024, November). Ground Truth in Machine Learning. Retrieved from https://www.lyzr.ai/glossaries/ground-truth-in-machine-learning/
Telnyx. (2025, January 6). Why ground truth matters in AI. Retrieved from https://telnyx.com/learn-ai/ground-truth
GIS Geography. (2025, April 10). Ground Truthing: Verify Remotely Collected Data. Retrieved from https://gisgeography.com/ground-truthing/
Atlas. Ground Truth Verification - Definitions & FAQs. Retrieved from https://atlas.co/glossary/ground-truth-verification/
Esri. (2025). Ground Truth Definition. GIS Dictionary. Retrieved from https://support.esri.com/en-us/gis-dictionary/ground-truth
Sustainability Methods. Ground Truth. Retrieved from https://sustainabilitymethods.org/index.php/Ground_Truth
Malvern Panalytical. (2025). Ground Truthing | Ground Truth Data Collection. Retrieved from https://www.malvernpanalytical.com/en/products/measurement-type/ground-truthing
Global Growth Insights. (2024). Data Annotation Tool Market Size, Share & Growth Report [2025-2033]. Retrieved from https://www.globalgrowthinsights.com/market-reports/data-annotation-tool-market-104931
Grand View Research. (2023). Data Annotation Tools Market Size And Share Report, 2030. Retrieved from https://www.grandviewresearch.com/industry-analysis/data-annotation-tools-market
Expert Market Research. (2024). Data Annotation Tools Market Size and Forecast to 2033. Retrieved from https://www.expertmarketresearch.com/reports/data-annotation-tools-market
Research Nester. (2025, May 28). Data Annotation Tools Market size to hit $32.54 billion by 2037. Retrieved from https://www.researchnester.com/reports/data-annotation-tools-market/4763
Cognitive Market Research. (2025, September 19). Data Annotation and Labeling Market Report 2025. Retrieved from https://www.cognitivemarketresearch.com/data-annotation-and-labeling-market-report
Verified Market Research. (2024, November). Data Annotation Service Market Size, Share, Trends & Forecast. Retrieved from https://www.verifiedmarketresearch.com/product/data-annotation-service-market/
Mordor Intelligence. (2025, June 17). Data Annotation Tools Market Size & Outlook, 2025-2033. Retrieved from https://www.mordorintelligence.com/industry-reports/data-annotation-tools-market
Straits Research. (2025, March 24). Data Annotation Tools Market Size & Outlook, 2025-2033. Retrieved from https://straitsresearch.com/report/data-annotation-tools-market
IMARC Group. (2024). Healthcare Data Annotation Tools Market Report by 2033. Retrieved from https://www.imarcgroup.com/healthcare-data-annotation-tools-market
Contrary Research. (2025, July 8). Deep Dive: Tesla, Waymo, and the Great Sensor Debate. Retrieved from https://research.contrary.com/deep-dive/tesla-waymo-and-the-great-sensor-debate
IEEE Spectrum. (2024, December 11). Robotaxis Are Blazing the Trail for Self-Driving Cars. Retrieved from https://spectrum.ieee.org/robotaxi
Think Autonomous. (2025, September 10). Tesla vs Waymo - Who is closer to Level 5 Autonomous Driving? Retrieved from https://www.thinkautonomous.ai/blog/tesla-vs-waymo-two-opposite-visions/
IPG Media Lab. (2024, October 25). The Near Future of Self-Driving Cars. Medium. Retrieved from https://medium.com/ipg-media-lab/teslas-cybercab-waymo-s-robotaxi-and-the-nearing-future-of-self-driving-cars-e9a34ac6a86b
MishTalk. (2025, January 7). Waymo Picked Up 4 Million Rides in 2024, Tesla Had Zero. Retrieved from https://mishtalk.com/economics/waymo-picked-up-4-million-rides-in-2024-tesla-had-zero/
Impakter. (2025, August 26). Tesla vs. Waymo: The Trillion Dollar Robotaxi Battle. Retrieved from https://impakter.com/tesla-vs-waymo-the-trillion-dollar-robotaxi-battle/
Radiological Society of North America. (2024). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): 2024 Update. Radiology: Artificial Intelligence. Retrieved from https://pubs.rsna.org/doi/full/10.1148/ryai.240300
PMC. (2024). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): 2024 Update. Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC11304031/
The Lancet eClinicalMedicine. (2025, May 12). Artificial intelligence for diagnostics in radiology practice: a rapid systematic scoping review. Retrieved from https://www.thelancet.com/journals/eclinm/article/PIIS2589-5370(25)00160-9/fulltext
RSNA Radiology. (2024). Using AI to Improve Radiologist Performance in Detection of Abnormalities on Chest Radiographs. Retrieved from https://pubs.rsna.org/doi/full/10.1148/radiol.230860
ScienceDirect. (2024, July 12). Large-Scale Study on AI's Impact on Identifying Chest Radiographs with No Actionable Disease in Outpatient Imaging. Academic Radiology, 31(12), 5237-5247. Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S1076633224003908
Radsource. (2025, July 7). The Evolving Role of AI in Radiology in 2024. Retrieved from https://radsource.us/ai-in-radiology/
AIM Multiple. Top 6 Radiology AI Use Cases for Improved Diagnostics. Retrieved from https://research.aimultiple.com/radiology-ai/
arXiv. (2024, July 16). Minority Reports: Balancing Cost and Quality in Ground Truth Data Annotation. Retrieved from https://arxiv.org/pdf/2504.09341
ResearchGate. (2024). Inter-annotator Agreement. Retrieved from https://www.researchgate.net/publication/318176345_Inter-annotator_Agreement
PMC. (2023). Assessing Inter-Annotator Agreement for Medical Image Segmentation. Retrieved from https://pmc.ncbi.nlm.nih.gov/articles/PMC10062409/
Innovatiana. (2025). Discover about "Ground Truth" in Data Science and AI. Retrieved from https://www.innovatiana.com/en/post/ground-truth-in-ai
Innovatiana. Inter-Annotator Agreement: a key metric in Labeling. Retrieved from https://www.innovatiana.com/en/post/inter-annotator-agreement
Emergent Mind. Inter-Annotator Agreement (IAA). Retrieved from https://www.emergentmind.com/topics/inter-annotator-agreement-iaa
California Learning Resource Network. (2025, January 26). What is Ground Truth in Machine Learning. Retrieved from https://www.clrn.org/what-is-ground-truth-in-machine-learning/
arXiv. (2025, July 31). Beyond Agreement: Rethinking Ground Truth in Educational AI Annotation. Retrieved from https://arxiv.org/html/2508.00143
LHN CBC. Assessing Inter-Annotator Agreement for Medical Image Segmentation. Retrieved from https://lhncbc.nlm.nih.gov/LHC-publications/PDF/Assessing_Inter-Annotator_Agreement_for_Medical_Image_Segmentation.pdf
$50
Product Title
Product Details goes here with the simple product description and more information can be seen by clicking the see more button. Product Details goes here with the simple product description and more information can be seen by clicking the see more button
$50
Product Title
Product Details goes here with the simple product description and more information can be seen by clicking the see more button. Product Details goes here with the simple product description and more information can be seen by clicking the see more button.
$50
Product Title
Product Details goes here with the simple product description and more information can be seen by clicking the see more button. Product Details goes here with the simple product description and more information can be seen by clicking the see more button.
Comments