HistoGPT
by AI Agent
— Automating high-quality pathology reports from histology images.
Regulatory Status Disclosed
Overview
HistoGPT is an innovative AI tool designed to transform dermatopathology by automating the generation of high-quality pathology reports directly from gigapixel whole slide images (WSIs). This vision-language foundation model processes immense spatial data from images and translates intricate morphological features into coherent, clinically relevant textual descriptions. It aims to streamline the traditionally labor-intensive and time-consuming process of pathology reporting, enhancing diagnostic accuracy and efficiency for dermatological diseases, including cancer. HistoGPT was developed by a multidisciplinary team led by Helmholtz Munich researchers Dr. Tingying Peng and Dr. Carsten Marr, and dermatologists at University Hospital Muenster and University of Freiburg, Dr. Stephan A. Braun and Dr. Kilian Eyerich. Some sources also associate it with Metrum AI.
Reviewed by Pouyan Golshani, MD — Interventional Radiologist
Key Features
- Automated pathology report generation from WSIs
- Vision-language model for image and text understanding
- Disease classification
- Tumor subtype prediction
- Tumor thickness estimation
- Text-to-image gradient-attention maps for explainability
- Zero-shot learning for new cases without additional training
- Handles gigapixel whole slide images (WSIs)
- Generates human-level written reports
- Can be queried for additional details using prompts
Use Cases
- Automating dermatopathology report generation
- Assisting pathologists in routine evaluations
- Providing a second opinion for pathologists
- Enhancing diagnostic accuracy and efficiency in dermatological diseases, including cancer
- Standardizing pathology reporting
- Training native-language reporting models on institutional archives
What Physicians Need to Know
Evidence Base
HistoGPT is a vision-language foundation model trained on an extensive dataset of 15,129 whole slide images from 6,705 dermatology patients, along with their corresponding pathology reports, from the Department of Dermatology at the Technical University of Munich. [1, 3, 7, 9, 10] This dataset includes 167 skin diseases. [3] The model integrates vision and language by extracting features from gigapixel images and combining them with a large language model via cross-attention mechanisms. [2, 3, 4]
Clinical Validation Studies
HistoGPT's diagnostic reports have been rigorously validated in multi-center studies, demonstrating robust capabilities in real-world scenarios. [1, 6, 7, 9] It successfully predicted critical tumor characteristics, including tumor subtypes, tumor thickness, and tumor margins, in a zero-shot fashion (without additional training for new cases). [1, 5, 6, 7, 9, 10] Evaluations at leading medical institutions like the Mayo Clinic (USA), University Hospital Mu00fcnster (Germany), and Radboud University Medical Center (Netherlands) confirmed its accuracy for common neoplastic epithelial lesions such as basal cell carcinoma, melanocytic nevus, actinic keratosis, and squamous cell carcinoma. [1, 9] The generated reports matched or exceeded the diagnostic accuracy of board-certified dermatopathologists. [2, 5, 6, 7, 9, 10] Independent expert reviews and natural language processing metrics confirmed the high quality of HistoGPT's reports, finding them very similar to human-written reports, especially for common and well-known cancer types. [1, 3, 5, 6, 7, 9, 10]
Alert Fatigue Management
While HistoGPT is designed to automate pathology report generation and assist in diagnosis, the provided information does not specifically detail features for alert fatigue management. [1, 2, 3, 4, 5, 6, 7, 9, 10] However, by streamlining the reporting process and providing accurate, standardized reports, it has the potential to reduce pathologist workload, which can be a contributing factor to alert fatigue in other clinical decision support systems. [1, 9, 16, 21]
Differential Diagnosis Support
HistoGPT synthesizes visual embeddings from whole slide images into structured reports that can include lesion characterization and differential diagnoses. [2] It can predict disease classification and discriminate between tumor subtypes. [3, 4, 9, 10] The model's ability to generate comprehensive reports and provide disease classification directly supports the differential diagnosis process for dermatological conditions. [2, 3, 4, 9, 10]
Guideline Update Frequency
The provided information does not specify a guideline update frequency for HistoGPT. [1, 2, 3, 4, 5, 6, 7, 9, 10] As an AI model, its knowledge base is derived from its training data. [1, 2, 3, 7, 9] Updates to its performance would likely come from retraining with new or expanded datasets and model refinements. [1, 2, 3, 5, 9]
Clinical Workflow Integration
HistoGPT is designed to integrate into the clinical workflow by assisting pathologists in evaluating, reporting, and understanding routine dermatopathology cases. [3, 5, 6, 7, 9, 10] It generates human-level written reports, provides disease classification, predicts tumor characteristics, and offers text-to-image gradient-attention maps for explainability. [3, 9, 10] This output can serve as a second opinion or a first draft for the final report, and can be used to fill in standardized templates. [3, 10, 12] The open-source nature of the project also invites community collaboration for further refinement and integration. [2, 5]
Decision Audit Trail
HistoGPT is designed to be fully interpretable, with every word or phrase in the output text visualizable in the original image through text-to-image gradient-attention maps. [3, 4, 9, 10] This explainability feature can contribute to an audit trail by showing the basis of its diagnostic assertions to human reviewers. [2, 3, 9] While the documentation for HistoGPT itself doesn't explicitly detail a 'decision audit trail' feature in the context of a platform-level compliance, its interpretability supports auditable workflows. [2, 3, 4, 9, 11, 20]
Physician Tip
HistoGPT can significantly streamline your dermatopathology reporting by generating high-quality initial drafts and providing a 'second opinion' with detailed image-to-text explanations. Leverage its ability to predict tumor characteristics and classify diseases to enhance efficiency and consistency in your diagnoses. Remember that while HistoGPT offers advanced support, human oversight and final review remain crucial for patient care.
HistoGPT is a vision-language model that processes whole slide images and generates pathology reports. Its integration into existing clinical systems would involve feeding digitized histology images to the model and incorporating its generated reports into electronic health records or pathology information systems. The open-source nature of the project suggests potential for custom integrations and development by the scientific community. [2, 5]
Details
| Category | Clinical Decision Support & Reference, Dermatology AI, Pathology AI |
| Pricing | Unknown — unknown |
| Deployment | Can be deployed on local machines |
| Compliance | |
| BAA Available | Unknown AI-estimated |
| HIPAA Compliant | Unknown AI-estimated |
| FDA Status |
Not applicable AI-estimated HistoGPT itself does not have FDA clearance. While digital pathology solutions for primary diagnostic use have received FDA clearance (e.g., Philips IntelliSite Pathology Solution in 2017), HistoGPT is a research tool and its regulatory pathway as a medical device remains uncertain. |
| Integrations | |
| EHR | Not specified |
| Specialties | Dermatology, Oncology, Pathology |
What the Web Says
HistoGPT is a vision language model designed to automate the generation of dermatopathology reports from whole slide images. It aims to reduce the time-consuming, labor-intensive, and often non-standardized manual process currently performed by pathologists. The model has been evaluated in multi-center clinical studies and shows promise in assisting with routine dermatopathology cases by providing accurate reports and predictions for common malignancies.
Overall: MixedStrengths
- Generates human-level pathology reports for common and homogeneous malignancies.
- Accurately predicts tumor subtypes, tumor thickness, and tumor margins in a zero-shot fashion.
- Can serve as a second opinion and provide a first draft for pathologists, potentially reducing workload.
- Offers model explainability through text-to-image gradient-attention maps.
- Outperforms some other state-of-the-art vision language models in certain tasks.
- Generalizes well to diverse cohorts, including different countries, scanner types, and staining techniques.
Limitations
- Sometimes produces anatomically inconsistent or diagnostically mismatched descriptions.
- May exhibit cross-turn inconsistency and self-contradictory hallucinations in multi-turn conversational settings.
- Less effective with complex cases compared to routine ones.
- A competing model, SlideChat, has shown superior performance on most tasks in a comprehensive evaluation.
Based on reviews from: Nature Communications, Bioengineer.org, PMC, medRxiv, Helmholtz Munich
Last updated: 2026-09-13
Ratings & Reviews
No reviews yet. Be the first to review this tool!
Rate HistoGPT
Press & Coverage
Helmholtz Munich
HistoGPT: Advancing AI-Powered Pathology Reporting in Dermatopathology
HistoGPT is a new AI tool developed by Helmholtz Munich researchers and dermatologists that automates the generation of high-quality pathology reports from histology images, aiming to transform dermatopathology. The model has shown proven accuracy in clinical applications, generating reports on par with human-written reports for common and well-known types of cancer.
2025-05
Nature Communications
Generating dermatopathology reports from gigapixel whole slide images with HistoGPT
This article details HistoGPT, a foundation vision language model for dermatopathology that generates human-level written reports, provides disease classification, and predicts tumor characteristics from whole slide images. The model was trained on a large dataset of dermatopathology cases and validated in an international, multi-center clinical study.
2025-05
Lumea
Top AI Companies for Digital Derm Pathology
Lumea highlights HistoGPT as a significant new solution in digital dermatopathology, noting its ability to automate complex clinical documentation by generating the first draft of pathology reports. Developed by Helmholtz Munich and TUM, HistoGPT was featured in Nature Communications (2025) for its breakthrough in AI-powered reporting.
2026-03
medRxiv
Generating clinical-grade pathology reports from gigapixel whole slide images with HistoGPT
This preprint introduces HistoGPT, a vision language model that generates pathology reports from multiple full-resolution histology images, aiming to address the time-consuming and non-standardized nature of traditional histopathology reporting. The model has been evaluated in real-world medical cohorts and clinical evaluations, showing potential as an AI assistant for pathologists.
2024-06
medRxiv
Generating highly accurate pathology reports from gigapixel whole slide images with HistoGPT
This preprint describes HistoGPT as a vision language model that generates pathology reports from digitized slides, matching the quality of human-written reports. The research highlights HistoGPT's ability to generalize across international cohorts and predict tumor subtypes and thickness in a zero-shot fashion.
2024-03
Helmholtz Munich
New AI Model Enhances Speed and Accuracy in Medical Diagnoses
This article from Helmholtz Munich mentions HistoGPT in its 'Related news' section, indicating its role in advancing AI-powered pathology reporting in dermatopathology. It highlights the broader efforts in developing AI models for faster and more accurate medical diagnoses.
2025-03
MDPI
Conditional Generative AI in Oncology Diagnostics
This review article discusses HistoGPT as a dermatopathology-specific model trained on over 15,000 gigapixel WSIs, capable of generating complete pathology reports and demonstrating zero-shot inference of structured report elements. It emphasizes HistoGPT's clinical relevance in melanoma diagnostics and its comparable performance to human-written reports.
2026-04
unknown
Artificial Intelligence in Pathology: Advancing Large Models for Scalable Applications
This article discusses the impact of AI on medical research and pathology studies, mentioning HistoGPT as a model for generating clinical-grade pathology reports from gigapixel whole slide images. It highlights the trend of applying large-scale AI models in pathology.
2025-08
Frequently Asked Questions
HistoGPT is designed to act as an AI assistant, generating human-level written reports, providing disease classification, discriminating tumor subtypes, and predicting tumor depth and surgical margins directly from whole slide images (WSIs). This output can serve as a second opinion or a first draft for your final pathology report, streamlining the process and potentially reducing manual labor.
While promising, HistoGPT has limitations including potential instability in quantitative attribute estimation (e.g., numeric hallucination) and a tendency towards diagnostic overspecification. It may also exhibit errors resembling known diagnostic pitfalls in routine pathology. Furthermore, it has primarily been trained and tested on dermatological samples, limiting its generalizability to other tissue types.
HistoGPT has demonstrated high accuracy in generating diagnostic reports for common neoplastic epithelial lesions, matching or exceeding the diagnostic accuracy of board-certified dermatopathologists in rigorous validation studies. It has been evaluated in multi-center clinical studies and shown to accurately predict tumor subtypes, thickness, and margins in a 'zero-shot' fashion, meaning it can analyze new cases without additional training.
The regulatory pathway for approving pathology-specific multimodal large language models as medical devices remains uncertain. However, HistoGPT incorporates attention mechanisms and explainability modules to reveal the basis of its diagnostic assertions, aiming to build trust and support the collaborative nature of medical decision-making. The model's architecture also supports continuous learning to adapt to evolving clinical guidelines.
While other AI models exist for image classification or text generation at the patch level, HistoGPT is unique in its ability to generate comprehensive reports directly from entire gigapixel whole slide images, and even from multiple images simultaneously. This integrated vision-language approach allows it to process vast spatial data and translate intricate morphological features into coherent, clinically relevant textual descriptions, unlike conventional AI models trained separately on image or text.
The provided information does not specify the pricing model for HistoGPT. However, pricing for pathology services, in general, can be influenced by factors such as H&E slide preparation, supplies for biopsy submission, grossing services, and delivery options. For specific pricing, it is recommended to fill out a quote form.
HistoGPT has demonstrated the ability to predict tumor subtypes and tumor thickness in a 'zero-shot' fashion, meaning it can accurately analyze new cases without requiring additional training. While trained on a diverse corpus of skin pathologies, including common and rarer diagnoses, its performance on samples rarely seen during training may highlight limitations of current deep learning techniques.
Suggest an Edit →
| Last Verified: 2026-09-11 | First Added: 2026-09-11
Investors who backed HistoGPT
Funded through the company that built this tool.
