20 Oct 2026 · 5 min read

NLP in Clinical Notes: What AI Can Extract and What It Cannot

Free-text clinical notes contain information that structured EHR fields never capture. Natural language processing can extract much of it - but the gaps, especially around negation and clinical context, are larger than most implementations acknowledge.

NLP in Clinical Notes: What AI Can Extract and What It Cannot

A typical hospital discharge summary contains more clinically relevant information than the structured fields in the electronic health record that feeds most research databases. The structured fields capture diagnosis codes, procedure codes, medication lists, and vital signs. The free-text narrative captures the clinical reasoning, the uncertainty, the nuance, the things the clinician noticed but did not code, and the patient's own words when they were recorded. For decades, this information was inaccessible for research at scale. Natural language processing changed that - partially, unevenly, and with limitations that are frequently underestimated.

What NLP Can Extract Reliably

Clinical NLP systems have become genuinely competent at a set of well-defined extraction tasks. Named entity recognition - identifying mentions of diagnoses, medications, procedures, anatomical structures, and laboratory values in free text - achieves high precision and recall on common entities in well-trained systems. Systems like cTAKES (Clinical Text Analysis and Knowledge Extraction System) and MedSpaCy, both developed specifically for clinical language, can extract these entities with F1 scores above 0.85 in most evaluations on standard benchmarks.

The extraction tasks that NLP handles well are those with consistent surface forms - medication names, standard diagnoses, and laboratory test names that appear in predictable patterns. Where the language is formulaic and the vocabulary is controlled, NLP performs reliably.

Where NLP Struggles: Negation and Uncertainty

The clinical note contains more negated statements than almost any other text type. "No evidence of metastasis." "Patient denies chest pain." "CT scan negative for pulmonary embolism." Extracting these mentions and incorrectly classifying them as positive findings - assigning a diagnosis of metastasis, chest pain, or PE to the patient - produces systematic errors that can entirely invalidate a real-world evidence study.

Negation detection has been an active research area for two decades, and systems like NegEx handle common negation patterns reliably. But negation in clinical notes includes pre-negation, post-negation, and negation within complex syntactic structures that simpler rule-based systems miss. Uncertainty expressions - "possible pneumonia," "cannot exclude," "query," "rule out" - are equally challenging. A patient noted as having "possible heart failure" is not a patient with heart failure, but a naive extraction system may count them as one. Studies on the i2b2 challenge datasets have consistently shown that negation and uncertainty handling remain major sources of error in clinical NLP systems.

Context and Temporality

Clinical notes routinely describe past history, family history, and hypothetical scenarios in the same text that describes the current clinical situation. An extraction system that does not distinguish between "the patient has a family history of colon cancer" and "the patient has colon cancer" will systematically misclassify family history mentions as patient diagnoses. Similarly, the temporal relationship between clinical events matters for most research questions - whether a complication preceded or followed a treatment is fundamental - and extracting temporal relationships from free text remains substantially harder than extracting entities.

The gap between what a clinical NLP system extracts and what it actually means clinically is widest precisely in the dimensions that matter most for understanding patient trajectories: the sequence of events, the certainty of diagnoses, and the context in which findings arose.

HIPAA, GDPR, and the Data Access Problem

Clinical notes are among the most sensitive personal data that exists. They contain not only medical information but social history, mental health details, family circumstances, and observations that patients share with clinicians in a relationship of trust. HIPAA in the United States and GDPR in Europe impose legal constraints on the use of this data for research, requiring either specific consent, de-identification, or data use agreements that limit access to authorised research teams within defined purposes.

De-identification of free-text clinical notes - removing or replacing names, dates, addresses, and other identifying information - is technically feasible but imperfect. NLP-based de-identification systems miss identifiers, particularly when they appear in unusual forms or when the identifying information is clinical rather than demographic. Research using de-identified clinical NLP data should acknowledge the residual re-identification risk, particularly for patients with rare conditions or unusual clinical histories.

MIMIC and i2b2: The Research Datasets

The MIMIC-III and MIMIC-IV datasets (Medical Information Mart for Intensive Care) from Beth Israel Deaconess Medical Center are the most widely used public clinical databases for NLP research. They contain de-identified clinical notes, structured EHR data, and outcomes for ICU patients, and they have enabled a large body of research on clinical NLP methods. The i2b2 (Informatics for Integrating Biology and the Bedside) challenge datasets have provided annotated corpora for NLP shared tasks in negation detection, temporal reasoning, de-identification, and clinical concept extraction.

These datasets are valuable but limited in generalisability - MIMIC is a single-centre ICU database, and NLP models trained on it may not transfer directly to outpatient notes, community hospital records, or notes from different health systems with different documentation cultures.

At NousLab, we see clinical NLP as a powerful tool for real-world evidence generation, used critically. The extraction is only as good as the validation, and the validation should always include domain experts reviewing samples where the clinical context is complex. See how NousLab approaches real-world evidence and clinical data.

Jesus Arias
Jesus Arias
Founder & CEO at NousLab

Join the Research Revolution

See how NousLab can accelerate your organization's research

AI