The structured data problem in healthcare

Healthcare generates more clinical data today than at any point in history. And clinicians can meaningfully access less of it than you might expect.

That is not a storage problem. The data exists. It lives inside electronic health records, in lab systems, in PDF attachments, in notes written by dozens of clinicians across dozens of encounters. The problem is that most of it is unstructured, and unstructured data cannot be searched, queried, compared, or reasoned over without a human reading through it manually.

This piece focuses on a technical thread from a recent Clinical Innovation Team deep dive:

1. What structured data actually means in a clinical setting
2. What SNOMED is and why it matters
3. The two very different problems of retrospective retrieval and forward-looking structuring, and where the limits of each approach sit.

What "unstructured" means in practice

When a clinician documents a patient encounter, the output is usually a free-text note. It describes what the patient reported, what the clinician observed, what was decided, and what should happen next. It is rich, nuanced, and written for another clinician to read. It is also, from a data perspective, almost entirely opaque.

A note that says "patient reports lower back pain radiating to the left leg, onset approximately three weeks ago, worsening over the last 48 hours" contains precise clinical meaning. But a computer cannot extract that meaning, compare it to a note from six months ago, or flag it as a progression pattern without significant effort.

EHR systems do have structured fields. Allergies are supposed to be entered in a dedicated section. Diagnoses get coded. Medications appear in a structured list. But these fields are often incomplete, inconsistently used, or out of date, because the actual clinical information flows primarily through free-text notes. The structured data is a downstream artifact of the documentation process, and it is frequently not maintained with the same care as the notes themselves.

The result is a system where the most important clinical information is technically present but functionally buried.

SNOMED and the case for a common language

SNOMED CT (Systematized Nomenclature of Medicine, Clinical Terms) is the most comprehensive clinical terminology system in the world. It contains several hundred thousand concepts covering diseases, findings, procedures, organisms, medications, and more, each with a unique identifier and defined relationships to other concepts.

The promise of SNOMED is interoperability and searchability. If a peanut allergy is always encoded as the same concept, regardless of which clinician documented it, which hospital system recorded it, or what free-text phrasing was used, it becomes possible to search for that allergy across an entire patient record, across institutions, and across time.

For certain categories of information, this works well. Allergies are the clearest example. A well-structured allergy record captures not just the allergen but the reaction type, the severity, and critically, the source: whether it came from the patient's self-report, a formal test, or a clinician's clinical suspicion. That distinction matters because it affects how much weight to assign the information. A suspected allergy buried in a note three years ago is different from a confirmed allergy on record.

The same logic applies to diagnoses, medications, observations, family history, and procedures. These are all categories where structured coding adds real value, and where the absence of structure creates real risk.

For a deeper look at what SNOMED CT is and why it matters for AI in healthcare, see SNOMED CT: healthcare's new common language, and why AI should care.

The limits of full structuring

The more ambitious version of the argument, structuring everything, runs into problems quickly.

SNOMED is comprehensive enough that "speaks French" is a valid concept in the system. You could encode that. In practice, the moment you start encoding that kind of information, you have probably lost the point. Not all clinical information is equal, and not all of it benefits from being converted into a code.

Clinical notes express something that a list of codes cannot. A note captures uncertainty, clinical reasoning, the distinction between "this is definitely the diagnosis" and "we are ruling this out." It captures temporal relationships within a single encounter: the sequence in which symptoms appeared, which came first, which triggered concern. A SNOMED code for lower back pain tells you the concept was present. It does not tell you whether the pain started acutely or built gradually over weeks, or whether the clinician was more or less worried about it than the last time it was documented.

The deeper technical challenge is that SNOMED itself is not as clean as it appears. A peanut allergy can be expressed as a single concept, as a composition of allergen and reaction as separate codes, or as any number of variant codes that mean roughly the same thing. Structuring for retrieval does not eliminate ambiguity; it moves the ambiguity from the note into the coding layer.

The team's honest assessment: full structuring is probably not possible and possibly not desirable. The more useful question is which categories of clinical information are high-value enough to structure, and which are better handled through other means.

Two problems that need different solutions

The most important distinction from the deep dive was this: retrospective data retrieval and forward-looking data creation are different problems. They look similar on the surface but require different approaches to solve.

Retrospective retrieval is the problem of making sense of everything documented before today, across a patient's entire history. That data is overwhelmingly unstructured. It lives in notes, PDFs, scanned documents, and formats that were never designed to be machine-readable. The challenge is finding the needle in the haystack: locating a prior medication reaction buried in a note from four years ago, or identifying when a symptom first appeared across a record that spans decades.

Full SNOMED coding of historical notes is not a practical solution here. Running every historical note through a coding model is computationally expensive and produces a representation that is still incomplete. A semantic search approach, building an index that allows meaning-based retrieval rather than keyword matching, is more practical for this problem. The goal is not to perfectly structure everything that was ever written, but to make it retrievable enough that a clinician can surface relevant history quickly when they need it.

Forward-looking structuring is a different problem entirely. Here, the question is how to ensure that clinical information captured from this point forward ends up in structured form, so that it is available and usable for every subsequent encounter. An agent-based approach is well-suited to this: during a consultation, as a clinician documents, an AI agent extracts clinical concepts, proposes structured entries, and presents them for one-click confirmation into the EHR. The clinician stays in their workflow. The structured data gets created as a byproduct.

These two problems are related but not the same, and conflating them leads to solutions that try to do too much and end up doing neither well.

The timestamp problem

Even when you solve the extraction challenge, there is a subtler problem underneath it.

Structured data tells you what. It does not always tell you when, in the clinically meaningful sense.

A note has a timestamp: the date it was written. That is not the same as the date the clinical events it describes actually occurred. A note written today might document a symptom the patient first noticed three weeks ago, a medication change that happened during a prior admission, or a family history that has been stable for decades.

When you extract a SNOMED code from that note and attach the note's timestamp, you are recording when the documentation happened, not when the clinical event happened. For building a timeline, that distinction matters significantly. Lower back pain coded to a note from last Tuesday is not the same as lower back pain that started last Tuesday.

The more useful approach is to extract not just the concept but the temporal reference within the note: what time anchors does the text contain, and can they be resolved to actual dates? In some cases this is straightforward. In others, it is ambiguous. The honest system acknowledges that distinction rather than collapsing everything to the note date.

What this means in practice

Structured data is a foundation, not a solution on its own.

The goal is a view of a patient's history that is coherent, ordered, and useful at the moment of care. Structured data, done well, makes that view possible for the categories it captures cleanly: allergies, diagnoses, medications, key observations. For those categories, coded entries are preferable to free text because they are searchable, comparable, and transferable across systems.

But the clinical record is larger than those categories. A lot of what matters is in the notes, and the notes need to remain readable and retrievable without being fully replaced by codes.

The practical architecture is an intelligence layer that sits on top of both: something that can query structured data where it exists, retrieve relevant content from unstructured notes where it does not, and surface a combined view that is ordered chronologically, filtered by clinical context, and honest about where gaps exist.

Building that layer means solving the retrospective retrieval problem and the forward-looking structuring problem separately, with different tools applied to each, while keeping the clinician in the loop at the points that matter.

This piece is part of Corti's ongoing series exploring clinical workflows where AI infrastructure can make a meaningful difference. The ideas here draw from problem exploration sessions run by Corti's Clinical Innovation Team.