“I can build this with Claude in a weekend.” What month 6 actually looks like

Pick almost any engineer today and they can wire up an impressive demo in a weekend: take a clinical PDF, send it to a large language model, and get back tidy structured data. It works. It looks like the problem is solved.

And for that one clean PDF, it basically is. The demo is real, and it is genuinely useful for deciding whether to build at all. The trouble is that the weekend demo is the easy five percent. The other ninety-five lives in the parts the demo quietly skipped.

Month one: the documents are not clean

The first surprise arrives the moment you feed in real records. Production inputs are faxes, scans of scans, and handwriting, not clean PDFs, and models that read tidy documents well fall off sharply on messy ones. So now you are building a stack the demo never needed:

• Image cleanup for poor-quality scans
• Layout detection for pages with unusual structure
• A way to catch and re-check the pages the model misread

The model was the quick part. The plumbing around it is the actual project.

Month two: reading is not the same as being right

Next you learn that pulling text off a page tells you nothing about whether the result is clinically correct. A diagnosis has to be coded to the exact level the note supports, and the wrong level of detail is simply the wrong answer. Getting that right means encoding real healthcare logic, the kind that lives in experienced coders, not writing a cleverer prompt.

Month three: healthcare does not forgive the boring parts

Then come the rules software cannot skip. Patient data is protected under HIPAA, so every step needs secure handling, access controls, and logging. And because a wrong code can mean a repaid claim, you cannot just produce an answer, you have to prove where it came from. That means an audit trail linking every output back to the exact words in the source document. None of this appears in a demo, and all of it is mandatory.

Month four and beyond: it has to work every time

A demo runs once, on an input you chose. Production runs on everything, all day. That means:

• Service-level guarantees you have to hit
• Thousands of document types, each with its own odd edge cases
• Monitoring for when accuracy quietly drifts
• Someone on call when it breaks at 2 a.m.

This is ordinary software-reliability work, and there is a lot of it.

The part with no finish line

Even then you are not done, because the rules underneath keep moving:

• ICD codes get revised
• Payer policies change
• Federal deadlines like the 2027 FHIR requirements arrive

A system tuned on last year's rules quietly makes wrong decisions this year. The weekend project has become a permanent team.

What the numbers say

This is not a pessimistic guess, it is the base rate. RAND found that more than 80% of AI projects fail, about twice the rate of other IT projects. Gartner expects at least 30% of generative-AI pilots to be abandoned after the proof-of-concept stage. And the people who can do this work are not cheap: experienced ML engineers in the US run well over $150,000 each, so even a small in-house team costs north of a million dollars a year before it ships anything.

None of this means you should not prototype. The weekend demo is the right way to learn the shape of the problem. It simply is not the same as owning a production pipeline, which is a product in its own right, with its own team and its own roadmap. The honest question is not "can we build a demo?" It is "do we want to spend the next two years maintaining one?"