Recipe 1.7 Architecture and Implementation: Prescription Label OCR ๐ถ
Companion to Recipe 1.7: Prescription Label OCR ๐ถ. This page covers the AWS architecture, services, prerequisites, and pseudocode. For the problem framing and the conceptual approach, start with the main recipe.
The AWS Implementation
Why These Services
Amazon Textract for OCR and key-value extraction. The same justification as Recipe 1.1 applies here: the AnalyzeDocument API with the FORMS feature type understands the 2D spatial structure of the label and returns matched key-value pairs rather than a flat string of characters. For a label that prints "SIG: Take 1 tab PO BID," FORMS mode will pair the key "SIG" with the value "Take 1 tab PO BID" as a matched unit. That spatial relationship detection is exactly what makes the downstream normalization tractable. Basic OCR gives you characters; FORMS gives you structure.
Amazon Comprehend Medical for medication entity extraction and RxNorm linking. Comprehend Medical's DetectEntitiesV2 API is trained specifically on clinical and pharmaceutical text. When you pass it a string like "Lisinopril 10mg oral tablet," it identifies the MEDICATION entity, pulls out its attributes (dosage: "10mg", route: "oral"), and returns the RxNorm concept IDs that correspond to the detected entity. This is the linkage between raw OCR text and the clinical ontology layer that interoperability requires. A general-purpose NLP model would not reliably handle medication entity extraction or RxNorm mapping; Comprehend Medical is the purpose-built tool for this.
Amazon S3 for image storage. Prescription label images contain PHI: patient name, date of birth (sometimes), medication, prescriber, pharmacy. They need encrypted at-rest storage with an audit trail. S3 with SSE-KMS and CloudTrail logging is the standard answer. S3 event notifications provide a clean trigger to kick off extraction without polling.
AWS Lambda for orchestration. The extraction pipeline is a short-lived sequence of API calls: fetch the image from S3, call Textract, parse the response, normalize fields, decode SIG, call Comprehend Medical, assemble the structured record, write to DynamoDB. Lambda fits this workload exactly: stateless, event-driven, scales with request volume, and you pay only for execution time. For member-facing synchronous use (upload image, get structured record back immediately), put API Gateway in front.
API Security. The API Gateway endpoint accepting label uploads must require authentication (Cognito User Pools or IAM SigV4). Add a usage plan with rate limits and configure WAF rules to block oversized requests and malformed content types. A public API accepting prescription label images without authentication is a PHI ingest endpoint that could be abused for data exfiltration or denial-of-service attacks.
Amazon DynamoDB for medication record storage. The access patterns for medication records are point lookups: find all records for this member, find this specific Rx number, find all records with a given NDC. DynamoDB's key-value model handles these well. It's fully managed, encrypts at rest by default, and is on the AWS HIPAA eligible services list.
Architecture Diagram
flowchart LR
A[๐ฑ Member App] -->|Label Photo| B[S3 Bucket\nrx-labels/]
B -->|S3 Event| C[Lambda\nrx-label-extractor]
C -->|AnalyzeDocument\nFORMS| D[Amazon Textract]
D -->|Key-Value Pairs| C
C -->|Medication Text| E[Comprehend Medical\nDetectEntitiesV2]
E -->|Entities + RxNorm| C
C -->|Structured Rx Record| F[DynamoDB\nmedication-records]
C -->|Structured JSON| G[API Response\nto Caller]
style B fill:#f9f,stroke:#333
style D fill:#ff9,stroke:#333
style E fill:#f96,stroke:#333
style F fill:#9ff,stroke:#333
Deployment topology note: The diagram above shows the asynchronous model: member app uploads to S3, S3 event triggers Lambda, result lands in DynamoDB. For member-facing synchronous use (upload image, get structured record back in the HTTP response), the member app POSTs the image to API Gateway, which invokes Lambda directly. Lambda calls Textract and Comprehend Medical, assembles the record, writes to DynamoDB, and returns the structured JSON in the response. Latency: 2-5 seconds. S3 storage still happens inside Lambda for the audit trail, but S3 events are not the trigger. For asynchronous bulk/background processing, member app uploads directly to S3 via presigned URL, S3 event triggers Lambda, and the member app polls a status endpoint or receives a push notification when the record is ready.
Prerequisites
| Requirement | Details |
|---|---|
| AWS Services | Amazon Textract, Amazon Comprehend Medical, Amazon S3, AWS Lambda, Amazon DynamoDB |
| IAM Permissions | textract:AnalyzeDocument, comprehendmedical:DetectEntitiesV2, s3:GetObject, s3:PutObject, dynamodb:PutItem, dynamodb:GetItem |
| BAA | AWS BAA signed (required: prescription labels contain PHI including patient name, medication, and prescriber) |
| Encryption | S3: SSE-KMS; DynamoDB: encryption at rest enabled (default); CloudWatch Log Groups: KMS encryption via kmsKeyId parameter (required because Lambda log output contains PHI); all API calls over TLS |
| Log Sanitization | Lambda log output is PHI in this pipeline. All CloudWatch log groups for this function must use KMS encryption. In production, structured logging must redact or omit extracted field values. Log only non-PHI signals: image key suffix (not full path), confidence score ranges, boolean flags, latency, error codes. Never log medication names, dosages, prescriber names, or patient identifiers at any log level. |
| EXIF Stripping | Prescription label photos from smartphones contain EXIF metadata including GPS coordinates (often the member's home address), which qualifies as PHI. Strip EXIF data before storing images in S3. Preferred: client-side stripping before upload. Defense-in-depth fallback: Lambda-side stripping (e.g., Pillow image.save() without EXIF copy) before passing the image to Textract. |
| API Gateway | If exposing via API Gateway: minimum TLS policy set to TLS_1_2 on the custom domain; HTTP endpoint disabled; all traffic encrypted in transit. Authentication required (Cognito User Pools or IAM SigV4). WAF rules to block oversized requests and malformed content types. |
| Upload Path | Recommended pattern for mobile PHI upload: presigned S3 URL with short expiry (5-15 minutes), scoped to a single object key, generated by an authenticated API endpoint that requires a valid member authentication token. The presigned URL generation endpoint must not be publicly accessible. |
| VPC | Production: Lambda in VPC with VPC endpoints for S3, Textract, Comprehend Medical, DynamoDB, and CloudWatch Logs. Without the CloudWatch Logs endpoint, Lambda cannot write audit logs from a private subnet. |
| Retry Configuration | All AWS API calls in this pipeline should use the SDK's built-in retry configuration with exponential backoff. For boto3: configure botocore.config.Config(retries={'max_attempts': 3, 'mode': 'adaptive'}) on each client. |
| S3 Lifecycle | Configure S3 lifecycle rules: transition raw label images to S3 Glacier Instant Retrieval after 90 days; delete after your organization's HIPAA retention period (typically 6 years from date of service). The DynamoDB record with confidence scores and raw extracted values suffices for most audit and appeal scenarios. |
| CloudTrail | Enabled: log all Textract, Comprehend Medical, and S3 API calls for HIPAA audit trail |
| Sample Data | Synthetic prescription labels. Create samples across major pharmacy chains (CVS, Walgreens, Rite Aid, independent) with varied fonts and layouts. Never use real member labels in development. The FDA NDC Database provides real NDC codes for use in synthetic test data. |
| Cost Estimate | Textract AnalyzeDocument (FORMS): $0.05/page. Comprehend Medical DetectEntitiesV2: $0.01 per 100 characters (full label text runs 300-500 characters: ~$0.03-$0.05). Total: ~$0.08-$0.10 per label. Lambda and DynamoDB costs are negligible at this scale. Note: VPC Interface endpoints for Textract, Comprehend Medical, and CloudWatch Logs add ~$44/month fixed overhead in a 2-AZ deployment. This cost is volume-independent. Below ~10,000 labels/month, endpoint overhead is material; above 100,000 labels/month, it is negligible. |
Ingredients
| AWS Service | Role |
|---|---|
| Amazon Textract | Extracts key-value pairs from the label image using FORMS mode |
| Amazon Comprehend Medical | Identifies MEDICATION entities and returns RxNorm concept IDs via DetectEntitiesV2. Comprehend Medical is available in a subset of AWS regions. Verify your target region supports it before selecting your deployment region. |
| Amazon S3 | Stores incoming label images; encrypted at rest with KMS |
| AWS Lambda | Orchestrates the full pipeline: Textract extraction, field normalization, SIG parsing, RxNorm mapping |
| Amazon DynamoDB | Stores structured medication records for downstream lookup |
| AWS KMS | Manages encryption keys for S3 and DynamoDB |
| Amazon CloudWatch | Logs, metrics, and alarms for extraction failures and latency |
Code
Reference implementations: The following AWS sample repos demonstrate the patterns used in this recipe:
amazon-textract-code-samples: General Textract code samples including FORMS extraction and key-value pair parsingamazon-textract-textractor: Python SDK wrapper that simplifies calling and parsing Textract responses (installable via pip asamazon-textract-textractor)amazon-textract-and-amazon-comprehend-medical-claims-example: Healthcare-specific example combining Textract and Comprehend Medical for structured data extraction from medical documents
Walkthrough
Step 1: Textract extraction. When a label image arrives in the S3 bucket, the pipeline wakes up automatically and sends it to Amazon Textract for analysis. As in Recipe 1.1, the critical choice is requesting FORMS extraction rather than basic OCR. FORMS mode understands spatial relationships: it recognizes that "SIG" and "Take 1 tab PO BID" appear adjacent to each other on the label, and it returns them as a matched key-value pair rather than two disconnected strings. Prescription labels are single-page and synchronous processing is appropriate: results come back in under 3 seconds. Skip FORMS and request plain text detection, and everything downstream falls apart: you're trying to reconstruct structure from a flat string, and accuracy drops significantly.
FUNCTION extract_label(bucket, key):
// Send the label image to Textract for intelligent analysis.
// "bucket" is the name of the S3 storage container; "key" is the filename/path.
response = call Textract.AnalyzeDocument with:
document = S3 object at bucket/key // locate the image in cloud storage
features = ["FORMS"] // FORMS mode: return matched label-value pairs,
// not just a flat string of characters
RETURN response
Step 2: Parse key-value pairs. Textract returns a collection of text blocks connected by relationship links. This step walks that structure and assembles the matched key-value pairs, along with confidence scores indicating how clearly each piece of text was read. The output is a map from raw label text (whatever the pharmacy printed, e.g., "SIG", "Rx #", "Dispense Date") to extracted value text, with a confidence score for each pair. Think of it as sorting through labeled index cards and connecting each label to its matching answer. Skip this step and you're left with raw building blocks; no downstream logic can use them.
FUNCTION parse_key_value_pairs(textract_response):
// Pull out all detected text regions from Textract's response.
blocks = textract_response.Blocks
// Build a lookup index: block ID -> block data.
// Textract connects labels to values by referencing block IDs.
block_map = build map of block.Id -> block for all blocks
// This holds our results: label text -> { value text, confidence score }.
key_values = empty map
FOR each block in blocks:
// Only process KEY_VALUE_SET blocks that are the KEY side of a pair.
// Textract marks each pair half as KEY (the label) or VALUE (the answer).
IF block.BlockType == "KEY_VALUE_SET" AND block is a KEY entity:
// Assemble the label text (e.g., "SIG", "Rx Number", "Dispense Date")
key_text = get concatenated text from block's CHILD blocks in block_map
// Follow the link to the paired VALUE block
value_block = follow block's VALUE relationship to find the linked value block
// Assemble the value text (e.g., "Take 1 tab PO BID", "7284910", "02/28/2026")
value_text = get concatenated text from value_block's CHILD blocks in block_map
// Record the lower of the two confidence scores.
// If either the key or the value was hard to read, flag both.
confidence = minimum of (block.Confidence, value_block.Confidence)
key_values[key_text] = { value: value_text, confidence: confidence }
RETURN key_values
Step 3: Normalize pharmacy fields. Every pharmacy chain prints prescription labels differently. "Drug Name," "Medication," and "Rx" all mean the same field. "SIG" and "Directions" and "Instructions" all point to the patient instruction line. "Refills" and "Refills Remaining" and "Refills Left" all contain the same count. This step maps whatever labels Textract found on a given label to a consistent set of canonical field names. The mapping table (RX_FIELD_MAP) is the operational knowledge base of this recipe: it encodes real-world pharmacy label layouts and requires maintenance as new layouts are encountered. The NDC field is especially important to capture correctly: it is printed on most labels in 10-digit format and is the most reliable structured identifier for the specific drug dispensed. Skip this step and you have accurate text with no reliable way to use it across chains.
RX_FIELD_MAP = {
"drug_name": ["drug name", "medication", "medication name", "drug", "product", "item", "drug/product"],
"dosage": ["strength", "dosage", "dose", "potency"],
"quantity": ["qty", "quantity", "qty dispensed", "disp qty", "#"],
"directions": ["sig", "directions", "instructions", "take", "use", "dir"],
"prescriber": ["prescriber", "doctor", "physician", "prescribed by", "dr.", "provider"],
"pharmacy": ["pharmacy", "store", "dispensed by", "location"],
"rx_number": ["rx #", "rx number", "prescription #", "rx no", "prescription number", "rx", "rx num"],
"refills": ["refills", "refills remaining", "refills left", "rfl", "ref"],
"days_supply": ["days supply", "day supply", "days", "supply"],
"date_filled": ["date filled", "fill date", "dispensed", "disp date", "date"],
"ndc": ["ndc", "ndc #", "national drug code", "ndc code"],
"lot_number": ["lot", "lot #", "lot number"]
}
FUNCTION normalize_rx_fields(raw_kv):
normalized = empty map
FOR each canonical_name, variants in RX_FIELD_MAP:
FOR each raw_key, raw_val in raw_kv:
// Compare case-insensitively and strip whitespace
IF lowercase(trim(raw_key)) is in variants:
normalized[canonical_name] = {
value: trim(raw_val.value),
confidence: raw_val.confidence
}
BREAK // found a match for this canonical field
RETURN normalized
Step 4: Decode SIG abbreviations. The directions field from a prescription label reads like "Take 1 TAB PO BID x 14d PRN pain." A human pharmacist reads this instantly. A downstream care management system or FHIR document cannot. This step decodes the pharmacy abbreviation shorthand in the directions field into plain language. It works as a word-level lookup: split the directions string on spaces, check each word against the SIG codebook, substitute the decoded meaning if found, and reassemble the string. The output is human-readable and machine-processable text that downstream systems can display to members and parse for structured frequency and route information. Skip this step and your medication records carry an abbreviation string that every consumer of the data has to decode independently, inconsistently.
SIG_CODES = {
// Frequency codes
"qd": "once daily",
"qdaily": "once daily",
"bid": "twice daily",
"tid": "three times daily",
"qid": "four times daily",
"qhs": "at bedtime",
"prn": "as needed",
"stat": "immediately",
"q4h": "every 4 hours",
"q6h": "every 6 hours",
"q8h": "every 8 hours",
"q12h": "every 12 hours",
"ud": "as directed",
// Route codes
"po": "by mouth",
"sl": "under the tongue",
"pr": "rectally",
"top": "topically",
"inh": "inhaled",
"inj": "by injection",
// Timing codes
"ac": "before meals",
"pc": "after meals",
"hs": "at bedtime",
// Dose form codes
"tab": "tablet",
"tabs": "tablets",
"cap": "capsule",
"caps": "capsules",
"ml": "milliliter",
"gtt": "drop",
"gtts": "drops",
"supp": "suppository",
"soln": "solution",
"susp": "suspension"
}
FUNCTION decode_sig(raw_sig):
// Split the directions string on whitespace
words = split raw_sig on whitespace
// For each word, strip punctuation then check the codebook (case-insensitive)
decoded = []
FOR each word in words:
clean = strip leading and trailing punctuation from word
lookup = lowercase(clean)
IF lookup is in SIG_CODES:
append SIG_CODES[lookup] to decoded
ELSE:
// Pass through any word that isn't a recognized abbreviation
// (numbers, drug names, durations like "14d", custom text)
append word to decoded
RETURN join decoded with single space
Step 5: Map to RxNorm via Comprehend Medical. This step takes the full normalized label text and passes it through Comprehend Medical's DetectEntitiesV2 API. DetectEntitiesV2 is trained on clinical text to identify MEDICATION entities and their attributes (dosage, route, frequency), and it returns RxNorm concept IDs that correspond to each detected medication. Passing the full label text (not just drug name and dosage) gives the model surrounding clinical context (route, frequency, indication) that helps disambiguate similar drug names and confirm the correct dosage form. The RxNorm concept ID is the clinical-equivalence identifier: it's the same for Lisinopril 10mg oral tablet regardless of manufacturer, package size, or dispensing pharmacy. This is what downstream systems need for medication reconciliation, drug interaction checking, and formulary matching. A confidence threshold filters out low-confidence mappings.
RXNORM_CONFIDENCE_THRESHOLD = 0.70 // discard low-confidence RxNorm mappings
FUNCTION map_to_rxnorm(normalized_fields):
// Assemble the full normalized label text for maximum entity context.
// More surrounding context improves Comprehend Medical's accuracy
// for medication entity detection and RxNorm concept selection.
medication_text = concatenate all values from normalized_fields
separated by spaces
// e.g., "Amoxicillin 500mg Take 1 CAP PO TID x 7d Dr. Sarah Chen 0 00093-4155-21"
// Call Comprehend Medical to detect medication entities and RxNorm concepts.
// DetectEntitiesV2 handles clinical vocabulary and understands medication entity structure.
response = call ComprehendMedical.DetectEntitiesV2 with:
text = medication_text
rxnorm_mappings = []
FOR each entity in response.Entities:
// Only process MEDICATION category entities.
// DetectEntitiesV2 also detects MEDICAL_CONDITION, TEST_TREATMENT_PROCEDURE, etc.
// We want only the medication entities for this step.
IF entity.Category == "MEDICATION":
// Each entity can carry one or more RxNorm concept candidates, ranked by confidence.
FOR each concept in entity.RxNormConcepts:
IF concept.Score >= RXNORM_CONFIDENCE_THRESHOLD:
// Record the mapping: text as detected, RxNorm ID, concept type, description
append to rxnorm_mappings:
{
detected_text: entity.Text, // what Comprehend Medical read
rxnorm_id: concept.Code, // standard RxNorm concept ID
description: concept.Description, // e.g., "lisinopril 10 MG Oral Tablet"
concept_type: concept.Type, // TTY: "SCD", "IN", "SBD", etc.
confidence: round(concept.Score, 3)
}
// Return the list of matched concepts, sorted by confidence descending.
// The first entry is the highest-confidence RxNorm match.
RETURN sort rxnorm_mappings by confidence descending
Step 6: Validate NDC and compute refill metrics. Before writing the final record, two quick validation steps add significant downstream value. First, validate the extracted NDC code: NDC codes have a well-defined format (either 10-digit or 11-digit with hyphens) and can be verified against a known pattern. A malformed NDC indicates either an extraction error or a label format you haven't seen before. Flag it rather than silently passing a bad identifier downstream. Second, compute refill metrics from the raw label fields. "Refills: 3" tells you the remaining count. The days supply field tells you how long one fill lasts. Together, they give you the days of medication coverage remaining, which is directly useful for medication adherence programs and care gap identification. These calculations are simple arithmetic, but doing them here centralizes the logic so every consumer of the medication record gets the same computed values.
FUNCTION validate_ndc(ndc_raw):
// Remove hyphens and whitespace for validation
ndc_clean = remove all hyphens and spaces from ndc_raw
// Standard NDC is 10 digits. Some systems use an 11-digit representation.
// Validate by checking that the cleaned value is 10 or 11 numeric digits.
IF ndc_clean matches pattern "^[0-9]{10,11}$":
RETURN { valid: true, ndc_normalized: ndc_clean }
ELSE:
RETURN { valid: false, ndc_raw: ndc_raw, error: "NDC format not recognized" }
FUNCTION compute_refill_metrics(refills_remaining_str, days_supply_str):
// Parse the raw string values from the label (e.g., "3", "30")
refills_remaining = parse integer from refills_remaining_str
days_supply = parse integer from days_supply_str
// Total days of medication coverage if all refills are filled:
// current fill + remaining refills, each covering days_supply
total_days_remaining = (1 + refills_remaining) * days_supply
RETURN {
refills_remaining: refills_remaining,
days_supply: days_supply,
total_days_remaining: total_days_remaining
// total_days_remaining drives downstream adherence gap detection
}
Step 7: Assemble and store the medication record. The final step assembles all pipeline outputs into a single record and writes it to the database. Every field carries both the raw extracted value (what the label actually said) and the normalized or decoded value (what it means). This dual representation is important for auditability: when a care coordinator or pharmacist reviews a record, they can see both what the label printed and how the system interpreted it. Any field that fell below the confidence threshold, any NDC that failed validation, and any failed RxNorm mappings are recorded in a flags array so downstream systems and review queues know exactly what needs a human eye.
CONFIDENCE_THRESHOLD = 90.0 // same threshold as Recipe 1.1; fields below this go to human review
FUNCTION store_medication_record(image_key, normalized_fields, rxnorm_mappings, ndc_validation, refill_metrics):
// Separate high-confidence fields from those needing review
clean_fields = { field: data.value for field, data in normalized_fields
where data.confidence >= CONFIDENCE_THRESHOLD }
flagged_fields = [ { field: field, extracted_value: data.value, confidence: data.confidence }
for field, data in normalized_fields
where data.confidence < CONFIDENCE_THRESHOLD ]
// Add NDC validation flags if needed
IF ndc_validation.valid == false:
append to flagged_fields: { field: "ndc", issue: ndc_validation.error }
write record to database table "medication-records":
image_key = image_key
extraction_timestamp = current UTC timestamp (ISO 8601)
fields = clean_fields
directions_decoded = decoded SIG text from Step 4
ndc_validated = ndc_validation
rxnorm_mappings = rxnorm_mappings // list of matched RxNorm concepts
refill_metrics = refill_metrics // days coverage, refills remaining
flagged_fields = flagged_fields
needs_review = (length of flagged_fields > 0)
Curious how this looks in Python? The pseudocode above covers the concepts. If you'd like to see sample Python code that demonstrates these patterns using boto3, check out the Python Example. It walks through each step with inline comments and notes on what you'd need to change for a real deployment.
Expected Results
Sample output for a typical printed label:
{ "image_key": "rx-labels/2026/03/01/label-00182.jpg", "extraction_timestamp": "2026-03-01T14:22:08Z", "fields": { "drug_name": "Amoxicillin", "dosage": "500mg", "quantity": "21", "directions": "Take 1 CAP PO TID x 7d", "rx_number": "7284910", "prescriber": "Dr. Sarah Chen", "pharmacy": "CVS Pharmacy #4821", "date_filled": "02/28/2026", "ndc": "00093-4155-21" }, "directions_decoded": "Take 1 capsule by mouth three times daily x 7d", "ndc_validated": { "valid": true, "ndc_normalized": "00093415521" }, "rxnorm_mappings": [ { "detected_text": "Amoxicillin 500mg", "rxnorm_id": "723", "description": "Amoxicillin 500 MG Oral Capsule", "concept_type": "SCD", "confidence": 0.964 } ], "refill_metrics": { "refills_remaining": 0, "days_supply": 7, "total_days_remaining": 7 }, "flagged_fields": [], "needs_review": false }
Performance benchmarks:
| Metric | Typical Value |
|---|---|
| End-to-end latency | 2-4 seconds |
| Field extraction accuracy (flat, well-lit labels) | 93-98% |
| Field extraction accuracy (curved/worn labels) | 75-90% |
| SIG decoding accuracy | 95-99% (known abbreviations) |
| RxNorm mapping accuracy | 88-96% (clean OCR input; degrades 2-5 points on curved/worn labels) |
| NDC extraction accuracy | 95-99% (when present and printed clearly) |
| Cost per label | ~$0.08-$0.10 |
| Throughput | ~30 labels/second (Lambda concurrency limited) |
Where it struggles: Curved label photos where text near the bottle edges is distorted. Partially peeled or worn labels where key fields (often the NDC or Rx number) are damaged. Compounding pharmacy labels, which have non-standard formats and often hand-typed dosage instructions. Labels that mix brand name and generic name in different font sizes, which can confuse the drug name extraction. Medication names that are very similar (Hydroxyzine vs. Hydroxyurea: easy to confuse when OCR has low confidence on one or two characters). And labels photographed at steep angles despite your best UX guidance in the app.
Why This Isn't Production-Ready
The pseudocode and architecture above demonstrate the pattern. A real deployment needs additional work in a few specific areas.
EXIF metadata on mobile photos. Members take photos of their pill bottles with smartphones. Those photos contain EXIF metadata including GPS coordinates, which is effectively the member's home address. Strip EXIF data before storing the image in S3. A one-line Pillow call (image.getexif().clear()) or an S3 Lambda@Edge function handles this, but skipping it means you're storing location data alongside PHI with no business justification.
Log sanitization. The Lambda functions extract medication names, dosages, and prescriber information. Without explicit log sanitization, those values appear in CloudWatch Logs at INFO level. Log the field names ("extracted: drug_name, dosage, refills_remaining") but not the values. Encrypt CloudWatch Log Groups with a CMK and restrict access to authorized personnel.
SIG parsing coverage. The abbreviations in the walkthrough are the common ones, but a real codebook needs ~150+ entries including routes (IM, IV, SubQ), ophthalmic (OD/OS/OU), and compound frequencies (q4-6h). The parser also needs to handle punctuation attached to tokens ("BID.", "PRN/pain"). Build logging around unrecognized tokens: capture them, review regularly, and expand the codebook. This is ongoing maintenance, not one-time setup.
RxNorm concept selection. DetectEntitiesV2 returns multiple RxNorm candidate concepts, ranked by confidence. The walkthrough returns all concepts above the threshold, including the concept type (TTY: IN vs. SCD vs. SBD). In practice, you often want the highest-confidence single concept for downstream use, filtered by concept type. Decide whether you want the most specific concept (SCD or SBD, matching strength and dose form) or the ingredient-level concept (IN, generalizes across packages). That choice depends on your use case: formulary matching needs specificity (SCD/SBD); interaction checking works at ingredient level (IN). Filter the returned concept_type field based on your downstream requirements.
Dead Letter Queue. Same gap as Recipe 1.1: Lambda on S3 events is asynchronous, and failed events retry and disappear. In a medication management pipeline, a silently dropped label means a gap in the member's medication record with no visible signal. Configure an SQS dead letter queue and set a CloudWatch alarm on queue depth.
Idempotency. S3 delivers event notifications at least once. Without a conditional write in DynamoDB (check for existing record with the same image_key before writing), the same label photo can create duplicate records. Use a conditional expression on the DynamoDB write.
NDC validation goes further than format checking. A 10-digit string in the right format is a well-formed NDC. Whether it corresponds to a real drug product requires a lookup against the FDA NDC database. For medication reconciliation programs, consider validating extracted NDCs against a regularly-refreshed copy of the FDA NDC dataset.
Variations and Extensions
Drug interaction checking. After mapping to RxNorm, cross-reference the detected medication against the member's full active medication list and flag potential drug-drug interactions. The National Library of Medicine's RxNorm API includes drug interaction data via the Drug Interaction API endpoint (no separate subscription required). This turns the label scan into a real-time safety check: member scans a new prescription, system checks it against their medication history before they take their first dose.
Formulary matching and cost transparency. Take the extracted NDC and RxNorm concept and look them up against the member's plan formulary tier table. Return the copay, the tier, and (if applicable) a lower-cost therapeutically equivalent alternative. This is the use case that generates the most immediate member value: "here's what this medication will cost you, and here's a $4 alternative." Recipe 3.3 (Medication Reconciliation) builds this out as a full pipeline.
Multi-label medication list building. Accept a batch of label photos in sequence, run each through the pipeline, deduplicate (same RxNorm concept from different fill dates is one medication, not two), and produce a reconciled medication list in FHIR MedicationStatement format. This is the care transition use case: a patient with six medications holds up each bottle in turn, and the app builds a complete reconciled medication list that can be sent to the receiving care team. Recipe 3.3 covers this integration.
Additional Resources
AWS Documentation:
- Amazon Textract AnalyzeDocument API Reference
- Amazon Textract FORMS Feature Type
- Amazon Textract Pricing
- Amazon Comprehend Medical DetectEntitiesV2 API Reference
- Amazon Comprehend Medical RxNorm Ontology Linking
- Amazon Comprehend Medical Pricing
- AWS HIPAA Eligible Services Reference
- Architecting for HIPAA Security and Compliance on AWS
AWS Sample Repos:
amazon-textract-code-samples: General Textract code samples including FORMS extraction patternsamazon-textract-textractor: Python SDK wrapper for Textract that simplifies calling and parsing responses; installable via pipamazon-textract-and-amazon-comprehend-medical-claims-example: Healthcare-specific pipeline combining Textract extraction with Comprehend Medical NLP for structured data extraction from medical documents
External References:
- NLM RxNorm Technical Documentation: Official NLM documentation for RxNorm concept structure, relationships, and the NDC-to-RxNorm mapping database
- NLM RxNav Drug Interaction API: Free API for drug-drug interaction checking using RxNorm concept IDs; relevant for the drug interaction variation
- FDA NDC Database: Official FDA source for NDC code validation; downloadable in bulk for offline lookup
- NCPDP SCRIPT Standard: The industry standard for electronic prescribing, useful context for understanding what structured prescription data looks like downstream
- Pharmacy Abbreviations Reference (USP): United States Pharmacopeia maintains authoritative references on pharmaceutical abbreviations and nomenclature
Estimated Implementation Time
| Tier | Scope | Time |
|---|---|---|
| Basic | Single-format label extraction with Textract FORMS, basic SIG abbreviation lookup table, Comprehend Medical entity extraction with RxNorm linking, Lambda orchestration, DynamoDB storage, no NDC validation or multi-format support | 2-4 weeks |
| Production-ready | Multi-format label support (retail, mail-order, hospital), complete SIG normalization with dosing schedule parsing, NDC cross-validation against FDA database, confidence thresholds with pharmacist review queue, audit logging, error handling for partial extractions, API Gateway with authentication | 2-4 months |
| With variations | Drug interaction checking via RxNav API, multi-language label support, refill tracking with temporal logic, integration with e-prescribing systems (NCPDP SCRIPT), real-time formulary coverage lookup | 2-3 months beyond production-ready |
โ Main Recipe 1.7 ยท Python Example ยท Chapter Preface