ResetaTapat: Handwritten prescription decoder, FDA generic switcher, and blister-pack verifier

Problem Statement

Out-of-pocket medicine costs push hundreds of thousands of Filipino households into poverty every year. Although the Generics Act of 1988 (RA 6675) requires generic name prescribing and the DOH Maximum Drug Retail Price (MDRP) and PhilHealth GAMOT package subsidize essential drugs, ordinary patients struggle to read cursive medical shorthand ("b.i.d. p.c.", "q8h PRN"), compare prices between a ₱48 branded tablet and a ₱4 FDA-approved generic equivalent, or know which Rural Health Unit (RHU) has free stock. As a result, patients frequently buy only two or three tablets of a seven-day antibiotic course—fueling dangerous antimicrobial resistance—or accidentally take double doses when switching between branded and generic blister packs.

Proposed Solution

ResetaTapat is a pocket medication advocate for Filipino patients and barangay health workers. It works in three simple steps from two smartphone photos:

1. Handwritten Prescription Decoding & Generic Savings: The user photographs their paper reseta. A multimodal vision-language model constrained by the Philippine National Formulary (PNF) and FDA Philippines registry extracts the active generic ingredient, strength, dosage form, and Latin frequency shorthand, immediately showing the price difference between branded options, DOH Drug Price Reference Index (DPRI) ceilings, and nearby RHUs offering the medicine free under PhilHealth Konsulta/GAMOT.
2. Visual Pill & Blister-Pack Verification: After buying the medicine, the user snaps a photo of the blister pack or tablet. Using few-shot metric learning modeled on the ePillID architecture, ResetaTapat verifies the tablet’s imprint, shape, color, and foil batch details against the prescribed generic drug and checks for dangerous duplicate ingredients across the patient’s existing maintenance medicines.
3. Spoken Dialect Dosing Schedule: Generates a visual sun/noon/moon color-coded card and spoken Tagalog, Cebuano, or Ilocano audio instructions (“Inumin ang isang tableta ng Losartan 50mg tuwing alas-siyete ng umaga pagkatapos mag-almusal”).

Target Market

• Primary users: Filipino patients buying maintenance medicines (hypertension, diabetes, cholesterol) or pediatric antibiotics, senior citizens, caregivers, and Barangay Health Workers.
• Public-sector users: Municipal Rural Health Units, PhilHealth GAMOT partner pharmacies, DOH Botika ng Bayan outlets, and LGU senior citizen offices (OSCA).
• Community benefit: Cuts household medicine spending by 50 to 80 percent through verified generic switching while preventing dispensing errors and incomplete antibiotic courses.

How AI Is Used

• Medical handwriting OCR with formulary grounding: Uses a vision-language model constrained against a trie of FDA Philippines registered generic/brand names and standard dosage strengths so it never hallucinates a non-existent drug name; ambiguous handwriting triggers a confidence warning to ask the pharmacist.
• Low-shot fine-grained pill and blister identification: Adapts the ePillID multi-view metric learning network to match pill shape, color, score lines, and blister foil text even when new local generic manufacturers have only a few reference images.
• Drug-duplication and interaction checker: Flags when a patient is about to take two different brand names that contain the exact same active generic ingredient (for example, two different paracetamol or amlodipine brands).
• Tagalog and Bisaya medical translator: Converts Latin prescription abbreviations (sig: 1 tab PO BID x 7d) into plain, spoken everyday instructions.

Community & Ecosystem Integration

• Direct support for RA 6675 (Generics Act) and Universal Health Care (RA 11223): Empowers citizens at the pharmacy counter with official DOH Drug Price Reference Index and PhilHealth GAMOT coverage facts.
• Zero-hardware accessibility: Runs as a lightweight mobile web app and Messenger bot accessible on any budget smartphone.
• BHW home-visit mode: Lets Barangay Health Workers audit an elderly resident’s plastic bag of mixed maintenance pills in 30 seconds to spot expired or duplicate medications.

Demo Scenario (3 minutes for judges)

1. Place a real doctor’s handwritten prescription (for a branded antihypertensive and antibiotic) and three different blister packs on the judge table.
2. Snap the handwritten reseta. Within 4 seconds, ResetaTapat decodes all three drugs, translates "1 tab PO BID" into Tagalog, and shows that switching to FDA-certified generics drops the 30-day cost from ₱2,850 to ₱490—or ₱0 at the listed RHU 1.2 km away.
3. Snap two blister packs from the table: one correct generic equivalent and one look-alike tablet with a different active ingredient.
4. ResetaTapat approves the first pack in green and flashes a red Wrong Medicine / Duplicate Ingredient voice warning on the second pack.
5. Play the generated Tagalog voice reminder and print the visual sun/moon pill card for a senior citizen.

30-Hour Build Plan

Suggested stack: Python FastAPI, Gemini Multimodal API (with strict PNF JSON schema), PyTorch + timm (ePillID metric learning), PostgreSQL, Next.js PWA, Google/Edge TTS. Public data / APIs: DOH Drug Price Reference Index (DPRI), Philippine National Formulary (PNF) Essential Medicines List, FDA Philippines Verification Portal dataset, ePillID open benchmark dataset, OpenFDA interaction rules.

Research & Literature References

1. Naoto Usuyama, Natalia Larios Delgado, Amanda K. Hall, and Jessica Lundin, “ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 462-463, 2020. DOI: 10.1109/CVPRW50498.2020.00463
   • How ResetaTapat uses this study: Uses the ePillID low-shot metric-learning architecture (bilinear/embedding similarity across appearance, shape, color, and imprint text) so the verifier accurately identifies local generic tablets and blister packs using only a few reference images per manufacturer.

Risks & Privacy Notes

• Pharmacist and physician loop: ResetaTapat never alters a prescribed drug molecule or dosage. It only displays bioequivalent generic options sharing the exact same active ingredient, dosage form, and strength under RA 6675, and requires a licensed pharmacist to dispense.
• Abstain on illegible handwriting: If OCR confidence on drug name or milligram strength falls below 90 percent, the app refuses to guess and instructs the user to confirm directly with the prescribing doctor or pharmacist.
• Data Privacy Act of 2012: Prescription photos contain sensitive health information. Patient names are cropped or masked on-device before cloud inference, and no prescription images are retained after the session.

AI Use Disclosure (draft)

ResetaTapat uses a formulary-constrained vision-language model to transcribe handwritten prescriptions, a metric-learning vision model based on ePillID to verify medicine blister packs and tablets, and text-to-speech to read dosing schedules in Tagalog. AI coding assistants helped structure the PNF lookup API and frontend UI. All generic equivalence mappings strictly follow the Philippine National Formulary.

Rubric Scoring