Tindahan Copilot: Snap a receipt or a shelf, get Taglish business advice, and build a credit history

Problem Statement

MSMEs make up about 99.5% of Philippine businesses, according to DTI, and sari-sari stores, estimated at over a million nationwide, are the retail backbone of barangays. Most owners, often women and senior citizens, keep records in a notebook or in their heads. They run out of fast-moving items, overstock slow ones, lose track of “utang” (customer credit), and can’t prove income. Without financial records they can’t access formal credit, so many turn to “5-6” informal lenders charging very high interest. Typhoons and price spikes wipe out thin margins, and owners have no data to plan restocking or pricing.

Proposed Solution

Tindahan Copilot is a lightweight Android or web app with a chat interface in Taglish and local languages. Capture: the owner photographs supplier receipts or delivery slips, and vision AI extracts items, quantities, and prices automatically. Periodic shelf photos estimate stock levels by recognizing common FMCG products. Sales can be logged by voice (“Nakabenta ako ng 5 Lucky Me at 2 Coke”). Insights: the app forecasts demand per SKU using sales history, paydays, school calendars, fiestas, and PAGASA typhoon advisories, then suggests what to restock and how much. It flags slow movers and margin leaks and tracks utang with gentle reminders. Coach: weekly Taglish voice and text tips, such as “Mag-stock ng extra bigas at de-lata, may bagyo sa Huwebes” (stock extra rice and canned goods, a typhoon is coming Thursday). Credit path: with the owner’s consent, consistent records generate a transparent “Tindahan Score” and a shareable report for microfinance institutions, banks, and digital lenders.

Target Market

• Primary users: sari-sari store owners and micro-retailers (estimated at over a million stores), plus carinderias and small market vendors.
• B2B buyers: FMCG distributors and brands (demand signals, route planning), microfinance institutions, rural and digital banks, and e-wallets.
• B2G buyers: DTI (Negosyo Centers), LGU livelihood offices, and BSP financial inclusion programs.

How AI Is Used

• Vision and OCR plus LLM receipt parsing: turns messy handwritten or printed receipts into structured inventory. Manual entry is exactly why owners abandon bookkeeping apps.
• Object detection on shelf photos: estimates stock of common SKUs without barcode scanning.
• Speech and LLM understanding (Taglish): voice sales logging and conversational Q&A (“Ano pinakamabenta ngayong buwan?” / What sold best this month?).
• Time-series forecasting (e.g., LightGBM or Prophet with exogenous features): predicts SKU demand around paydays, holidays, and typhoons. Gut feel fails during disruptions.
• Alternative credit scoring (gradient boosting with explainability): generates an interpretable score from cash flow regularity, inventory turnover, and utang repayment, opening credit to people without formal records.

PLDT / Smart / ePLDT / Rokid Angle

• Smart and PLDT Enterprise: MSME connectivity bundles, Smart prepaid promos for store owners, and SMS restock alerts. Sari-sari stores are already major prepaid load retailers.
• ePLDT cloud: hosts the vision models, forecasting, and sensitive financial data in PH data centers, and powers a data and insight platform for FMCG and financial clients.
• Ecosystem fit: connects naturally with PLDT’s digital finance partners for loan disbursement and e-wallet payments.
• Rokid (stretch): AR glasses for distributor field agents to scan shelves hands-free and auto-generate restock orders.

Demo Scenario (3 minutes for judges)

1. “Aling Nena,” 58, runs a sari-sari store in Tacloban. She shows her worn utang notebook.
2. Live, she photographs a crumpled supplier receipt, and 12 items appear in inventory in seconds.
3. She snaps a shelf photo, and the app detects low stock of instant noodles and canned sardines.
4. A PAGASA advisory shows a tropical depression approaching. The forecast advises: “Dagdagan ng 2 kahon ang sardinas at 1 sako ng bigas bago Huwebes” (add 2 boxes of sardines and 1 sack of rice before Thursday).
5. She asks by voice, “Sino pa ang may utang?” (Who still owes me?) and gets a list with polite SMS reminder drafts.
6. Fast-forward 6 months of synthetic data: her Tindahan Score is 720, with an explanation of the factors. A microfinance partner offers a ₱20,000 working-capital loan at fair rates instead of 5-6.
7. Close: “Ang tindahan mo, may data na. May kinabukasan na.” (Your store has data now. It has a future.)

30-Hour Build Plan

Suggested stack: Flutter or React Native (or a PWA), FastAPI, Gemini or GPT-4o-class vision LLM for receipts and shelves (YOLOv8 as fallback), Whisper for Taglish voice, LightGBM or Prophet, SHAP, PostgreSQL, Semaphore or Twilio SMS. Public data / APIs: PAGASA weather and tropical cyclone bulletins, PSA OpenSTAT CPI and retail price data, the DA Bantay Presyo price monitoring reports, the DTI suggested retail price bulletins, the national holiday calendar, Hugging Face receipt datasets (e.g., CORD and SROIE) and retail product detection datasets (e.g., SKU-110K).

Risks & Privacy Notes

• OCR errors on handwritten receipts: show a confirm-and-edit screen and learn from corrections. Shelf detection is limited to common SKUs at first.
• Forecast errors from sparse data: start with category priors from synthetic or pooled data and show confidence ranges.
• Credit scoring fairness and regulation: use an explainable model, keep it opt-in, give owners the right to see and contest their score, and partner with BSP-regulated lenders. Never share data without consent.
• Data Privacy Act 2012: financial data and the names of customers who owe money are sensitive. Store customer names locally or hashed, encrypt data, host in PH, and practice data minimization and erasure on request.
• Clause 24 compliance: the demo uses synthetic stores, customers, and sales. Public receipt datasets are used under their licenses. AI components are disclosed.

AI Use Disclosure (draft)

We use a vision LLM (Gemini or GPT-4o-class) and/or YOLOv8 for receipt and shelf understanding, Whisper for Taglish voice input, LightGBM or Prophet for demand forecasting, and gradient boosting with SHAP for the explainable credit score. An LLM helped generate synthetic sales data and draft this proposal, and coding assistants helped with boilerplate. Teammates validated model outputs. No real customer or financial data was used.

Rubric Scoring