TawagTulay: Zero-data voice agent that executes public service transactions on any keypad phone

Problem Statement

Digital government portals and AI apps in the Philippines assume every citizen owns a smartphone, pays for mobile data, and can navigate complex web forms in English. In reality, millions of smallholder farmers, municipal fisherfolk, senior citizens, and low-income workers rely on basic 2G keypad phones or have zero data balance when they need government help most. Even SMS chatbots require typing long structured codes and paying per text message. As a result, a farmer whose rice field was flattened by a typhoon must spend 200 to 500 pesos on tricycle and jeepney fare and lose a full day of work lining up at a municipal office just to file a Philippine Crop Insurance Corporation notice of loss or book a PhilHealth Konsulta primary care appointment.

Proposed Solution

TawagTulay eliminates the smartphone, mobile data, and literacy barrier by delivering an agentic AI system over an ordinary cellular voice call. A citizen dials a toll-free Smart shortcode or triggers a missed-call callback from any basic keypad phone.

First, TawagTulay verifies the caller’s registered SIM number in the background using telco-level Silent Network Authentication aligned with GSMA Open Gateway standards, removing the need for vulnerable SMS one-time passwords or memorized logins. Second, a low-latency conversational voice agent greets the caller in Tagalog, Cebuano, or Ilocano. Unlike legacy interactive voice response trees (“Press 1 for…”), TawagTulay uses deterministic LLM tool calling to execute real government workflows during the call: checking PhilHealth Konsulta clinic availability, booking an appointment slot, logging a PCIC crop-damage claim with barangay coordinates, or checking DSWD beneficiary status. Before committing any transaction, the agent reads back the details for verbal confirmation and sends a reference number via Smart SMS the moment the call ends.

Target Market

• Primary users: Over 25 million Filipinos using basic feature phones, prepaid voice-only SIMs, or living in low-data rural barangays, particularly farmers, fisherfolk, senior citizens, and 4Ps households.
• B2G buyers: PhilHealth (Konsulta / YAKAP primary care registration), Department of Agriculture and PCIC (crop insurance claims), DSWD, SSS/GSIS, and 1,600+ municipal local government units.
• B2B buyers: Rural banks, micro-insurance providers, and agricultural cooperatives serving unbanked members.

How AI Is Used

• Dialect-aware speech recognition and synthesis: Transcribes conversational Tagalog, Cebuano, Ilocano, and code-switched Taglish over narrowband 8 kHz telephony audio and responds with natural, unhurried voice synthesis.
• Constrained agentic tool calling: Maps natural spoken intent (“Nabaha po ang kalahating ektarya ng palay ko sa Barangay San Isidro kahapon”) into strict JSON schema calls against agency APIs while prompting only for missing required fields.
• Telco-verified identity binding: Combines SIM network verification with conversational slot verification (birth year and barangay) so the AI agent can safely perform state-changing transactions on the caller’s behalf.
• Automated SMS receipt and LGU queue sync: Summarizes the completed transaction into a 160-character SMS ticket that the citizen presents at the rural health unit or municipal agriculture office.

PLDT / Smart / ePLDT / Rokid Angle

• Smart Communications: Monetizes Smart’s core cellular voice network, toll-free enterprise shortcodes, A2P SMS receipts, and GSMA Open Gateway Number Verification APIs across 100M+ SIMs.
• PLDT Enterprise: Provides a turnkey citizen-access fulfillment layer that PLDT Enterprise can sell to national agencies (PhilHealth, DA, DSWD) and provincial capitals under the E-Governance Act (RA 12254).
• ePLDT: Hosts the SIP voice gateway, speech models, and agency integration connectors on sovereign VITRO data centers with full audit logging.
• De La Salle University: Builds on DLSU Center for Language Technologies and human-computer interaction work on low-resource Philippine languages and inclusive public systems.
• Rokid Philippines: Municipal front-desk clerks wearing Rokid AR glasses see the incoming caller’s pre-filled voice transaction ticket and SMS reference code when the citizen arrives for their scheduled visit.

Demo Scenario (3 minutes for judges)

1. Hold up a ₱600 Nokia keypad phone with mobile data turned off. Dial the TawagTulay number live on stage with speakerphone enabled.
2. The dashboard on screen shows the incoming call authenticated via SIM Number Verification in under 1 second without asking for an OTP.
3. The voice agent answers in natural Tagalog: “Magandang araw, Mang Ramon. Nakita kong rehistrado ang numero mo sa Barangay San Roque. Ano pong maipaglilingkod ko sa PhilHealth o sa agrikultura ngayong araw?”
4. The caller speaks in Tagalog: “Ipapa-check up ko sana yung ubo ng nanay ko bukas sa health center, at irereport ko rin yung nasirang palay namin nung bagyo.”
5. Live on the big screen, judges watch the agent execute two structured tool calls: book_rhu_konsulta_slot() and file_pcic_notice_of_loss(). The agent confirms the 9:00 AM clinic slot and claim ticket verbally, hangs up, and the keypad phone immediately beeps with the official SMS reference receipt.

30-Hour Build Plan

Suggested stack: Python FastAPI, Asterisk / Twilio Programmable Voice / LiveKit SIP, OpenAI Realtime or Gemini Multimodal Live + Whisper-large-v3-turbo, PostgreSQL, Next.js operator dashboard, SMS gateway. Public data / APIs: PhilHealth Konsulta public package guidelines, Philippine Crop Insurance Corporation (PCIC) notice-of-loss form fields, PSA Philippine Standard Geographic Code (PSGC) barangay directory, GSMA Open Gateway Number Verification specification.

Research & Literature References

1. Neil Patel, Deepti Chittamuru, Anupam Jain, Paresh Dave, and Tapan S. Parikh, “Avaaj Otalo: A Field Study of an Interactive Voice Forum for Small Farmers in Rural India,” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 733-742, 2010. DOI: 10.1145/1753326.1753434
   • How TawagTulay uses this study: Applies the empirical finding that rural farmers overwhelmingly prefer synchronous, natural-language voice calls over SMS or complex touchtone menus, while solving the human-moderation bottleneck documented in Avaaj Otalo by using function-calling speech agents.
2. Asaf Tzachor, Medha Devare, Catherine Richards, Pieter Pypers, Aniruddha Ghosh, Jawoo Koo, S. Johal, and Brian King, “Large language models and agricultural extension services,” Nature Food, Vol. 4, pp. 941-948, 2023. DOI: 10.1038/s43016-023-00867-x
   • How TawagTulay uses this study: Implements the study’s core design requirements for deploying LLMs to smallholder farmers: voice-first delivery in local dialects, strict grounding in verified government schemas to prevent hallucinated advice, and human-in-the-loop auditability.

Risks & Privacy Notes

• Zero unconfirmed state changes: The voice agent cannot submit a final claim or cancel an appointment until the caller explicitly says “Oo, tama” after hearing the read-back summary, followed by an immediate SMS receipt.
• Data Privacy Act of 2012: Voice recordings are transcribed in memory and redacted of sensitive health details before storage. Full audio logs are retained only with explicit verbal consent at the start of the call and auto-purged after 30 days.
• SIM swap and shared family phones: In rural households where one keypad phone is shared, the agent verifies the specific beneficiary’s birth year and barangay alongside the SIM number before accessing personal records.
• Clause 24 compliance: Hackathon demos use synthetic citizen profiles and sandbox agency databases with zero real personal data.
• Strict 30-hour feasibility constraint: Live GSMA Open Gateway Number Verification requires carrier-level provisioning unavailable to public hackathon teams, so the 30-hour build must mock the network token handshake over a Twilio PSTN trunk.

AI Use Disclosure (draft)

TawagTulay uses speech-to-text and text-to-speech models for narrowband Tagalog and Cebuano telephony audio, and a function-calling LLM to extract structured form fields and trigger mock agency APIs. AI coding assistants helped build the SIP WebSocket audio bridge and dashboard UI. All transaction schemas and verbal confirmation guardrails were written and verified by the team.

Rubric Scoring