MATA: Hands-free AR damage assessment that writes the RDANA report for you

Problem Statement

After a disaster, LGUs must run a Rapid Damage Assessment and Needs Analysis (RDANA) within the first hours to days. The results drive relief requests, calamity declarations, and NDRRMC situation reports. Today, RDANA teams walk through debris with clipboards, paper forms, and phones, then re-type everything at the municipal hall. Classifying damage as “partially” or “totally” damaged is subjective and differs between assessors. After major events such as the 2013 Bohol earthquake, Typhoon Odette (Rai) in 2021, and the 2025 Cebu and Davao Oriental earthquakes, delays in damage counts slowed relief and housing aid. Responders need both hands free, and the data has to be consistent and fast.

Proposed Solution

MATA is an AR field app for Rokid glasses, with an Android phone mode. As an assessor walks a street, the camera streams frames to a vision model that detects and labels damage: roof loss, wall cracks (shear/diagonal), collapsed or soft stories, tilted columns, broken utility poles, and debris blocking roads. The glasses overlay a color-coded damage class, using a simplified ATC-20-style green/yellow/red tag, and the assessor confirms by voice (“Confirm, totally damaged, family of five, needs tarpaulin”). Speech-to-text and an LLM turn voice notes into structured fields: households affected, casualties, and needs (food, water, shelter, medical). Each entry is geotagged and timestamped with photo evidence. At the end of the walk, MATA auto-fills an RDANA/sitrep-ready report and dashboard for the MDRRMO, exportable to standard OCD templates. It works offline and syncs when signal returns.

Target Market

• Primary users: MDRRMO/CDRRMO staff, barangay DRRM committees, municipal engineers, and DSWD and DPWH field teams.
• Buyers (B2G/B2B): LGUs (about 1,600 cities and municipalities), OCD regional offices, DPWH, DepEd (school damage assessment), and insurers and property-loss adjusters (B2B).
• Secondary: Humanitarian NGOs (Red Cross, IOM shelter cluster) and utilities and telcos assessing tower and pole damage.

How AI Is Used

• Vision detection and classification: Detects damage types and severity in real time. Without it, AR is just a camera, and classification stays slow and subjective.
• Speech-to-text (Taglish/Bisaya): Lets assessors stay hands-free in debris, which is critical for safety and speed.
• LLM structuring and report drafting: Converts voice notes and detections into RDANA fields and a narrative summary. This removes the manual re-typing step that causes the delay.
• Aggregation and anomaly detection: Flags duplicate structures and low-confidence classifications for human review.

PLDT / Smart / ePLDT / Rokid Angle

• Rokid AR glasses are the hero device. MATA is a flagship enterprise use case that Rokid Philippines can bundle for government and insurance clients.
• Smart 5G/LTE plus PLDT satellite and portable cell sites for syncing in disaster zones. This fits with Smart’s existing disaster response deployments.
• ePLDT cloud hosts the RDANA dashboard and image archive, with GPU inference available for heavier models.
• PLDT Enterprise B2G sales: A device + connectivity + SaaS bundle for LGUs, DPWH, and insurers.

Demo Scenario (3 minutes for judges)

1. Set the scene with a 10-second photo montage of the 2025 Cebu earthquake damage. “The municipal hall needs numbers by tonight.”
2. A teammate wearing Rokid glasses walks past printed large-format photos of damaged houses set up as a “street.” The overlay labels “Diagonal shear crack – UNSAFE (red)” and “Roof partially blown off – yellow.”
3. The teammate speaks: “Confirm red. Five residents, one senior, need tent and water.” The fields fill in live on the projector dashboard.
4. After three houses, they say “Generate report.” A completed RDANA-style PDF and map appear with counts, needs, and photos.
5. Close on the comparison: “Clipboard: two days. MATA: twenty minutes.”

30-Hour Build Plan

Suggested stack: Rokid SDK / Android (Kotlin), YOLOv8/RT-DETR for detection plus a multimodal LLM (Gemini/GPT-class) for severity explanations, Whisper for STT, FastAPI, PostgreSQL/PostGIS, React dashboard, ReportLab for PDF generation. Public data / APIs: xBD/xView2 building damage dataset, Hugging Face disaster-damage image datasets, PHIVOLCS earthquake bulletins, NDRRMC sitreps and OCD RDANA templates, OpenStreetMap building footprints, PSA PSGC codes.

Risks & Privacy Notes

• Model misclassification: AI only suggests a tag. A human assessor must confirm every tag, low-confidence results are flagged, and MATA is positioned as a rapid assessment tool, not a structural engineering certification.
• Rokid SDK and hardware limits: Run inference on a paired phone or edge device, and keep phone-only mode as a fallback.
• Connectivity loss: Offline-first local storage with later sync.
• Data Privacy Act 2012: Images may capture people and house numbers. Blur faces automatically on-device, store only what RDANA requires, encrypt in transit and at rest, and use role-based access for LGU staff.
• Clause 24 compliance: Train and demo only on public, licensed datasets and staged photos. No victim personal data goes into external AI tools.

AI Use Disclosure (draft)

MATA uses an object-detection model (e.g., YOLO) and a multimodal LLM for damage classification, Whisper-class speech recognition for voice notes, and an LLM to draft RDANA reports. AI coding assistants helped write boilerplate code and draft documentation. All damage classifications and reports are confirmed by human assessors, and the team reviewed all AI-assisted code and text.

Rubric Scoring