RésuméEmail me

MDAMCR

Multimodal data analysis from murder case reports: public news video, audio and articles about body discoveries in Texas, turned into a source linked dataset on one workstation.

Year
2026
Role
Co author, pipeline and evaluation
Team
Team of two, UH Computational Biomedicine Lab, with an IEEE format report
Stack
LLaMA 3.2 3B, Faster-Whisper, Moondream2, MiniLM embeddings, rapidfuzz, scikit-learn, MongoDB, all local

How it works

Phase 3 of 4: grouping

Matched at pass 2 and bound into the case folder

  1. 1. Names
  2. 2. Fingerprint
  3. 3. Embeddings
  4. 4. LLM review
  • B-cubed F1 0.889
  • Pairwise F1 0.615
  • Body found F1 0.871

Sample case 1 of 3

Source cardCheck passedCheck flaggedNews sources become source linked cases; automation only pre screens, a person signs off.

The system

From news reports to reviewed cases A four-stage pipeline: an adaptive agent loop collects article and video links and repeats until each city is covered, extraction turns each source into a 9-field JSON record with one small LLM call and a vision fallback, a four-pass cascade gives every source a case ID, and validation runs rule checks and cross-source agreement before a mandatory human review signs off on the published dataset. Input Tooling Guard Output loops up to 5 rounds 1 Collection Adaptive agent loop Per-city search targets LLaMA 3.2 3B writes queries YouTube API, Google News RSS LLM relevance filter City coverage check Collected URLs stored in MongoDB 2 Extraction Two branches, one LLM call Article text via newspaper3k Video: yt-dlp, Faster-Whisper One LLaMA 3.2 3B call on the body-discovery window Moondream2 on 3 frames if confidence < 0.5 9-field JSON per source 3 Grouping 4-pass cascade Extracted records, every source Name fuzzy ≥ 70, city + date, MiniLM + agglomerative LLM reviews edges 0.3 to 0.8 at most 500 calls One Case ID per source 4 Validation Rules, agreement, a person Grouped cases with their sources 7 rule checks, cross-source agreement HIGH / MED / LOW Mandatory human review queue, every case cases.csv, flagged cases, validation report Automation pre screens; a person signs off. Stage handoff Coverage feedback Human sign-off gate

The job

The same event gets reported many times, by different outlets, in video and in text, with names spelled differently and places described loosely. The task: collect those reports, extract the facts from each, and group the sources that describe the same case, with every claim linked back to where it came from.

Everything runs on one workstation with open weight models, at no cloud cost. Because the subject is real deaths, the dataset never leaves the lab and every case goes past a person before it counts.

Collection and extraction

An adaptive agent loop writes search queries, pulls YouTube and Google News, filters for relevance and checks coverage across 20 Texas cities, then asks again where coverage is thin, up to five rounds.

Articles are scraped; videos are transcribed with Faster-Whisper and cut to the discovery window. One LLaMA 3.2 3B call per source returns nine structured fields, and when confidence is low on a video, a Moondream2 pass reads three frames and is kept only if it does better.

A four pass cascade

Grouping runs four passes, each a fallback for the one before: names canonicalised with fuzzy matching, then a city and date fingerprint, then sentence embeddings with agglomerative clustering, then an LLM that reviews only the ambiguous pairs, capped at 500 calls a run.

Validation adds seven rule checks and a cross source agreement score. Anything that trips a flag lands in a human review queue; automation is pre screening only.

Results and what they taught

On 50 hand labelled sources and 44 true cases, grouping reached a B cubed F1 of 0.889 and a pairwise F1 of 0.615, against trivial baselines of 0.000 and 0.019. Recall was perfect; precision is the open work.

The lesson I keep: embedding similarity is the wrong final arbiter. It merged events that only shared topic words across different cities, so geography has to be a hard constraint, not a feature.

Next, SocialOps →