EMPATHAR.COM
Real-time social battery detection — MediaPipe AR overlay in the browser
Most of social intelligence is invisible. We read signals unconsciously: crossed arms, a fading smile, restless movement. EmpathAR makes those signals visible in real time, rendering a live Social Battery overlay directly over the people in your camera view.
OBJECTIVE
EmpathAR uses your camera to detect posture, expression, and movement in real time — translating them into a live Social Battery overlay rendered directly over the people in your frame. Most of social intelligence is invisible. EmpathAR makes those signals visible, without telling you what to feel.
APPROACH
Two MediaPipe models run simultaneously per frame: a pose landmarker and a face landmarker. Their outputs are weighted and fused into a single battery score using an asymmetric blend algorithm — raw scores ease gradually into the current value rather than snapping instantly, preventing flicker from a single outlier frame.
The Social Battery panel lists all five states with contextual action prompts — Energized down to Needs Space — while the live overlay tracks facial landmarks and speaking state in real time.
Five named states — from Energized down to Needs Space — each paired with a contextual action prompt that shifts based on what the model is reading.
Examples of Contextual Action Prompts
✦ Energized
"They're lit up — ask: 'What's fueling this for you?'"
"Great energy right now — match it and build on it"
◐ Engaged
"Keep it alive: 'What's the hardest part of that?'"
"Open it up: 'What are you working on these days?'"
● Present
"They're engaged — ask: 'What do you think the next step is?'"
"Invite their opinion: 'What's your take on this?'"
○ Fading
"Shift gears: 'What's something totally different you love?'"
"A genuine compliment right now could shift everything"
↓ Needs Space
"They're running low — ease the pace a little"
"Let the silence be okay — not every gap needs filling"
BUILT
Real-time AR overlay — Social Battery score and named state rendered directly over people in frame via WebGL, no app install required
Three-signal model — posture (33 landmarks), expression (52 blendshapes), and movement velocity fused into a single 0–100% score with asymmetric smoothing: draining is fast, charging is slow
Multi-person tracking — P1/P2/P3 labels with contextual toast notifications that fire as battery state changes
Fully on-device — Google MediaPipe runs locally via WebAssembly; no video, images, or biometric data ever leaves the browser
Impact / Reflections
EmpathAR debuted at a hackathon and earned enough traction to push further. Next steps include testing the overlay in VR with headsets and a potential collaboration with ARTEX for Milan Design Week.
Two things I'd do differently: person tracking via shirt color is fragile — it breaks in low light or when two people wear similar colors, and a more robust re-identification method would be the first thing to revisit. I'd also add per-signal calibration so users can weight posture, expression, and movement to fit how they actually read a room rather than trusting a fixed heuristic.
What's still open: view the roadmap →