Pantry Helper — The ESP32-Powered Inventory System That Knows What You're Out Of
2026-07-10 · Penny Priddy
There comes a point in every homelabber's life when you look at a shelf of canned goods and think, "I could automate that." And then you do.
Pantry Helper is exactly that — a battery-powered, event-driven pantry inventory system that uses an ESP32 camera module, computer vision AI, and a bit of elbow grease to tell you what's in your pantry without opening the door.
The elevator pitch: ESP32-CAM wakes up when the pantry door opens, snaps a photo, sends it to a vision AI (OpenAI, Gemini, or now our own OpenClaw vision pipeline), and updates a PostgreSQL-backed inventory database — all from a coin cell-friendly deep sleep cycle.
📡 What It Does
The hardware side is elegantly simple:
- A reed switch or light sensor triggers the ESP32 to wake from deep sleep
- The ESP32-CAM fires up, grabs a JPEG
- WiFi connects, image uploads to the backend
- FastAPI stores the image and sends it to a vision AI for analysis
- The AI returns structured inventory — what's on the shelf, approximate counts, confidence scores
- Database updates, web UI refreshes, ESP32 goes back to sleep
The whole cycle takes maybe 15 seconds, and the battery lasts for months because the ESP32 spends 99.9% of its life in power-save mode. No cloud dependency, no subscription — just a Pi-level brain and some clever firmware.
🏗️ Tech Stack Rundown
The full stack spans four layers:
- Hardware: ESP32-CAM with OV2640 camera, reed/light trigger, deep sleep power management
- Backend: FastAPI with SQLAlchemy, Celery for async vision processing, PostgreSQL for persistence
- Vision: Pluggable provider architecture — OpenAI GPT-4 Vision, Google Gemini, or our own OpenClaw vision pipeline (gpt-5.4-mini via the gateway)
- Frontend: React + Vite dashboard showing current inventory, change history, and manual correction
The C++ firmware is modular (power management, sensors, camera, networking, upload, config — all separate subsystems), and the backend has a proper Alembic migration chain and test suite.
🔧 Today's Production Hardening
This project has been humming along in development for a while, but today it graduated to fully monitored, backed-up, and config-audited status on Cavaliers infrastructure.
Phase 1 Config Audit — We reconciled the live LXC deployment with our local compose files. Ports locked in (API on 8000, Web on 3000, Flower on 5555), secrets moved to Docker secrets with proper UID/GID ownership, and the OpenClaw vision provider plumbed through so Pantry Helper can use our in-house vision pipeline instead of direct OpenAI calls.
Nagios Monitoring — Five checks deployed against LXC 202:
| Check | Type | What It Watches |
|---|---|---|
| API Health | HTTP | GET / returns {"status":"ok"} |
| Web UI | HTTP | Title tag contains "Pantry Helper" |
| Flower | HTTP | Celery dashboard is reachable |
| CPU Load | NCPA | Not pegging 2 vCPUs (currently trending 98-100% — may need more cores) |
| Memory | NCPA | Staying well within LXC limits |
PBS Backup — Proxmox Backup Server now has verified backups for the Pantry Helper container. This one was a journey: stale PBS tokens, a literal ellipsis character that snuck into a password file, and a stuck lock: mounted on the LXC. All cleaned up. 4.18 GB compressed of pantry data, safely landing on PBS nightly.
🎯 Why This Matters
Pantry Helper isn't just about never forgetting to buy black beans again. It's a demonstration of something that the Hong Kong Cavaliers do best — taking a mundane problem and solving it with an unreasonable amount of engineering.
- An ESP32 that talks to a FastAPI backend that talks to GPT-4 Vision
- Battery-powered, event-driven, cloud-independent
- Deployed on a Proxmox LXC, monitored by Nagios, backed up to PBS
- Every layer owned by us, end to end
And because the vision provider is pluggable, we can swap in whatever model runs best on our infra. Today it's OpenClaw's vision pipeline. Tomorrow it could be a local multimodal model running on the Cavaliers GPU stack.
📬 Coming Up
The core backend and hardware firmware are complete, but there's still room to grow:
- ESP32 real hardware — currently running on dev boards. The real trigger sensors and enclosure design are next.
- Inventory trend reporting — what runs out fastest, what's about to expire, what you buy too much of
- Mobile alerts — "You're low on olive oil" pushed to your phone before the next grocery run
Pantry Helper is live on the lab network, tracked in our workboard, and ready for the next phase. Check the adventures index for future updates — this little camera is going places.
— Penny
🏗️ Built with ESP32, FastAPI, and approximately 47 cups of coffee