Self-Hosted Nutrition // calorie tracking without the SaaS
#architecture#api#self-hosted#build-your-own

Building a calorie tracker app from scratch: architecture overview

If you're going to build your own tracker, here's the architecture that won't paint you into a corner.

By Alex Brennan · Published

Audience

Someone who has used a commercial tracker, has used a FOSS tracker, doesn’t love either, and is considering building their own. Specifically: a self-hosted tracker for personal or small-group use, not a product to sell.

This is the architecture document I wish I’d had two years ago.

Constraints to get straight

Before any design decisions, decide these:

  1. Single-user or multi-user? Single-user is dramatically simpler. If you can stay there, do.
  2. Online-only or offline-first mobile? Offline-first is harder; for travel and gym use it’s worth it.
  3. Native mobile app or web? Native is better for camera, barcode, push. Web is faster to ship and revisit.
  4. Sync model: server-as-truth or eventual? Server-as-truth is simpler and right for single-user.
  5. One device or many? Many devices need real sync; one device can be a SQLite file.

For our reference design we assume: single user, offline-first mobile + web companion, server-as-truth, two devices (phone + browser).

Stack

The stack we’d recommend if starting fresh in 2026:

  • Database: Postgres 16. SQLite if it’s literally one device.
  • Server: Python (FastAPI) or Go. Either is fine.
  • Mobile: Flutter or PWA. Both viable; Flutter if you specifically need camera/barcode beyond what PWA permits.
  • Search: pg_trgm in Postgres for the indexed description search.
  • Sync: REST + a “last-modified-since” pull model. CRDTs are overkill for a single-user diary.
  • Auth: a single API token in environment / a system keyring on phones. Stop here unless you have a real multi-user need.
  • Backups: nightly pg_dump → off-site Borg/restic.

Schema

Five tables get you most of the way.

CREATE TABLE foods (
    id BIGSERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    brand TEXT,
    barcode TEXT,
    kcal_100g REAL,
    protein_100g REAL,
    carbs_100g REAL,
    fat_100g REAL,
    source TEXT,      -- 'usda', 'off', 'custom'
    source_id TEXT,
    created_at TIMESTAMPTZ DEFAULT now(),
    updated_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE servings (
    id BIGSERIAL PRIMARY KEY,
    food_id BIGINT REFERENCES foods(id),
    label TEXT,        -- '1 medium', '1 cup'
    grams REAL NOT NULL
);

CREATE TABLE diary_entries (
    id BIGSERIAL PRIMARY KEY,
    day DATE NOT NULL,
    meal_slot TEXT,    -- 'breakfast' | 'lunch' | 'dinner' | 'snack'
    food_id BIGINT REFERENCES foods(id),
    grams REAL NOT NULL,
    note TEXT,
    created_at TIMESTAMPTZ DEFAULT now(),
    updated_at TIMESTAMPTZ DEFAULT now()
);

CREATE TABLE recipes (
    id BIGSERIAL PRIMARY KEY,
    name TEXT NOT NULL,
    yield_grams REAL,
    notes TEXT
);

CREATE TABLE recipe_components (
    id BIGSERIAL PRIMARY KEY,
    recipe_id BIGINT REFERENCES recipes(id),
    food_id BIGINT REFERENCES foods(id),
    grams REAL NOT NULL
);

CREATE INDEX idx_foods_name_trgm ON foods USING gin (name gin_trgm_ops);
CREATE INDEX idx_foods_barcode ON foods (barcode);
CREATE INDEX idx_diary_day ON diary_entries (day);

That’s enough to log meals, search foods, scan barcodes, and assemble recipes. Add columns as you need them — micronutrients, custom serving units, etc.

Server endpoints

Minimum useful set:

GET  /api/foods/search?q=...
GET  /api/foods/{id}
GET  /api/foods/barcode/{gtin}
POST /api/foods                          (create custom food)
PUT  /api/foods/{id}

GET  /api/diary?day=YYYY-MM-DD
POST /api/diary
PUT  /api/diary/{id}
DEL  /api/diary/{id}

GET  /api/sync/since?ts=...              (for pull-sync)
POST /api/sync/push                      (offline mobile pushes a batch)

About 200 lines of FastAPI plus the Pydantic models.

Sync model (offline-first)

The simplest offline-first pattern that works for a single user:

  1. Mobile keeps a local SQLite DB shadowing the server’s diary tables.
  2. On wake, mobile pulls /api/sync/since?ts=<last_pull> for foods and diary.
  3. Mobile applies pull, deduping on id (or a UUID if you go that route).
  4. Mobile stores any locally-created entries (with a temporary negative ID).
  5. On wake, mobile pushes locally-created entries to /api/sync/push, gets back permanent IDs, updates local rows.
  6. Conflict resolution: server wins. (Single-user sync, conflicts shouldn’t happen often; when they do, server’s row wins and the mobile gets the discrepancy in the next pull.)

This is not CRDT. It’s not multi-master. It’s not great for collaborative editing. It’s perfectly fine for “me logging my own food on my own phone.”

For the food search:

SELECT id, name, brand, kcal_100g
FROM foods
WHERE name % :query
ORDER BY similarity(name, :query) DESC,
         (CASE WHEN brand IS NULL THEN 0 ELSE 1 END),
         length(name)
LIMIT 25;

The similarity() ordering plus the brand-priority and shortest-name tie-breaker get you reasonable results. For better fuzzy matching, add unaccent and lowercase normalisation in the index.

If your DB grows past a few hundred thousand items (likely if you load FDC + OFF), Meilisearch in front of Postgres is a clean upgrade. Don’t start there.

Camera and barcode

Avoid building your own. Use the platform’s API:

  • Flutter: mobile_scanner package. ZXing-backed. Works on iOS and Android.
  • Web (PWA): BarcodeDetector API in Chrome and Safari. Limited Firefox support.

Don’t roll your own ZXing wrapper. The maintained packages handle the camera permissions and the focus loops correctly.

Photo recognition

Don’t build this. As of 2026, FOSS-quality photo recognition for calorie estimation is approximately 30% MAPE on a weighed bench. That’s worse than a user manually entering the food.

If you must do this, the pieces look like:

  1. Quantised vision-language model on-device (~800 MB+).
  2. Prompt template with a fixed list of food categories.
  3. Postprocessing to map to your foods table.

The accuracy isn’t there. We have prototyped this twice and recommend you don’t, in 2026, on FOSS. The relevant commercial-validation data is discussed in our state-of-the-art piece but doesn’t change the FOSS picture.

What to skip at v1

  • Push notifications. Not needed for a personal tracker.
  • Multi-language support. You speak one language. Skip i18n until you don’t.
  • Account management. One user is one user.
  • Social/share features. Don’t.
  • Wearable integrations. Hard, brittle. Add via export-and-merge later.
  • A “coach” UI layer. You don’t need it.

What to over-engineer up front

Two things, in our experience, deserve more care than they seem to:

  1. Backups. From day one, dump the DB nightly to an off-site location. We have lost a personal tracker DB to disk failure on a side project; the lesson stuck.
  2. Schema migrations. Use Alembic from the first day. Even a one-table schema needs migrations once you start adding columns, and retrofitting Alembic is painful.

Time-box

A reasonable v1 of this design takes about 25–30 hours of work for someone who knows the languages involved. Roughly:

  • Schema and migrations: 3 hours
  • API: 8 hours
  • Mobile (PWA path): 10 hours
  • Mobile (Flutter path): 18 hours
  • Sync: 6 hours
  • Backups: 2 hours
  • Polish, deploy: 3 hours

Most of the time is in the mobile client. If you don’t need a mobile client and you’re fine with a phone-friendly web UI, halve that.

Where to host it

For a personal v1: a $4 Hetzner CX22 plus the home Pi as backup. See Personal VPS vs homelab.

References