Self-Hosted Nutrition // calorie tracking without the SaaS

Methodology

A standing document. Updated when the test bench changes. Last revised 3 September 2026 — added the exit test and the programmatic-access test, and the rule about negative claims.

Test hardware

Sample food set

For barcode and database testing, we use the same 30-item shopping list, refreshed quarterly. It is intentionally heterogeneous:

The list lives in a private repo and rotates so apps cannot game it. Each item is logged on every app under test using whatever the app's "easiest" path is (barcode, search, AI estimate).

Scoring

Privacy testing

For commercial apps, we run them once on the GrapheneOS phone with a NextDNS profile capturing all DNS, and a mitmproxy session capturing TLS-decryptable HTTPS where the app's certificate pinning permits. Captures are logged for 24 hours of app use and then archived. We publish destination domains and observed payload categories — never the full capture.

A 24-hour capture tells you where the bytes go. It tells you nothing about retention on day nine. Where an operator states a retention policy we report it as stated, labelled as documented rather than verified, and we do not upgrade a stated policy into an audited one because we liked the phrasing.

The exit test (data portability)

Added September 2026, after several years of scoring apps on what they collect and never on whether you can leave with it. Every commercial app under test now gets two additional checks:

An app with an aggressive privacy policy and a working export is a smaller problem than an app with a clean policy and no way out, because every objection to the first is one you can act on. We weight it accordingly.

Programmatic access testing

Also new for September 2026. The question is whether software you control can make authenticated reads against your own account — not a scrape, not a screenshot, not a partner agreement you cannot apply for.

What we check: whether an end user can obtain credentials from published documentation; what auth flow is used; what the granularity of consent is; whether the surface is read-only or can also write; and whether revocation is documented as enforced per request or only at token expiry. Where such an endpoint exists we connect it on a live test account and exercise every documented call.

We treat a read-only surface as a feature rather than a limitation. An integration that can write into a health diary can also silently corrupt one, and a corrupted food log is not something a user notices in time to fix.

Negative claims

We do not publish "app X does not do Y" on the basis that a page failed to mention Y. A capability absent from marketing copy is not a capability absent from the product. We have made that mistake — we once inferred, from a page that simply did not mention it, that an app lacked a data-export feature it had shipped all along, and the inference propagated through several pieces before a reader corrected it — and the rule since is that a negative capability claim requires either an explicit statement from the operator or a failed test on a live account. Otherwise the cell reads "not verified this round", which is less satisfying and more honest.

Third-party accuracy figures

We run our own 30-item bench, but we do not pretend it is a validation study. Where an independent laboratory has published one, we cite it and we follow three rules:

FOSS evaluation

For an open-source app to be reviewed, we compile from source on a clean container, run on the test phone, and read at least the data-layer and network-layer source. Apps with no commits in the last 12 months are flagged "stale." Apps with no commits in 24 months are flagged "abandoned."

What we do not do

Conflicts of interest

The editor has personally paid (and then cancelled) MyFitnessPal Premium, Cronometer Gold, and MacroFactor. He has run the OpenNutriTracker codebase on a personal phone for over a year. He has briefly trialed PlateLens for the purpose of writing the "Would I pay" piece, and a free-tier PlateLens account is currently kept open — unpaid — because the export and read-only API tests described above need a live account to run against. He has not received any compensation, free subscriptions, or merchandise from any of the above.

No app under test has been given advance notice of an audit, a draft, or a right of reply before publication. Corrections after publication are handled under the policy on the about page and are dated in the affected piece.