From 7efe277fba4ba3292243b90d3033e049a91f9c91 Mon Sep 17 00:00:00 2001 From: Claudio Ortolina Date: Mon, 4 May 2026 16:12:43 +0100 Subject: [PATCH] Add research docs about nerves deployment --- ...doc-10 - Deepseek-v4-Pro-xhigh-analysis.md | 222 ++++++++ .../doc-8 - Opus-4.7-xhigh-analysis.md | 119 +++++ .../doc-9 - GPT-5.5-high-analysis.md | 500 ++++++++++++++++++ 3 files changed, 841 insertions(+) create mode 100644 backlog/docs/nerves-deployment/doc-10 - Deepseek-v4-Pro-xhigh-analysis.md create mode 100644 backlog/docs/nerves-deployment/doc-8 - Opus-4.7-xhigh-analysis.md create mode 100644 backlog/docs/nerves-deployment/doc-9 - GPT-5.5-high-analysis.md diff --git a/backlog/docs/nerves-deployment/doc-10 - Deepseek-v4-Pro-xhigh-analysis.md b/backlog/docs/nerves-deployment/doc-10 - Deepseek-v4-Pro-xhigh-analysis.md new file mode 100644 index 00000000..41eb129c --- /dev/null +++ b/backlog/docs/nerves-deployment/doc-10 - Deepseek-v4-Pro-xhigh-analysis.md @@ -0,0 +1,222 @@ +--- +id: doc-10 +title: 'Deepseek v4 Pro, xhigh analysis' +type: other +created_date: '2026-05-04 15:11' +--- +# Nerves Deployment Feasibility Analysis for Music Library + +> Research report — do not implement. May 2026. + +## Overview + +This report assesses the viability of deploying the Music Library application (a +Phoenix LiveView app backed by SQLite + Litestream) as a Nerves firmware image. +Three areas were identified as potential challenges: + +1. **Libraries with native extensions (NIFs)** — do they offer precompiled ARM + binaries or can they be cross-compiled? +2. **SQLite extensions** — `vec0` (sqlite-vec) and `unicode` — already support + Linux/ARM on paper, but are they tested on Nerves targets? +3. **Data replication** — the production instance uses Litestream for backup. How + would a Nerves device replicate data, and what happens with write conflicts? + +Each area is analysed below, followed by a summary of risks and recommended next +steps. + +--- + +## 1. Native-extensions audit + +Every dependency that ships a NIF or binary artefact was reviewed for ARM / +Nerves compatibility. Dev‑only tools (esbuild, tailwind, credo, etc.) are +excluded — they never reach the firmware. + +| Dependency | Purpose | NIF type | Precompiled ARM? | Nerves feasibility | +|---|---|---|---|---| +| **exqlite** → `ecto_sqlite3` | SQLite driver | C NIF (`cc_precompiler`) | ✅ `aarch64-linux-gnu/musl` (no `armv7-*`) | **🟢 Good** — `force_build: true` compiles from source inside the Nerves toolchain. Explicitly mentioned in Exqlite’s README as an embedded use-case. | +| **vix** → libvips | Image processing (covers, thumbnails) | C NIF (`elixir_make`) | ❌ No ARM binaries advertised | **🔴 High risk** — libvips is ~30 MB with transitive deps (libjpeg, libpng, libwebp, etc.). Each needs Buildroot integration or cross‑compilation. Biggest blocker for a lean firmware. | +| **mdex** → comrak + ammonia | Markdown → HTML | Rust NIF (RustlerPrecompiled) | ⚠️ Standard Rustler targets (`aarch64-unknown-linux-gnu`, possibly `arm-*`) | **🟡 Needs cross‑compilation** — RustlerPrecompiled targets use different triples than Nerves (`aarch64-nerves-linux-gnu`). Precompiled `.so` likely won’t load. Cross‑compile from source with a Nerves‑aware Rust toolchain. | +| **lumis** → tree‑sitter | Syntax highlighting (in mdex) | Rust NIF (RustlerPrecompiled) | Same as mdex | **🟡 Same situation** — part of the mdex stack. Tree‑sitter grammars add per‑language compilation but the NIF itself is manageable. | +| **typst** | PDF generation (tracklists) | Rust NIF (RustlerPrecompiled) | Same as mdex | **🔴 Heavy** — a full typesetting engine. Large binary, plus the same target‑triple mismatch. Likely overkill for embedded PDFs. | +| **dominant_colors** → kmeans_colors | Colour extraction from covers | Rust NIF (`rustler_precompiled ~> 0.7`) | Same as mdex | **🟢 Small** — trivial NIF. Cross‑compilation is straightforward. Low risk. | +| **esbuild** | JS bundling | Go binary | — | **N/A** — dev only, never deployed. | +| **tailwind** | CSS generation | Go binary | — | **N/A** — dev only, never deployed. | + +### Target‑architecture specifics + +| Target | CPU | Nerves toolchain triple | Precompiled‑binary match? | +|---|---|---|---| +| RPi 4 / 5 | Cortex‑A72 / A76 (64‑bit) | `aarch64-nerves-linux-gnu` | ⚠️ `aarch64-linux-gnu` *may* be ABI‑compatible (exqlite). Rust NIFs need rebuild. | +| RPi 3 | Cortex‑A53 (32‑bit) | `armv7-nerves-linux-gnueabihf` | ❌ No precompiled binaries match. Everything must be compiled from source. | + +**Recommendation**: Target RPi 4 or 5 (64‑bit) first — the precompiled ecosystem is +far better and the 32‑bit path adds unnecessary friction. + +--- + +## 2. SQLite extensions + +The application loads two run‑time extensions via Exqlite’s `load_extensions` +config: + +| Extension | Source | ARM status | Notes | +|---|---|---|---| +| **vec0** | [sqlite-vec](https://github.com/asg017/sqlite-vec) | ✅ **Good** | “Written in pure C, no dependencies, runs anywhere SQLite runs (… Raspberry Pis, etc.)” — explicitly tested on ARM. Compilable for any Nerves target. | +| **unicode** | likely [sqlean](https://github.com/nalgeon/sqlean) (`text` module or bundle) | ✅ **ARM64 available** | Sqlean ships `sqlean-linux-arm64.zip`. For ARMv7, compile from C source. | + +### Loading mechanism + +The project already uses an architecture‑aware directory layout: + +```elixir +config :myapp, Myapp.Repo, + load_extensions: ["./priv/sqlite/#{arch_dir}/vector0"] +``` + +This pattern extends naturally to Nerves — compile the `.so` files with the Nerves +cross‑compiler, place them in the appropriate arch subdirectory, and Exqlite will +load them at connection time. + +**Unknown**: Whether the `unicode` extension is built from the exact same sqlean +release as production, or compiled independently. Sync the build flags and version +to avoid subtle behavioural differences. + +--- + +## 3. Data replication & conflict resolution + +This is the **hardest problem** and the most architecturally open of the three. + +### What Litestream does (and doesn’t) + +| Capability | Litestream | Needed for device ↔ cloud sync | +|---|---|---| +| Unidirectional backup (app → S3) | ✅ | ❌ | +| Disaster recovery (restore from S3) | ✅ | ❌ | +| Bidirectional sync | ❌ | ✅ | +| Conflict detection / resolution | ❌ | ✅ | +| Multi‑writer support | ❌ | ✅ | + +Litestream is purely an asynchronous backup tool. It continuously streams WAL pages +to S3. It has **no mechanism** for a second instance to push writes back, no merge +strategy, and no conflict detection. Adding a Nerves device that writes to the same +database file would corrupt the Litestream generation (or create a parallel one that +cannot be merged). + +### What LiteFS does (and doesn’t) + +[LiteFS](https://github.com/superfly/litefs) (same creators, different project) +provides multi‑node SQLite replication with a primary/replica model: + +- ✅ Automatic failover via distributed lease (Consul) +- ✅ LTX‑based transaction‑log replication +- ✅ Built‑in HTTP proxy for write‑forwarding to the primary +- ❌ Requires FUSE (needs a Buildroot kernel module) +- ❌ Requires Consul or static leasing +- ❌ **Replicas are read‑only** — only one writer at a time +- ❌ Designed for same‑region clusters, not device ↔ cloud +- ❌ No offline‑first or intermittent‑connectivity support + +**LiteFS does not fit** the device‑sync use case. If the Nerves device goes offline, +accumulates writes, then reconnects, LiteFS has no way to merge those writes onto +the primary. + +### Realistic approaches + +#### A. Read‑only Nerves device (simplest, safest) + +The Nerves device is a read‑only replica. All writes happen on the production +server. The device periodically pulls a fresh database snapshot. + +- **Sync**: `GET /api/v1/backup` (already exists), Litestream restore, or HTTP + range‑request to fetch a compressed DB. +- **Pros**: Zero conflicts, simple to implement, leverages existing backup API. +- **Cons**: No local writes — can’t add records, update notes, or scrobble from + the device. + +#### B. Separate write domains (pragmatic) + +Never write the same record type from both sides: + +- **Production server**: records, embeddings, notes, record sets, artist info. +- **Nerves device**: scrobbles, listening stats, playback logs. +- Each instance has its own SQLite database. They sync via API calls (e.g., device + pushes scrobbles to production, production pushes new records to device). + +- **Pros**: No database‑level conflicts. Clear ownership boundaries. +- **Cons**: App‑level routing logic. Two separate databases on the device. + +#### C. CRDT‑backed data model (most robust) + +Model the subset of data writable on both sides using Conflict‑free Replicated Data +Types (CRDTs). + +- **Libraries**: [Automerge](https://automerge.org/), [Yjs](https://yjs.dev/), + custom CRDT stores. +- **Pros**: True offline‑first, automatic conflict resolution, no central + authority needed. +- **Cons**: Requires wrapping your data in CRDT types. No mature Elixir CRDT ↔ + SQLite bridge. Significant architecture change. + +#### D. Event sourcing with log shipping + +Write all mutations as an append‑only event log. Ship the log between instances. +Replay to reconstruct state. + +- **Pros**: Natural fit with SQLite (append to a `mutations` table). Full audit + trail. +- **Cons**: Requires event‑schema design, log compaction, eventual consistency + handling. + +#### E. Application‑level sync protocol + +Define a custom sync protocol with: + +- Vector clocks or timestamps for causality tracking. +- Last‑write‑wins (LWW) or manual conflict resolution. +- Periodic pull/push of changed records since the last sync point. + +- **Pros**: Domain‑specific, can optimise for exact data shapes. +- **Cons**: Most implementation effort. Easy to get wrong. + +### Litestream’s role on the device + +Even with a sync strategy in place, Litestream could still run on the Nerves device +for **local disaster recovery** — streaming the device’s own WAL to a separate S3 +path. This is independent from the production Litestream instance. + +--- + +## Summary of risks + +| Risk | Severity | Mitigation | +|---|---|---| +| **vix / libvips on Nerves** | 🔴 High | Replace image processing with a lighter alternative, or offload to the production server via API. | +| **Data‑sync strategy** | 🔴 High | No off‑the‑shelf solution. Start with read‑only replica (option A), iterate toward separate write domains (option B). | +| **typst on embedded** | 🟡 Medium | Switch to server‑side PDF generation; client downloads the result. | +| **Rust NIF cross‑compilation** | 🟡 Medium | Set up Nerves Rust cross‑compilation. Works but adds build complexity. Each library needs individual testing. | +| **ARMv7 (RPi 3) vs AArch64 (RPi 4/5)** | 🟡 Medium | RPi 4/5 (64‑bit) has much better precompiled‑binary support. RPi 3 requires everything compiled from source. | +| **Buildroot package deps (libvips)** | 🟡 Medium | Each transitive C library needs a Buildroot package definition or a custom cross‑compile step. | +| **Firmware size** | 🟡 Medium | libvips + typst + sqlite extensions → image could exceed 200 MB. Typical Nerves firmware is 20–80 MB. | +| **sqlite‑vec on ARM** | 🟢 Low | Explicitly supported. Pure C, no deps. Compiles trivially with the Nerves toolchain. | +| **exqlite on Nerves** | 🟢 Low | Designed for embedded use. Source compilation well‑supported and documented. | +| **dominant\_colors** | 🟢 Low | Tiny Rust NIF. Straightforward cross‑compilation. | + +--- + +## Recommended next steps (if proceeding) + +1. **Target RPi 4 or 5 (64‑bit)** — significantly better precompiled binary + ecosystem and larger RAM for Phoenix + Oban. +2. **Start with a read‑only firmware** — pull database snapshots from the + production server via the existing `/api/v1/backup` (or `/health`) endpoint. + Don’t write locally until a sync strategy is decided. +3. **Audit which features to keep on‑device** — consider dropping image + processing (vix) and PDF generation (typst) from the embedded build, or + replacing them with lighter alternatives / server‑side APIs. +4. **Benchmark firmware size incrementally** — start with bare Phoenix + SQLite, + add extensions, measure. Set a hard size budget (e.g., 120 MB). +5. **Design the sync protocol as a separate project** — this is the most complex + piece. Prototype with a read‑only client first, then evaluate CRDTs, event + sourcing, or application‑level sync based on actual write patterns. diff --git a/backlog/docs/nerves-deployment/doc-8 - Opus-4.7-xhigh-analysis.md b/backlog/docs/nerves-deployment/doc-8 - Opus-4.7-xhigh-analysis.md new file mode 100644 index 00000000..6582a518 --- /dev/null +++ b/backlog/docs/nerves-deployment/doc-8 - Opus-4.7-xhigh-analysis.md @@ -0,0 +1,119 @@ +--- +id: doc-8 +title: 'Opus 4.7, xhigh analysis' +type: other +created_date: '2026-05-04 15:08' +updated_date: '2026-05-04 15:10' +--- +# Nerves Deployment Research — Findings + +Three parallel research agents covered NIFs, SQLite extensions, and Litestream replication semantics. The user's concerns were well-founded: there are real blockers, but most are solvable with custom Nerves system work. The Litestream question reframes the architecture more than expected. + +## 1. Native libraries (NIFs on aarch64-linux-musl) + +| Library | aarch64-gnu | aarch64-musl | Status | +|---|---|---|---| +| `mdex` | ✅ | ✅ | Fine | +| `lumis` | ✅ | ✅ | Fine | +| `typst` | ✅ | ✅ | **Nerves-aware** — has `@nerves_rust_target_triple_mapping` in mix.exs | +| `ecto_sqlite3` / `exqlite` | ✅ | ✅ | Driver fine; designed with embedded use in mind. Extensions are the issue (see §2) | +| `dominant_colors` | likely | unknown | Verify GitHub releases; if missing, request musl target from maintainer | +| `vix` | partial | ❌ | **Major blocker** | +| `cloak_ecto` | n/a | n/a | Pure Elixir + `:crypto` | + +**`vix` is the dominant blocker.** Two compounding issues: + +- It loads the NIF at *compile time* (macro-driven introspection), which breaks cross-compilation on stock Nerves systems. fhunleth (Nerves core) discussed this in a 2023 ElixirForum thread; the conclusion was "no easy Elixir-level fix." +- libvips itself is not in `nerves_system_rpi4` / `nerves_system_rpi5`. You'd need a **custom Nerves system** with `BR2_PACKAGE_LIBVIPS=y` in Buildroot, then `VIX_COMPILATION_MODE=PLATFORM_PROVIDED_LIBVIPS`. + +This is doable (15–30 min Buildroot rebuild, well-documented) but it commits you to maintaining a custom Nerves system. That single decision affects everything downstream — once you're building custom systems, ICU and sqlite-vec become "while we're at it" additions. + +**General Phoenix-on-Nerves status (2025):** Pattern is established but niche. Underjord's "LiveView on Nerves" guide is the canonical reference; a poncho project (sibling `ui` + `firmware`) is preferred over umbrella. No widely-cited "production Phoenix LiveView on Nerves at non-hobby scale" guide exists — most published examples are thermostats and birdhouses. + +## 2. SQLite extensions + +Both extensions are blockers on stock Nerves systems for the same root reason — Nerves uses musl, prebuilt binaries assume glibc. + +**`sqlite-vec` (`vec0`):** + +- Upstream publishes `linux-aarch64` but it's **glibc** — won't `dlopen` on Nerves. +- No musl prebuilt exists; PR #199 fixes the musl compile but is **unmerged**. +- The `sqlite_vec` Hex wrapper (Joel Paul Koch's) downloads the glibc binary and would fail at runtime. +- Mitigation: cross-compile inside Buildroot, ship via `rootfs_overlay/`. Or compile statically into exqlite at firmware build time (recommended pattern from the SQLCipher-on-Nerves ElixirForum thread). + +**`unicode` (ICU) extension:** + +- ICU is ~30 MB and deliberately excluded from Nerves base systems. +- Requires a custom Nerves system with `BR2_PACKAGE_ICU=y`. +- If you're already building custom for `vix`, the marginal cost is minimal. + +**Good news for `exqlite`:** Driver itself ships precompiled musl NIFs (since v0.13.7), supports `load_extension`, and has been used on Nerves in production (the SQLCipher thread). The complexity is purely in shipping the extension `.so`s built against the same musl toolchain. + +## 3. Litestream replication — this is the architectural pivot + +**Headline:** Litestream is *not* a multi-master replicator. It's a one-way backup tool. If the Nerves device writes locally, those writes are lost on the next restore — no conflict detection, no reconciliation, no merge. + +**Two viable replica modes today (both newer than the docs you may remember):** + +1. **VFS read replica** (stable, in `litestream.io/guides/vfs/`): Nerves device queries S3-backed replica directly. ~1 minute staleness matches your sync interval. Writes return `"litestream is a read only vfs"` at the SQLite layer. **Requires CGO** — verify this works in your Nerves build. +2. **Restore-then-poll**: simpler, no CGO. Restore on startup, periodically re-restore. Live replication via the embedded HTTP server existed but the example repo was **archived November 2023** — treat as experimental. + +**For the architectures you'd consider:** + +- **Read-only Nerves device** (browse music library at home, no edits) — Litestream VFS replica works, modulo CGO. This is the simplest path. +- **Read-write Nerves device** (record listening, edit notes) — Litestream alone is insufficient. Three options: + 1. **Split databases**: music library is read-only via Litestream; a separate small SQLite on the device captures writes and pushes to the production API out-of-band. Cleanest fit for your existing app shape, since scrobbles are already a separate domain. + 2. **LiteFS** (Fly.io's project): true primary/replica with write-forwarding. Requires FUSE — not viable on Nerves without kernel module support, so likely off the table. + 3. **Writes go directly to production API**: Nerves device calls the prod Phoenix JSON API for any write; reads are local. Requires the device to be online for writes (acceptable for a home device). +- **Other distributed-SQLite options** — rqlite (Raft, kills write throughput), Marmot (NATS-based CDC, eventual consistency), cr-sqlite (CRDTs). All exist; all are heavier than the problem warrants for one home device. + +## Synthesis: where the unknowns concentrate + +The three concerns aren't independent — they collapse into **one decision**: are you willing to maintain a custom Nerves system? + +- If **yes**: `vix`, ICU, and `sqlite-vec` are all solvable in the same Buildroot config. The remaining unknowns are minor (`dominant_colors` musl target, exact poncho structure for the existing umbrella-style codebase). +- If **no**: the application as currently architected cannot run on Nerves. You'd have to drop image processing, drop FTS unicode collation, drop similarity search — at which point it's a different application. + +The **biggest non-obvious unknown** is the Litestream architecture decision. Going read-only on the Nerves side is dramatically simpler than supporting writes, and it covers most of the value (browsing, search, AI chat against the local collection). Splitting writes to a separate small DB that pushes to prod is the natural compromise — and your codebase is already structured to support it (scrobbles are a separate context, notes are a separate schema). + +**Suggested next step if you want to proceed:** prototype a custom `nerves_system_rpi5` with libvips + ICU on a feature branch, before touching the application. That single artifact resolves three of the four major risks; everything else is application-layer work that can be done iteratively. + +--- + +## Sources + +### NIF / Phoenix-on-Nerves +- [Underjord: LiveView on Nerves](https://underjord.io/liveview-on-nerves.html) +- [ElixirForum: Cross Compiling with NIF as compilation dependency (vix/Nerves thread)](https://elixirforum.com/t/cross-compiling-with-nif-as-compilation-dependency/59821) +- [Nerves docs: Compiling Non-BEAM Code](https://hexdocs.pm/nerves/compiling-non-beam-code.html) +- [rustler_precompiled Precompilation Guide](https://hexdocs.pm/rustler_precompiled/precompilation_guide.html) +- [mdex GitHub releases (NIF targets)](https://github.com/leandrocp/mdex/releases) +- [lumis GitHub releases](https://github.com/leandrocp/lumis) +- [typst Elixir GitHub releases](https://github.com/Hermanverschooten/typst/releases) +- [vix HexDocs README](https://hexdocs.pm/vix/readme.html) +- [exqlite README](https://hexdocs.pm/exqlite/readme.html) +- [NervesConf 2024 Talks thread](https://elixirforum.com/t/nervesconf-2024-talks/75205) +- [Customizing Your Nerves System](https://hexdocs.pm/nerves/customizing-systems.html) + +### SQLite extensions +- [Releases · asg017/sqlite-vec](https://github.com/asg017/sqlite-vec/releases) +- [Fix for musl compile PR #199 · asg017/sqlite-vec](https://github.com/asg017/sqlite-vec/pull/199) +- [Build and install exqlite with sqlcipher into nerves rpi5 image — ElixirForum](https://elixirforum.com/t/build-and-install-exqlite-with-sqlcipher-into-nerves-rpi5-image/75089) +- [Concerning warning compiling under Nerves · Issue #166 · elixir-sqlite/exqlite](https://github.com/elixir-sqlite/exqlite/issues/166) +- [Compiling sqlite-vec — Alex Garcia](https://alexgarcia.xyz/sqlite-vec/compiling.html) +- [GitHub — joelpaulkoch/sqlite_vec](https://github.com/joelpaulkoch/sqlite_vec) +- [Advanced Configuration — Nerves](https://hexdocs.pm/nerves/advanced-configuration.html) + +### Litestream +- [Live Read Replication — Litestream (tip)](https://tip.litestream.io/guides/read-replica/) +- [VFS Read Replicas — Litestream](https://litestream.io/guides/vfs/) +- [How it works — Litestream](https://litestream.io/how-it-works/) +- [Tips & Caveats — Litestream](https://litestream.io/tips/) +- [Alternatives — Litestream](https://litestream.io/alternatives/) +- [Litestream: Revamped — Fly.io Blog](https://fly.io/blog/litestream-revamped/) +- [benbjohnson/litestream-read-replica-example (archived 2023)](https://github.com/benbjohnson/litestream-read-replica-example) +- [Live read replicas — Issue #8](https://github.com/benbjohnson/litestream/issues/8) +- [LiteFS vs Litestream vs rqlite vs dqlite on VPS in 2025](https://onidel.com/blog/sqlite-replication-vps-2025) +- [Marmot distributed SQLite replicator](https://github.com/maxpert/marmot) +- [mvSQLite](https://github.com/losfair/mvsqlite) +- [cr-sqlite](https://github.com/vlcn-io/cr-sqlite) diff --git a/backlog/docs/nerves-deployment/doc-9 - GPT-5.5-high-analysis.md b/backlog/docs/nerves-deployment/doc-9 - GPT-5.5-high-analysis.md new file mode 100644 index 00000000..49364950 --- /dev/null +++ b/backlog/docs/nerves-deployment/doc-9 - GPT-5.5-high-analysis.md @@ -0,0 +1,500 @@ +--- +id: doc-9 +title: 'GPT 5.5, high analysis' +type: other +created_date: '2026-05-04 15:11' +--- +# Nerves Deployment Research Report + +Date: 2026-05-04 + +## Scope + +This report explores the feasibility and risk areas for deploying this Phoenix/SQLite music library application as part of a Nerves application. No implementation work was performed. + +The research focused on: + +- Native dependencies and NIF availability for ARM/Nerves targets. +- SQLite loadable extensions on Linux ARM. +- Litestream replication behavior, especially conflicts and local writes. +- Project-specific architecture implications based on the current application docs and dependency graph. + +## Project Context + +The application is a Phoenix LiveView app backed by three SQLite databases: + +- `MusicLibrary.Repo`: main application data. +- `MusicLibrary.BackgroundRepo`: Oban jobs. +- `MusicLibrary.TelemetryRepo`: persistent telemetry metrics. + +The main repo loads two SQLite extensions at runtime: + +- `unicode` +- `vec0` + +The application currently ships precompiled extension binaries for: + +- `darwin-amd64` +- `darwin-arm64` +- `linux-amd64` +- `linux-arm64` + +The app also uses several native or precompiled-native dependencies: + +- `exqlite`, via `ecto_sqlite3` +- `vix`, backed by libvips +- `dominant_colors`, via RustlerPrecompiled +- `mdex`, via RustlerPrecompiled +- `lumis`, via RustlerPrecompiled +- `typst`, via RustlerPrecompiled + +## Summary + +A Nerves deployment appears feasible as a read-only or mostly read-only edge deployment, but it should not be treated as a second writable production node. + +The central architectural constraint is that Litestream is not an application-level sync or conflict-resolution system. It is physical SQLite/WAL replication. If the Nerves device writes to its local database, those writes either remain local, conflict with read-replica/follow mode, or create a divergent database history. + +The safest complementary architecture is: + +1. The Nerves device keeps a local read model restored from production. +2. User writes are sent to the production app through an authenticated API. +3. Production remains the single SQLite writer. +4. Litestream brings the resulting production state back down to the device. +5. Offline-capable writes, if required, are handled with an application-level outbox and explicit conflict rules. + +## Native Dependency Challenges + +### Nerves Cross-Compilation Model + +Nerves sets environment variables that are intended to support cross-compiling C/C++ NIFs and ports, including `CC`, `CFLAGS`, `LDFLAGS`, `CROSSCOMPILE`, `NERVES_SDK_SYSROOT`, `TARGET_ARCH`, `TARGET_OS`, and `TARGET_ABI`. + +This is helpful for dependencies that use standard `elixir_make`, `cc_precompiler`, Rustler, or RustlerPrecompiled flows. It does not guarantee that every dependency will compile cleanly, especially if the dependency needs system libraries that are not present in the Nerves system. + +Source: https://hexdocs.pm/nerves/environment-variables.html + +### Exqlite / Ecto SQLite + +`ecto_sqlite3` depends on `exqlite`. Exqlite documents precompiled artifacts and native build options. It can also be forced to use a system SQLite through `EXQLITE_USE_SYSTEM`, `EXQLITE_SYSTEM_CFLAGS`, and `EXQLITE_SYSTEM_LDFLAGS`. + +Risk level: medium. + +Likely challenge: + +- Verifying that the selected Nerves target receives a compatible Exqlite NIF. +- Deciding whether to use bundled SQLite or a system SQLite from the Nerves image. +- Ensuring SQLite compile-time options still satisfy app requirements, especially FTS5 and loadable extensions. + +Source: https://hexdocs.pm/exqlite/readme.html + +### Vix / Libvips + +`vix` is the highest-risk native dependency in this application. + +Vix includes prebuilt binaries for macOS and Linux, and it can alternatively use a platform-provided libvips by setting: + +```sh +VIX_COMPILATION_MODE=PLATFORM_PROVIDED_LIBVIPS +``` + +However, on Nerves this may require a custom Nerves system that includes libvips and its development headers. Vix also has historically been tricky in cross-compilation contexts because it depends on libvips introspection during compilation. + +Risk level: high. + +Likely challenge: + +- Official Nerves systems may not include libvips. +- A custom Buildroot/Nerves system may be needed. +- Optional libvips codec support may differ from production. +- Image processing behavior may not match production unless libvips versions and build options are controlled. + +Sources: + +- https://hexdocs.pm/vix/readme.html +- https://hexdocs.pm/nerves/customizing-systems.html +- https://elixirforum.com/t/cross-compiling-with-nif-as-compilation-dependency/59821 + +### RustlerPrecompiled Dependencies + +RustlerPrecompiled supports Nerves-style target detection via `TARGET_ARCH`, `TARGET_ABI`, and `TARGET_OS`. Default targets include common ARM Linux triples such as: + +- `aarch64-unknown-linux-gnu` +- `aarch64-unknown-linux-musl` +- `arm-unknown-linux-gnueabihf` + +This is promising for `mdex`, `lumis`, `typst`, and `dominant_colors`, but every package still needs to publish compatible precompiled artifacts for the relevant target and NIF version. + +Risk level: medium. + +Specific concern: + +- `dominant_colors` is old and low-activity. It depends on `rustler_precompiled ~> 0.7`, and there is no guarantee that all current Nerves targets are covered. + +Sources: + +- https://hexdocs.pm/rustler_precompiled/RustlerPrecompiled.html +- https://hex.pm/packages/dominant_colors + +## SQLite Extension Challenges + +The app already includes `priv/sqlite_extensions/linux-arm64/unicode.so` and `vec0.so`. This is a good starting point, but it does not prove Nerves compatibility. + +Open questions: + +- Were the `linux-arm64` extensions built against assumptions compatible with the target Nerves system? +- Are they dynamically linked against glibc or musl expectations that differ from the Nerves target? +- Do they require symbols unavailable in the SQLite build used by Exqlite? +- Does the target allow SQLite loadable extensions as used by Ecto/Exqlite? + +The current platform detection only maps Linux `aarch64` to `linux-arm64`. That means 32-bit ARM targets such as Raspberry Pi Zero-class systems would fail unless new binaries and platform mapping were added. + +Risk level: medium for aarch64, high for arm32. + +Recommended spike: + +- Build a minimal Nerves firmware for the intended target. +- Boot it on hardware or QEMU if possible. +- Run only the repo extension tests: + - `SELECT vec_version()` + - Unicode extension behavior currently covered by `test/music_library/repo_test.exs` + +## Nerves Filesystem and Runtime Considerations + +Nerves root filesystems are read-only by default. Persistent data should live on the writable data partition, typically `/data`. + +This affects: + +- `DATABASE_PATH` +- `BACKGROUND_DATABASE_PATH` +- `TELEMETRY_DATABASE_PATH` +- Litestream restore target paths +- Litestream buffers, if using VFS write mode +- Temporary files used by image/PDF processing + +Source: https://hexdocs.pm/nerves/faq.html + +The current application expects runtime environment variables for production configuration. A Nerves deployment would need a target-specific runtime configuration story for: + +- `SECRET_KEY_BASE` +- `CLOAK_ENCRYPTION_KEY` +- `LOGIN_PASSWORD` +- `API_TOKEN` +- External API keys, if those features are enabled +- SQLite DB paths under `/data` + +## Litestream Findings + +### What Litestream Does + +Litestream is a streaming replication and disaster recovery tool for SQLite. It runs as a separate process and continuously copies WAL data to a replica store such as S3-compatible object storage. + +It is physical replication, not row-level sync. + +Source: https://litestream.io/how-it-works/ + +### Restore Mode + +`litestream restore` can restore a database from a replica. It can also restore to a specific transaction ID or timestamp. + +`restore -f` enables follow mode, continuously applying new data as it becomes available. Litestream explicitly states that the restored database in follow mode should only be opened read-only. + +Source: https://litestream.io/reference/restore/ + +Implication: + +- Follow mode is a good fit for a local read replica. +- It is not a fit for local writes. + +### VFS Read Replica + +The Litestream VFS can serve reads directly from object storage. It is read-only by default, fetches pages on demand, caches them, and polls for new LTX files. + +Important constraints: + +- Write attempts fail unless write mode is enabled. +- It requires a CGO-enabled SQLite VFS extension. +- It is network-latency sensitive. +- It needs contiguous LTX coverage. +- Initial snapshot availability is required. + +Source: https://litestream.io/reference/vfs/ + +Implication for Nerves: + +- The VFS is probably more complex than a restore-to-`/data` approach. +- Building and loading `litestream-vfs` on Nerves would be a separate native extension challenge. +- A fully restored local DB is likely simpler and more robust for embedded use. + +### VFS Write Mode + +Litestream VFS write mode allows writes to a remote-first database by buffering local changes and periodically uploading them. + +However, Litestream documents that write mode assumes a single writer. Multiple writers trigger conflict detection, not conflict prevention or automatic conflict resolution. + +Limitations include: + +- Single-writer assumption. +- Sync latency. +- Local write buffer disk usage. +- Crash window for unsynced buffered writes. +- Application-owned conflict resolution. + +Source: https://litestream.io/guides/vfs-write-mode/ + +Implication: + +- VFS write mode should not be used to let production and a Nerves device both write to the same logical database unless an external single-writer lock exists. +- It does not solve offline bidirectional sync. + +## What Happens If the Device Writes? + +There are several possible scenarios. + +### Scenario 1: Device Restores Production DB Once and Writes Locally + +The device diverges from production. + +Production does not receive those writes. Later restores or updates from production may overwrite or invalidate the local changes depending on the restore strategy. + +There is no automatic merge. + +### Scenario 2: Device Uses `restore -f` and Also Writes + +This is unsupported. Litestream says follow-mode restored databases should only be opened read-only. + +Expected outcome: + +- Conflict or corruption risk. +- Follow mode may be unable to apply incoming production changes cleanly. + +### Scenario 3: Device Uses Litestream VFS Read Replica and Writes + +By default, writes fail because the VFS is read-only. + +### Scenario 4: Device Uses VFS Write Mode Against Same Replica Path as Production + +This is a multiple-writer design. Litestream detects conflicts but does not reconcile them. + +Expected outcome: + +- Conflict errors. +- Need for application retry/rebase logic. +- Possible divergent generations or failed sync depending on exact mode. + +### Scenario 5: Device Sends Writes to Production API + +This is the cleanest model. + +Production remains the only writer to the production SQLite database. The device reads a replicated copy and submits commands to production when it needs to mutate state. + +This works well when the device has network connectivity. + +### Scenario 6: Device Supports Offline Writes + +This requires application-level sync. + +A likely shape: + +- Device stores local commands/events in a separate outbox database. +- Each command has an idempotency key. +- The production API accepts commands and performs validation/conflict checks. +- Production returns accepted/rejected/conflicted status. +- Device periodically pushes pending commands. +- Device refreshes its read model from Litestream. + +This is not something Litestream provides. + +## Data Architecture Recommendations + +### Recommended Baseline: Read Replica + Write-Through API + +Use Litestream to hydrate the main application database onto the device under `/data`. + +Only replicate the main app DB: + +- Replicate `MusicLibrary.Repo`: yes. +- Replicate `MusicLibrary.BackgroundRepo`: probably no. +- Replicate `MusicLibrary.TelemetryRepo`: no. + +Make the local app behave as read-only for production-backed tables, or route mutations through the production API. + +Benefits: + +- Avoids multi-writer SQLite. +- Avoids Litestream conflict semantics. +- Keeps production as source of truth. +- Preserves the current production deployment model. + +Challenges: + +- The UI may currently assume writes are local Ecto writes. +- Many LiveView workflows may need capability checks. +- Oban workers on device must be carefully disabled or narrowed. + +### Alternative: Device-Local Features Only + +Allow writes only to separate device-local tables/databases: + +- Device settings. +- Local telemetry. +- Local cache state. +- Sync metadata. +- Offline outbox. + +Keep production-replicated data read-only. + +This avoids mixing local-only rows into the production physical database copy. + +### Avoid: Bidirectional Litestream + +Do not attempt to use Litestream as a bidirectional sync system between production and Nerves. + +The core problem is that Litestream replicates database pages/WAL, not business operations. It has no understanding of records, notes, assets, chats, Oban jobs, timestamps, or conflict intent. + +## Application-Specific Concerns + +### Oban + +The app uses Oban heavily for imports, enrichments, external API calls, periodic refreshes, and maintenance. + +On a device: + +- Running production Oban jobs locally would be risky. +- Replicating the Oban database would be risky. +- Cron jobs that mutate the main database should likely be disabled. +- External API workers may be inappropriate if API secrets are not present. + +The Nerves deployment probably needs a distinct Oban configuration: + +- disabled entirely, or +- local-only queues, or +- only maintenance tasks that do not mutate production-backed data. + +### Secrets + +The main DB contains encrypted secrets via Cloak. If production data is restored to the device, encrypted rows come with it. + +Options: + +- Put the same `CLOAK_ENCRYPTION_KEY` on the device, which increases exposure risk. +- Do not replicate secrets. +- Keep replicated DB encrypted-but-unreadable for those features. +- Split secrets into a production-only database/table in a future architecture. + +### Assets + +Assets are stored as binary blobs in SQLite and content-addressed by hash. This is favorable for replication because immutable content has simpler conflict semantics. + +However: + +- Large assets affect restore time and storage wear. +- Cover refresh/image generation on-device may exercise Vix/libvips. + +### Embeddings and sqlite-vec + +The app uses OpenAI embeddings and `sqlite-vec`. + +Read-only similarity search on the device should be plausible if `vec0` loads. Generating embeddings on-device would require OpenAI credentials and write access, so it should probably stay production-side. + +### External APIs + +MusicBrainz, Last.fm, Discogs, Wikipedia, Brave Search, OpenAI, and Mailgun are integrated. + +On-device deployment should decide which of these are: + +- disabled, +- proxied through production, +- allowed directly from the device. + +For most production-backed data mutations, proxying through production is cleaner. + +## Proposed Research Spikes + +### Spike 1: Minimal Nerves Build + +Goal: + +- Determine whether the current dependency tree can build for the intended Nerves target. + +Checks: + +- `exqlite` NIF loads. +- RustlerPrecompiled dependencies load. +- Vix loads or fails with a clear path. +- Firmware size remains acceptable. + +### Spike 2: SQLite Extension Load Test + +Goal: + +- Verify `unicode.so` and `vec0.so` on target hardware. + +Checks: + +- `MusicLibrary.Repo.ensure_supported_platform!()` +- `SELECT vec_version()` +- representative FTS/unicode behavior +- vector similarity query + +### Spike 3: Read-Only Litestream Replica + +Goal: + +- Restore the production main DB to `/data`. +- Start app with the main repo opened read-only if possible. +- Confirm browsing/search/similarity workflows. + +Checks: + +- cold restore time +- incremental update lag +- disk usage +- behavior during network loss +- behavior during power loss + +### Spike 4: Write-Through Prototype + +Goal: + +- Pick one simple mutation, such as creating a note or updating wishlist state. +- Route it to production API instead of local DB. +- Observe how the replicated DB catches up. + +Checks: + +- UX during replication lag +- idempotency +- authentication +- error handling + +### Spike 5: Offline Outbox Design + +Goal: + +- Only if offline writes are a requirement. +- Design command schema, conflict policy, and reconciliation flow. + +Likely requirements: + +- globally unique command IDs +- entity version or `updated_at` checks +- tombstones for deletes +- conflict UI or deterministic server-side policy +- separation between read model DB and local outbox DB + +## Open Questions + +- Which hardware target is intended: Raspberry Pi 4/5 aarch64, Raspberry Pi Zero arm32, or something else? +- Does the device need to serve the full LiveView UI, or a constrained local UI? +- Is the device expected to work without internet? +- Are local writes required, or can all writes go through production? +- Should sensitive tables such as `secrets`, `chats`, and possibly API-derived metadata be present on the device? +- Is Vix/libvips actually needed on the device, or can image processing be production-only? +- Should the device run any Oban workers? +- Is the production S3-compatible bucket reachable from the device network? +- Is eventual consistency acceptable for the UI after write-through actions? + +## Conclusion + +The Nerves architecture is plausible if the device is treated as an edge read replica with carefully selected local behavior. The native dependency work is significant but bounded: Vix/libvips and SQLite extension ABI compatibility are the main technical spikes. + +The larger architectural decision is data ownership. Litestream can get production SQLite data onto the device and keep it fresh, but it should not be used as a bidirectional sync system. If the Nerves app must write production data, those writes should go through production or through an explicit application-level sync protocol.