cloud/music_library
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Run prettier on all relevant files

Browse Source
This commit is contained in:
Claudio Ortolina
2026-05-05 10:45:25 +01:00
parent 0b1f839091
commit 5888324cc7
17 changed files with 447 additions and 404 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+238 -235
View File
@@ -11,6 +11,7 @@ Uses SQLite (via `ecto_sqlite3`) with three databases: one for the app, one for
jobs (Oban), and one for telemetry metrics. All schemas use `binary_id` primary keys.
Key capabilities:
- Browse/search collected and wishlisted records
- Import metadata from MusicBrainz, enrich with Discogs/Wikipedia/Last.fm
- Scrobble tracks to Last.fm, import listening history
@@ -49,11 +50,11 @@ when `single_line_logging` is `true` — not a supervised process.
## Database & Repos
| Repo | DB file (dev) | Purpose |
|------|---------------|---------|
| `MusicLibrary.Repo` | `data/music_library_dev.db` | All application data |
| `MusicLibrary.BackgroundRepo` | `data/music_library_background_dev.db` | Oban job queue |
| `MusicLibrary.TelemetryRepo` | `data/music_library_telemetry_dev.db` | Telemetry metrics history (persistent across restarts) |
| Repo | DB file (dev) | Purpose |
| ----------------------------- | -------------------------------------- | ------------------------------------------------------ |
| `MusicLibrary.Repo` | `data/music_library_dev.db` | All application data |
| `MusicLibrary.BackgroundRepo` | `data/music_library_background_dev.db` | Oban job queue |
| `MusicLibrary.TelemetryRepo` | `data/music_library_telemetry_dev.db` | Telemetry metrics history (persistent across restarts) |
SQLite extensions loaded at runtime: `unicode`, `vec0` (vector search).
@@ -64,26 +65,27 @@ write to it directly; insert/update the `records` table instead.
## Schemas
| Schema | Table | PK | Key Fields |
|--------|-------|----|------------|
| `Records.Record` | `records` | `id` (binary_id) | title, type, format, cover_url, cover_hash, musicbrainz_id, genres[], release_date, purchased_at, dominant_colors[], embeds_many :artists |
| `Records.RecordEmbedding` | `record_embeddings` | `id` | embedding (Float32 vector), text_representation, belongs_to :record |
| `Records.SearchIndex` | `records_search_index` | `id` | FTS5 mirror of records (virtual, trigger-synced) |
| `Records.RecordRelease` | `record_releases` | none | record_id, release_id, cover_hash, purchased_at — read-only, no PK |
| `Records.ArtistRecord` | `artist_records` | composite | musicbrainz_id, record_id — DB view joining artists to records |
| `Artists.Artist` | — | — | Embedded schema (name, sort_name, musicbrainz_id, joinphrase) |
| `Artists.ArtistInfo` | `artist_infos` | `id` | musicbrainz_data, discogs_data, wikipedia_data, lastfm_data, image_data_hash |
| `Assets.Asset` | `assets` | `hash` (SHA256) | content (binary), format, properties (map) |
| `Notes.Note` | `notes` | `id` | entity (:record/:artist), content, musicbrainz_id |
| `RecordSets.RecordSet` | `record_sets` | `id` | name, description, has_many :items |
| `RecordSets.RecordSetItem` | `record_set_items` | `id` | position, belongs_to :record_set, belongs_to :record |
| `ScrobbleRules.ScrobbleRule` | `scrobble_rules` | `id` (integer) | type (:album/:artist), match_value, target_musicbrainz_id, enabled |
| `OnlineStoreTemplates.OnlineStoreTemplate` | `online_store_templates` | `id` | name, url_template, enabled |
| `Secrets.Secret` | `secrets` | `name` (string) | value (encrypted binary) |
| `Chats.Chat` | `chats` | `id` (binary_id) | entity (:record/:artist/:collection), musicbrainz_id, topic, has_many :messages |
| `Chats.Message` | `chat_messages` | `id` (binary_id) | role, content, position, belongs_to :chat |
| Schema | Table | PK | Key Fields |
| ------------------------------------------ | ------------------------ | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `Records.Record` | `records` | `id` (binary_id) | title, type, format, cover_url, cover_hash, musicbrainz_id, genres[], release_date, purchased_at, dominant_colors[], embeds_many :artists |
| `Records.RecordEmbedding` | `record_embeddings` | `id` | embedding (Float32 vector), text_representation, belongs_to :record |
| `Records.SearchIndex` | `records_search_index` | `id` | FTS5 mirror of records (virtual, trigger-synced) |
| `Records.RecordRelease` | `record_releases` | none | record_id, release_id, cover_hash, purchased_at — read-only, no PK |
| `Records.ArtistRecord` | `artist_records` | composite | musicbrainz_id, record_id — DB view joining artists to records |
| `Artists.Artist` | — | — | Embedded schema (name, sort_name, musicbrainz_id, joinphrase) |
| `Artists.ArtistInfo` | `artist_infos` | `id` | musicbrainz_data, discogs_data, wikipedia_data, lastfm_data, image_data_hash |
| `Assets.Asset` | `assets` | `hash` (SHA256) | content (binary), format, properties (map) |
| `Notes.Note` | `notes` | `id` | entity (:record/:artist), content, musicbrainz_id |
| `RecordSets.RecordSet` | `record_sets` | `id` | name, description, has_many :items |
| `RecordSets.RecordSetItem` | `record_set_items` | `id` | position, belongs_to :record_set, belongs_to :record |
| `ScrobbleRules.ScrobbleRule` | `scrobble_rules` | `id` (integer) | type (:album/:artist), match_value, target_musicbrainz_id, enabled |
| `OnlineStoreTemplates.OnlineStoreTemplate` | `online_store_templates` | `id` | name, url_template, enabled |
| `Secrets.Secret` | `secrets` | `name` (string) | value (encrypted binary) |
| `Chats.Chat` | `chats` | `id` (binary_id) | entity (:record/:artist/:collection), musicbrainz_id, topic, has_many :messages |
| `Chats.Message` | `chat_messages` | `id` (binary_id) | role, content, position, belongs_to :chat |
Last.fm schemas (separate, not Ecto-persisted to main DB):
- `LastFm.Track` — scrobbled tracks from Last.fm API responses
- `LastFm.Album`, `LastFm.Artist` — parsed API responses
@@ -91,78 +93,78 @@ Last.fm schemas (separate, not Ecto-persisted to main DB):
## Contexts (lib/music_library/)
| Context | Schemas | Responsibility |
|---------|---------|---------------|
| `Records` | Record, RecordEmbedding, SearchIndex | CRUD, search, import from MusicBrainz, cover/genre/color management, PubSub notifications |
| `Collection` | Record (via SearchIndex) | Querying collected records (purchased_at != nil), stats, collected artist IDs, collection summary for AI chat |
| `Wishlist` | Record (via SearchIndex) | Querying wishlisted records (purchased_at is nil) |
| `Artists` | ArtistInfo, ArtistRecord | Artist metadata from MusicBrainz/Discogs/Wikipedia/Last.fm, images, search |
| `Assets` | Asset | Binary asset storage (covers, artist images), cache tracking, pruning unreferenced assets |
| `Notes` | Note | Free-text notes for records and artists |
| `Chats` | Chat, Message, StreamProvider, RecordChat, ArtistChat, CollectionChat | Persistent AI chat conversations for records, artists, and the collection, streaming AI chat behaviour and entity-specific implementations |
| `RecordSets` | RecordSet, RecordSetItem | User-curated record groupings with ordering |
| `ScrobbleRules` | ScrobbleRule | Rules to remap Last.fm scrobble data to correct MusicBrainz IDs; searchable by match_value/target/description, orderable by alphabetical or inserted_at |
| `ScrobbleActivity` | — | Scrobbling releases/media/tracks to Last.fm |
| `ListeningStats` | (LastFm.Track, RecordRelease, ArtistRecord, ArtistInfo) | Scrobble persistence, refresh scheduling, listening analytics, track CRUD, search, listing: scrobble counts, artist play counts (from DB), recent activity, top albums/artists by period |
| `OnlineStoreTemplates` | OnlineStoreTemplate | URL templates for buying records online; searchable by name/description |
| `Errors` | ErrorTracker.Error, ErrorTracker.Occurrence | Read-only queries for production error data tracked by ErrorTracker; filtered listing with pagination, single error with preloaded occurrences and computed counts |
| `Search` | (cross-context) | Universal search dispatcher across collection, wishlist, artists, record sets (delegates to domain contexts) |
| `Secrets` | Secret | Encrypted key-value storage (CRUD + delete) |
| `BarcodeScan` | (Result struct) | Barcode → MusicBrainz lookup workflow, async batch import for multiple new records |
| `Maintenance` | (Oban.Job, LastFm.Track) | Background job monitoring, database vacuum/optimize, scrobble data quality diagnostics |
| Context | Schemas | Responsibility |
| ---------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Records` | Record, RecordEmbedding, SearchIndex | CRUD, search, import from MusicBrainz, cover/genre/color management, PubSub notifications |
| `Collection` | Record (via SearchIndex) | Querying collected records (purchased_at != nil), stats, collected artist IDs, collection summary for AI chat |
| `Wishlist` | Record (via SearchIndex) | Querying wishlisted records (purchased_at is nil) |
| `Artists` | ArtistInfo, ArtistRecord | Artist metadata from MusicBrainz/Discogs/Wikipedia/Last.fm, images, search |
| `Assets` | Asset | Binary asset storage (covers, artist images), cache tracking, pruning unreferenced assets |
| `Notes` | Note | Free-text notes for records and artists |
| `Chats` | Chat, Message, StreamProvider, RecordChat, ArtistChat, CollectionChat | Persistent AI chat conversations for records, artists, and the collection, streaming AI chat behaviour and entity-specific implementations |
| `RecordSets` | RecordSet, RecordSetItem | User-curated record groupings with ordering |
| `ScrobbleRules` | ScrobbleRule | Rules to remap Last.fm scrobble data to correct MusicBrainz IDs; searchable by match_value/target/description, orderable by alphabetical or inserted_at |
| `ScrobbleActivity` | — | Scrobbling releases/media/tracks to Last.fm |
| `ListeningStats` | (LastFm.Track, RecordRelease, ArtistRecord, ArtistInfo) | Scrobble persistence, refresh scheduling, listening analytics, track CRUD, search, listing: scrobble counts, artist play counts (from DB), recent activity, top albums/artists by period |
| `OnlineStoreTemplates` | OnlineStoreTemplate | URL templates for buying records online; searchable by name/description |
| `Errors` | ErrorTracker.Error, ErrorTracker.Occurrence | Read-only queries for production error data tracked by ErrorTracker; filtered listing with pagination, single error with preloaded occurrences and computed counts |
| `Search` | (cross-context) | Universal search dispatcher across collection, wishlist, artists, record sets (delegates to domain contexts) |
| `Secrets` | Secret | Encrypted key-value storage (CRUD + delete) |
| `BarcodeScan` | (Result struct) | Barcode → MusicBrainz lookup workflow, async batch import for multiple new records |
| `Maintenance` | (Oban.Job, LastFm.Track) | Background job monitoring, database vacuum/optimize, scrobble data quality diagnostics |
---
## Business Logic Modules
| Module | Purpose |
|--------|---------|
| `Records.SearchParser` | Parses search syntax: `artist:X`, `album:X`, `genre:"Y"`, `format:cd`, `type:album`, `purchase_year:2024`, `release_year:2024`, free text |
| `ListeningStats.SearchParser` | Parses scrobbled tracks search syntax: `record:X`, `album_mbid:X`, `artist_mbid:X`, `artist:X`, `album:X`, `track:X`, free text |
| `Records.Similarity` | Embedding generation and async enqueue (OpenAI, enriched with Last.fm tags, skips API call when text representation unchanged), artist-cascade regeneration when upstream metadata changes, cosine-distance search (sqlite-vec) |
| `Records.TracklistPdf` | Generates 120mm×120mm PDF tracklist from record + release data (Typst) |
| `Batch` | Generic batch runner: stream + transaction + error accumulation |
| `Records.Batch` | Batch operations: refresh all MusicBrainz data, generate all embeddings (uses `Batch`) |
| `Artists.Batch` | Batch refresh: MusicBrainz, Discogs, Wikipedia, Last.fm for all artists (uses `Batch`) |
| `Req.RateLimiter` | ETS-backed Req request step enforcing per-API minimum intervals between requests |
| `Req.RateLimiter.Clock` | Behaviour for time operations (allows test clock injection) |
| `Req.RateLimiter.SystemClock` | Real clock implementation using System.monotonic_time |
| `Assets.Cache` | ETS-based asset cache with TTL (7-day TTL, TTL-only invalidation since assets are content-addressable and immutable) |
| `Assets.Image` / `Assets.Transform` | Image processing via Vix (libvips) |
| `Colors.Extractor` | Behaviour for dominant color extraction (configurable, allows test stubbing) |
| `Colors.KMeansExtractor` | Color extraction via K-Means clustering (dominant_colors library), implements `Colors.Extractor` |
| `Chats.StreamProvider` | Behaviour for streaming AI chat (`stream_response/3` callback) |
| `Chats.RecordChat` | Chat implementation for records (OpenAI streaming, web search enabled) |
| `Chats.ArtistChat` | Chat implementation for artists (OpenAI streaming, uses Wikipedia/artist context) |
| `Chats.CollectionChat` | Chat implementation for the collection (OpenAI streaming via gpt-5.1, uses collection summary as context) |
| `Chats.Prompt` | Builds complete chat prompts by interpolating identity, content, and approach guidelines |
| `Country` | Country code (alpha-2, alpha-3, subdivision, IETF) to flag emoji conversion |
| `ErrorTracker.ErrorNotifier` | GenServer: attaches to ErrorTracker telemetry, skips muted errors, throttles repeated errors, dispatches email notifications |
| `ErrorTracker.ErrorNotifier.Email` | Builds and sends Swoosh error notification emails with stack trace formatting |
| `ErrorIgnorer` | ErrorTracker.Ignorer implementation: filters non-actionable errors (e.g., NoRouteError from bot scanners) |
| `MusicLibrary.ErrorResponse` | Behaviour for structured API error responses (`retryable?/1`, `retry_delay_seconds/1`) — per-API `ErrorResponse` modules implement this |
| `MusicLibrary.HttpError` | Default HTTP status → kind mapping (`:rate_limit`, `:server_error`, `:timeout`, `:auth_error`, `:not_found`, `:client_error`, `:unknown`) used as baseline by per-API `ErrorResponse` modules |
| `MusicLibrary.RetryDelay` | Parses and clamps provider retry/reset headers into Oban snooze delays for structured API errors |
| `MusicLibrary.Worker.ErrorHandler` | Translates per-API `ErrorResponse` structs into Oban tuples — `{:snooze, seconds}` for retryable, `{:cancel, reason}` for permanent |
| `MusicLibraryWeb.RecordsOnThisDayEmail` | Builds and sends daily "records on this day" email with cover images, anniversary styling |
| `MusicLibrary.Mailer` | Swoosh mailer (Mailgun in prod, local adapter in dev) |
| `FormatNumber` | Number formatting utility |
| `QueryReporter` | Dev-only Ecto telemetry reporter: captures executed SQL to a log file with interpolated params, source locations, and timing. Activated at runtime via `start/1` / `stop/0` |
| `MusicLibrary.Logger.SingleLineFormatter` | Production-only Logger.Formatter safety net: replaces embedded newlines (`\n`) with escaped `\\n` in all log messages, ensuring every physical log line is exactly one log event |
| Module | Purpose |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Records.SearchParser` | Parses search syntax: `artist:X`, `album:X`, `genre:"Y"`, `format:cd`, `type:album`, `purchase_year:2024`, `release_year:2024`, free text |
| `ListeningStats.SearchParser` | Parses scrobbled tracks search syntax: `record:X`, `album_mbid:X`, `artist_mbid:X`, `artist:X`, `album:X`, `track:X`, free text |
| `Records.Similarity` | Embedding generation and async enqueue (OpenAI, enriched with Last.fm tags, skips API call when text representation unchanged), artist-cascade regeneration when upstream metadata changes, cosine-distance search (sqlite-vec) |
| `Records.TracklistPdf` | Generates 120mm×120mm PDF tracklist from record + release data (Typst) |
| `Batch` | Generic batch runner: stream + transaction + error accumulation |
| `Records.Batch` | Batch operations: refresh all MusicBrainz data, generate all embeddings (uses `Batch`) |
| `Artists.Batch` | Batch refresh: MusicBrainz, Discogs, Wikipedia, Last.fm for all artists (uses `Batch`) |
| `Req.RateLimiter` | ETS-backed Req request step enforcing per-API minimum intervals between requests |
| `Req.RateLimiter.Clock` | Behaviour for time operations (allows test clock injection) |
| `Req.RateLimiter.SystemClock` | Real clock implementation using System.monotonic_time |
| `Assets.Cache` | ETS-based asset cache with TTL (7-day TTL, TTL-only invalidation since assets are content-addressable and immutable) |
| `Assets.Image` / `Assets.Transform` | Image processing via Vix (libvips) |
| `Colors.Extractor` | Behaviour for dominant color extraction (configurable, allows test stubbing) |
| `Colors.KMeansExtractor` | Color extraction via K-Means clustering (dominant_colors library), implements `Colors.Extractor` |
| `Chats.StreamProvider` | Behaviour for streaming AI chat (`stream_response/3` callback) |
| `Chats.RecordChat` | Chat implementation for records (OpenAI streaming, web search enabled) |
| `Chats.ArtistChat` | Chat implementation for artists (OpenAI streaming, uses Wikipedia/artist context) |
| `Chats.CollectionChat` | Chat implementation for the collection (OpenAI streaming via gpt-5.1, uses collection summary as context) |
| `Chats.Prompt` | Builds complete chat prompts by interpolating identity, content, and approach guidelines |
| `Country` | Country code (alpha-2, alpha-3, subdivision, IETF) to flag emoji conversion |
| `ErrorTracker.ErrorNotifier` | GenServer: attaches to ErrorTracker telemetry, skips muted errors, throttles repeated errors, dispatches email notifications |
| `ErrorTracker.ErrorNotifier.Email` | Builds and sends Swoosh error notification emails with stack trace formatting |
| `ErrorIgnorer` | ErrorTracker.Ignorer implementation: filters non-actionable errors (e.g., NoRouteError from bot scanners) |
| `MusicLibrary.ErrorResponse` | Behaviour for structured API error responses (`retryable?/1`, `retry_delay_seconds/1`) — per-API `ErrorResponse` modules implement this |
| `MusicLibrary.HttpError` | Default HTTP status → kind mapping (`:rate_limit`, `:server_error`, `:timeout`, `:auth_error`, `:not_found`, `:client_error`, `:unknown`) used as baseline by per-API `ErrorResponse` modules |
| `MusicLibrary.RetryDelay` | Parses and clamps provider retry/reset headers into Oban snooze delays for structured API errors |
| `MusicLibrary.Worker.ErrorHandler` | Translates per-API `ErrorResponse` structs into Oban tuples — `{:snooze, seconds}` for retryable, `{:cancel, reason}` for permanent |
| `MusicLibraryWeb.RecordsOnThisDayEmail` | Builds and sends daily "records on this day" email with cover images, anniversary styling |
| `MusicLibrary.Mailer` | Swoosh mailer (Mailgun in prod, local adapter in dev) |
| `FormatNumber` | Number formatting utility |
| `QueryReporter` | Dev-only Ecto telemetry reporter: captures executed SQL to a log file with interpolated params, source locations, and timing. Activated at runtime via `start/1` / `stop/0` |
| `MusicLibrary.Logger.SingleLineFormatter` | Production-only Logger.Formatter safety net: replaces embedded newlines (`\n`) with escaped `\\n` in all log messages, ensuring every physical log line is exactly one log event |
---
## External API Integrations
| Module | API | Rate limit | Purpose |
|--------|-----|-----------|---------|
| `MusicBrainz` / `MusicBrainz.API` | musicbrainz.org | 1000 ms | Release/artist metadata, search |
| `LastFm` / `LastFm.API` | last.fm | 500 ms | Scrobbling, listening history, artist info (tags, similar artists), user profile/session validation |
| `Discogs` / `Discogs.API` | discogs.com | 2000 ms | Artist profiles, images |
| `Wikipedia` / `Wikipedia.API` | wikipedia.org | 1000 ms | Artist biographies |
| `BraveSearch` / `BraveSearch.API` | search.brave.com | 1000 ms | Cover art and artist image search |
| `OpenAI` / `OpenAI.API` | api.openai.com | 250 ms | Text embeddings for similarity, streaming chat via Responses API (gpt-4.1/gpt-5.1 + web search) |
| `MusicLibrary.Mailer` | Mailgun (via Swoosh) | — | Transactional email delivery (error notifications, daily digest) |
| Module | API | Rate limit | Purpose |
| --------------------------------- | -------------------- | ---------- | --------------------------------------------------------------------------------------------------- |
| `MusicBrainz` / `MusicBrainz.API` | musicbrainz.org | 1000 ms | Release/artist metadata, search |
| `LastFm` / `LastFm.API` | last.fm | 500 ms | Scrobbling, listening history, artist info (tags, similar artists), user profile/session validation |
| `Discogs` / `Discogs.API` | discogs.com | 2000 ms | Artist profiles, images |
| `Wikipedia` / `Wikipedia.API` | wikipedia.org | 1000 ms | Artist biographies |
| `BraveSearch` / `BraveSearch.API` | search.brave.com | 1000 ms | Cover art and artist image search |
| `OpenAI` / `OpenAI.API` | api.openai.com | 250 ms | Text embeddings for similarity, streaming chat via Responses API (gpt-4.1/gpt-5.1 + web search) |
| `MusicLibrary.Mailer` | Mailgun (via Swoosh) | — | Transactional email delivery (error notifications, daily digest) |
Each has a `Config` module reading from application env. All HTTP clients use `Req` with
per-API rate limiting (`Req.RateLimiter`, ETS-backed). In tests, all HTTP calls are
@@ -183,73 +185,73 @@ HTTP 429 into `:rate_limit` vs `:auth_error` by reading the body `code`
### Queues
| Queue | Concurrency | Purpose |
|-------|-------------|---------|
| `default` | 10 | General async tasks |
| `heavy_writes` | 1 | DB-intensive or serialized operations |
| `music_brainz` | 1 | MusicBrainz calls (rate-limited at Req layer via `Req.RateLimiter`) |
| `discogs` | 1 | Discogs calls (rate-limited at Req layer via `Req.RateLimiter`) |
| `wikipedia` | 1 | Wikipedia calls |
| `last_fm` | 1 | Last.fm calls (rate-limited at Req layer via `Req.RateLimiter`) |
| Queue | Concurrency | Purpose |
| -------------- | ----------- | ------------------------------------------------------------------- |
| `default` | 10 | General async tasks |
| `heavy_writes` | 1 | DB-intensive or serialized operations |
| `music_brainz` | 1 | MusicBrainz calls (rate-limited at Req layer via `Req.RateLimiter`) |
| `discogs` | 1 | Discogs calls (rate-limited at Req layer via `Req.RateLimiter`) |
| `wikipedia` | 1 | Wikipedia calls |
| `last_fm` | 1 | Last.fm calls (rate-limited at Req layer via `Req.RateLimiter`) |
### Plugins (prod)
| Plugin | Config | Purpose |
|--------|--------|---------|
| `Oban.Plugins.Pruner` | `max_age: 43200` (12h) | Prune completed/cancelled/discarded jobs older than 12 hours |
| `Oban.Plugins.Reindexer` | `schedule: "@weekly"` | Weekly reindex of Oban tables for query performance |
| `Oban.Plugins.Cron` | `timezone: "Europe/London"` | Scheduled recurring workers (see Cron Workers table) |
| Plugin | Config | Purpose |
| ------------------------ | --------------------------- | ------------------------------------------------------------ |
| `Oban.Plugins.Pruner` | `max_age: 43200` (12h) | Prune completed/cancelled/discarded jobs older than 12 hours |
| `Oban.Plugins.Reindexer` | `schedule: "@weekly"` | Weekly reindex of Oban tables for query performance |
| `Oban.Plugins.Cron` | `timezone: "Europe/London"` | Scheduled recurring workers (see Cron Workers table) |
### On-Demand Workers
| Worker | Queue | Trigger |
|--------|-------|---------|
| `FetchArtistInfo` | default | Artist page visit / import (also fetches Last.fm data inline) |
| `FetchArtistLastFmData` | last_fm | Manual / batch |
| `FetchArtistImage` | heavy_writes | Artist info fetched |
| `RefreshCover` | heavy_writes | Manual action / import |
| `ImportFromMusicbrainzRelease` | music_brainz | Barcode scan batch import (2+ new records) |
| Worker | Queue | Trigger |
| ----------------------------------- | ------------ | --------------------------------------------------------------------------- |
| `FetchArtistInfo` | default | Artist page visit / import (also fetches Last.fm data inline) |
| `FetchArtistLastFmData` | last_fm | Manual / batch |
| `FetchArtistImage` | heavy_writes | Artist info fetched |
| `RefreshCover` | heavy_writes | Manual action / import |
| `ImportFromMusicbrainzRelease` | music_brainz | Barcode scan batch import (2+ new records) |
| `ImportFromMusicbrainzReleaseGroup` | music_brainz | Cart-style multi-record import in AddRecord component (2+ records selected) |
| `PopulateGenres` | heavy_writes | Manual action (chains → GenerateRecordEmbedding) |
| `GenerateRecordEmbedding` | heavy_writes | Manual / after genre population (delegates to Similarity, skips unchanged) |
| `RecordRefreshMusicBrainzData` | music_brainz | Manual / batch |
| `ArtistRefreshMusicBrainzData` | music_brainz | Manual / batch |
| `ArtistRefreshDiscogsData` | discogs | Manual / batch |
| `ArtistRefreshWikipediaData` | wikipedia | Manual / batch |
| `PruneArtistInfo` | default | Record deleted (cleanup orphaned artist data) |
| `RecordRefreshAllMusicBrainzData` | music_brainz | Manual / cron (bulk refresh via Records.Batch) |
| `RecordGenerateAllEmbeddings` | heavy_writes | Manual / cron (bulk generate via Records.Batch) |
| `ArtistRefreshAllMusicBrainzData` | music_brainz | Manual / cron (bulk refresh via Artists.Batch) |
| `ArtistRefreshAllDiscogsData` | discogs | Manual / cron (bulk refresh via Artists.Batch) |
| `ArtistRefreshAllWikipediaData` | wikipedia | Manual / cron (bulk refresh via Artists.Batch) |
| `RefreshScrobbles` | last_fm | Cron / manual (fetch recent Last.fm scrobbles) |
| `BackfillScrobbledTracks` | heavy_writes | Manual (self-chaining batch import) |
| `SendRecordsOnThisDayEmail` | default | Cron (daily "records on this day" email) |
| `PopulateGenres` | heavy_writes | Manual action (chains → GenerateRecordEmbedding) |
| `GenerateRecordEmbedding` | heavy_writes | Manual / after genre population (delegates to Similarity, skips unchanged) |
| `RecordRefreshMusicBrainzData` | music_brainz | Manual / batch |
| `ArtistRefreshMusicBrainzData` | music_brainz | Manual / batch |
| `ArtistRefreshDiscogsData` | discogs | Manual / batch |
| `ArtistRefreshWikipediaData` | wikipedia | Manual / batch |
| `PruneArtistInfo` | default | Record deleted (cleanup orphaned artist data) |
| `RecordRefreshAllMusicBrainzData` | music_brainz | Manual / cron (bulk refresh via Records.Batch) |
| `RecordGenerateAllEmbeddings` | heavy_writes | Manual / cron (bulk generate via Records.Batch) |
| `ArtistRefreshAllMusicBrainzData` | music_brainz | Manual / cron (bulk refresh via Artists.Batch) |
| `ArtistRefreshAllDiscogsData` | discogs | Manual / cron (bulk refresh via Artists.Batch) |
| `ArtistRefreshAllWikipediaData` | wikipedia | Manual / cron (bulk refresh via Artists.Batch) |
| `RefreshScrobbles` | last_fm | Cron / manual (fetch recent Last.fm scrobbles) |
| `BackfillScrobbledTracks` | heavy_writes | Manual (self-chaining batch import) |
| `SendRecordsOnThisDayEmail` | default | Cron (daily "records on this day" email) |
### Cron Workers
| Schedule | Worker | Queue |
|----------|--------|-------|
| Every 12h | `ApplyScrobbleRules` | heavy_writes |
| Every 12h | `PruneAssetCache` | default |
| Daily 2 AM | `PruneAssets` | default |
| Daily 3 AM | `RepoVacuum` | heavy_writes |
| Daily 4 AM | `RepoOptimize` | heavy_writes |
| Monthly 1st, 6 AM | `RecordRefreshAllMusicBrainzData` | music_brainz |
| Monthly 1st, 7 AM | `RecordGenerateAllEmbeddings` | heavy_writes |
| Monthly 1st, 8 AM | `ArtistRefreshAllMusicBrainzData` | music_brainz |
| Monthly 1st, 9 AM | `ArtistRefreshAllDiscogsData` | discogs |
| Monthly 1st, 10 AM | `ArtistRefreshAllWikipediaData` | wikipedia |
| Daily 7 AM | `SendRecordsOnThisDayEmail` | default |
| Every 5 min | `RefreshScrobbles` | last_fm |
| Schedule | Worker | Queue |
| ------------------ | --------------------------------- | ------------ |
| Every 12h | `ApplyScrobbleRules` | heavy_writes |
| Every 12h | `PruneAssetCache` | default |
| Daily 2 AM | `PruneAssets` | default |
| Daily 3 AM | `RepoVacuum` | heavy_writes |
| Daily 4 AM | `RepoOptimize` | heavy_writes |
| Monthly 1st, 6 AM | `RecordRefreshAllMusicBrainzData` | music_brainz |
| Monthly 1st, 7 AM | `RecordGenerateAllEmbeddings` | heavy_writes |
| Monthly 1st, 8 AM | `ArtistRefreshAllMusicBrainzData` | music_brainz |
| Monthly 1st, 9 AM | `ArtistRefreshAllDiscogsData` | discogs |
| Monthly 1st, 10 AM | `ArtistRefreshAllWikipediaData` | wikipedia |
| Daily 7 AM | `SendRecordsOnThisDayEmail` | default |
| Every 5 min | `RefreshScrobbles` | last_fm |
---
## PubSub Topics
| PubSub | Topic Pattern | Message | Used By |
|--------|---------------|---------|---------|
| `:music_library` | `"records:#{id}"` | `{:update, record}` | CollectionLive.Show, WishlistLive.Show — real-time record updates |
| PubSub | Topic Pattern | Message | Used By |
| ---------------- | -------------------------- | ------------------- | ------------------------------------------------------------------ |
| `:music_library` | `"records:#{id}"` | `{:update, record}` | CollectionLive.Show, WishlistLive.Show — real-time record updates |
| `:music_library` | `"listening_stats:update"` | `%{track_count: n}` | StatsLive.Index, ScrobbledTracksLive.Index — new scrobbles arrived |
---
@@ -259,91 +261,92 @@ HTTP 429 into `:rate_limit` vs `:auth_error` by reading the body `code`
### Router Structure
All authenticated routes live inside a single `live_session` with three `on_mount` hooks:
- `StaticAssets` — detects app updates, shows toast
- `GetTimezone` — reads timezone from connect params
- `ShowToast` — enables `put_toast!/2` in LiveViews
### LiveViews
| LiveView | Route | Purpose |
|----------|-------|---------|
| `StatsLive.Index` | `/` | Dashboard: counts, recent activity, records on this day |
| `CollectionLive.Index` | `/collection` | Browse/search collected records (grid/list, paginated) |
| `CollectionLive.Show` | `/collection/:id` | Record detail: metadata, scrobbles, similar, colors |
| `WishlistLive.Index` | `/wishlist` | Browse/search wishlisted records |
| `WishlistLive.Show` | `/wishlist/:id` | Wishlist record detail with store links |
| `ArtistLive.Show` | `/artists/:musicbrainz_id` | Artist bio, discography, similar artists |
| `RecordSetLive.Index` | `/record-sets` | Browse/manage curated record sets |
| `RecordSetLive.Show` | `/record-sets/:id` | Set detail with reorderable items |
| `ScrobbleLive.Index` | `/scrobble` | Search MusicBrainz release groups to scrobble |
| `ScrobbleLive.ReleaseGroupShow` | `/scrobble/:rg_id` | List releases within a release group |
| `ScrobbleLive.ReleaseShow` | `/scrobble/:rg_id/releases/:release_id` | Select tracks and scrobble (uses `Release` live_component) |
| `ScrobbledTracksLive.Index` | `/scrobbled-tracks` | Browse/search Last.fm history |
| `ScrobbleRulesLive.Index` | `/scrobble-rules` | Browse/search/sort scrobble remapping rules (paginated, 50 per page) |
| `OnlineStoreTemplateLive.Index` | `/online-store-templates` | Manage store URL templates |
| `MaintenanceLive.Index` | `/maintenance` | Admin: batch jobs, DB maintenance, Last.fm connection |
| LiveView | Route | Purpose |
| ------------------------------- | --------------------------------------- | -------------------------------------------------------------------- |
| `StatsLive.Index` | `/` | Dashboard: counts, recent activity, records on this day |
| `CollectionLive.Index` | `/collection` | Browse/search collected records (grid/list, paginated) |
| `CollectionLive.Show` | `/collection/:id` | Record detail: metadata, scrobbles, similar, colors |
| `WishlistLive.Index` | `/wishlist` | Browse/search wishlisted records |
| `WishlistLive.Show` | `/wishlist/:id` | Wishlist record detail with store links |
| `ArtistLive.Show` | `/artists/:musicbrainz_id` | Artist bio, discography, similar artists |
| `RecordSetLive.Index` | `/record-sets` | Browse/manage curated record sets |
| `RecordSetLive.Show` | `/record-sets/:id` | Set detail with reorderable items |
| `ScrobbleLive.Index` | `/scrobble` | Search MusicBrainz release groups to scrobble |
| `ScrobbleLive.ReleaseGroupShow` | `/scrobble/:rg_id` | List releases within a release group |
| `ScrobbleLive.ReleaseShow` | `/scrobble/:rg_id/releases/:release_id` | Select tracks and scrobble (uses `Release` live_component) |
| `ScrobbledTracksLive.Index` | `/scrobbled-tracks` | Browse/search Last.fm history |
| `ScrobbleRulesLive.Index` | `/scrobble-rules` | Browse/search/sort scrobble remapping rules (paginated, 50 per page) |
| `OnlineStoreTemplateLive.Index` | `/online-store-templates` | Manage store URL templates |
| `MaintenanceLive.Index` | `/maintenance` | Admin: batch jobs, DB maintenance, Last.fm connection |
### LiveComponents
| Component | Used In | Purpose |
|-----------|---------|---------|
| `RecordForm` | Collection/Wishlist (edit) | Record editing: cover search, genre autocomplete, color picker, file upload |
| `ArtistLive.Form` | ArtistLive.Show | Edit artist image (upload + Brave image search) |
| `RecordSetLive.Form` | RecordSetLive.Index | Create/edit record set |
| `RecordSetLive.RecordPicker` | RecordSetLive.Show | Search and add records to set |
| `ScrobbledTracksLive.Form` | ScrobbledTracksLive.Index | Edit scrobbled track |
| `ScrobbleRulesLive.Form` | ScrobbleRulesLive.Index | Create/edit scrobble rule |
| `ScrobbleRulePicker` | ScrobbledTracksLive.Index, StatsLive.Index | Search records and create scrobble rules inline |
| `OnlineStoreTemplateLive.Form` | OnlineStoreTemplateLive.Index | Create/edit store template |
| `StatsLive.TopByPeriod` | StatsLive.TopAlbums, StatsLive.TopArtists | Generic period-tabbed stats display (7d, 30d, 90d, 1y, all-time) |
| `StatsLive.TopAlbums` | StatsLive.Index | Top albums by period (uses TopByPeriod) |
| `StatsLive.TopArtists` | StatsLive.Index | Top artists by period (uses TopByPeriod) |
| `UniversalSearchLive.Index` | Layout (global) | Cmd+K search modal with quick actions (add to wishlist/collection, scrobble, collection chat) |
| `Chat` | CollectionLive.Index, CollectionLive.Show, WishlistLive.Show, ArtistLive.Show | AI chat sheet (OpenAI streaming, configurable per entity) |
| `Notes` | CollectionLive.Show, WishlistLive.Show, ArtistLive.Show | Markdown note rendering and editing |
| `AddRecord` | CollectionLive.Index, WishlistLive.Index | MusicBrainz import interface |
| `BarcodeScanner` | CollectionLive.Index | Barcode scanning UI (uses barcode-detector JS) |
| `Release` | CollectionLive.Show, ScrobbleLive.ReleaseShow | MusicBrainz release display with scrobble (form-based with auto-recovery) |
| Component | Used In | Purpose |
| ------------------------------ | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| `RecordForm` | Collection/Wishlist (edit) | Record editing: cover search, genre autocomplete, color picker, file upload |
| `ArtistLive.Form` | ArtistLive.Show | Edit artist image (upload + Brave image search) |
| `RecordSetLive.Form` | RecordSetLive.Index | Create/edit record set |
| `RecordSetLive.RecordPicker` | RecordSetLive.Show | Search and add records to set |
| `ScrobbledTracksLive.Form` | ScrobbledTracksLive.Index | Edit scrobbled track |
| `ScrobbleRulesLive.Form` | ScrobbleRulesLive.Index | Create/edit scrobble rule |
| `ScrobbleRulePicker` | ScrobbledTracksLive.Index, StatsLive.Index | Search records and create scrobble rules inline |
| `OnlineStoreTemplateLive.Form` | OnlineStoreTemplateLive.Index | Create/edit store template |
| `StatsLive.TopByPeriod` | StatsLive.TopAlbums, StatsLive.TopArtists | Generic period-tabbed stats display (7d, 30d, 90d, 1y, all-time) |
| `StatsLive.TopAlbums` | StatsLive.Index | Top albums by period (uses TopByPeriod) |
| `StatsLive.TopArtists` | StatsLive.Index | Top artists by period (uses TopByPeriod) |
| `UniversalSearchLive.Index` | Layout (global) | Cmd+K search modal with quick actions (add to wishlist/collection, scrobble, collection chat) |
| `Chat` | CollectionLive.Index, CollectionLive.Show, WishlistLive.Show, ArtistLive.Show | AI chat sheet (OpenAI streaming, configurable per entity) |
| `Notes` | CollectionLive.Show, WishlistLive.Show, ArtistLive.Show | Markdown note rendering and editing |
| `AddRecord` | CollectionLive.Index, WishlistLive.Index | MusicBrainz import interface |
| `BarcodeScanner` | CollectionLive.Index | Barcode scanning UI (uses barcode-detector JS) |
| `Release` | CollectionLive.Show, ScrobbleLive.ReleaseShow | MusicBrainz release display with scrobble (form-based with auto-recovery) |
### Shared Component Modules (lib/music_library_web/components/)
| Module | Purpose |
|--------|---------|
| `CoreComponents` | Forms, buttons, icons, tables, flash messages |
| `Layouts` | Application layout templates, navigation components (`dropdown_nav/1`) |
| `RecordComponents` | Record cards, cover images, artist images, labels, grids, release status icon badge, shared show-page sections (title, external links, genres, releases, timestamps, debug) |
| `ChartComponents` | Charts for stats dashboard |
| `StatsComponents` | Stats dashboard widgets (`section/1` layout, counters, album preview, records on this day) |
| `ScrobbleComponents` | Scrobble activity displays: status badges, import dropdowns, metadata tooltips, and record-matching UI |
| `SearchComponents` | Search result rendering |
| `CartComponents` | `cart_sidebar/1` — shared cart aside used by `AddRecord` and `BarcodeScanner` |
| `Pagination` | Pagination UI and logic |
| Module | Purpose |
| -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CoreComponents` | Forms, buttons, icons, tables, flash messages |
| `Layouts` | Application layout templates, navigation components (`dropdown_nav/1`) |
| `RecordComponents` | Record cards, cover images, artist images, labels, grids, release status icon badge, shared show-page sections (title, external links, genres, releases, timestamps, debug) |
| `ChartComponents` | Charts for stats dashboard |
| `StatsComponents` | Stats dashboard widgets (`section/1` layout, counters, album preview, records on this day) |
| `ScrobbleComponents` | Scrobble activity displays: status badges, import dropdowns, metadata tooltips, and record-matching UI |
| `SearchComponents` | Search result rendering |
| `CartComponents` | `cart_sidebar/1` — shared cart aside used by `AddRecord` and `BarcodeScanner` |
| `Pagination` | Pagination UI and logic |
### Web Utility Modules (lib/music_library_web/)
| Module | Purpose |
|--------|---------|
| `ErrorMessages` | Maps internal error terms (atoms, structs) to user-friendly gettext strings via `friendly_message/1` |
| `Markdown` | Markdown-to-HTML conversion (MDEx with ammonia sanitization) with `[[double bracket]]` link syntax and streaming document support for chat |
| `Duration` | Milliseconds to human-readable duration formatting |
| `Auth` | Authentication plugs: login password check, API token validation, session enforcement |
| `ArtistLive.Biography` | Artist biography building/rendering from Wikipedia and Last.fm data |
| `LiveHelpers.Params` | URL query param parsing: pagination, search query, sort order, display mode, fallback index |
| `LiveHelpers.IndexActions` | Shared index page logic (search, pagination, import, delete, display mode) for Collection/Wishlist index pages, parameterized by config map |
| `LiveHelpers.RecordActions` | Shared record action handlers (refresh cover, genres, embeddings, MusicBrainz data) for Collection/Wishlist show pages |
| — | Logster v2 handles `[:phoenix, :socket_connected]` telemetry — not a custom module |
| Module | Purpose |
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `ErrorMessages` | Maps internal error terms (atoms, structs) to user-friendly gettext strings via `friendly_message/1` |
| `Markdown` | Markdown-to-HTML conversion (MDEx with ammonia sanitization) with `[[double bracket]]` link syntax and streaming document support for chat |
| `Duration` | Milliseconds to human-readable duration formatting |
| `Auth` | Authentication plugs: login password check, API token validation, session enforcement |
| `ArtistLive.Biography` | Artist biography building/rendering from Wikipedia and Last.fm data |
| `LiveHelpers.Params` | URL query param parsing: pagination, search query, sort order, display mode, fallback index |
| `LiveHelpers.IndexActions` | Shared index page logic (search, pagination, import, delete, display mode) for Collection/Wishlist index pages, parameterized by config map |
| `LiveHelpers.RecordActions` | Shared record action handlers (refresh cover, genres, embeddings, MusicBrainz data) for Collection/Wishlist show pages |
| — | Logster v2 handles `[:phoenix, :socket_connected]` telemetry — not a custom module |
### Controllers
| Controller | Routes | Purpose |
|------------|--------|---------|
| `SessionController` | `/login`, `/sessions/create` | Login/logout |
| `HealthController` | `/health` | Health check |
| `LastFmController` | `/auth/last_fm/callback` | Last.fm OAuth |
| `ArchiveController` | `/backup`, `/api/v1/backup` | Database backup download (API route requires token) |
| `AssetController` | `/assets/:transform_payload`, `/public/assets/:transform_payload`, `/api/v1/assets/:transform_payload` | Serve images with transforms (public route for emails, API route requires token) |
| `CollectionController` | `/api/v1/collection/*` | JSON API for collection queries |
| `ErrorController` | `/api/v1/errors`, `/api/v1/errors/:id` | JSON API for production error queries (requires Bearer token) |
| Controller | Routes | Purpose |
| ---------------------- | ------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------- |
| `SessionController` | `/login`, `/sessions/create` | Login/logout |
| `HealthController` | `/health` | Health check |
| `LastFmController` | `/auth/last_fm/callback` | Last.fm OAuth |
| `ArchiveController` | `/backup`, `/api/v1/backup` | Database backup download (API route requires token) |
| `AssetController` | `/assets/:transform_payload`, `/public/assets/:transform_payload`, `/api/v1/assets/:transform_payload` | Serve images with transforms (public route for emails, API route requires token) |
| `CollectionController` | `/api/v1/collection/*` | JSON API for collection queries |
| `ErrorController` | `/api/v1/errors`, `/api/v1/errors/:id` | JSON API for production error queries (requires Bearer token) |
---
@@ -355,26 +358,26 @@ All authenticated routes live inside a single `live_session` with three `on_moun
### JS Hooks
| Hook | Type | Purpose |
|------|------|---------|
| `FormatNumber` | External (`assets/js/hooks/`) | Client-side number formatting |
| `UniversalSearchNavigation` | External | Keyboard navigation in search modal (via `create-navigation-hook` factory) |
| `RecordPickerNavigation` | External | Keyboard navigation in record picker (via `create-navigation-hook` factory) |
| `RulePickerNavigation` | External | Keyboard navigation in scrobble rule picker (via `create-navigation-hook` factory) |
| `SortableList` | External (`assets/js/hooks/`) | Drag-and-drop reordering of record set items (uses sortablejs) |
| `LiveToast` | External (via `createLiveToastHook`) | Toast notification rendering |
| Various `.ColocatedHooks` | Colocated (in .heex) | Inline hooks prefixed with `.` (includes `.ScrollBottom` for Chat) |
| Hook | Type | Purpose |
| --------------------------- | ------------------------------------ | ---------------------------------------------------------------------------------- |
| `FormatNumber` | External (`assets/js/hooks/`) | Client-side number formatting |
| `UniversalSearchNavigation` | External | Keyboard navigation in search modal (via `create-navigation-hook` factory) |
| `RecordPickerNavigation` | External | Keyboard navigation in record picker (via `create-navigation-hook` factory) |
| `RulePickerNavigation` | External | Keyboard navigation in scrobble rule picker (via `create-navigation-hook` factory) |
| `SortableList` | External (`assets/js/hooks/`) | Drag-and-drop reordering of record set items (uses sortablejs) |
| `LiveToast` | External (via `createLiveToastHook`) | Toast notification rendering |
| Various `.ColocatedHooks` | Colocated (in .heex) | Inline hooks prefixed with `.` (includes `.ScrollBottom` for Chat) |
### JS Event Listeners (app.js)
All events are namespaced with `music_library:` prefix.
| Event | Action |
|-------|--------|
| `music_library:clipcopy` | Copy text to clipboard |
| `music_library:scroll_top` | Scroll window to top |
| `music_library:confetti` | Trigger canvas-confetti animation |
| `music_library:download` | Decode base64 blob and trigger browser file download (dispatched via `push_event`, prefixed `phx:` on client) |
| Event | Action |
| -------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `music_library:clipcopy` | Copy text to clipboard |
| `music_library:scroll_top` | Scroll window to top |
| `music_library:confetti` | Trigger canvas-confetti animation |
| `music_library:download` | Decode base64 blob and trigger browser file download (dispatched via `push_event`, prefixed `phx:` on client) |
### NPM Dependencies
@@ -390,26 +393,26 @@ All events are namespaced with `music_library:` prefix.
### Test Support
| Module | Purpose |
|--------|---------|
| `ConnCase` | HTTP test setup, auto-logged-in session |
| `DataCase` | Database test setup with Ecto sandbox |
| Module | Purpose |
| ----------------- | ------------------------------------------- |
| `ConnCase` | HTTP test setup, auto-logged-in session |
| `DataCase` | Database test setup with Ecto sandbox |
| `LiveTestHelpers` | `escape/1` for HTML-escaped text assertions |
### Fixture Modules (test/support/fixtures/)
| Module | Creates |
|--------|---------|
| `MusicLibrary.RecordsFixtures` | Records with MusicBrainz data |
| `MusicLibrary.RecordSetsFixtures` | Record sets with items |
| `MusicLibrary.OnlineStoreTemplatesFixtures` | Store templates |
| `MusicLibrary.ArtistInfoFixtures` | ArtistInfo records |
| `ScrobbleRulesFixtures` | Scrobble rules |
| `ScrobbledTracksFixtures` | Last.fm tracks |
| `Discogs.ArtistFixtures` | Discogs API responses |
| `LastFm.ArtistFixtures` | Last.fm API responses |
| `MusicBrainz.*Fixtures` | MusicBrainz API responses |
| `Wikipedia.Fixtures` | Wikipedia API responses |
| Module | Creates |
| ------------------------------------------- | ----------------------------- |
| `MusicLibrary.RecordsFixtures` | Records with MusicBrainz data |
| `MusicLibrary.RecordSetsFixtures` | Record sets with items |
| `MusicLibrary.OnlineStoreTemplatesFixtures` | Store templates |
| `MusicLibrary.ArtistInfoFixtures` | ArtistInfo records |
| `ScrobbleRulesFixtures` | Scrobble rules |
| `ScrobbledTracksFixtures` | Last.fm tracks |
| `Discogs.ArtistFixtures` | Discogs API responses |
| `LastFm.ArtistFixtures` | Last.fm API responses |
| `MusicBrainz.*Fixtures` | MusicBrainz API responses |
| `Wikipedia.Fixtures` | Wikipedia API responses |
### Test Styles
docs/production-infrastructure.md
+56 -53
View File
@@ -19,12 +19,12 @@ push via GitHub Actions.
## Hosting
| Component | Technology |
|-----------|-----------|
| Orchestration | Coolify (self-hosted) |
| Container runtime | Docker |
| SSL termination | Coolify reverse proxy |
| HTTP redirect | HTTP 301 → HTTPS (enforced by app config) |
| Component | Technology |
| ----------------- | ----------------------------------------- |
| Orchestration | Coolify (self-hosted) |
| Container runtime | Docker |
| SSL termination | Coolify reverse proxy |
| HTTP redirect | HTTP 301 → HTTPS (enforced by app config) |
The Docker image is a multi-stage build:
@@ -42,11 +42,11 @@ Fluxon UI (licensed dependency) is fetched during build via Docker build secrets
Three separate SQLite databases, each managed by its own Ecto repo:
| Repo | Purpose | Cache size | Pool size |
|------|---------|------------|-----------|
| `MusicLibrary.Repo` | Application data | 128 MB | `$POOL_SIZE` (default 5) |
| `MusicLibrary.BackgroundRepo` | Oban job queue | 16 MB | `$POOL_SIZE` (default 5) |
| `MusicLibrary.TelemetryRepo` | Persistent telemetry metrics | 4 MB | 2 |
| Repo | Purpose | Cache size | Pool size |
| ----------------------------- | ---------------------------- | ---------- | ------------------------ |
| `MusicLibrary.Repo` | Application data | 128 MB | `$POOL_SIZE` (default 5) |
| `MusicLibrary.BackgroundRepo` | Oban job queue | 16 MB | `$POOL_SIZE` (default 5) |
| `MusicLibrary.TelemetryRepo` | Persistent telemetry metrics | 4 MB | 2 |
All databases use incremental auto-vacuum. Paths are configured via environment variables
(`DATABASE_PATH`, `BACKGROUND_DATABASE_PATH`, `TELEMETRY_DATABASE_PATH`).
@@ -60,13 +60,13 @@ All databases use incremental auto-vacuum. Paths are configured via environment
Configured inline in `compose.yaml`. Runs as a separate Docker Compose service
(`litestream/litestream:0.5.11-scratch`) sharing the database volume.
| Setting | Value |
|---------|-------|
| S3 endpoint | `https://nbg1.your-objectstorage.com` |
| Bucket | `ffmusiclibrary` |
| Sync interval | 60 minutes |
| Retention | 672 hours (28 days) |
| Healthcheck | `litestream databases` every 30s (timeout 10s, 3 retries) |
| Setting | Value |
| ------------- | --------------------------------------------------------- |
| S3 endpoint | `https://nbg1.your-objectstorage.com` |
| Bucket | `ffmusiclibrary` |
| Sync interval | 60 minutes |
| Retention | 672 hours (28 days) |
| Healthcheck | `litestream databases` every 30s (timeout 10s, 3 retries) |
Credentials via environment: `LITESTREAM_ACCESS_KEY_ID`, `LITESTREAM_SECRET_ACCESS_KEY`.
@@ -132,10 +132,10 @@ Fluxon (private dependency) is configured with a dedicated registry entry.
### Deployment credentials
| Secret/Variable | Purpose |
|-----------------|---------|
| `COOLIFY_TOKEN` | API authentication (GitHub secret) |
| `COOLIFY_HOST` | Coolify server address (GitHub variable) |
| Secret/Variable | Purpose |
| ------------------ | ---------------------------------------- |
| `COOLIFY_TOKEN` | API authentication (GitHub secret) |
| `COOLIFY_HOST` | Coolify server address (GitHub variable) |
| `COOLIFY_APP_UUID` | Application identifier (GitHub variable) |
---
@@ -144,32 +144,32 @@ Fluxon (private dependency) is configured with a dedicated registry entry.
### Required
| Variable | Purpose |
|----------|---------|
| `DATABASE_PATH` | Absolute path to main SQLite database |
| `BACKGROUND_DATABASE_PATH` | Absolute path to background jobs database |
| `TELEMETRY_DATABASE_PATH` | Absolute path to telemetry database |
| `SECRET_KEY_BASE` | Phoenix secret (`mix phx.gen.secret`) |
| `CLOAK_ENCRYPTION_KEY` | Base64-encoded 32-byte AES key for encrypted secrets |
| `LOGIN_PASSWORD` | Web login password |
| `API_TOKEN` | Bearer token for API endpoints |
| `MAILGUN_API_KEY` | Mailgun API key |
| Variable | Purpose |
| -------------------------- | ---------------------------------------------------- |
| `DATABASE_PATH` | Absolute path to main SQLite database |
| `BACKGROUND_DATABASE_PATH` | Absolute path to background jobs database |
| `TELEMETRY_DATABASE_PATH` | Absolute path to telemetry database |
| `SECRET_KEY_BASE` | Phoenix secret (`mix phx.gen.secret`) |
| `CLOAK_ENCRYPTION_KEY` | Base64-encoded 32-byte AES key for encrypted secrets |
| `LOGIN_PASSWORD` | Web login password |
| `API_TOKEN` | Bearer token for API endpoints |
| `MAILGUN_API_KEY` | Mailgun API key |
### Optional
| Variable | Default | Purpose |
|----------|---------|---------|
| `SERVICE_FQDN_WEB` | `example.com` | Application domain |
| `PORT` | `4000` | HTTP listen port |
| `POOL_SIZE` | `5` | Database connection pool size |
| `MAILGUN_DOMAIN` | `mailgun.fullyforged.com` | Mailgun sending domain |
| `DEFAULT_TIMEZONE` | `Europe/London` | Application timezone |
| `OPENAI_KEY` | — | OpenAI API (embeddings, chat) |
| `DISCOGS_PERSONAL_ACCESS_TOKEN` | — | Discogs API |
| `BRAVE_SEARCH_API_KEY` | — | Brave Search API |
| `LAST_FM_API_KEY` | — | Last.fm API |
| `LAST_FM_SHARED_SECRET` | — | Last.fm scrobbling auth |
| `LAST_FM_USER` | — | Last.fm username |
| Variable | Default | Purpose |
| ------------------------------- | ------------------------- | ----------------------------- |
| `SERVICE_FQDN_WEB` | `example.com` | Application domain |
| `PORT` | `4000` | HTTP listen port |
| `POOL_SIZE` | `5` | Database connection pool size |
| `MAILGUN_DOMAIN` | `mailgun.fullyforged.com` | Mailgun sending domain |
| `DEFAULT_TIMEZONE` | `Europe/London` | Application timezone |
| `OPENAI_KEY` | — | OpenAI API (embeddings, chat) |
| `DISCOGS_PERSONAL_ACCESS_TOKEN` | — | Discogs API |
| `BRAVE_SEARCH_API_KEY` | — | Brave Search API |
| `LAST_FM_API_KEY` | — | Last.fm API |
| `LAST_FM_SHARED_SECRET` | — | Last.fm scrobbling auth |
| `LAST_FM_USER` | — | Last.fm username |
---
@@ -276,22 +276,25 @@ Pi extensions provide additional tools for production observability without manu
or browser access. Each extension reads its own environment variables from the pi
runtime environment (not server-side config).
| Extension | Tools | Env vars |
|-----------|-------|----------|
| `prod-logs` | `fetch_production_logs` | `PI_COOLIFY_HOST`, `PI_COOLIFY_APP_UUID`, `PI_COOLIFY_TOKEN` |
| `prod-errors` | `fetch_production_errors`, `fetch_production_error`, `/prod-errors` | `PI_API_TOKEN`, `PI_SERVICE_FQDN_WEB` |
| Extension | Tools | Env vars |
| ------------- | ------------------------------------------------------------------- | ------------------------------------------------------------ |
| `prod-logs` | `fetch_production_logs` | `PI_COOLIFY_HOST`, `PI_COOLIFY_APP_UUID`, `PI_COOLIFY_TOKEN` |
| `prod-errors` | `fetch_production_errors`, `fetch_production_error`, `/prod-errors` | `PI_API_TOKEN`, `PI_SERVICE_FQDN_WEB` |
**`prod-logs` env vars:**
- `PI_COOLIFY_HOST` — Coolify server base URL (e.g., `https://coolify.example.com`)
- `PI_COOLIFY_APP_UUID` — Application UUID in Coolify
- `PI_COOLIFY_TOKEN` — Coolify API Bearer token
**`prod-errors` tools and command:**
- `fetch_production_errors` — List/filter errors via LLM tool
- `fetch_production_error` — Single error detail via LLM tool
- `/prod-errors` — Interactive TUI for browsing errors (list, detail, filter toggles)
**`prod-errors` env vars:**
- `PI_API_TOKEN` — Must match the `API_TOKEN` env var on the production server (used for Bearer auth on `/api/v1/*`)
- `PI_SERVICE_FQDN_WEB` — Production domain with protocol (e.g., `https://musiclibrary.claudio-ortolina.org`, no trailing slash)
@@ -301,10 +304,10 @@ runtime environment (not server-side config).
Mailgun via Swoosh (`Swoosh.Adapters.Mailgun`).
| Setting | Value |
|---------|-------|
| From | `postmaster@mailgun.fullyforged.com` |
| To | `claudio@fullyforged.com` |
| Setting | Value |
| ------- | ------------------------------------ |
| From | `postmaster@mailgun.fullyforged.com` |
| To | `claudio@fullyforged.com` |
Used for error notifications and the daily "records on this day" digest email.