cloud/music_library
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Deduplicate and consolidate project docs

Browse Source 
Remove redundant sections from production-infrastructure.md that
duplicated architecture.md (Oban queues/plugins/cron, external API
integrations, SQLite extensions, scheduled maintenance). Absorb API
rate limits into architecture.md's integration table. Replace tool
versions table with pointer to mise.toml. Add two conventions from
project history: scoped test setup and single-path shared components.
This commit is contained in:
Claudio Ortolina
2026-04-09 15:19:32 +01:00
parent 67c01ab1fa
commit eeb0763466
3 changed files with 14 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+11 -10
View File
@@ -142,17 +142,18 @@ Last.fm schemas (separate, not Ecto-persisted to main DB):
## External API Integrations
| Module | API | Purpose |
|--------|-----|---------|
| `MusicBrainz` / `MusicBrainz.API` | musicbrainz.org | Release/artist metadata, search |
| `LastFm` / `LastFm.API` | last.fm | Scrobbling, listening history, artist info (tags, similar artists), user profile/session validation |
| `Discogs` / `Discogs.API` | discogs.com | Artist profiles, images |
| `Wikipedia` / `Wikipedia.API` | wikipedia.org | Artist biographies |
| `BraveSearch` / `BraveSearch.API` | search.brave.com | Cover art and artist image search |
| `OpenAI` / `OpenAI.API` | api.openai.com | Text embeddings for similarity, streaming chat via Responses API (gpt-4.1 + web search), rate-limited |
| `MusicLibrary.Mailer` | Mailgun (via Swoosh) | Transactional email delivery (error notifications, daily digest) |
| Module | API | Rate limit | Purpose |
|--------|-----|-----------|---------|
| `MusicBrainz` / `MusicBrainz.API` | musicbrainz.org | 1000 ms | Release/artist metadata, search |
| `LastFm` / `LastFm.API` | last.fm | 500 ms | Scrobbling, listening history, artist info (tags, similar artists), user profile/session validation |
| `Discogs` / `Discogs.API` | discogs.com | 2000 ms | Artist profiles, images |
| `Wikipedia` / `Wikipedia.API` | wikipedia.org | 1000 ms | Artist biographies |
| `BraveSearch` / `BraveSearch.API` | search.brave.com | 1000 ms | Cover art and artist image search |
| `OpenAI` / `OpenAI.API` | api.openai.com | 250 ms | Text embeddings for similarity, streaming chat via Responses API (gpt-4.1 + web search) |
| `MusicLibrary.Mailer` | Mailgun (via Swoosh) | — | Transactional email delivery (error notifications, daily digest) |
Each has a `Config` module reading from application env. In tests, all HTTP calls are
Each has a `Config` module reading from application env. All HTTP clients use `Req` with
per-API rate limiting (`Req.RateLimiter`, ETS-backed). In tests, all HTTP calls are
stubbed via `Req.Test` (configured in `config/test.exs`).
---
docs/production-infrastructure.md
+1 -71
View File
@@ -48,15 +48,6 @@ Three separate SQLite databases, each managed by its own Ecto repo:
All databases use incremental auto-vacuum. Paths are configured via environment variables
(`DATABASE_PATH`, `BACKGROUND_DATABASE_PATH`, `TELEMETRY_DATABASE_PATH`).
SQLite extensions loaded at runtime: `unicode`, `vec0` (vector search).
### Scheduled maintenance
| Schedule | Worker | Purpose |
|----------|--------|---------|
| Daily 3 AM | `RepoVacuum` | Reclaim unused space |
| Daily 4 AM | `RepoOptimize` | Run `PRAGMA optimize` |
---
## Backups
@@ -238,61 +229,6 @@ Used for error notifications and the daily "records on this day" digest email.
---
## External API Integrations
All HTTP clients use `Req` with per-API rate limiting (`Req.RateLimiter`, ETS-backed).
| API | Rate limit | Purpose |
|-----|-----------|---------|
| MusicBrainz | 1000 ms cooldown | Release/artist metadata, search |
| Last.fm | 500 ms cooldown | Scrobbling, listening history, artist tags |
| Discogs | 2000 ms cooldown | Artist profiles, images |
| Wikipedia | 1000 ms cooldown | Artist biographies |
| Brave Search | 1000 ms cooldown | Cover art, artist image search |
| OpenAI | 250 ms cooldown | Text embeddings (similarity), streaming chat |
---
## Background Jobs
Oban with `Oban.Engines.Lite` (SQLite-backed, process-based).
### Queues
| Queue | Concurrency | Purpose |
|-------|-------------|---------|
| `default` | 10 | General async tasks |
| `heavy_writes` | 1 | Serialized DB-intensive operations |
| `music_brainz` | 1 | MusicBrainz API calls |
| `discogs` | 1 | Discogs API calls |
| `wikipedia` | 1 | Wikipedia API calls |
| `last_fm` | 1 | Last.fm API calls |
### Plugins
- **Pruner**: Removes completed/cancelled/discarded jobs older than 12 hours
- **Reindexer**: Weekly Oban table reindex
- **Cron**: Scheduled jobs (timezone: `Europe/London`)
### Cron schedule
| Schedule | Worker |
|----------|--------|
| Every 12h | `ApplyScrobbleRules` |
| Every 12h | `PruneAssetCache` |
| Daily 2 AM | `PruneAssets` |
| Daily 3 AM | `RepoVacuum` |
| Daily 4 AM | `RepoOptimize` |
| Daily 7 AM | `SendRecordsOnThisDayEmail` |
| Monthly 1st, 6 AM | `RecordRefreshAllMusicBrainzData` |
| Monthly 1st, 7 AM | `RecordGenerateAllEmbeddings` |
| Monthly 1st, 8 AM | `ArtistRefreshAllMusicBrainzData` |
| Monthly 1st, 9 AM | `ArtistRefreshAllDiscogsData` |
| Every 5 min | `RefreshScrobbles` |
| Monthly 1st, 10 AM | `ArtistRefreshAllWikipediaData` |
---
## Encryption
Cloak vault (`MusicLibrary.Vault`) with AES.GCM cipher for at-rest encryption of
@@ -303,10 +239,4 @@ secrets stored in the `secrets` table. Key configured via `CLOAK_ENCRYPTION_KEY`
## Tool Versions
Defined in `mise.toml`:
| Tool | Version |
|------|---------|
| Elixir | 1.20.0-rc.3-otp-28 |
| Erlang | 28.4.1 |
| Node.js | 24.13.0 |
Defined in `mise.toml`.
docs/project-conventions.md
+2
View File
@@ -29,6 +29,7 @@ Rules extracted from commit history that are specific to this project and not al
- Extract when duplicated 3+ times. Identical template markup in 3+ places becomes a function component.
- Delete thin wrapper modules with a single caller -- inline them instead.
- Parameterize the differences when extracting shared logic.
- **When modifying an existing shared component**, refactor all callers to the unified new behavior rather than adding a mode/variant attr. Keep shared components to a single render path.
- Private helpers go at module bottom, public functions first.
## Template / UI
@@ -70,6 +71,7 @@ Rules extracted from commit history that are specific to this project and not al
## Testing
- **Feature-specific test setup stays in the test modules that need it**, not in shared case templates (DataCase, ConnCase). Only add to shared templates when every test using that template genuinely needs it.
- **`@tag :logged_out`** for public endpoint tests. **`@tag :capture_log`** on tests with expected error log output.
- **Fixture modules** use `System.unique_integer([:positive])` for unique names and call through context functions (not raw `Repo.insert`).
- **Verify outcomes through context modules**, not just UI assertions. Delete tests assert both `refute has_element?` and `assert_raise Ecto.NoResultsError`.