cloud/music_library
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ML-144: enrich issue description and add done requirements

Browse Source
This commit is contained in:
Claudio Ortolina
2026-04-24 07:52:22 +01:00
parent 6322428c83
commit 7ad09736cf
1 changed files with 11 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backlog/tasks/ml-144 - Implement-magazine-issue-ingestion-pipeline.md
+11 -3
View File
@@ -4,6 +4,7 @@ title: Implement magazine issue ingestion pipeline
status: To Do
assignee: []
created_date: '2026-04-22 15:38'
updated_date: '2026-04-24 06:51'
labels:
- feature
- magazines
@@ -21,7 +22,7 @@ Build a pipeline to ingest music magazine issues (PDF), extract structured metad
## Approach (Approved)
**Approach 1 — Text-first, single LLM pass with structured output:**
- Extract text locally with pdftotext (poppler, shell from Elixir)
- Extract text locally with pdftotext (poppler, shell from Elixir) or [kreuzberg](https://hex.pm/packages/kreuzberg)
- Feed chunks to OpenAI Responses API with JSON schema for structured output
- One Oban worker per issue, one API call per chunk
- Cost estimate: ~$0.01–0.10 per issue with gpt-4o-mini
@@ -32,8 +33,8 @@ Build a pipeline to ingest music magazine issues (PDF), extract structured metad
3. Extract text locally (pdftotext)
4. Split by section boundaries (page breaks, headings)
5. Per-chunk LLM call requesting structured payload:
- `mentions`: [{type: "artist"|"album"|"track", name, artist_name?, release_date?, context_snippet}]
- `topic`: "review"|"news"|"upcoming"|"feature"
- `mentions`: `[{type: "artist"|"album"|"track", name, artist_name?, release_date?, context_snippet}]`
- `topic`: `"review"|"news"|"upcoming"|"feature"`
6. Persist to new `Magazines` context (Issue, Mention schemas)
7. Resolve each mention against existing records/artists by name (MusicBrainz lookup fallback for unknowns)
@@ -61,3 +62,10 @@ Build a pipeline to ingest music magazine issues (PDF), extract structured metad
- [ ] #5 Magazine mentions surface on record/artist detail pages
- [ ] #6 Cost per issue documented and under budget estimate
<!-- AC:END -->
## Definition of Done
<!-- DOD:BEGIN -->
- [ ] #1 I can open a page in the application, drag and drop a magazine PDF, and start ingestion
- [ ] #2 For each artist/album in the collection and Wishlist, I can see relevant informational articles coming from ingested magazines
- [ ] #3 I can see a list of ALL ingested items, even those which don't have direct correlation with existing records and artists
<!-- DOD:END -->