cloud/music_library
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ML-144: enrich issue description and add done requirements

Browse Source
This commit is contained in:
Claudio Ortolina
2026-04-24 07:52:22 +01:00
parent 6322428c83
commit 7ad09736cf
1 changed files with 11 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backlog/tasks/ml-144 - Implement-magazine-issue-ingestion-pipeline.md
+11 -3
View File
@@ -4,6 +4,7 @@ title: Implement magazine issue ingestion pipeline
status: To Do status: To Do
assignee: [] assignee: []
created_date: '2026-04-22 15:38' created_date: '2026-04-22 15:38'
updated_date: '2026-04-24 06:51'
labels: labels:
- feature - feature
- magazines - magazines
@@ -21,7 +22,7 @@ Build a pipeline to ingest music magazine issues (PDF), extract structured metad
## Approach (Approved) ## Approach (Approved)
**Approach 1 — Text-first, single LLM pass with structured output:** **Approach 1 — Text-first, single LLM pass with structured output:**
- Extract text locally with pdftotext (poppler, shell from Elixir) - Extract text locally with pdftotext (poppler, shell from Elixir) or [kreuzberg](https://hex.pm/packages/kreuzberg)
- Feed chunks to OpenAI Responses API with JSON schema for structured output - Feed chunks to OpenAI Responses API with JSON schema for structured output
- One Oban worker per issue, one API call per chunk - One Oban worker per issue, one API call per chunk
- Cost estimate: ~$0.01–0.10 per issue with gpt-4o-mini - Cost estimate: ~$0.01–0.10 per issue with gpt-4o-mini
@@ -32,8 +33,8 @@ Build a pipeline to ingest music magazine issues (PDF), extract structured metad
3. Extract text locally (pdftotext) 3. Extract text locally (pdftotext)
4. Split by section boundaries (page breaks, headings) 4. Split by section boundaries (page breaks, headings)
5. Per-chunk LLM call requesting structured payload: 5. Per-chunk LLM call requesting structured payload:
- `mentions`: [{type: "artist"|"album"|"track", name, artist_name?, release_date?, context_snippet}] - `mentions`: `[{type: "artist"|"album"|"track", name, artist_name?, release_date?, context_snippet}]`
- `topic`: "review"|"news"|"upcoming"|"feature" - `topic`: `"review"|"news"|"upcoming"|"feature"`
6. Persist to new `Magazines` context (Issue, Mention schemas) 6. Persist to new `Magazines` context (Issue, Mention schemas)
7. Resolve each mention against existing records/artists by name (MusicBrainz lookup fallback for unknowns) 7. Resolve each mention against existing records/artists by name (MusicBrainz lookup fallback for unknowns)
@@ -61,3 +62,10 @@ Build a pipeline to ingest music magazine issues (PDF), extract structured metad
- [ ] #5 Magazine mentions surface on record/artist detail pages - [ ] #5 Magazine mentions surface on record/artist detail pages
- [ ] #6 Cost per issue documented and under budget estimate - [ ] #6 Cost per issue documented and under budget estimate
<!-- AC:END --> <!-- AC:END -->
## Definition of Done
<!-- DOD:BEGIN -->
- [ ] #1 I can open a page in the application, drag and drop a magazine PDF, and start ingestion
- [ ] #2 For each artist/album in the collection and Wishlist, I can see relevant informational articles coming from ingested magazines
- [ ] #3 I can see a list of ALL ingested items, even those which don't have direct correlation with existing records and artists
<!-- DOD:END -->