Commit Graph

579 Commits

Author SHA1 Message Date
Claudio Ortolina c4ebe636ff Fix ML-143 review: security, worker, and test improvements
Closes ML-143 review findings:
- cast_id/1 uses Integer.parse instead of String.to_integer
- add_to_cart validates against server-side release_groups
- JS.push payload trimmed to {id, format} only
- Oban.insert_all result checked for partial failures
- handle_async success resets @importing? to false
- ImportFromMusicbrainzReleaseGroup gets unique constraint
- Record.parse_datetime/1 made public with safe match
- Workers delegate to Record.parse_datetime/1
- Test coverage: change_format, clear_cart, wishlist single import
2026-04-20 15:15:29 +01:00
Claudio Ortolina 3f5ce060b1 Implement ML-143: Cart-style multi record import
Closes ML-143.
2026-04-20 15:15:29 +01:00
Claudio Ortolina 45236e4b10 Convert ErrorIgnorer tests to doctests 2026-04-19 07:50:39 +01:00
Claudio Ortolina bbe1722c9f Handle unencodable structs in QueryReporter params 2026-04-17 07:10:45 +01:00
Claudio Ortolina 58c60193f2 Use pipe 2026-04-16 10:38:08 +01:00
Claudio Ortolina 2124c982f7 Handle possible non-existent record id in search filter 2026-04-15 15:10:28 +01:00
Claudio Ortolina c816f2de4a Add scrobbled tracks search parser
Introduces ListeningStats.SearchParser with NimbleParsec supporting
record:, album_mbid:, artist_mbid:, artist:, album:, track: filters.
Links scrobble count on record page to filtered scrobbled tracks view.
2026-04-15 14:56:38 +01:00
Claudio Ortolina b9612a2a26 Fix records on this day for partial release dates
SQLite interprets bare years (e.g. "1970") as Julian day numbers
in strftime, producing wrong month-day matches. Filter to only
full YYYY-MM-DD dates before comparing.
2026-04-15 07:29:58 +01:00
Claudio Ortolina d75508531e Generate parse_type/parse_format from module attributes 2026-04-14 16:56:04 +01:00
Claudio Ortolina 2bb755b1c8 Parse matching record fields in Records.Record 2026-04-14 16:51:06 +01:00
Claudio Ortolina 888b5a68e6 Update all callers to use matching_records
Migrate StatsLive.Index, ScrobbledTracksLive.Index, and TopAlbums to
consume matching_records list instead of scalar record IDs. Remove the
derive_legacy_record_ids bridge function. TopAlbums now uses a dropdown
for multi-record badges with play count display.
2026-04-14 16:14:59 +01:00
Claudio Ortolina 2c6e6e7ca2 Return matching_records list from top_albums_attach_metadata
Apply the same release-group JSON aggregation pattern to top albums
queries. Add JSON parsing in get_top_albums/1 and
get_top_albums_by_days/2.
2026-04-14 16:05:51 +01:00
Claudio Ortolina cfd267627c Return matching_records list from tracks_with_record_info_query
Replace scalar collected_record_id/wishlisted_record_id subqueries with
a single matching_records JSON array grouped by release group
(records.musicbrainz_id). Add parse_matching_records/1 for JSON parsing
and derive_legacy_record_ids/2 as a temporary bridge for existing
templates. Update tests to use shared musicbrainz_id fixtures.
2026-04-14 14:17:02 +01:00
Claudio Ortolina 9f4135661e Replace scalar record IDs with matching_records JSON array in tracks query
Replaces collected_record_id and wishlisted_record_id correlated
subqueries in tracks_with_record_info_query with a single
matching_records json_group_array subquery that returns all records
sharing the same release ID. Adds parse_matching_records/1 for JSON
deserialization. Updates recent_activity/2 and list_tracks/1 to parse
the JSON at the boundary. Derives legacy collected_record_id and
wishlisted_record_id from matching_records so existing LiveViews keep
working. The get_top_albums test is temporarily skipped pending the
top_albums_attach_metadata update.
2026-04-14 14:12:14 +01:00
Claudio Ortolina 6a6410bcaa Have the LLM use square bracketed links for internal references 2026-04-13 14:24:36 +01:00
Claudio Ortolina 96bb0fc403 Try and control ChatGPT's tone and "helpfulness" 2026-04-13 14:09:43 +01:00
Claudio Ortolina 9b44b7050f Count unique catalog entries in collection summary
The record_count now reflects the number of grouped catalog lines
the model sees, not raw DB rows. Two copies of the same album on
different formats count as 1 catalog entry.
2026-04-13 13:12:30 +01:00
Claudio Ortolina 54dab5fbdf Use gpt 5.1 to cater for collection summary context size 2026-04-13 12:43:42 +01:00
Claudio Ortolina f9448cac6f Revert "Use smaller model so that we don't exhaust tokens"
This reverts commit 9c0e11bc84.
2026-04-13 12:34:17 +01:00
Claudio Ortolina 8108645718 Shrink collection summary to fit OpenAI token limit
Deduplicate records by musicbrainz_id (merging formats)
and cap genres to 3 per record, reducing the summary from
~32K to ~24K estimated tokens.
2026-04-13 12:33:15 +01:00
Claudio Ortolina 9c0e11bc84 Use smaller model so that we don't exhaust tokens 2026-04-13 12:06:24 +01:00
Claudio Ortolina 34f484cc81 Add CollectionChat StreamProvider 2026-04-13 10:09:50 +01:00
Claudio Ortolina 24a3cc9265 Add Collection.collection_summary/0 2026-04-13 10:08:01 +01:00
Claudio Ortolina d845ab93dd Add collection sentinel UUID to Chats context 2026-04-13 10:06:16 +01:00
Claudio Ortolina 55be2a4bab Add :collection to chat entity enum 2026-04-13 10:04:37 +01:00
Claudio Ortolina 88906e10ad Add ex_slop, configure it, run it and fix issues 2026-04-12 22:07:27 +01:00
Claudio Ortolina 02761428c1 Format map and list query params as JSON 2026-04-11 23:57:40 +01:00
Claudio Ortolina d8c84e787b Optimize ListeningStats query performance
Three independent refactors of lib/music_library/listening_stats.ex,
measured against the dev DB (104k tracks) via bench/listening_stats.exs:

| Query                           | Baseline  | After     | Speedup |
|---------------------------------|-----------|-----------|---------|
| recent_activity(tz, 100)        | 38.98 ms  | 2.87 ms   | 13.6x   |
| list_tracks(page 1, 200)        | 47.30 ms  | 4.81 ms   | 9.8x    |
| get_top_artists_by_days(7)      | 66.51 ms  | 1.02 ms   | 65x     |
| get_top_artists_by_days(30)     | 67.41 ms  | 2.84 ms   | 23.7x   |
| get_top_artists_by_days(365)    | 76.03 ms  | 25.57 ms  | 3.0x    |
| get_top_albums_by_days(7)       | 92.63 ms  | 1.61 ms   | 57.5x   |
| get_top_albums_by_days(30)      | 94.52 ms  | 3.69 ms   | 25.6x   |
| get_top_albums_by_days(365)     | 105.23 ms | 32.81 ms  | 3.2x    |
| get_top_artists(limit: 10)      | 123.96 ms | 101.68 ms | 1.22x   |
| get_top_albums(limit: 10)       | 209.64 ms | 108.23 ms | 1.94x   |

tracks_with_record_info_query/0 now uses correlated scalar subqueries
against record_releases and artist_records instead of materializing
helper subqueries on every call. The cost scales with the outer LIMIT,
not with the size of record_releases.

top_albums_base_query/0 and top_artists_base_query/0 are replaced by
aggregate_query + attach_metadata pairs. The pattern is aggregate first,
attach metadata second: GROUP BY runs against the raw track scan, then
a tiny outer SELECT attaches record_releases / artist_infos lookups for
the <= 10 result rows via correlated subqueries.

tracks_since_query/1 wraps the date-filtered inner scan with limit: -1.
SQLite cannot flatten a subquery that has a LIMIT, so it materializes
the date-bounded subset and the optimizer uses the timestamp index for
the range scan instead of the album/artist composite index. This trick
replaces SQLite's WITH ... AS MATERIALIZED (which ecto_sqlite3 doesn't
expose).

All json_extract(?, '\$.path') fragments use the canonical form rather
than the equivalent ? ->> '\$.path' shorthand. SQLite's index matcher
requires the GROUP BY expression to match the index expression
textually to use the composite index for natural ordering.

collected_releases_query/0 and wishlisted_releases_query/0 are removed
from Collection and Wishlist — they had a single internal caller that
no longer exists after the refactor.

New regression tests lock the semantics that the optimized queries
must preserve:
- count(DISTINCT scrobbled_at_uts) — 579 duplicate timestamps exist in
  the dev DB from rapid Last.fm scrobbles, so replacing with count(*)
  would silently change results
- :artist_id key in list_tracks result maps — ScrobbledTracksLive.Index
  destructures it even though the template body never references it

Closes #148
2026-04-11 23:36:32 +01:00
Claudio Ortolina 3da0865725 Drop Records embedding wrappers 2026-04-11 21:43:00 +01:00
Claudio Ortolina a608be7387 Move regenerate_artist_embeddings into Similarity 2026-04-11 21:43:00 +01:00
Claudio Ortolina 789ce04b87 Use MusicBrainz.get_all_releases in Records 2026-04-11 21:40:32 +01:00
Claudio Ortolina 1c720c8394 Add query timing metadata to reporter output 2026-04-10 10:30:05 +01:00
Claudio Ortolina 27148c6507 Add dev query reporter and skill
Ecto telemetry reporter that captures executed SQL queries
to a log file with interpolated parameters and source
locations. Activated/deactivated at runtime via Tidewave
for LLM-driven query analysis workflows.
2026-04-10 10:27:12 +01:00
Claudio Ortolina 59937dadb8 Remove duplication around searching scrobbled tracks
Fixes #157
2026-04-09 06:57:24 +01:00
Claudio Ortolina edc4e51d1d Optimize record set compaction
Closes #154
2026-04-06 18:38:56 +01:00
Claudio Ortolina a5df68c7ad Don't suppress errors when fetching artist Last.fm data
Closes #149
2026-04-05 15:47:36 +01:00
Claudio Ortolina 4be9db6417 Refactor DefaultPrompt into Chats.Prompt with build/2 2026-04-05 08:32:03 +01:00
Claudio Ortolina f9155f8c52 Try and control the chat responses style 2026-04-04 23:35:46 +01:00
Claudio Ortolina ec12be613b Extract a default prompt 2026-04-04 23:35:30 +01:00
Claudio Ortolina 86fd3134f1 Reduce repetitive LLM chat responses (#146)
Replace "always provide a link" with conditional linking and add
instructions to vary response style and avoid restating information
already covered in the conversation.

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-04 18:32:57 +01:00
Claudio Ortolina b9be62a500 Async barcode scan batch import for 2+ new records
When barcode scan results contain at least 2 new records,
enqueue individual Oban jobs instead of importing synchronously.
Wishlisted/collected/not_found results still process synchronously.
2026-04-04 10:20:50 +01:00
Claudio Ortolina 2f9460d2cb Regenerate embeddings only when the test representation changes 2026-04-04 08:32:29 +01:00
Claudio Ortolina 4bfe27e6c8 Cancel refresh cover job if image is not available 2026-04-03 12:43:50 +01:00
Claudio Ortolina 3f7da55e50 Add a release tracked badge to records in lists 2026-04-02 11:52:07 +01:00
Claudio Ortolina 7f406ce455 Replace deprecated {:discard, _} with {:cancel, _} 2026-04-01 09:38:05 +01:00
Claudio Ortolina 553ad2bbb9 Discard worker when no wikipedia entry is present 2026-04-01 08:52:28 +01:00
Claudio Ortolina 6fcca38e27 Make sure to pull artists info when record is updated
Catches the scenario when a new artist is added
2026-04-01 07:11:47 +01:00
Claudio Ortolina 7d65a7d4ab Make sure create record returns updated record 2026-04-01 07:06:33 +01:00
Claudio Ortolina c913dab692 Return controlled errors from genre population
Closes #142
2026-03-30 15:01:42 +01:00
Claudio Ortolina 92a36b915a Harden public asset endpoint against invalid payloads
Closes #143
2026-03-30 14:56:40 +01:00