Commit Graph

218 Commits

Author SHA1 Message Date
Claudio Ortolina cccd28a9f9 ML-165: implementation 2026-05-05 13:43:18 +01:00
Claudio Ortolina ba2566828c Fix FTS5 crashes when searching with special characters
Always wrap FTS5 query terms in phrase double-quotes instead of
checking a hardcoded list of special characters. Characters like
|, ;, +, #, @ were missing from the list, causing FTS5 syntax
errors (production errors #1 and #1398). Phrase-quoting every term
produces identical results for normal ASCII search (verified via
SQL) while safely handling any character FTS5 chokes on.

Add regression tests for bare special characters and special
characters mixed with normal words.
2026-05-05 12:05:12 +01:00
Claudio Ortolina 7c6760d7db ML-158: force single-line log output in production
Three-layer architecture:
- Logster v2 handles HTTP request and LiveView socket telemetry in logfmt format
- Custom formatter as safety net escaping any remaining embedded newlines
- Config flag (single_line_logging) gating Logster attachment to prod only

Dev environment retains multi-line format for readability.
2026-05-04 15:01:10 +01:00
Claudio Ortolina 93ac86f748 fix(errors): LIKE wildcard escaping, memory-efficient counts, context tests
- Escape LIKE wildcards (% and _) in search queries using fragment ESCAPE
  to prevent accidental pattern expansion and potential DoS vectors
- Use Repo.aggregate for occurrence_count instead of loading all
  occurrences into memory in get_error/1
- Use Repo.aggregate(:min, :inserted_at) for first_occurrence_at instead
  of fragile List.last/1 that depends on query ordering
- Add MusicLibrary.ErrorsTest with 15 tests covering list_errors/1
  (default, status/muted/search filters, wildcard escaping, pagination,
  empty results) and get_error/1 (with/without occurrences, not_found)
2026-05-04 14:11:46 +01:00
Claudio Ortolina 0b78f24fed ML-155: Compact collection summary and add stats preamble 2026-05-02 07:11:18 +01:00
Claudio Ortolina ca5b13ad8d Remove some uses of List.last 2026-05-01 17:03:42 +01:00
Claudio Ortolina 3b558044b4 refactor: extract Records sub-contexts (Search, Import, Enrichment)
Split the 450+ line Records context into focused modules:

- Records.Search — FTS5 search, SearchParser integration, genre listing
- Records.Import — MusicBrainz release/group import, release status
- Records.Enrichment — genre population, cover refresh, color extraction,
  MusicBrainz data refresh
- Records.Query — shared helpers (essential_fields, order_alphabetically macro)

Records module is now a facade with defdelegate for compile-time safety.
CRUD and PubSub stay directly in Records. Tests moved alongside each
sub-module. Zero callers changed, 886 tests pass.
2026-04-30 17:20:37 +01:00
Claudio Ortolina 3849b338f3 Improve performance of job counting in maintenance
- Use a single query for all workers
- Replace the manual Ecto query against Oban.Job with Oban.Job.query/1
2026-04-30 14:54:56 +01:00
Claudio Ortolina 9113df5879 ML-146: honour API retry headers 2026-04-25 07:39:49 +01:00
Claudio Ortolina d588e5738f Remove redundant tests 2026-04-25 06:57:03 +01:00
Claudio Ortolina a1c665b490 ML-21: Classify API errors as transient vs permanent 2026-04-24 14:02:15 +01:00
Claudio Ortolina 0d1f580379 Refactor TracklistPdf to take release alone 2026-04-23 07:48:53 +01:00
Claudio Ortolina f7f38d580e Enable unaliased modules ex_slop check 2026-04-22 20:37:09 +01:00
Claudio Ortolina c4ebe636ff Fix ML-143 review: security, worker, and test improvements
Closes ML-143 review findings:
- cast_id/1 uses Integer.parse instead of String.to_integer
- add_to_cart validates against server-side release_groups
- JS.push payload trimmed to {id, format} only
- Oban.insert_all result checked for partial failures
- handle_async success resets @importing? to false
- ImportFromMusicbrainzReleaseGroup gets unique constraint
- Record.parse_datetime/1 made public with safe match
- Workers delegate to Record.parse_datetime/1
- Test coverage: change_format, clear_cart, wishlist single import
2026-04-20 15:15:29 +01:00
Claudio Ortolina 3f5ce060b1 Implement ML-143: Cart-style multi record import
Closes ML-143.
2026-04-20 15:15:29 +01:00
Claudio Ortolina 78487b0085 Pin Wikipedia intro_html to fixture value
Map.has_key? verified the key existed but not the content. The
fixture is deterministic — assert the stored value equals
Wikipedia.Fixtures.article_extract_html/0.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 07c99992a3 Pin barcode_scan release assertions
Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 162c10e70a Pin fetch_artist_image asset lookup
Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina bce8560ce8 Pin fetch_artist_info stored-data assertions
Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 1f457f0f11 Pin prune_assets test to %Asset{hash: ^hash} match
Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 6adfbea336 Pin Records.populate_genres error to OpenAI prefix
Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 0ee0dcb118 Match decoded OpenAI error body, not raw JSON
Switch the 500-response stubs to Req.Test.json so Req decodes the
body, then match the map shape. Avoids coupling the assertion to
the exact JSON serialization.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 1429f7a826 Drop tautological delete.id checks
Repo.delete always returns the struct it was given, so deleted.id
== original.id is trivially true. Thread deleted.id through the
trailing get_*! call instead, so the binding is load-bearing.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina bb260437a1 Pin populate_genres error to OpenAI prefix
Match the string prefix "OpenAI API error:" that propagates from
OpenAI.gpt on non-2xx responses.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina a841ccf8e8 Tighten generate_record_embedding assertions
Assert stored embedding text matches the representation helper.
Pin the OpenAI-500 error path to the raw JSON body string.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina bbe0d5cd9e Pin artist_refresh_discogs_data assertions
Bind the returned ArtistInfo. For the no-discogs branch, assert
discogs_data was not touched.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina f9e3e8c5f8 Pin artist_refresh_wikipedia_data assertions
Bind perform_job results and assert id equality. For the unchanged case,
also assert wikipedia_data is the same as before the job ran.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 914fb4fbee Match specific errors on scrobble_rules duplicate
Pin the duplicate-rule error to the match_value field and "has already been
taken" message, matching the unique_constraint error_key in the changeset.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 9e29de55aa Pin online_store_templates delete assertion
Assert deleted.id == template.id rather than wildcard-matching the
returned struct.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina a90fdcf82f Tighten chats assertions
Pin invalid-attrs error to specific field, assert collection UUID round-trips
through Ecto.UUID.cast unchanged, and assert deleted.id == chat.id on delete.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 66c36f4d09 Tighten record_sets delete and duplicate assertions
Assert deleted.id == set.id on delete, and pin the duplicate error to the
specific field and message from the unique_constraint on record_set_id.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina 60e95804ad Tighten similarity test assertions
Bind returned embeddings and assert record_id / text_representation match.
Pin OpenAI error path to the actual raw string body returned by the stub.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina e0fe5051ec Pin asset dedup assertion to same-hash
The duplicate prevention test asserted `{:ok, _}` twice without checking
that both calls returned the same asset. Assert hash equality.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina ae036e79fb Tighten scrobble_activity assertions
Replace 9 `{:ok, _}` sites with specific response-body checks to verify
the context returns the Last.fm API payload and not just the success branch.

Refs #176
2026-04-20 06:50:04 +01:00
Claudio Ortolina beaf38fcfe Add record_release tests 2026-04-19 16:11:32 +01:00
Claudio Ortolina 45236e4b10 Convert ErrorIgnorer tests to doctests 2026-04-19 07:50:39 +01:00
Claudio Ortolina 22a47f4ca9 Add tests for previously uncovered modules 2026-04-19 07:47:20 +01:00
Claudio Ortolina 4c5a3393e2 Run query reporter tests with async false
Avoid getting spurious failures due to interference by other tests
2026-04-17 10:29:43 +01:00
Claudio Ortolina c816f2de4a Add scrobbled tracks search parser
Introduces ListeningStats.SearchParser with NimbleParsec supporting
record:, album_mbid:, artist_mbid:, artist:, album:, track: filters.
Links scrobble count on record page to filtered scrobbled tracks view.
2026-04-15 14:56:38 +01:00
Claudio Ortolina b9612a2a26 Fix records on this day for partial release dates
SQLite interprets bare years (e.g. "1970") as Julian day numbers
in strftime, producing wrong month-day matches. Filter to only
full YYYY-MM-DD dates before comparing.
2026-04-15 07:29:58 +01:00
Claudio Ortolina dd3bb47989 Add tests for mixed collected/wishlisted matching records 2026-04-14 16:07:14 +01:00
Claudio Ortolina 2c6e6e7ca2 Return matching_records list from top_albums_attach_metadata
Apply the same release-group JSON aggregation pattern to top albums
queries. Add JSON parsing in get_top_albums/1 and
get_top_albums_by_days/2.
2026-04-14 16:05:51 +01:00
Claudio Ortolina cfd267627c Return matching_records list from tracks_with_record_info_query
Replace scalar collected_record_id/wishlisted_record_id subqueries with
a single matching_records JSON array grouped by release group
(records.musicbrainz_id). Add parse_matching_records/1 for JSON parsing
and derive_legacy_record_ids/2 as a temporary bridge for existing
templates. Update tests to use shared musicbrainz_id fixtures.
2026-04-14 14:17:02 +01:00
Claudio Ortolina 9f4135661e Replace scalar record IDs with matching_records JSON array in tracks query
Replaces collected_record_id and wishlisted_record_id correlated
subqueries in tracks_with_record_info_query with a single
matching_records json_group_array subquery that returns all records
sharing the same release ID. Adds parse_matching_records/1 for JSON
deserialization. Updates recent_activity/2 and list_tracks/1 to parse
the JSON at the boundary. Derives legacy collected_record_id and
wishlisted_record_id from matching_records so existing LiveViews keep
working. The get_top_albums test is temporarily skipped pending the
top_albums_attach_metadata update.
2026-04-14 14:12:14 +01:00
Claudio Ortolina b422cccf09 Make format merge assertion order-independent 2026-04-13 13:17:03 +01:00
Claudio Ortolina 9b44b7050f Count unique catalog entries in collection summary
The record_count now reflects the number of grouped catalog lines
the model sees, not raw DB rows. Two copies of the same album on
different formats count as 1 catalog entry.
2026-04-13 13:12:30 +01:00
Claudio Ortolina 8108645718 Shrink collection summary to fit OpenAI token limit
Deduplicate records by musicbrainz_id (merging formats)
and cap genres to 3 per record, reducing the summary from
~32K to ~24K estimated tokens.
2026-04-13 12:33:15 +01:00
Claudio Ortolina 34f484cc81 Add CollectionChat StreamProvider 2026-04-13 10:09:50 +01:00
Claudio Ortolina 24a3cc9265 Add Collection.collection_summary/0 2026-04-13 10:08:01 +01:00
Claudio Ortolina d845ab93dd Add collection sentinel UUID to Chats context 2026-04-13 10:06:16 +01:00