Remove AI tooling, and related packages/docs
This commit is contained in:
Claudio Ortolina
@@ -1,79 +0,0 @@
|
||||
# Advanced Perceptually-Aware Color Extraction Algorithms
|
||||
|
||||
Modern image color extraction demands algorithms that understand visual perception, spatial context, and semantic importance rather than simple pixel frequency statistics. This comprehensive analysis covers state-of-the-art techniques specifically designed for extracting the perceptual "essence" of images like album covers, where artistic elements carry greater weight than raw pixel counts.
|
||||
|
||||
## Saliency-based attention algorithms prioritize visually important colors
|
||||
|
||||
**Graph-Based Visual Saliency (GBVS)** represents the most effective traditional approach for attention-guided color extraction. Unlike pixel-frequency methods, GBVS achieves **98% correlation with human attention patterns** by treating saliency as an equilibrium distribution of Markov chains on image graphs. The algorithm creates activation maps based on feature dissimilarity rather than simple center-surround differences, then normalizes these through random walks to produce probabilistic attention maps.
|
||||
|
||||
For practical implementation, GBVS can be integrated with color clustering through **saliency-weighted sampling**: instead of treating all pixels equally, pixels are sampled for k-means clustering proportionally to their saliency scores. This approach naturally emphasizes colors from visually important regions while de-emphasizing background areas.
|
||||
|
||||
Modern **transformer-based saliency models** like Vision Transformers provide even superior performance through global context modeling. These architectures capture long-range color relationships across entire images using self-attention mechanisms, avoiding the spatial limitations of traditional convolutional approaches. **CLIP-ViT models demonstrate particularly strong color understanding** when combined with natural language descriptions.
|
||||
|
||||
The key implementation strategy involves a multi-stage pipeline: generate saliency maps using GBVS or CNN-based models, apply attention-guided pixel sampling weighted by saliency scores, perform clustering in perceptual color spaces, and rank final palette colors by importance based on their saliency contributions.
|
||||
|
||||
## Perceptual color spaces align extraction with human vision
|
||||
|
||||
Traditional RGB-based color extraction fails because **equal RGB distances don't represent equal perceptual differences**. Perceptual color spaces like **CIELAB and CAM16-UCS** provide approximately uniform representations where numerical differences correspond to similar perceived color differences.
|
||||
|
||||
**CIELAB's superiority emerges from opponent color theory alignment** - the L* lightness channel, a* green-red axis, and b* blue-yellow axis match human visual system structure. Research shows the 'b' channel correlates most strongly with visual attention, making it particularly valuable for saliency-guided extraction.
|
||||
|
||||
**Advanced color difference metrics like CIEDE2000** dramatically improve clustering quality over simple Euclidean distance. For red-blue color comparisons, CIE2000 achieves perceptual accuracy scores of 52.9 versus 176.3 for basic RGB distance calculations. The algorithm incorporates hue rotation terms, neutral color compensation, and sophisticated weighting factors for lightness, chroma, and hue differences.
|
||||
|
||||
**Color appearance models like CAM16** add another sophistication layer by predicting how colors appear under different viewing conditions. These models include chromatic adaptation transforms (CAT16) that simulate human visual adaptation to different illuminants, ensuring consistent color extraction across varying lighting conditions.
|
||||
|
||||
Practical implementation involves converting RGB to CIELAB color space, applying CAT16 transforms for viewing condition normalization, using CIE2000 distance metrics in clustering algorithms, and weighting color contributions based on psychophysical principles like Weber-Fechner law applications.
|
||||
|
||||
## Spatial awareness through edge detection and region analysis
|
||||
|
||||
**Spatially-aware algorithms weight colors based on structural importance** rather than pure frequency. Edge-guided color weighting assigns higher importance to colors near strong boundaries, as these often represent perceptually significant object boundaries and artistic elements crucial in album cover design.
|
||||
|
||||
**SLIC superpixels provide optimal spatial coherence** by clustering pixels in combined spatial-color space [x, y, L, a, b]. The algorithm uses distance measures that balance color similarity with spatial proximity, creating regions that maintain both color consistency and spatial coherence. Each superpixel represents a meaningful color region whose contribution can be weighted by size, edge strength, and spatial position.
|
||||
|
||||
**Multi-scale analysis using Gaussian-Laplacian pyramids** captures color information at different hierarchical levels. Coarse scales identify major color themes while fine scales extract detailed variations. This approach proves particularly effective for complex artistic images where color significance varies across scales.
|
||||
|
||||
**Graph-based segmentation (Felzenszwalb-Huttenlocher)** provides adaptive region boundaries by representing images as weighted graphs where edge weights measure color dissimilarity between adjacent pixels. The algorithm merges regions based on internal homogeneity versus external differences, automatically adapting to image content complexity.
|
||||
|
||||
For album covers and artistic content, **figure-ground segmentation using GrabCut energy minimization** effectively separates foreground artistic elements from backgrounds. The algorithm combines Gaussian mixture models for color learning with spatial smoothness terms, enabling automatic identification of artistically important regions that should dominate the color palette.
|
||||
|
||||
## Deep learning enables semantic color understanding
|
||||
|
||||
**Modern CNN architectures like U-Net and ResNet** extract hierarchical color features that capture semantic relationships beyond simple clustering. CNN-based color palette extraction using pre-trained features achieves superior palette generation compared to pixel-based methods by understanding object-level color relationships.
|
||||
|
||||
**Vision transformers revolutionize color analysis** through global context modeling without spatial limitations. The self-attention mechanism captures long-range color dependencies across entire images, making them particularly effective for artistic content where color relationships span large spatial distances.
|
||||
|
||||
**Generative adversarial networks (GANs) trained on professional design databases** learn color harmony principles directly from human-curated content. Colormind's GAN architecture achieves **91%+ accuracy in user studies** by training on Adobe Color and Dribbble palettes, demonstrating how neural networks can internalize aesthetic color principles.
|
||||
|
||||
**Semantic segmentation models like DeepLab and Mask R-CNN** enable object-specific color analysis. These models identify semantically important regions (people, objects, text) and weight their color contributions more heavily than background areas, particularly valuable for album covers where specific elements carry artistic significance.
|
||||
|
||||
**End-to-end learned approaches** combine saliency detection, segmentation, and color extraction in unified frameworks. Models like SU2GE-Net use Swin TransformerV2 backbones for simultaneous saliency detection and color analysis, achieving state-of-the-art performance on complex artistic imagery.
|
||||
|
||||
## Industry implementations provide production-ready solutions
|
||||
|
||||
**Adobe's Creative Suite employs multi-modal color analysis** combining k-means clustering with Adobe Sensei AI for content-aware color matching. Their system uses multiple color spaces (sRGB, Adobe RGB, Lab, HSV) with neural networks for perceptual color space mapping and harmony suggestions based on color theory principles.
|
||||
|
||||
**Google's Vision API provides scalable cloud-based color extraction** using convolutional neural networks trained on massive image datasets. The service returns confidence scores and pixel fractions for each extracted color, enabling sophisticated weighting schemes for final palette generation.
|
||||
|
||||
**Apple's Vision Framework optimizes for on-device processing** using their Neural Engine for real-time color analysis. The system achieves sub-4ms processing times while maintaining privacy through local computation, making it ideal for interactive applications.
|
||||
|
||||
For practical implementation, **libvips integration offers 4-5x faster processing than ImageMagick** with comprehensive color space support and built-in color difference calculations (dE76, dE00, dECMC). The library's demand-driven processing architecture scales efficiently for production web applications.
|
||||
|
||||
## Implementation strategies for advanced color extraction
|
||||
|
||||
**Multi-stage processing pipelines** provide the most effective approach: begin with saliency detection using GBVS or transformer models, perform spatial segmentation using SLIC superpixels or semantic segmentation, extract regional color information weighted by saliency and spatial importance, cluster in perceptual color spaces using CIE2000 distances, and rank final colors by semantic and artistic importance.
|
||||
|
||||
**Performance optimization requires careful algorithmic choices**. For real-time applications, GBVS with OpenCV provides good balance of quality and speed. For highest quality results, transformer-based saliency with semantic segmentation yields superior artistic relevance. GPU acceleration enables real-time processing of high-resolution images through CUDA implementations.
|
||||
|
||||
**Color space selection impacts results significantly**. CIELAB provides good perceptual uniformity for most applications. CAM16-UCS offers superior performance when viewing conditions vary. OKLab represents the newest advancement with improved perceptual uniformity over CIELAB.
|
||||
|
||||
**Library integration strategies** depend on requirements: Google Vision API for cloud-scale processing, Apple Vision Framework for iOS/macOS applications, OpenCV with scikit-learn for custom implementations, and libvips for high-performance server applications.
|
||||
|
||||
## Future directions in perceptual color extraction
|
||||
|
||||
The field continues advancing through **multi-modal integration combining visual and semantic cues**. Vision-language models like CLIP demonstrate strong color understanding when guided by textual descriptions, enabling more sophisticated artistic color analysis.
|
||||
|
||||
**Personalized attention models** account for individual differences in color perception and aesthetic preferences. These systems adapt color extraction based on user feedback and demographic factors, particularly relevant for consumer applications.
|
||||
|
||||
**Foundation models for color analysis** leverage large-scale pre-training to understand color relationships across diverse domains. These models show promise for few-shot learning on specific artistic styles or cultural color preferences.
|
||||
|
||||
The convergence of attention-based methods, perceptual color science, spatial analysis, and deep learning creates a powerful toolkit for extracting truly meaningful color palettes from artistic content. These advanced techniques finally move beyond crude pixel counting to understand the visual and aesthetic essence that makes colors truly important in human perception.
|
||||
@@ -1,104 +0,0 @@
|
||||
# Record Management
|
||||
|
||||
- Import a record from a MusicBrainz release ID
|
||||
- Import a record from a MusicBrainz release group
|
||||
- Search records with filtering and pagination
|
||||
- Get random record from collection
|
||||
- Get latest purchased record
|
||||
- Refresh record metadata from MusicBrainz
|
||||
- Generate genres for a record using AI
|
||||
- Update cover art from MusicBrainz
|
||||
- Extract dominant colors from album artwork
|
||||
- Create, update, or delete records
|
||||
- Resize cover art to standard dimensions
|
||||
|
||||
# Collection Operations
|
||||
|
||||
- Add record to collection (mark as purchased)
|
||||
- Search collection with various filters
|
||||
- View collection statistics by format (CD, vinyl, etc.)
|
||||
- View collection statistics by type (album, EP, etc.)
|
||||
- Get top artists in collection by record count
|
||||
- Get top genres in collection by record count
|
||||
|
||||
# Wishlist Operations
|
||||
|
||||
- Add record to wishlist
|
||||
- Search wishlist records
|
||||
- Count total wishlist items
|
||||
- Check which releases are wishlisted
|
||||
|
||||
# Last.fm Integration
|
||||
|
||||
- Scrobble a release to Last.fm
|
||||
- Scrobble a specific disc/medium to Last.fm
|
||||
- Import listening history from Last.fm
|
||||
- Get top albums by time period (7/30/90/365 days)
|
||||
- Get top artists by time period
|
||||
- Get most recent scrobbled track for a record
|
||||
- Backfill historical Last.fm data
|
||||
- Refresh scrobbled tracks from Last.fm
|
||||
- Get Last.fm authentication session
|
||||
|
||||
# Search Operations
|
||||
|
||||
- Universal search across collection, wishlist, and artists
|
||||
- Search collection records
|
||||
- Search wishlist records
|
||||
- Search artists by name
|
||||
- Search records by barcode
|
||||
|
||||
# Artist Management
|
||||
|
||||
- Fetch artist information from MusicBrainz and Discogs
|
||||
- Get similar artists from Last.fm that are in collection
|
||||
- Download and cache artist images
|
||||
- View artist's records in collection
|
||||
- Clean up unused artist information
|
||||
|
||||
# Data Quality & Rules
|
||||
|
||||
- Create scrobble correction rules for album names
|
||||
- Create scrobble correction rules for artist names
|
||||
- Apply correction rules to scrobbled tracks
|
||||
- Count tracks affected by correction rules
|
||||
- Enable/disable correction rules
|
||||
|
||||
# Barcode Scanning
|
||||
|
||||
- Scan barcode and check collection/wishlist status
|
||||
- Import multiple scanned records in batch
|
||||
- Import single scanned record from barcode
|
||||
|
||||
# External Store Integration
|
||||
|
||||
- Create templates for online music stores
|
||||
- Generate purchase URLs from store templates
|
||||
- Manage enabled/disabled store templates
|
||||
|
||||
# Secrets Management
|
||||
|
||||
- Store encrypted API keys and credentials
|
||||
- Retrieve encrypted secrets by name
|
||||
|
||||
# Background Processing
|
||||
|
||||
- Queue color extraction jobs
|
||||
- Queue artist info fetching jobs
|
||||
- Queue cover art refresh jobs
|
||||
- Queue metadata refresh jobs
|
||||
- Queue artist image download jobs
|
||||
- Clean up unused artist information
|
||||
|
||||
# Statistics & Analytics
|
||||
|
||||
- View listening statistics by time period
|
||||
- Compare listening trends across multiple periods
|
||||
- Count records by format, type, artist, or genre
|
||||
- Get collection overview statistics
|
||||
|
||||
# Data Import/Export
|
||||
|
||||
- Import batch tracks from Last.fm API
|
||||
- Parse Obsidian music notes with frontmatter
|
||||
- Export listening data (via statistics views)
|
||||
@@ -1,186 +0,0 @@
|
||||
# Database Structure
|
||||
|
||||
<!--toc:start-->
|
||||
|
||||
- [Database Structure](#database-structure)
|
||||
- [Entity Relationship Diagram](#entity-relationship-diagram)
|
||||
- [Tables Description](#tables-description)
|
||||
- [Records](#records)
|
||||
- [Records Search Index](#records-search-index)
|
||||
- [Artist Infos](#artist-infos)
|
||||
- [Views](#views)
|
||||
- [Artist Records View](#artist-records-view)
|
||||
- [Triggers](#triggers)
|
||||
- [Indices](#indices)
|
||||
- [Notes](#notes)
|
||||
- [WHY ONE TABLE?](#why-one-table)
|
||||
<!--toc:end-->
|
||||
|
||||
This document describes the database structure of the Music Library application.
|
||||
|
||||
## Entity Relationship Diagram
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
RECORDS {
|
||||
uuid id PK
|
||||
enum type
|
||||
enum format
|
||||
string title
|
||||
string cover_url
|
||||
blob cover_data
|
||||
string cover_hash
|
||||
uuid musicbrainz_id
|
||||
map musicbrainz_data
|
||||
string[] genres
|
||||
string release_date
|
||||
datetime purchased_at
|
||||
string selected_release_id
|
||||
string[] release_ids
|
||||
string[] included_release_group_ids
|
||||
Artist[] artists
|
||||
datetime inserted_at
|
||||
datetime updated_at
|
||||
}
|
||||
|
||||
RECORDS_SEARCH_INDEX {
|
||||
uuid id PK
|
||||
enum type
|
||||
enum format
|
||||
string title
|
||||
string normalized_title
|
||||
Artist[] artists
|
||||
string normalized_artists
|
||||
string[] genres
|
||||
uuid musicbrainz_id
|
||||
string[] release_ids
|
||||
string[] included_release_group_ids
|
||||
string cover_hash
|
||||
datetime purchased_at
|
||||
string release_date
|
||||
}
|
||||
|
||||
ARTIST_INFOS {
|
||||
uuid id PK
|
||||
map musicbrainz_data
|
||||
map discogs_data
|
||||
blob image_data
|
||||
string image_data_hash
|
||||
integer image_data_width
|
||||
datetime inserted_at
|
||||
datetime updated_at
|
||||
}
|
||||
|
||||
ARTIST_RECORDS {
|
||||
uuid musicbrainz_id
|
||||
uuid record_id
|
||||
map artist
|
||||
}
|
||||
|
||||
RECORDS ||--o{ RECORDS_SEARCH_INDEX : "syncs via triggers"
|
||||
RECORDS ||--o{ ARTIST_INFOS : "references via musicbrainz_id"
|
||||
RECORDS ||--o{ ARTIST_RECORDS : "extracted via view"
|
||||
```
|
||||
|
||||
## Tables Description
|
||||
|
||||
### Records
|
||||
|
||||
The main table storing music records. Key features:
|
||||
|
||||
- Uses UUID as primary key
|
||||
- Stores basic record information (title, type [album, ep, live, compilation, single, other], format [cd, backup, vinyl, blu_ray, dvd, multi])
|
||||
- Includes MusicBrainz integration with IDs and additional data (musicbrainz_id, musicbrainz_data)
|
||||
- Stores cover image data and URLs (cover_url, cover_data, cover_hash)
|
||||
- Embeds artists data as an array of objects (each with musicbrainz_id, name, sort_name, disambiguation)
|
||||
- Includes timestamps for record keeping
|
||||
- Tracks purchase status via `purchased_at` field
|
||||
- Stores release information including multiple release IDs (`release_ids`), included release group IDs (`included_release_group_ids`), and a selected release ID (`selected_release_id`)
|
||||
- Maintains release date information (`release_date`)
|
||||
|
||||
### Records Search Index
|
||||
|
||||
A virtual FTS5 (Full Text Search) table that mirrors the records table for efficient searching:
|
||||
|
||||
- Automatically synced with the records table via triggers
|
||||
- Optimized for full-text search operations
|
||||
- Contains most fields from the records table, including:
|
||||
- id, type, format, title, normalized_title, artists, normalized_artists, genres, musicbrainz_id, release_ids, included_release_group_ids, cover_hash, purchased_at, release_date
|
||||
- `normalized_title` and `normalized_artists` are unaccented versions for improved search
|
||||
- Some fields are marked as UNINDEXED for efficiency
|
||||
|
||||
### Artist Infos
|
||||
|
||||
A table that stores additional artist information:
|
||||
|
||||
- Uses UUID as primary key
|
||||
- Stores MusicBrainz and Discogs data for artists
|
||||
- Maintains artist image data with dimensions and hash
|
||||
- Includes timestamps for record keeping
|
||||
|
||||
### Views
|
||||
|
||||
#### Artist Records View
|
||||
|
||||
A view that extracts artist information from the embedded JSON in the records table:
|
||||
|
||||
```sql
|
||||
CREATE VIEW artist_records AS
|
||||
SELECT json_each.value ->> '$.musicbrainz_id' AS musicbrainz_id,
|
||||
records.id AS record_id,
|
||||
json_each.value as artist
|
||||
FROM records,
|
||||
json_each(records.artists)
|
||||
```
|
||||
|
||||
This view is crucial for querying artist information as it:
|
||||
|
||||
- Extracts individual artists from the embedded JSON array in the records table
|
||||
- Provides a normalized view of the artist-record relationships
|
||||
- Makes it easier to query records by artist
|
||||
- Maintains the relationship between records and their artists without requiring a separate join table
|
||||
|
||||
### Triggers
|
||||
|
||||
The following triggers maintain the search index:
|
||||
|
||||
1. `records_after_insert`: Inserts new record data into search index after record creation
|
||||
2. `records_after_update`: Updates record data in search index after record updates
|
||||
|
||||
### Indices
|
||||
|
||||
The following indices are maintained for performance:
|
||||
|
||||
1. On `records`:
|
||||
- `type`
|
||||
- `format`
|
||||
- `title`
|
||||
- `musicbrainz_id`
|
||||
- `purchased_at`
|
||||
- `included_release_group_ids`
|
||||
- `release_ids`
|
||||
|
||||
## Notes
|
||||
|
||||
1. The database uses SQLite as the primary database.
|
||||
2. Artists data is embedded directly in the records table as an array of objects, not a separate table.
|
||||
3. The search index is implemented using SQLite's FTS5 extension for efficient full-text search capabilities, with normalized (unaccented) fields for better search.
|
||||
4. Where needed, queries use SQLite's `unicode` extension to filter/sort over UTF-8 data.
|
||||
5. The database supports both collection and wishlist functionality through the `purchased_at` field:
|
||||
- Records with `purchased_at IS NOT NULL` are in the collection
|
||||
- Records with `purchased_at IS NULL` are in the wishlist
|
||||
6. The schema uses `release_date` for clarity
|
||||
7. The `selected_release_id` field tracks the primary release for a record
|
||||
8. The `artist_infos` table stores additional artist metadata and images
|
||||
9. The `artist_records` view provides a normalized way to query artist-record relationships
|
||||
|
||||
## WHY ONE TABLE?
|
||||
|
||||
In traditional relational database design, you would split out artists into a
|
||||
separate table, and associate them with records via a join table. So why
|
||||
sticking with one table?
|
||||
|
||||
1. You only need to backup/export one table.
|
||||
2. Re-fetching data from MusicBrainz becomes trivial, as it just needs to update one field and everything else cascades accordingly.
|
||||
3. Traditional efficiency design constraints [do not apply to
|
||||
SQLite](https://www.sqlite.org/np1queryprob.html), so it makes it easier to experiment with alternative database designs.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,264 +0,0 @@
|
||||
╭─────────────────────────────────────────────────────────╮
|
||||
│ ✻ Welcome to Claude Code! │
|
||||
│ │
|
||||
│ /help for help, /status for your current setup │
|
||||
│ │
|
||||
│ cwd: /Users/cloud/github.com/cloud8421/music_library │
|
||||
╰─────────────────────────────────────────────────────────╯
|
||||
|
||||
> Can you optimize this query?
|
||||
|
||||
SELECT s0."scrobbled_at_uts", s0."musicbrainz_id", s0."title", s0."cover_url", s0."scrobbled_at_label", s0."artist", s0."album",
|
||||
s0."last_fm_data", s1."record_id", s2."record_id", s3."musicbrainz_id", coalesce(s1."cover_hash", s2."cover_hash") FROM
|
||||
"scrobbled_tracks" AS s0 LEFT OUTER JOIN (SELECT records.id AS "record_id", records.cover_hash AS "cover_hash", sf0."value" AS
|
||||
"release_id" FROM records, json_each(records.release_ids) AS sf0 WHERE (records.purchased_at IS NOT NULL)) AS s1 ON
|
||||
s1."release_id" = s0."album" ->> '$.musicbrainz_id' LEFT OUTER JOIN (SELECT records.id AS "record_id", records.cover_hash AS
|
||||
"cover_hash", sf0."value" AS "release_id" FROM records, json_each(records.release_ids) AS sf0 WHERE (records.purchased_at IS
|
||||
NULL)) AS s2 ON s2."release_id" = s0."album" ->> '$.musicbrainz_id' LEFT OUTER JOIN (SELECT DISTINCT sa0."musicbrainz_id" AS
|
||||
"musicbrainz_id", sa0."record_id" AS "record_id", sa0."artist" AS "artist" FROM "artist_records" AS sa0) AS s3 ON (s2."record_id"
|
||||
= s3."record_id") OR (s1."record_id" = s3."record_id") ORDER BY s0."scrobbled_at_uts" DESC LIMIT 100
|
||||
|
||||
⏺ I'll analyze this query to identify optimization opportunities. Let me first examine the current database schema and understand
|
||||
the query structure.
|
||||
⎿ * LastFm.Album at lib/last_fm/album.ex:1
|
||||
* LastFm.Artist at lib/last_fm/artist.ex:1
|
||||
* LastFm.Track at lib/last_fm/track.ex:1
|
||||
… +11 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Search(pattern: "scrobbled_tracks")
|
||||
⎿ Found 11 files (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/last_fm/track.ex)
|
||||
⎿ Read 95 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Read(priv/repo/migrations/20250530203701_create_scrobbled_tracks.exs)
|
||||
⎿ Read 19 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/music_library/records/record.ex)
|
||||
⎿ Read 276 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Now let me find where this query is being generated to understand the context:
|
||||
|
||||
⏺ Search(pattern: "FROM.*scrobbled_tracks.*AS s0")
|
||||
⎿ Found 0 files
|
||||
|
||||
⏺ Search(pattern: "scrobbled_tracks.*LEFT OUTER JOIN", output_mode: "content")
|
||||
⎿ Found 0 lines
|
||||
|
||||
⏺ Search(pattern: "json_each.*release_ids")
|
||||
⎿ Found 3 files (ctrl+o to expand)
|
||||
|
||||
⏺ Search(pattern: "scrobbled_tracks.*s0", output_mode: "content")
|
||||
⎿ Found 0 lines
|
||||
|
||||
⏺ Let me search in the last_fm.ex file where this query might be generated:
|
||||
|
||||
⏺ Read(lib/last_fm.ex)
|
||||
⎿ Read 87 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Let me search for more specific patterns to find where this query is generated:
|
||||
|
||||
⏺ Search(pattern: "purchased_at IS NOT NULL")
|
||||
⎿ Found 2 files (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/music_library/collection.ex)
|
||||
⎿ Read 127 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Search(pattern: "coalesce.*cover_hash")
|
||||
⎿ Found 1 file (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/music_library/scrobble_activity.ex)
|
||||
⎿ Read 472 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Found it! The query comes from the recent_activity/2 function in MusicLibrary.ScrobbleActivity. Let me examine the database
|
||||
indexes to understand the current optimization opportunities:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name, sql FROM sqlite_master WHERE type='index' AND tbl_name IN
|
||||
('scrobbled_tracks', 'records') ORDER BY tbl_name, name")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name", "sql"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name, sql FROM sqlite_master WHERE type='index' ORDER BY name")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name", "sql"],
|
||||
… +7 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Let me check what indexes exist specifically for the tables we're working with:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('scrobbled_tracks')")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('records')")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name FROM sqlite_master WHERE type='table' ORDER BY name")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Let me check if the scrobbled_tracks table actually exists and examine its structure:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: ".tables")
|
||||
⎿ Error: Failed to execute query: %Exqlite.Error{message: "near \".\": syntax error", statement: ".tables"}
|
||||
|
||||
⏺ Bash(mix ecto.migrate)
|
||||
⎿ 21:39:52.643 [info] Migrations already up
|
||||
|
||||
21:39:52.646 [debug] Exqlite: loading extension `/Users/cloud/github.com/cloud8421/music_library/_build/dev/lib/music_library
|
||||
… +5 lines (ctrl+o to expand)
|
||||
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%scrobbled%' OR name
|
||||
LIKE '%records%'")
|
||||
⎿ %Exqlite.Result{command: :execute, columns: ["name"], rows: [], num_rows: 0}
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name FROM sqlite_master WHERE type='table'", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name"],
|
||||
… +28 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Now let me check the existing indexes:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('scrobbled_tracks')", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('records')", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +18 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_info('scrobbled_tracks_scrobbled_at_uts_title_index')", repo:
|
||||
"MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seqno", "cid", "name"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_info('records_release_ids_index')", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seqno", "cid", "name"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Query Optimization Analysis
|
||||
|
||||
The query from MusicLibrary.ScrobbleActivity.recent_activity/2 has several optimization opportunities:
|
||||
|
||||
Current Issues:
|
||||
|
||||
1. Complex subqueries with json_each(): The query uses json_each() on release_ids arrays for both collection and wishlist joins,
|
||||
which is expensive for each row.
|
||||
2. Multiple LEFT JOINs with subqueries: Three separate subqueries are executed and joined, creating a cartesian product that's
|
||||
then filtered.
|
||||
3. JSON extraction in WHERE clauses: Using fragment("? ->> '$.musicbrainz_id'", t.album) requires JSON parsing for every row.
|
||||
4. Missing indexes: No composite indexes for the JSON-based join conditions.
|
||||
|
||||
Recommended Optimizations:
|
||||
|
||||
1. Add Strategic Indexes
|
||||
|
||||
-- Index for album musicbrainz_id JSON extraction (most critical)
|
||||
CREATE INDEX idx_scrobbled_tracks_album_mbid
|
||||
ON scrobbled_tracks((album ->> '$.musicbrainz_id'));
|
||||
|
||||
-- Composite index for the main query
|
||||
CREATE INDEX idx_scrobbled_tracks_scrobbled_at_album_mbid
|
||||
ON scrobbled_tracks(scrobbled_at_uts DESC, (album ->> '$.musicbrainz_id'));
|
||||
|
||||
2. Rewrite the Query Structure
|
||||
|
||||
Instead of multiple subqueries, use CTEs and reduce the number of joins:
|
||||
|
||||
def recent_activity_optimized(timezone, limit \\ 100) do
|
||||
# Single CTE for all releases with their collection/wishlist status
|
||||
releases_cte = """
|
||||
WITH releases AS (
|
||||
SELECT
|
||||
r.id as record_id,
|
||||
r.cover_hash,
|
||||
r.purchased_at IS NOT NULL as is_collected,
|
||||
sf.value as release_id
|
||||
FROM records r, json_each(r.release_ids) sf
|
||||
)
|
||||
"""
|
||||
|
||||
query = from t in Track,
|
||||
left_join: r in fragment("""
|
||||
(SELECT record_id, cover_hash, release_id,
|
||||
CASE WHEN is_collected THEN record_id END as collected_record_id,
|
||||
CASE WHEN NOT is_collected THEN record_id END as wishlisted_record_id
|
||||
FROM releases)
|
||||
"""),
|
||||
on: r.release_id == fragment("? ->> '$.musicbrainz_id'", t.album),
|
||||
left_join: ar in ArtistRecord,
|
||||
on: ar.record_id == r.record_id,
|
||||
order_by: [desc: t.scrobbled_at_uts],
|
||||
limit: ^limit,
|
||||
select: %{
|
||||
track: t,
|
||||
collected_record_id: r.collected_record_id,
|
||||
wishlisted_record_id: r.wishlisted_record_id,
|
||||
artist_id: ar.musicbrainz_id,
|
||||
cover_hash: r.cover_hash
|
||||
}
|
||||
end
|
||||
|
||||
3. Alternative: Materialized View Approach
|
||||
|
||||
Create a materialized view that pre-joins release data:
|
||||
|
||||
CREATE VIEW release_lookup AS
|
||||
SELECT
|
||||
r.id as record_id,
|
||||
r.cover_hash,
|
||||
r.purchased_at IS NOT NULL as is_collected,
|
||||
sf.value as release_id
|
||||
FROM records r, json_each(r.release_ids) sf;
|
||||
|
||||
CREATE INDEX idx_release_lookup_release_id ON release_lookup(release_id);
|
||||
|
||||
4. Denormalization Option
|
||||
|
||||
Consider adding a release_musicbrainz_id field directly to scrobbled_tracks to avoid JSON extraction:
|
||||
|
||||
# In migration
|
||||
add :album_musicbrainz_id, :string
|
||||
create index(:scrobbled_tracks, [:album_musicbrainz_id])
|
||||
create index(:scrobbled_tracks, [:scrobbled_at_uts, :album_musicbrainz_id])
|
||||
|
||||
⏺ Immediate Impact Recommendations:
|
||||
|
||||
High Impact, Low Risk:
|
||||
1. Add the JSON extraction index: CREATE INDEX idx_scrobbled_tracks_album_mbid ON scrobbled_tracks((album ->>
|
||||
'$.musicbrainz_id'))
|
||||
2. Add composite index: CREATE INDEX idx_scrobbled_tracks_scrobbled_at_album_mbid ON scrobbled_tracks(scrobbled_at_uts DESC,
|
||||
(album ->> '$.musicbrainz_id'))
|
||||
|
||||
Medium Impact, Medium Risk:
|
||||
3. Rewrite query to use a single CTE instead of multiple subqueries
|
||||
4. Create the release_lookup view for better join performance
|
||||
|
||||
High Impact, Higher Risk:
|
||||
5. Denormalize by adding album_musicbrainz_id column to eliminate JSON extraction entirely
|
||||
|
||||
The most critical optimization is adding indexes for the JSON extraction operations, which should provide immediate performance
|
||||
improvements with no code changes required.
|
||||
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,546 +0,0 @@
|
||||
# Online Store Quick Check System - Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines the implementation of a system to quickly check for the presence of wishlisted records on configurable online record stores. The system will add an expandable section to record details pages with links to online stores, generated from user-configurable templates.
|
||||
|
||||
## Requirements Analysis
|
||||
|
||||
### Core Features
|
||||
|
||||
- **Expandable section on record details**: Collapsed by default, shows links to online stores
|
||||
- **Configurable store templates**: User-managed templates stored in database
|
||||
- **Template variables**: Support for record artist and title as search parameters
|
||||
- **User management interface**: Dedicated section for managing store templates
|
||||
|
||||
### Example Use Case
|
||||
|
||||
Create a template for Amazon UK search: `https://www.amazon.co.uk/s?k={artist}+{title}+vinyl`
|
||||
When viewing a record by "Pink Floyd" with title "The Wall", generate: `https://www.amazon.co.uk/s?k=Pink+Floyd+The+Wall+vinyl`
|
||||
|
||||
## Database Schema Design
|
||||
|
||||
### Online Store Templates Table
|
||||
|
||||
Following the application's established patterns:
|
||||
|
||||
```elixir
|
||||
# Migration: create_online_store_templates.exs
|
||||
defmodule MusicLibrary.Repo.Migrations.CreateOnlineStoreTemplates do
|
||||
use Ecto.Migration
|
||||
|
||||
def change do
|
||||
create table(:online_store_templates, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :name, :string, null: false
|
||||
add :description, :text
|
||||
add :url_template, :text, null: false
|
||||
add :enabled, :boolean, default: true, null: false
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
create index(:online_store_templates, [:enabled])
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Schema Module
|
||||
|
||||
```elixir
|
||||
# lib/music_library/online_store_templates/online_store_template.ex
|
||||
defmodule MusicLibrary.OnlineStoreTemplates.OnlineStoreTemplate do
|
||||
use Ecto.Schema
|
||||
import Ecto.Changeset
|
||||
|
||||
@primary_key {:id, :binary_id, autogenerate: true}
|
||||
@foreign_key_type :binary_id
|
||||
|
||||
schema "online_store_templates" do
|
||||
field :name, :string
|
||||
field :description, :string
|
||||
field :url_template, :string
|
||||
field :enabled, :boolean, default: true
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
@doc false
|
||||
def changeset(template, attrs) do
|
||||
template
|
||||
|> cast(attrs, [:name, :description, :url_template, :enabled])
|
||||
|> validate_required([:name, :url_template])
|
||||
|> validate_length(:name, min: 1, max: 100)
|
||||
|> validate_length(:url_template, min: 1, max: 500)
|
||||
|> validate_url_template()
|
||||
end
|
||||
|
||||
defp validate_url_template(changeset) do
|
||||
if template = get_field(changeset, :url_template) do
|
||||
case URI.parse(template) do
|
||||
%URI{scheme: scheme} when scheme in ["http", "https"] -> changeset
|
||||
_ -> add_error(changeset, :url_template, "must be a valid HTTP or HTTPS URL")
|
||||
end
|
||||
else
|
||||
changeset
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Context Module
|
||||
|
||||
```elixir
|
||||
# lib/music_library/online_store_templates.ex
|
||||
defmodule MusicLibrary.OnlineStoreTemplates do
|
||||
@moduledoc """
|
||||
The OnlineStoreTemplates context.
|
||||
"""
|
||||
|
||||
import Ecto.Query, warn: false
|
||||
alias MusicLibrary.Repo
|
||||
alias MusicLibrary.OnlineStoreTemplates.OnlineStoreTemplate
|
||||
|
||||
@doc """
|
||||
Returns the list of enabled online store templates ordered by name.
|
||||
"""
|
||||
def list_enabled_templates do
|
||||
OnlineStoreTemplate
|
||||
|> where([t], t.enabled == true)
|
||||
|> order_by([t], [asc: t.name])
|
||||
|> Repo.all()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Returns the list of all online store templates for management.
|
||||
"""
|
||||
def list_templates do
|
||||
OnlineStoreTemplate
|
||||
|> order_by([t], [asc: t.name])
|
||||
|> Repo.all()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Gets a single online store template.
|
||||
"""
|
||||
def get_template!(id), do: Repo.get!(OnlineStoreTemplate, id)
|
||||
|
||||
@doc """
|
||||
Creates an online store template.
|
||||
"""
|
||||
def create_template(attrs \\ %{}) do
|
||||
%OnlineStoreTemplate{}
|
||||
|> OnlineStoreTemplate.changeset(attrs)
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Updates an online store template.
|
||||
"""
|
||||
def update_template(%OnlineStoreTemplate{} = template, attrs) do
|
||||
template
|
||||
|> OnlineStoreTemplate.changeset(attrs)
|
||||
|> Repo.update()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Deletes an online store template.
|
||||
"""
|
||||
def delete_template(%OnlineStoreTemplate{} = template) do
|
||||
Repo.delete(template)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Returns an `%Ecto.Changeset{}` for tracking online store template changes.
|
||||
"""
|
||||
def change_template(%OnlineStoreTemplate{} = template, attrs \\ %{}) do
|
||||
OnlineStoreTemplate.changeset(template, attrs)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Generates a URL from a template by replacing variables with record data.
|
||||
"""
|
||||
def generate_url(template, record) do
|
||||
artists_string = record.artists |> Enum.map(& &1.name) |> Enum.join(" ")
|
||||
|
||||
template.url_template
|
||||
|> String.replace("{artist}", URI.encode(artists_string))
|
||||
|> String.replace("{title}", URI.encode(record.title))
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## User Interface Design
|
||||
|
||||
### Management Interface
|
||||
|
||||
Following the Scrobble Rules pattern for consistency:
|
||||
|
||||
#### Route Addition
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/router.ex
|
||||
# Add to the authenticated scope:
|
||||
live "/online-store-templates", OnlineStoreTemplateLive.Index, :index
|
||||
live "/online-store-templates/new", OnlineStoreTemplateLive.Index, :new
|
||||
live "/online-store-templates/:id/edit", OnlineStoreTemplateLive.Index, :edit
|
||||
```
|
||||
|
||||
#### Navigation Integration
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/components/layouts/app.html.heex
|
||||
# Add to the "More" dropdown menu:
|
||||
<.dropdown_link href={~p"/online-store-templates"}>
|
||||
Online Store Templates
|
||||
</.dropdown_link>
|
||||
```
|
||||
|
||||
#### LiveView Index Page
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/live/online_store_template_live/index.ex
|
||||
defmodule MusicLibraryWeb.OnlineStoreTemplateLive.Index do
|
||||
use MusicLibraryWeb, :live_view
|
||||
|
||||
alias MusicLibrary.OnlineStoreTemplates
|
||||
alias MusicLibrary.OnlineStoreTemplates.OnlineStoreTemplate
|
||||
|
||||
@impl true
|
||||
def mount(_params, _session, socket) do
|
||||
{:ok, stream(socket, :templates, OnlineStoreTemplates.list_templates())}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_params(params, _url, socket) do
|
||||
{:noreply, apply_action(socket, socket.assigns.live_action, params)}
|
||||
end
|
||||
|
||||
defp apply_action(socket, :edit, %{"id" => id}) do
|
||||
socket
|
||||
|> assign(:page_title, "Edit Online Store Template")
|
||||
|> assign(:template, OnlineStoreTemplates.get_template!(id))
|
||||
end
|
||||
|
||||
defp apply_action(socket, :new, _params) do
|
||||
socket
|
||||
|> assign(:page_title, "New Online Store Template")
|
||||
|> assign(:template, %OnlineStoreTemplate{})
|
||||
end
|
||||
|
||||
defp apply_action(socket, :index, _params) do
|
||||
socket
|
||||
|> assign(:page_title, "Online Store Templates")
|
||||
|> assign(:template, nil)
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_info({MusicLibraryWeb.OnlineStoreTemplateLive.FormComponent, {:saved, template}}, socket) do
|
||||
{:noreply, stream_insert(socket, :templates, template)}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_event("delete", %{"id" => id}, socket) do
|
||||
template = OnlineStoreTemplates.get_template!(id)
|
||||
{:ok, _} = OnlineStoreTemplates.delete_template(template)
|
||||
|
||||
{:noreply, stream_delete(socket, :templates, template)}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_event("toggle-enabled", %{"id" => id}, socket) do
|
||||
template = OnlineStoreTemplates.get_template!(id)
|
||||
{:ok, updated_template} = OnlineStoreTemplates.update_template(template, %{enabled: !template.enabled})
|
||||
|
||||
{:noreply, stream_insert(socket, :templates, updated_template)}
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
#### Template Design
|
||||
|
||||
```heex
|
||||
<!-- lib/music_library_web/live/online_store_template_live/index.html.heex -->
|
||||
<div class="px-4 py-6 sm:px-6 lg:px-8">
|
||||
<div class="sm:flex sm:items-center">
|
||||
<div class="sm:flex-auto">
|
||||
<h1 class="text-base font-semibold text-gray-900 dark:text-white">
|
||||
Online Store Templates
|
||||
</h1>
|
||||
<p class="mt-2 text-sm text-gray-700 dark:text-gray-300">
|
||||
Manage templates for generating links to online record stores.
|
||||
</p>
|
||||
</div>
|
||||
<div class="mt-4 sm:ml-16 sm:mt-0 sm:flex-none">
|
||||
<.button patch={~p"/online-store-templates/new"}>
|
||||
Add Template
|
||||
</.button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="mt-8 flow-root">
|
||||
<div class="-mx-4 -my-2 overflow-x-auto sm:-mx-6 lg:-mx-8">
|
||||
<div class="inline-block min-w-full py-2 align-middle sm:px-6 lg:px-8">
|
||||
<ul role="list" class="divide-y divide-gray-200 dark:divide-gray-800" phx-update="stream" id="templates">
|
||||
<li
|
||||
:for={{id, template} <- @streams.templates}
|
||||
id={id}
|
||||
class="flex items-center justify-between gap-x-6 py-5"
|
||||
>
|
||||
<div class="min-w-0">
|
||||
<div class="flex items-start gap-x-3">
|
||||
<p class="text-sm font-semibold text-gray-900 dark:text-white">
|
||||
<%= template.name %>
|
||||
</p>
|
||||
<.badge :if={template.enabled} color="success" size="sm">
|
||||
Enabled
|
||||
</.badge>
|
||||
<.badge :if={!template.enabled} color="warning" size="sm">
|
||||
Disabled
|
||||
</.badge>
|
||||
</div>
|
||||
<div class="mt-1 flex items-center gap-x-2 text-xs text-gray-500 dark:text-gray-400">
|
||||
<p class="truncate"><%= template.url_template %></p>
|
||||
</div>
|
||||
<p :if={template.description} class="mt-1 text-xs text-gray-500 dark:text-gray-400">
|
||||
<%= template.description %>
|
||||
</p>
|
||||
</div>
|
||||
<div class="flex flex-none items-center gap-x-4">
|
||||
<.button
|
||||
phx-click="toggle-enabled"
|
||||
phx-value-id={template.id}
|
||||
color={if template.enabled, do: "warning", else: "success"}
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
>
|
||||
<%= if template.enabled, do: "Disable", else: "Enable" %>
|
||||
</.button>
|
||||
<.button
|
||||
patch={~p"/online-store-templates/#{template}/edit"}
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
>
|
||||
Edit
|
||||
</.button>
|
||||
<.button
|
||||
phx-click={JS.push("delete", value: %{id: template.id})}
|
||||
data-confirm="Are you sure?"
|
||||
color="danger"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
>
|
||||
Delete
|
||||
</.button>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<.modal :if={@live_action in [:new, :edit]} id="template-modal" show on_cancel={JS.patch(~p"/online-store-templates")}>
|
||||
<.live_component
|
||||
module={MusicLibraryWeb.OnlineStoreTemplateLive.FormComponent}
|
||||
id={@template.id || :new}
|
||||
title={@page_title}
|
||||
action={@live_action}
|
||||
template={@template}
|
||||
patch={~p"/online-store-templates"}
|
||||
/>
|
||||
</.modal>
|
||||
```
|
||||
|
||||
### Record Details Integration
|
||||
|
||||
Add to the wishlist show page template:
|
||||
|
||||
```heex
|
||||
<!-- lib/music_library_web/live/wishlist_live/show.html.heex -->
|
||||
<!-- Add this section after the existing record details -->
|
||||
|
||||
<div class="mt-8">
|
||||
<details class="px-4 text-zinc-700 hover:text-zinc-500 dark:text-zinc-400 dark:hover:text-zinc-300">
|
||||
<summary class="text-sm font-medium cursor-pointer">
|
||||
Check Online Stores
|
||||
</summary>
|
||||
<div class="mt-4 space-y-2">
|
||||
<div
|
||||
:for={template <- @online_store_templates}
|
||||
class="flex items-center justify-between py-2 px-3 bg-gray-50 dark:bg-gray-800 rounded-lg"
|
||||
>
|
||||
<span class="text-sm font-medium text-gray-900 dark:text-white">
|
||||
<%= template.name %>
|
||||
</span>
|
||||
<.button
|
||||
href={OnlineStoreTemplates.generate_url(template, @record)}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
class="ml-2"
|
||||
>
|
||||
<.icon name="hero-arrow-top-right-on-square" class="h-4 w-4" />
|
||||
Search
|
||||
</.button>
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
```
|
||||
|
||||
### LiveView Updates
|
||||
|
||||
Update the wishlist show LiveView to load templates:
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/live/wishlist_live/show.ex
|
||||
# Add to the mount function:
|
||||
online_store_templates = MusicLibrary.OnlineStoreTemplates.list_enabled_templates()
|
||||
|
||||
socket =
|
||||
socket
|
||||
|> assign(:record, record)
|
||||
|> assign(:online_store_templates, online_store_templates)
|
||||
|> assign(:page_title, record.title)
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Database Foundation
|
||||
|
||||
1. Create migration for `online_store_templates` table
|
||||
2. Create schema module with validation
|
||||
3. Create context module with CRUD operations
|
||||
4. Run migration and test basic functionality
|
||||
|
||||
### Phase 2: Management Interface
|
||||
|
||||
1. Create LiveView modules for template management
|
||||
2. Create form component for template editing
|
||||
3. Add routes to router
|
||||
4. Add navigation link to main layout
|
||||
5. Style templates using existing design patterns
|
||||
|
||||
### Phase 3: Record Details Integration
|
||||
|
||||
1. Update wishlist show LiveView to load templates
|
||||
2. Add expandable section to show template
|
||||
3. Implement URL generation logic
|
||||
4. Test template variable replacement
|
||||
|
||||
### Phase 4: Testing & Refinement
|
||||
|
||||
1. Add comprehensive tests for context functions
|
||||
2. Add LiveView tests for management interface
|
||||
3. Test URL generation with various record data
|
||||
4. Validate template functionality end-to-end
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### URL Template Validation
|
||||
|
||||
- Validate URL templates contain only HTTP/HTTPS schemes
|
||||
- Sanitize user input in templates
|
||||
- Encode record data properly when generating URLs
|
||||
|
||||
### Access Control
|
||||
|
||||
- Templates management requires authentication
|
||||
- Template visibility follows existing permission patterns
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- Context module functions (CRUD operations)
|
||||
- URL generation with various record data
|
||||
- Template validation logic
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- LiveView template management workflow
|
||||
- Template display on record details pages
|
||||
- URL generation end-to-end
|
||||
|
||||
### Example Test Cases
|
||||
|
||||
```elixir
|
||||
defmodule MusicLibrary.OnlineStoreTemplatesTest do
|
||||
use MusicLibrary.DataCase
|
||||
|
||||
alias MusicLibrary.OnlineStoreTemplates
|
||||
|
||||
describe "generate_url/2" do
|
||||
test "replaces artist and title variables correctly" do
|
||||
template = %OnlineStoreTemplate{
|
||||
url_template: "https://example.com/search?q={artist}+{title}"
|
||||
}
|
||||
|
||||
record = %Record{
|
||||
title: "Dark Side of the Moon",
|
||||
artists: [%Artist{name: "Pink Floyd"}]
|
||||
}
|
||||
|
||||
assert OnlineStoreTemplates.generate_url(template, record) ==
|
||||
"https://example.com/search?q=Pink+Floyd+Dark+Side+of+the+Moon"
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Database Migration Commands
|
||||
|
||||
```bash
|
||||
# Create migration
|
||||
mix ecto.gen.migration create_online_store_templates
|
||||
|
||||
# Run migration
|
||||
mix ecto.migrate
|
||||
|
||||
# Rollback if needed
|
||||
mix ecto.rollback
|
||||
```
|
||||
|
||||
## Default Template Examples
|
||||
|
||||
Consider seeding the database with common store templates:
|
||||
|
||||
```elixir
|
||||
# priv/repo/seeds.exs additions
|
||||
alias MusicLibrary.OnlineStoreTemplates
|
||||
|
||||
templates = [
|
||||
%{
|
||||
name: "Amazon UK",
|
||||
description: "Search Amazon UK for vinyl records",
|
||||
url_template: "https://www.amazon.co.uk/s?k={artist}+{title}+vinyl&i=popular"
|
||||
},
|
||||
%{
|
||||
name: "Discogs",
|
||||
description: "Search Discogs marketplace",
|
||||
url_template: "https://www.discogs.com/search/?q={artist}+{title}&type=all"
|
||||
},
|
||||
%{
|
||||
name: "eBay",
|
||||
description: "Search eBay for records",
|
||||
url_template: "https://www.ebay.co.uk/sch/i.html?_nkw={artist}+{title}+vinyl"
|
||||
}
|
||||
]
|
||||
|
||||
Enum.each(templates, &OnlineStoreTemplates.create_template/1)
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
This implementation provides a clean, maintainable system for checking online stores that:
|
||||
|
||||
- Follows existing application patterns and conventions
|
||||
- Provides a user-friendly management interface
|
||||
- Integrates seamlessly with the existing record details pages
|
||||
- Supports extensibility for future enhancements
|
||||
- Maintains security and validation standards
|
||||
|
||||
The system is designed to be conservative, focusing on core functionality while following the established UI and database patterns throughout the application.
|
||||
|
||||
@@ -1,279 +0,0 @@
|
||||
# Scrobbled Tracks Data Transformation Rules - Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines the implementation of functionality to manage arbitrary rules for fixing data in the `scrobbled_tracks` table. The system will support two types of transformations:
|
||||
|
||||
1. **Album Rule**: Update the album MusicBrainz ID of all tracks belonging to a specific album (identified by album title)
|
||||
2. **Artist Rule**: Update the artist MusicBrainz ID of all tracks belonging to a specific artist (identified by artist name)
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### Existing Database Schema
|
||||
|
||||
- `scrobbled_tracks` table exists with the following structure:
|
||||
- `scrobbled_at_uts` (integer, primary key)
|
||||
- `musicbrainz_id` (string) - track MusicBrainz ID
|
||||
- `title` (string) - track title
|
||||
- `cover_url` (string)
|
||||
- `scrobbled_at_label` (string)
|
||||
- `artist` (map) - embedded artist data with `musicbrainz_id` and `name`
|
||||
- `album` (map) - embedded album data with `musicbrainz_id` and `title`
|
||||
- `last_fm_data` (map) - raw Last.fm API response
|
||||
|
||||
### Existing Schemas
|
||||
|
||||
- `LastFm.Track` - Schema for scrobbled tracks
|
||||
- `LastFm.Artist` - Embedded artist schema
|
||||
- `LastFm.Album` - Embedded album schema
|
||||
|
||||
### Navigation Structure
|
||||
|
||||
- Main navigation has Stats, Collection, Wishlist sections
|
||||
- "More" dropdown contains development tools and utilities
|
||||
- Uses Phoenix LiveView for interactive components
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### 1. Database Migration
|
||||
|
||||
**File**: `priv/repo/migrations/YYYYMMDDHHMMSS_create_scrobble_rules.exs`
|
||||
|
||||
Create `scrobble_rules` table with:
|
||||
|
||||
- `id` (primary key)
|
||||
- `type` (string) - "album" or "artist"
|
||||
- `match_value` (string) - album title or artist name to match
|
||||
- `target_musicbrainz_id` (string) - MusicBrainz ID to set
|
||||
- `enabled` (boolean) - whether rule is active
|
||||
- `description` (text) - optional description for the rule
|
||||
- `inserted_at` (timestamp)
|
||||
- `updated_at` (timestamp)
|
||||
|
||||
Indexes:
|
||||
|
||||
- `type, match_value` (compound index for efficient matching)
|
||||
- `enabled` (for filtering active rules)
|
||||
|
||||
### 2. Schema Module
|
||||
|
||||
**File**: `lib/music_library/scrobble_rules/scrobble_rule.ex`
|
||||
|
||||
Create Ecto schema with:
|
||||
|
||||
- Field definitions matching migration
|
||||
- Validations:
|
||||
- `type` must be "album" or "artist"
|
||||
- `match_value` required and non-empty
|
||||
- `target_musicbrainz_id` required and non-empty
|
||||
- `enabled` defaults to true
|
||||
- Changeset functions for create/update operations
|
||||
|
||||
### 3. Context Module
|
||||
|
||||
**File**: `lib/music_library/scrobble_rules.ex`
|
||||
|
||||
Implement context functions:
|
||||
|
||||
- `list_scrobble_rules(opts \\ [])` - list all rules with optional filtering
|
||||
- `get_scrobble_rule!(id)` - get specific rule by ID
|
||||
- `create_scrobble_rule(attrs)` - create new rule
|
||||
- `update_scrobble_rule(rule, attrs)` - update existing rule
|
||||
- `delete_scrobble_rule(rule)` - delete rule
|
||||
- `change_scrobble_rule(rule, attrs \\ %{})` - create changeset
|
||||
- `list_enabled_rules()` - get only enabled rules for worker
|
||||
- `apply_album_rule(rule)` - apply album transformation
|
||||
- `apply_artist_rule(rule)` - apply artist transformation
|
||||
- `apply_all_rules()` - apply all enabled rules
|
||||
|
||||
### 4. Navigation Update
|
||||
|
||||
**File**: `lib/music_library_web/components/layouts/app.html.heex`
|
||||
|
||||
Add new navigation link in the "More" dropdown:
|
||||
|
||||
- Icon: `hero-adjustments-horizontal`
|
||||
- Label: "Scrobble Rules"
|
||||
- Route: `/scrobble-rules`
|
||||
- Position: After "Oban" link, before separator
|
||||
|
||||
### 5. Router Configuration
|
||||
|
||||
**File**: `lib/music_library_web/router.ex`
|
||||
|
||||
Add new LiveView routes in the authenticated scope:
|
||||
|
||||
- `/scrobble-rules` - index page
|
||||
- `/scrobble-rules/new` - create new rule
|
||||
- `/scrobble-rules/:id/edit` - edit existing rule
|
||||
|
||||
### 6. LiveView Implementation
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobble_rules_live/index.ex`
|
||||
|
||||
Implement LiveView with:
|
||||
|
||||
- List view showing all rules with status indicators
|
||||
- Create/Edit forms with validation
|
||||
- Delete confirmation
|
||||
- Filter controls (by type, enabled/disabled)
|
||||
- Manual rule application triggers
|
||||
- Real-time updates via PubSub
|
||||
|
||||
**Template**: `lib/music_library_web/live/scrobble_rules_live/index.html.heex`
|
||||
|
||||
- Table layout with rule details
|
||||
- Action buttons (Edit, Delete, Apply)
|
||||
- Form components for create/edit
|
||||
- Status indicators for enabled/disabled rules
|
||||
|
||||
### 7. Form Components
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobble_rules_live/form_component.ex`
|
||||
|
||||
Create reusable form component:
|
||||
|
||||
- Type selection (album/artist)
|
||||
- Match value input with validation
|
||||
- Target MusicBrainz ID input
|
||||
- Description textarea
|
||||
- Enable/disable toggle
|
||||
- Save/Cancel actions
|
||||
|
||||
Where possible, use the components available at <https://docs.fluxonui.com/1.1.4/overview.html>.
|
||||
|
||||
### 8. Periodic Worker
|
||||
|
||||
**File**: `lib/music_library/scrobble_rules/worker.ex`
|
||||
|
||||
Implement Oban worker:
|
||||
|
||||
- Scheduled to run periodically (e.g., every 30 minutes)
|
||||
- Fetches all enabled rules
|
||||
- Applies each rule in sequence
|
||||
- Logs application results
|
||||
- Handles errors gracefully
|
||||
- Tracks last application time
|
||||
|
||||
### 9. Testing Strategy
|
||||
|
||||
#### Unit Tests
|
||||
|
||||
**File**: `test/music_library/scrobble_rules_test.exs`
|
||||
|
||||
- Test all context functions
|
||||
- Test rule application logic
|
||||
- Test edge cases and error handling
|
||||
|
||||
**File**: `test/music_library/scrobble_rules/scrobble_rule_test.exs`
|
||||
|
||||
- Test schema validations
|
||||
- Test changeset functions
|
||||
- Test constraint validations
|
||||
|
||||
#### Integration Tests
|
||||
|
||||
**File**: `test/music_library_web/live/scrobble_rules_live_test.exs`
|
||||
|
||||
- Test LiveView interactions
|
||||
- Test form submissions
|
||||
- Test real-time updates
|
||||
- Test navigation and routing
|
||||
|
||||
#### Worker Tests
|
||||
|
||||
**File**: `test/music_library/scrobble_rules/worker_test.exs`
|
||||
|
||||
- Test job execution
|
||||
- Test rule application
|
||||
- Test error handling
|
||||
- Test scheduling
|
||||
|
||||
### 11. Additional Considerations
|
||||
|
||||
#### Data Integrity
|
||||
|
||||
- Ensure atomic operations when applying rules
|
||||
- Validate MusicBrainz IDs before applying
|
||||
- Log all transformations for audit trail
|
||||
|
||||
#### Performance
|
||||
|
||||
- Index optimization for rule matching
|
||||
- Batch processing for large datasets
|
||||
- Rate limiting for external API calls
|
||||
|
||||
#### User Experience
|
||||
|
||||
- Clear feedback on rule application
|
||||
- Preview mode to show what would change
|
||||
- Bulk operations for managing multiple rules
|
||||
- Export/import functionality for rule sets
|
||||
|
||||
#### Security
|
||||
|
||||
- Input validation and sanitization
|
||||
- Rate limiting on rule creation
|
||||
- Audit logging for all changes
|
||||
|
||||
## Database Schema Summary
|
||||
|
||||
```sql
|
||||
-- scrobble_rules table
|
||||
CREATE TABLE scrobble_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
type TEXT NOT NULL CHECK (type IN ('album', 'artist')),
|
||||
match_value TEXT NOT NULL,
|
||||
target_musicbrainz_id TEXT NOT NULL,
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
description TEXT,
|
||||
inserted_at TIMESTAMP NOT NULL,
|
||||
updated_at TIMESTAMP NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX idx_scrobble_rules_type_match ON scrobble_rules(type, match_value);
|
||||
CREATE INDEX idx_scrobble_rules_enabled ON scrobble_rules(enabled);
|
||||
```
|
||||
|
||||
## File Structure Summary
|
||||
|
||||
```
|
||||
lib/music_library/
|
||||
├── scrobble_rules/
|
||||
│ ├── scrobble_rule.ex # Schema
|
||||
│ └── worker.ex # Oban worker
|
||||
└── scrobble_rules.ex # Context
|
||||
|
||||
lib/music_library_web/
|
||||
└── live/
|
||||
└── scrobble_rules_live/
|
||||
├── index.ex # LiveView
|
||||
├── index.html.heex # Template
|
||||
└── form_component.ex # Form component
|
||||
|
||||
priv/repo/migrations/
|
||||
└── YYYYMMDDHHMMSS_create_scrobble_rules.exs
|
||||
|
||||
test/
|
||||
├── music_library/
|
||||
│ ├── scrobble_rules_test.exs
|
||||
│ └── scrobble_rules/
|
||||
│ ├── scrobble_rule_test.exs
|
||||
│ └── worker_test.exs
|
||||
└── music_library_web/
|
||||
└── live/
|
||||
└── scrobble_rules_live_test.exs
|
||||
```
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. Database migration and schema
|
||||
2. Context module with basic CRUD operations
|
||||
3. LiveView interface for rule management
|
||||
4. Navigation and routing updates
|
||||
5. Periodic worker implementation
|
||||
6. Comprehensive testing
|
||||
7. Documentation and deployment
|
||||
|
||||
This plan ensures a robust, maintainable solution that integrates seamlessly with the existing Music Library application architecture.
|
||||
|
||||
@@ -1,278 +0,0 @@
|
||||
# Scrobbled Tracks Management UI Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Build a complete UI for managing scrobbled tracks with index, edit, and delete functionality. The interface will be accessible through the "More" menu in the top navigation and will follow existing patterns from collection and wishlist management.
|
||||
|
||||
## 1. Database & Context Layer
|
||||
|
||||
### Existing Schema
|
||||
|
||||
The `LastFm.Track` schema (`lib/last_fm/track.ex`) already exists with:
|
||||
|
||||
- Primary key: `scrobbled_at_uts` (Unix timestamp)
|
||||
- Fields: `musicbrainz_id`, `title`, `cover_url`, `scrobbled_at_label`
|
||||
- Embedded schemas: `artist` and `album`
|
||||
- Raw `last_fm_data` map
|
||||
|
||||
### Context Functions Needed
|
||||
|
||||
Add to `LastFm` module (`lib/last_fm.ex`):
|
||||
|
||||
- `list_tracks/1` - Paginated track listing with search support
|
||||
- `get_track!/1` - Individual track retrieval by `scrobbled_at_uts`
|
||||
- `update_track/2` - Track metadata updates
|
||||
- `delete_track/1` - Track deletion
|
||||
- `search_tracks_count/1` - Count for pagination totals
|
||||
|
||||
## 2. LiveView Components
|
||||
|
||||
### Index LiveView
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobbled_tracks_live/index.ex`
|
||||
|
||||
**Pattern**: Follow `CollectionLive.Index` structure
|
||||
|
||||
**Features**:
|
||||
|
||||
- Paginated list (200 items per page)
|
||||
- Search functionality by track title/artist name
|
||||
- Stream-based updates for performance (`phx-update="stream"`)
|
||||
- Delete confirmation modals
|
||||
- Edit links to individual track editing
|
||||
|
||||
**Default Parameters**:
|
||||
|
||||
```elixir
|
||||
@default_tracks_list_params %{
|
||||
query: "",
|
||||
page: 1,
|
||||
page_size: 200,
|
||||
order: :scrobbled_at
|
||||
}
|
||||
```
|
||||
|
||||
### Edit Form Component
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobbled_tracks_live/form_component.ex`
|
||||
|
||||
**Pattern**: Follow existing form components (e.g., `ScrobbleRulesLive.FormComponent`)
|
||||
|
||||
**Editable Fields**:
|
||||
|
||||
- Track title
|
||||
- Artist name
|
||||
- Album title
|
||||
- Scrobbled date/time
|
||||
- Cover URL (optional)
|
||||
|
||||
## 3. Templates & UI Design
|
||||
|
||||
### Index Template
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobbled_tracks_live/index.html.heex`
|
||||
|
||||
**Layout**: Responsive card/list layout similar to collection
|
||||
|
||||
**Components**:
|
||||
|
||||
- Search bar at top with debounced input
|
||||
- Track cards displaying:
|
||||
- Cover image (with fallback for missing covers)
|
||||
- Track title
|
||||
- Artist name
|
||||
- Album title
|
||||
- Scrobbled date/time
|
||||
- Edit/Delete action buttons
|
||||
- Pagination component at bottom using `MusicLibraryWeb.Components.Pagination`
|
||||
- Empty state message when no tracks found
|
||||
|
||||
### Form Component Template
|
||||
|
||||
**Features**:
|
||||
|
||||
- Form inputs for editable track metadata
|
||||
- Client-side and server-side validation
|
||||
- Cancel/Save buttons with proper form handling
|
||||
- Error display for validation failures
|
||||
|
||||
## 4. Routing & Navigation
|
||||
|
||||
### Router Updates
|
||||
|
||||
**File**: `lib/music_library_web/router.ex`
|
||||
|
||||
Add to the logged-in scope around line 65:
|
||||
|
||||
```elixir
|
||||
live "/scrobbled-tracks", ScrobbledTracksLive.Index, :index
|
||||
live "/scrobbled-tracks/:scrobbled_at_uts/edit", ScrobbledTracksLive.Index, :edit
|
||||
```
|
||||
|
||||
### Navigation Integration
|
||||
|
||||
**File**: `lib/music_library_web/components/layouts/app.html.heex`
|
||||
|
||||
Add to the "More" dropdown menu (after line 58):
|
||||
|
||||
```elixir
|
||||
<.dropdown_link href={~p"/scrobbled-tracks"}>
|
||||
<.icon name="hero-musical-note" class="h-4 w-4 mr-2" aria-hidden="true" data-slot="icon" />
|
||||
{gettext("Scrobbled Tracks")}
|
||||
</.dropdown_link>
|
||||
```
|
||||
|
||||
## 5. Implementation Details
|
||||
|
||||
### Pagination Integration
|
||||
|
||||
- **Pattern**: Follow `CollectionLive.Index` pagination approach
|
||||
- **Component**: Reuse `MusicLibraryWeb.Components.Pagination`
|
||||
- **Parameters**: Support `query`, `page`, `page_size`, and `order`
|
||||
- **URL Structure**: `/scrobbled-tracks?page=2&query=artist&page_size=50`
|
||||
|
||||
### Stream Implementation
|
||||
|
||||
- Use `stream(:tracks, tracks)` for efficient rendering of large track lists
|
||||
- Implement `stream_delete(socket, :tracks, track)` for track deletion
|
||||
- Handle pagination with `reset: true` for filtered results
|
||||
- Use unique DOM IDs based on `scrobbled_at_uts`
|
||||
|
||||
### Search Functionality
|
||||
|
||||
- Search across track title, artist name, and album title
|
||||
- Use database LIKE queries for basic string matching
|
||||
- Debounced search input with LiveView `phx-change` events
|
||||
- Reset to page 1 when search query changes
|
||||
|
||||
### Delete Confirmation
|
||||
|
||||
- Use modal overlay for delete confirmation
|
||||
- Show track details (title, artist, date) in confirmation modal
|
||||
- Implement optimistic UI updates with `stream_delete`
|
||||
- Handle deletion errors gracefully
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Validate track existence before edit operations
|
||||
- Handle database constraint violations
|
||||
- Display user-friendly error messages
|
||||
- Graceful fallbacks for missing track data
|
||||
|
||||
## 6. File Structure
|
||||
|
||||
```
|
||||
lib/music_library_web/live/scrobbled_tracks_live/
|
||||
├── index.ex # Main LiveView controller
|
||||
├── index.html.heex # Index template with track listing
|
||||
└── form_component.ex # Edit form component
|
||||
|
||||
docs/plans/
|
||||
└── scrobbled-tracks-crud.md # This plan document
|
||||
```
|
||||
|
||||
## 7. Implementation Steps
|
||||
|
||||
1. **Create context functions** in `MusicLibrary.ScrobbleActivity` module for CRUD operations
|
||||
2. **Implement pagination logic** following existing `CollectionLive.Index` patterns
|
||||
3. **Create LiveView** with stream-based track listing and search
|
||||
4. **Add search functionality** with query parameter handling and debouncing
|
||||
5. **Implement edit form** with validation and error handling
|
||||
6. **Add delete functionality** with confirmation modal
|
||||
7. **Update navigation** to include "Scrobbled Tracks" link in More dropdown
|
||||
8. **Add routing** for index and edit actions
|
||||
9. **Style components** using Tailwind CSS and Fluxon UI components
|
||||
10. **Add comprehensive tests** following existing test patterns
|
||||
|
||||
## 8. Technical Considerations
|
||||
|
||||
### Primary Key Handling
|
||||
|
||||
- Use `scrobbled_at_uts` as unique identifier in routes and database queries
|
||||
- Handle potential timestamp collisions gracefully
|
||||
- Consider compound keys if needed for uniqueness
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
- Use LiveView streams for efficient rendering of large lists
|
||||
- Implement database indexes on searchable fields
|
||||
- Paginate results to avoid memory issues
|
||||
- Use debounced search to reduce database load
|
||||
|
||||
### Data Integrity
|
||||
|
||||
- Validate required fields (track title, artist name)
|
||||
- Handle embedded schema updates (artist, album data)
|
||||
- Preserve Last.fm API data integrity
|
||||
- Consider soft deletes vs hard deletes
|
||||
|
||||
### User Experience
|
||||
|
||||
- Consistent styling with existing collection/wishlist pages
|
||||
- Responsive design for mobile/tablet viewing
|
||||
- Loading states during pagination and search
|
||||
- Clear feedback for user actions (edit, delete)
|
||||
- Accessible keyboard navigation
|
||||
|
||||
### Security & Permissions
|
||||
|
||||
- Ensure proper authentication via `:logged_in` pipeline
|
||||
- Validate user permissions for track modifications
|
||||
- Sanitize input data to prevent XSS
|
||||
- Use Phoenix CSRF protection
|
||||
|
||||
## 9. Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- Context function tests for CRUD operations
|
||||
- LiveView event handling tests
|
||||
- Form validation tests
|
||||
- Pagination logic tests
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- Full user flow tests (search, edit, delete)
|
||||
- Navigation integration tests
|
||||
- Error handling scenarios
|
||||
- Responsive layout tests
|
||||
|
||||
### Test Files
|
||||
|
||||
```
|
||||
test/music_library_web/live/scrobbled_tracks_live_test.exs
|
||||
test/last_fm_test.exs (extend existing)
|
||||
test/support/fixtures/scrobbled_tracks_fixtures.ex
|
||||
```
|
||||
|
||||
## 10. Localization
|
||||
|
||||
Add gettext entries for:
|
||||
|
||||
- Page titles and headings
|
||||
- Form labels and placeholders
|
||||
- Error messages
|
||||
- Action button text
|
||||
- Empty state messages
|
||||
- Navigation menu item
|
||||
|
||||
## 11. Future Enhancements
|
||||
|
||||
### Potential Extensions
|
||||
|
||||
- Bulk operations (delete multiple tracks)
|
||||
- Export functionality (CSV, JSON)
|
||||
- Advanced filtering (date ranges, specific artists)
|
||||
- Scrobble statistics and analytics
|
||||
- Integration with music streaming services
|
||||
- Track matching with collection records
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
- Virtual scrolling for very large lists
|
||||
- Background track synchronization
|
||||
- Caching for frequently accessed data
|
||||
- Search index optimization
|
||||
|
||||
This plan ensures consistency with existing codebase patterns while providing a complete, user-friendly interface for scrobbled tracks management.
|
||||
|
||||
@@ -1,345 +0,0 @@
|
||||
# Universal Search Modal Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines the implementation of a universal search modal that provides unified search across Records (Collection), Records (Wishlist), and Artists. The modal will be accessible via a search icon in the top navigation bar and a keyboard shortcut.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Core Features
|
||||
- Modal accessible via:
|
||||
- Search icon in top navigation bar
|
||||
- Keyboard shortcut (Ctrl/Cmd + K)
|
||||
- Search across three entity types:
|
||||
1. Records in Collection
|
||||
2. Records in Wishlist
|
||||
3. Artists
|
||||
- Real-time search as user types
|
||||
- Keyboard navigation support
|
||||
- Mobile-responsive design
|
||||
|
||||
### Search Functionality
|
||||
- Leverage existing FTS5 search infrastructure
|
||||
- Support existing search syntax (artist:, album:, genre:, etc.)
|
||||
- Show search results grouped by entity type
|
||||
- Limit results per category (e.g., 5 per type)
|
||||
- Click-to-navigate to full results or individual records
|
||||
|
||||
## Technical Architecture
|
||||
|
||||
### 1. Modal Component Structure
|
||||
|
||||
**New Components:**
|
||||
- `UniversalSearchModal` - Main modal component
|
||||
- `SearchResultGroup` - Groups results by type (Collection, Wishlist, Artists)
|
||||
- `SearchResultItem` - Individual result display
|
||||
- `SearchInput` - Enhanced search input with keyboard shortcuts
|
||||
|
||||
**Component Location:**
|
||||
```
|
||||
lib/music_library_web/components/
|
||||
├── search_components.ex # New universal search components
|
||||
└── layouts/app.html.heex # Add search icon to navigation
|
||||
```
|
||||
|
||||
### 2. LiveView Integration
|
||||
|
||||
**New LiveView Module:**
|
||||
```
|
||||
lib/music_library_web/live/universal_search_live/
|
||||
├── index.ex # Main search LiveView
|
||||
└── index.html.heex # Search modal template
|
||||
```
|
||||
|
||||
**Integration Points:**
|
||||
- Mount as child LiveView in main layout
|
||||
- Handle modal open/close state
|
||||
- Manage search query and results
|
||||
- Handle keyboard navigation
|
||||
|
||||
### 3. Backend Search Implementation
|
||||
|
||||
**New Context Functions:**
|
||||
```elixir
|
||||
# In lib/music_library/search.ex (new module)
|
||||
def universal_search(query, opts \\ [])
|
||||
def search_collection(query, limit \\ 5)
|
||||
def search_wishlist(query, limit \\ 5)
|
||||
def search_artists(query, limit \\ 5)
|
||||
```
|
||||
|
||||
**Search Strategy:**
|
||||
- Reuse existing `Records.search_collection/2` and `Records.search_wishlist/2`
|
||||
- Extend `Artists.search/2` or create new artist search function
|
||||
- Combine results with type metadata
|
||||
- Implement result limiting and pagination
|
||||
|
||||
### 4. Database Considerations
|
||||
|
||||
**Leverage Existing Infrastructure:**
|
||||
- Use existing `records_search_index` FTS5 table
|
||||
- Utilize existing search parser for tagged queries
|
||||
- Consider artist search optimization (may need artist search index)
|
||||
|
||||
**Artist Search Enhancement:**
|
||||
- Current artist search may need optimization
|
||||
- Consider adding FTS5 index for artist_infos table if performance issues
|
||||
- Leverage existing artist_records view for artist-record relationships
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### 1. Modal Behavior
|
||||
|
||||
**Desktop:**
|
||||
- Modal opens centered on screen
|
||||
- Size: `max-w-2xl` (responsive)
|
||||
- Overlay with backdrop blur
|
||||
- Esc key to close, click outside to close
|
||||
|
||||
**Mobile:**
|
||||
- Full-screen modal on mobile (`sm:max-w-2xl max-w-full`)
|
||||
- Touch-friendly result items
|
||||
- Swipe down to close (if feasible)
|
||||
- Virtual keyboard considerations
|
||||
|
||||
### 2. Search Interface
|
||||
|
||||
**Search Input:**
|
||||
- Placeholder: "Search records and artists..."
|
||||
- Auto-focus when modal opens
|
||||
- Debounced search (300ms delay)
|
||||
- Clear button when text present
|
||||
- Loading indicator during search
|
||||
|
||||
**Search Results:**
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Search: "radiohead ok computer" │
|
||||
├─────────────────────────────────────┤
|
||||
│ COLLECTION (3 results) │
|
||||
│ • OK Computer - Radiohead │
|
||||
│ • Kid A - Radiohead │
|
||||
│ • In Rainbows - Radiohead │
|
||||
│ View all 12 collection results → │
|
||||
├─────────────────────────────────────┤
|
||||
│ WISHLIST (1 result) │
|
||||
│ • Hail to the Thief - Radiohead │
|
||||
│ View all 1 wishlist results → │
|
||||
├─────────────────────────────────────┤
|
||||
│ ARTISTS (1 result) │
|
||||
│ • Radiohead │
|
||||
│ View artist page → │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 3. Keyboard Navigation
|
||||
|
||||
**Keyboard Shortcuts:**
|
||||
- `Ctrl/Cmd + K` - Open modal
|
||||
- `Esc` - Close modal
|
||||
- `↑/↓` - Navigate results
|
||||
- `Enter` - Select highlighted result
|
||||
- `Tab` - Move between sections
|
||||
|
||||
**Implementation:**
|
||||
- Use Phoenix LiveView's `phx-window-keydown` for global shortcuts
|
||||
- Implement focus management with JavaScript hooks
|
||||
- Maintain accessibility standards (ARIA labels, focus indicators)
|
||||
|
||||
### 4. Result Navigation
|
||||
|
||||
**Click Actions:**
|
||||
- Record results → Navigate to record show page
|
||||
- Artist results → Navigate to artist page
|
||||
- "View all X results" → Navigate to respective list page with search applied
|
||||
|
||||
**URL Strategy:**
|
||||
- Don't change URL for modal open/close
|
||||
- When navigating to results, apply search query to destination page
|
||||
- Use existing search parameter patterns
|
||||
|
||||
## Database Schema Impact
|
||||
|
||||
**No Schema Changes Required:**
|
||||
- Leverage existing tables and search infrastructure
|
||||
- May add database indexes for performance if needed
|
||||
|
||||
**Performance Considerations:**
|
||||
- Monitor query performance with combined searches
|
||||
- Consider query result caching for common searches
|
||||
- Implement search result limits to prevent slow queries
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### 1. Unit Tests
|
||||
|
||||
**Search Logic Tests:**
|
||||
```elixir
|
||||
# test/music_library/search_test.exs
|
||||
describe "universal_search/2" do
|
||||
test "searches across all entity types"
|
||||
test "limits results per category"
|
||||
test "handles empty queries"
|
||||
test "handles special characters"
|
||||
test "respects search syntax (artist:, album:, etc.)"
|
||||
end
|
||||
```
|
||||
|
||||
### 2. Integration Tests
|
||||
|
||||
**LiveView Tests:**
|
||||
```elixir
|
||||
# test/music_library_web/live/universal_search_live_test.exs
|
||||
describe "UniversalSearchLive" do
|
||||
test "opens modal on keyboard shortcut"
|
||||
test "searches as user types"
|
||||
test "displays results grouped by type"
|
||||
test "navigates to selected results"
|
||||
test "handles keyboard navigation"
|
||||
test "closes modal on escape"
|
||||
end
|
||||
```
|
||||
|
||||
### 3. Component Tests
|
||||
|
||||
**Search Components:**
|
||||
```elixir
|
||||
# test/music_library_web/components/search_components_test.exs
|
||||
describe "SearchResultGroup" do
|
||||
test "renders results with proper grouping"
|
||||
test "shows 'view all' link when more results available"
|
||||
test "handles empty result sets"
|
||||
end
|
||||
```
|
||||
|
||||
### 4. JavaScript/Hook Tests
|
||||
|
||||
**Browser Tests:**
|
||||
- Test keyboard shortcuts work globally
|
||||
- Test focus management
|
||||
- Test modal open/close behavior
|
||||
- Test mobile touch interactions
|
||||
|
||||
## Mobile Considerations
|
||||
|
||||
### 1. Layout Adaptations
|
||||
|
||||
**Mobile Modal:**
|
||||
- Full-screen on small devices
|
||||
- Reduced padding and margins
|
||||
- Larger touch targets for results
|
||||
- Simplified result display (fewer details)
|
||||
|
||||
**Touch Interactions:**
|
||||
- Swipe down to close modal
|
||||
- Tap to select results
|
||||
- Pull-to-refresh for search results
|
||||
|
||||
### 2. Performance
|
||||
|
||||
**Mobile Optimization:**
|
||||
- Shorter debounce delays on mobile
|
||||
- Reduced result limits on mobile
|
||||
- Lazy loading for large result sets
|
||||
- Optimize image loading for album covers
|
||||
|
||||
### 3. Virtual Keyboard
|
||||
|
||||
**iOS/Android Considerations:**
|
||||
- Modal positioning with virtual keyboard
|
||||
- Proper input focus management
|
||||
- Prevent zoom on input focus
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Core Modal Infrastructure
|
||||
1. Create basic modal component
|
||||
2. Add search icon to navigation
|
||||
3. Implement modal open/close
|
||||
4. Add keyboard shortcut support
|
||||
|
||||
### Phase 2: Search Backend
|
||||
1. Create universal search context
|
||||
2. Implement combined search functions
|
||||
3. Add result limiting and grouping
|
||||
4. Optimize query performance
|
||||
|
||||
### Phase 3: Search Interface
|
||||
1. Implement search input with debouncing
|
||||
2. Add search result display
|
||||
3. Implement keyboard navigation
|
||||
4. Add result selection and navigation
|
||||
|
||||
### Phase 4: Mobile & Polish
|
||||
1. Mobile layout optimizations
|
||||
2. Touch interaction improvements
|
||||
3. Performance optimizations
|
||||
4. Accessibility enhancements
|
||||
|
||||
### Phase 5: Testing & Documentation
|
||||
1. Comprehensive test coverage
|
||||
2. Performance testing
|
||||
3. Mobile device testing
|
||||
4. Documentation updates
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Functionality:**
|
||||
- Modal opens/closes reliably
|
||||
- Search results are accurate and fast
|
||||
- Keyboard navigation works smoothly
|
||||
- Mobile experience is touch-friendly
|
||||
|
||||
**Performance:**
|
||||
- Search queries complete in <500ms
|
||||
- Modal opens in <100ms
|
||||
- No janky animations or interactions
|
||||
- Efficient database queries
|
||||
|
||||
**Usability:**
|
||||
- Easy to discover and use
|
||||
- Intuitive keyboard shortcuts
|
||||
- Clear result presentation
|
||||
- Seamless navigation to results
|
||||
|
||||
## Risks and Mitigation
|
||||
|
||||
**Performance Risks:**
|
||||
- Complex queries may be slow
|
||||
- **Mitigation:** Implement query limits, caching, and monitoring
|
||||
|
||||
**Mobile Experience:**
|
||||
- Small screen constraints
|
||||
- **Mitigation:** Responsive design, touch-friendly interfaces
|
||||
|
||||
**Accessibility:**
|
||||
- Keyboard navigation complexity
|
||||
- **Mitigation:** Follow ARIA guidelines, comprehensive testing
|
||||
|
||||
**Search Accuracy:**
|
||||
- Users may expect Google-like search
|
||||
- **Mitigation:** Clear documentation of search syntax, good defaults
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
**Advanced Features:**
|
||||
- Search history and suggestions
|
||||
- Saved searches
|
||||
- Search filters and facets
|
||||
- Search result previews
|
||||
|
||||
**Integration:**
|
||||
- Integration with barcode scanning
|
||||
- Voice search capabilities
|
||||
- External search (MusicBrainz, Last.fm)
|
||||
|
||||
**Analytics:**
|
||||
- Search usage analytics
|
||||
- Popular search terms
|
||||
- Search success rates
|
||||
|
||||
## Conclusion
|
||||
|
||||
This universal search modal will significantly improve the user experience by providing quick access to all content types from any page. The implementation leverages existing infrastructure while adding a modern, accessible search interface that works well on both desktop and mobile devices.
|
||||
|
||||
The phased approach ensures a solid foundation while allowing for iterative improvements based on user feedback and usage patterns.
|
||||
@@ -1,219 +0,0 @@
|
||||
# Scrobble Rule Application Optimization
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the optimization made to the scrobble rule application system to improve performance when applying multiple rules.
|
||||
|
||||
## Problem
|
||||
|
||||
Previously, when applying all enabled scrobble rules, each rule was applied independently with a separate database UPDATE query. With N rules, this resulted in N separate database operations, which became inefficient as the number of rules grew.
|
||||
|
||||
### Previous Implementation
|
||||
|
||||
```elixir
|
||||
def apply_all_rules do
|
||||
list_enabled_rules()
|
||||
|> Enum.map(fn rule ->
|
||||
case apply_rule(rule) do
|
||||
{:ok, count} -> {:ok, {rule.type, rule.match_value, count}}
|
||||
{:error, reason} -> {:error, {rule.type, rule.match_value, reason}}
|
||||
end
|
||||
end)
|
||||
end
|
||||
```
|
||||
|
||||
This approach meant:
|
||||
- 10 rules = 10 separate UPDATE queries
|
||||
- 100 rules = 100 separate UPDATE queries
|
||||
- Each query had to scan the entire `scrobbled_tracks` table
|
||||
|
||||
## Solution
|
||||
|
||||
The optimization groups rules by type (album/artist) and applies all rules of each type in a single database query using SQLite's CASE statement.
|
||||
|
||||
### New Implementation
|
||||
|
||||
```elixir
|
||||
def apply_all_rules do
|
||||
enabled_rules = list_enabled_rules()
|
||||
|
||||
# Group rules by type
|
||||
{album_rules, artist_rules} =
|
||||
Enum.split_with(enabled_rules, fn rule -> rule.type == :album end)
|
||||
|
||||
# Apply all album rules in one query
|
||||
apply_all_album_rules(album_rules)
|
||||
|
||||
# Apply all artist rules in one query
|
||||
apply_all_artist_rules(artist_rules)
|
||||
end
|
||||
```
|
||||
|
||||
This approach means:
|
||||
- 10 album rules + 10 artist rules = 2 total UPDATE queries (one for albums, one for artists)
|
||||
- 100 album rules + 100 artist rules = 2 total UPDATE queries
|
||||
- Each query still scans the table once, but updates all matching records in a single pass
|
||||
|
||||
## Technical Details
|
||||
|
||||
### SQL Generation
|
||||
|
||||
The optimization dynamically builds a CASE statement for all rules of the same type:
|
||||
|
||||
**Example for 2 album rules:**
|
||||
|
||||
```sql
|
||||
UPDATE scrobbled_tracks
|
||||
SET album = CASE
|
||||
WHEN json_extract(album, '$.title') = 'Dark Side of the Moon'
|
||||
THEN json_set(album, '$.musicbrainz_id', '12345678-1234-1234-1234-123456789012')
|
||||
WHEN json_extract(album, '$.title') = 'Wish You Were Here'
|
||||
THEN json_set(album, '$.musicbrainz_id', 'abcdef12-3456-7890-abcd-ef1234567890')
|
||||
ELSE album
|
||||
END
|
||||
WHERE json_extract(album, '$.title') IN ('Dark Side of the Moon', 'Wish You Were Here')
|
||||
```
|
||||
|
||||
The WHERE clause uses IN to filter only tracks that match any of the rules, avoiding unnecessary updates.
|
||||
|
||||
### Functions Added
|
||||
|
||||
#### 1. `apply_all_album_rules/1`
|
||||
Applies all album rules in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_album_rules([ScrobbleRule.t()]) :: {:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
#### 2. `apply_all_artist_rules/1`
|
||||
Applies all artist rules in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_artist_rules([ScrobbleRule.t()]) :: {:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
#### 3. `apply_all_album_rules/2`
|
||||
Applies all album rules to a specific set of tracks in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_album_rules([ScrobbleRule.t()], [Track.t()]) ::
|
||||
{:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
#### 4. `apply_all_artist_rules/2`
|
||||
Applies all artist rules to a specific set of tracks in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_artist_rules([ScrobbleRule.t()], [Track.t()]) ::
|
||||
{:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
### Use Cases
|
||||
|
||||
The optimization handles two main use cases:
|
||||
|
||||
#### 1. Periodic Full Application
|
||||
The `ApplyScrobbleRules` Oban worker runs every 30 minutes and applies all rules to all tracks:
|
||||
|
||||
```elixir
|
||||
ScrobbleRules.apply_all_rules()
|
||||
```
|
||||
|
||||
#### 2. Incremental Application on New Tracks
|
||||
When new tracks are inserted via `LastFm.Feed.update/1`, rules are applied to just those tracks:
|
||||
|
||||
```elixir
|
||||
tracks
|
||||
|> ScrobbleRules.apply_all_rules()
|
||||
```
|
||||
|
||||
Both use cases now benefit from batched rule application.
|
||||
|
||||
## Performance Impact
|
||||
|
||||
### Expected Improvements
|
||||
|
||||
For a database with:
|
||||
- 10,000 scrobbled tracks
|
||||
- 20 enabled rules (10 album + 10 artist)
|
||||
|
||||
**Before:**
|
||||
- 20 UPDATE queries
|
||||
- Each query scans ~10,000 rows
|
||||
- Total: ~200,000 row scans
|
||||
|
||||
**After:**
|
||||
- 2 UPDATE queries (1 for albums, 1 for artists)
|
||||
- Each query scans ~10,000 rows
|
||||
- Total: ~20,000 row scans
|
||||
|
||||
**Result:** ~10x reduction in database operations for this scenario.
|
||||
|
||||
The benefit scales with the number of rules:
|
||||
- 5 rules: ~2.5x improvement
|
||||
- 10 rules: ~5x improvement
|
||||
- 20 rules: ~10x improvement
|
||||
- 100 rules: ~50x improvement
|
||||
|
||||
## Trade-offs
|
||||
|
||||
### Return Value Semantics
|
||||
|
||||
With the batched approach, all rules of the same type report the same aggregate count (total rows updated) rather than individual per-rule counts:
|
||||
|
||||
```elixir
|
||||
# Before: Each rule gets its own count
|
||||
[
|
||||
{:ok, {:album, "Album 1", 5}}, # 5 tracks updated
|
||||
{:ok, {:album, "Album 2", 3}}, # 3 tracks updated
|
||||
]
|
||||
|
||||
# After: Both rules report the total updated
|
||||
[
|
||||
{:ok, {:album, "Album 1", 8}}, # 8 total tracks updated
|
||||
{:ok, {:album, "Album 2", 8}}, # 8 total tracks updated
|
||||
]
|
||||
```
|
||||
|
||||
This is an acceptable trade-off because:
|
||||
1. The logging still shows how many rules were applied
|
||||
2. The total tracks updated is more meaningful for understanding impact
|
||||
3. Getting individual counts would require separate queries, negating the optimization
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
The API remains fully backward compatible:
|
||||
- All existing functions (`apply_rule/1`, `apply_album_rule/1`, etc.) still work
|
||||
- Return value format is unchanged: `{:ok, {type, match_value, count}}`
|
||||
- Error handling is unchanged
|
||||
- Tests continue to pass
|
||||
|
||||
## Testing
|
||||
|
||||
New tests were added to verify the optimization:
|
||||
|
||||
1. `apply_all_album_rules/1` applies multiple album rules in one query
|
||||
2. `apply_all_artist_rules/1` applies multiple artist rules in one query
|
||||
3. Empty list handling returns `{:ok, 0}`
|
||||
4. `apply_all_rules/0` correctly batches rules by type
|
||||
5. All existing tests continue to pass
|
||||
|
||||
## Future Considerations
|
||||
|
||||
### Potential Enhancements
|
||||
|
||||
1. **Parallel Execution**: Album and artist rule applications could run in parallel since they're independent
|
||||
2. **Query Optimization**: Could add indexes on `json_extract(album, '$.title')` and `json_extract(artist, '$.name')` if performance is still a concern
|
||||
3. **Metrics**: Add telemetry events to track batch sizes and execution times
|
||||
4. **Rule Ordering**: If rule priority matters in the future, the CASE statement naturally handles this (first match wins)
|
||||
|
||||
### Scalability
|
||||
|
||||
The current approach scales well up to ~100 rules per type. Beyond that, consider:
|
||||
- SQL query length limits (SQLite's default limit is 1MB, which allows ~10,000 rules)
|
||||
- Transaction size and memory usage
|
||||
- Potential chunking if needed (apply rules in batches of N)
|
||||
|
||||
## Conclusion
|
||||
|
||||
This optimization significantly improves the performance of scrobble rule application by reducing database operations from O(N) to O(1) per rule type, where N is the number of rules. The implementation maintains backward compatibility while providing substantial performance benefits that scale with the number of rules.
|
||||
Reference in New Issue
Block a user