Remove AI tooling, and related packages/docs
This commit is contained in:
Claudio Ortolina
@@ -1,866 +0,0 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Project Overview
|
||||
|
||||
Music Library is an Elixir/Phoenix application for managing a personal music collection. It allows users to:
|
||||
|
||||
- Add records from MusicBrainz, with optional override of specific pieces of data
|
||||
- Manage a collection and a wishlist of records, with ways to quickly search and filter based on records' metadata
|
||||
- Integration with Last.fm:
|
||||
- display latest scrobbles, and where possible connect them with records in the collection or wishlist
|
||||
- scrobble a record
|
||||
- store a local copy of the complete scrobble history, and setup rules to fix its data as needed
|
||||
- Barcode scanning functionality for adding records
|
||||
- Artist information management with automatic fetching and caching
|
||||
- Online store templates for quick external lookups
|
||||
- Notes system for tracking custom information about records
|
||||
- Assets management with image transformation and caching
|
||||
- Color extraction from album artwork
|
||||
- Some basic stats (top albums, top artists, collection insights)
|
||||
- Universal search across collection, wishlist, and artists
|
||||
- Similar records discovery using AI-powered embeddings (using SQLite vector extension)
|
||||
- All data stored in a single SQLite database for portability and ease of backup/restore
|
||||
|
||||
## Development Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- [mise-en-place](https://mise.jdx.dev) is used for environment management
|
||||
- Requires Erlang, Elixir, and Node.js (managed by mise)
|
||||
- Uses [Fluxon UI](https://fluxonui.com/) - requires valid credentials
|
||||
|
||||
### Environment Configuration
|
||||
|
||||
Required environment variables:
|
||||
|
||||
- `LAST_FM_USER`: Last.fm username used to populate the Scrobble Activity
|
||||
- `LAST_FM_API_KEY`: Last.fm API key used to fetch the Scrobble Activity (secret)
|
||||
- `OPENAI_KEY`: OpenAI API key used to populate genres (secret)
|
||||
- `FLUXON_KEY_FINGERPRINT`: Fluxon license fingerprint
|
||||
- `FLUXON_LICENSE_KEY`: Fluxon license key (secret)
|
||||
- `LOGIN_PASSWORD`: Password for accessing the application (in production, secret)
|
||||
|
||||
In development, the default password is `change me`.
|
||||
|
||||
Create a `mise.local.toml` with required variables (samples in `mise.toml`).
|
||||
|
||||
### Initial Setup
|
||||
|
||||
```sh
|
||||
# Install required tools (Erlang, Elixir, Node.js)
|
||||
mise install
|
||||
|
||||
# Setup dependencies and database
|
||||
mise run dev:setup
|
||||
```
|
||||
|
||||
## Common Commands
|
||||
|
||||
### Development
|
||||
|
||||
IF YOU CAN CONNECT TO THE MCP SERVER, THEN THE APPLICATION IS ALREADY RUNNING.
|
||||
|
||||
Otherwise you can run:
|
||||
|
||||
```sh
|
||||
# Run the Phoenix server with IEx console
|
||||
mise run dev:console
|
||||
|
||||
# Run static checks (format, credo, gettext)
|
||||
mise run dev:lint
|
||||
|
||||
# Show outdated dependencies
|
||||
mise run dev:outdated
|
||||
|
||||
# Update dependencies
|
||||
mise run dev:update
|
||||
|
||||
# Fix translation file conflicts
|
||||
mise run dev:fix-translations
|
||||
|
||||
# Render README.md with GitHub Flavored Markdown preview
|
||||
mise run dev:readme
|
||||
```
|
||||
|
||||
### Testing
|
||||
|
||||
```sh
|
||||
# Run all tests
|
||||
mix test
|
||||
|
||||
# Run tests via mise (alternative)
|
||||
mise run dev:test
|
||||
|
||||
# Run a specific test file
|
||||
mix test test/path/to/test_file.exs
|
||||
|
||||
# Run a specific test (line number)
|
||||
mix test test/path/to/test_file.exs:42
|
||||
|
||||
# Run previously failed tests
|
||||
mix test --failed
|
||||
```
|
||||
|
||||
### Database
|
||||
|
||||
```sh
|
||||
# Setup database (create and migrate)
|
||||
mix ecto.setup
|
||||
|
||||
# Reset database (drop, create, and migrate)
|
||||
mix ecto.reset
|
||||
|
||||
# Run migrations
|
||||
mix ecto.migrate
|
||||
|
||||
# Create a new migration
|
||||
mix ecto.gen.migration migration_name
|
||||
|
||||
# Rollback last migration
|
||||
mix ecto.rollback
|
||||
```
|
||||
|
||||
### Production & CI
|
||||
|
||||
YOU'RE NOT ALLOWED TO TOUCH PRODUCTION IN ANY POSSIBLE WAY.
|
||||
|
||||
Available commands for monitoring (view only):
|
||||
|
||||
- `mise run prod:backup` - Backup the production database to local dev env
|
||||
- `mise run prod:prune-backups` - Remove locally downloaded production database backups
|
||||
- `mise run prod:test` - Run HTTP tests against production
|
||||
- `mise run ci:watch` - Watch CI running
|
||||
|
||||
Deployment is handled via Coolify using a Docker Compose strategy.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Database Structure
|
||||
|
||||
The application uses SQLite with two databases:
|
||||
|
||||
- Main database (`Repo`): Primary data storage
|
||||
- Background database (`BackgroundRepo`): For background jobs and workers
|
||||
|
||||
Key tables and schemas:
|
||||
|
||||
- `records` table stores all record data with embedded JSON for artists
|
||||
- `artist_infos` table stores additional artist metadata (bio, images, etc.)
|
||||
- `artist_records` view provides normalized artist-record relationships
|
||||
- `records_search_index` virtual FTS5 table for efficient full-text searching
|
||||
- `record_embeddings` table stores AI-generated embeddings for similarity search (uses sqlite-vec extension)
|
||||
- `scrobble_rules` table for managing Last.fm scrobble data corrections
|
||||
- `online_store_templates` table for external store URL templates
|
||||
- `notes` table for custom record annotations
|
||||
- `assets` table for cached images with transformations
|
||||
- `secrets` table for encrypted sensitive data (uses Cloak)
|
||||
- Collection vs. wishlist differentiated by `purchased_at` field (NULL = wishlist)
|
||||
|
||||
Tables are synchronized with triggers, and multiple indices exist for performance.
|
||||
|
||||
### Code Organization
|
||||
|
||||
The application follows standard Phoenix/Elixir structure:
|
||||
|
||||
- `lib/music_library`: Core application logic
|
||||
- `records/`: Record and collection management (includes batch operations, search parsing, embeddings, similarity)
|
||||
- `artists/`: Artist data handling and artist info management
|
||||
- `collection.ex`: Collection-specific functionality
|
||||
- `wishlist.ex`: Wishlist-specific functionality
|
||||
- `search.ex`: Universal search across collection, wishlist, and artists
|
||||
- `scrobble_activity.ex`: Last.fm scrobble tracking and activity
|
||||
- `scrobble_rules/`: Rules for correcting scrobble data
|
||||
- `barcode_scan/`: Barcode scanning features
|
||||
- `assets/`: Asset caching, image transformation
|
||||
- `colors/`: Extract colors from images (album artworks)
|
||||
- `notes/`: Note management for records
|
||||
- `online_store_templates/`: External store URL templates
|
||||
- `secrets/`: Encrypted secrets management (uses Cloak/Vault)
|
||||
- `worker/`: Background job workers (Oban)
|
||||
- `fetch_artist_image.ex`, `fetch_artist_info.ex`, `prune_artist_info.ex`
|
||||
- `record_refresh_music_brainz_data.ex`, `refresh_cover.ex`
|
||||
- `extract_colors.ex`, `populate_genres.ex`
|
||||
- `generate_record_embedding.ex`: Generate AI embeddings for similarity search
|
||||
- `apply_scrobble_rules.ex`, `prune_asset_cache.ex`
|
||||
|
||||
- External API integrations:
|
||||
- `lib/music_brainz`: MusicBrainz API client (primary metadata source)
|
||||
- `lib/discogs`: Discogs API client
|
||||
- `lib/last_fm`: Last.fm API client (scrobbles, artist info)
|
||||
- `lib/open_ai`: OpenAI API client (genre population, embeddings generation)
|
||||
|
||||
- `lib/music_library_web`: Web interface (Phoenix)
|
||||
- `live/`: LiveView implementations
|
||||
- `collection_live/`, `wishlist_live/`: Collection and wishlist views
|
||||
- `artist_live/`: Artist detail pages
|
||||
- `scrobble_live/`, `scrobble_rules_live/`, `scrobbled_tracks_live/`: Scrobble management
|
||||
- `stats_live/`: Statistics (top albums, top artists)
|
||||
- `universal_search_live/`: Universal search interface
|
||||
- `online_store_template_live/`: Store template management
|
||||
- `components/`: UI components (Fluxon-based)
|
||||
- `record_components.ex`, `artist_components.ex`, `scrobble_components.ex`
|
||||
- `search_components.ex`, `stats_components.ex`, `chart_components.ex`
|
||||
- `barcode_scanner.ex`, `notes.ex`, `pagination.ex`
|
||||
- `record_form.ex`, `add_record.ex`, `release.ex`
|
||||
- `controllers/`: Traditional Phoenix controllers
|
||||
- `collection_controller.ex`: Collection export endpoints
|
||||
- `asset_controller.ex`: Asset serving
|
||||
- `archive_controller.ex`: Archive management
|
||||
- `health_controller.ex`: Health checks
|
||||
- `last_fm_controller.ex`: Last.fm OAuth callback
|
||||
- `session_controller.ex`: Authentication
|
||||
- `hooks/`: LiveView hooks (`get_timezone.ex`, `static_assets.ex`)
|
||||
|
||||
## Git Workflow
|
||||
|
||||
Run the following before commits to ensure code quality:
|
||||
|
||||
```sh
|
||||
# Run static checks to ensure code quality
|
||||
mise run dev:precommit
|
||||
```
|
||||
|
||||
To set up a pre-commit hook:
|
||||
|
||||
```sh
|
||||
mise generate git-pre-commit --write --task=dev:precommit
|
||||
```
|
||||
|
||||
### Guidelines
|
||||
|
||||
You are an expert in Elixir, Phoenix, Sqlite, LiveView, and Tailwind CSS.
|
||||
|
||||
Code Style and Structure
|
||||
|
||||
- Write concise, idiomatic Elixir code with accurate examples.
|
||||
- Follow Phoenix conventions and best practices.
|
||||
- Use functional programming patterns and leverage immutability.
|
||||
- Prefer higher-order functions and recursion over imperative loops.
|
||||
- Use descriptive variable and function names (e.g., user_signed_in?, calculate_total).
|
||||
- Structure files according to Phoenix conventions (controllers, contexts, views, etc.).
|
||||
- Where possible use Fluxon components instead of rolling your own
|
||||
- Use Oban for background job processing (workers in `lib/music_library/worker/`)
|
||||
|
||||
Database design
|
||||
|
||||
- Do not add columns for properties that are not specified in the requirements. Keep database changes to a minimum.
|
||||
|
||||
Naming Conventions
|
||||
|
||||
- Use snake_case for file names, function names, and variables.
|
||||
- Use PascalCase for module names.
|
||||
- Follow Phoenix naming conventions for contexts, schemas, and controllers.
|
||||
|
||||
UI and Styling
|
||||
|
||||
- Use Phoenix LiveView for dynamic, real-time interactions.
|
||||
- Implement responsive design with Tailwind CSS. When screen space is limited,
|
||||
prefer content-focused flexible layouts over rigid tabular structures.
|
||||
Reorganize information hierarchically within each item, grouping related data
|
||||
visually while maintaining scanability and preserving all functionality.
|
||||
- Use Phoenix view helpers and templates to keep views DRY.
|
||||
- Use minimal markup and avoid nesting DIVs unnecessarily.
|
||||
- Always use `gettext/1` for user-visible text instead of hardcoded strings
|
||||
- Apply translations consistently across all UI text, error messages, page titles, and button labels
|
||||
- Use `:if` attributes over `<%= if %>` blocks in HEEx templates for cleaner, more readable code
|
||||
- Leverage built-in Phoenix LiveView features like `phx-disable-with` for loading states instead of manual state tracking
|
||||
|
||||
Performance Optimization
|
||||
|
||||
- Use database indexing effectively.
|
||||
- Implement caching strategies (ETS, Redis).
|
||||
- Use Ecto's preload to avoid N+1 queries.
|
||||
- Optimize database queries using preload, joins, or select.
|
||||
|
||||
Key Conventions
|
||||
|
||||
- Follow RESTful routing conventions.
|
||||
- Use contexts for organizing related functionality.
|
||||
- Implement GenServers for stateful processes and background jobs.
|
||||
- Use Tasks for concurrent, isolated jobs.
|
||||
|
||||
Testing
|
||||
|
||||
- Write comprehensive tests using ExUnit.
|
||||
- Follow TDD practices.
|
||||
- Run tests with `mix test` or `mise run dev:test`
|
||||
- Use `mix test --failed` to rerun failed tests
|
||||
|
||||
Security
|
||||
|
||||
- Authentication handled via session-based login (see `MusicLibraryWeb.Auth`)
|
||||
- Use strong parameters in controllers (params validation).
|
||||
- Protect against common web vulnerabilities (XSS, CSRF, SQL injection).
|
||||
- Sensitive data encrypted using Cloak (see `MusicLibrary.Vault` and `MusicLibrary.Secrets`)
|
||||
|
||||
Background Jobs
|
||||
|
||||
- Use Oban workers for all background tasks
|
||||
- Workers are in `lib/music_library/worker/`
|
||||
- Common worker patterns: fetching external data, pruning caches, applying rules
|
||||
|
||||
Follow the official Phoenix guides for best practices in routing, controllers, contexts, views, and other Phoenix components.
|
||||
|
||||
<!-- usage-rules-start -->
|
||||
<!-- usage-rules-header -->
|
||||
|
||||
# Usage Rules
|
||||
|
||||
**IMPORTANT**: Consult these usage rules early and often when working with the packages listed below.
|
||||
Before attempting to use any of these packages or to discover if you should use them, review their
|
||||
usage rules to understand the correct patterns, conventions, and best practices.
|
||||
|
||||
<!-- usage-rules-header-end -->
|
||||
|
||||
<!-- igniter-start -->
|
||||
## igniter usage
|
||||
_A code generation and project patching framework_
|
||||
|
||||
[igniter usage rules](deps/igniter/usage-rules.md)
|
||||
<!-- igniter-end -->
|
||||
<!-- usage_rules-start -->
|
||||
## usage_rules usage
|
||||
_A dev tool for Elixir projects to gather LLM usage rules from dependencies_
|
||||
|
||||
## Using Usage Rules
|
||||
|
||||
Many packages have usage rules, which you should *thoroughly* consult before taking any
|
||||
action. These usage rules contain guidelines and rules *directly from the package authors*.
|
||||
They are your best source of knowledge for making decisions.
|
||||
|
||||
## Modules & functions in the current app and dependencies
|
||||
|
||||
When looking for docs for modules & functions that are dependencies of the current project,
|
||||
or for Elixir itself, use `mix usage_rules.docs`
|
||||
|
||||
```
|
||||
# Search a whole module
|
||||
mix usage_rules.docs Enum
|
||||
|
||||
# Search a specific function
|
||||
mix usage_rules.docs Enum.zip
|
||||
|
||||
# Search a specific function & arity
|
||||
mix usage_rules.docs Enum.zip/1
|
||||
```
|
||||
|
||||
|
||||
## Searching Documentation
|
||||
|
||||
You should also consult the documentation of any tools you are using, early and often. The best
|
||||
way to accomplish this is to use the `usage_rules.search_docs` mix task. Once you have
|
||||
found what you are looking for, use the links in the search results to get more detail. For example:
|
||||
|
||||
```
|
||||
# Search docs for all packages in the current application, including Elixir
|
||||
mix usage_rules.search_docs Enum.zip
|
||||
|
||||
# Search docs for specific packages
|
||||
mix usage_rules.search_docs Req.get -p req
|
||||
|
||||
# Search docs for multi-word queries
|
||||
mix usage_rules.search_docs "making requests" -p req
|
||||
|
||||
# Search only in titles (useful for finding specific functions/modules)
|
||||
mix usage_rules.search_docs "Enum.zip" --query-by title
|
||||
```
|
||||
|
||||
|
||||
<!-- usage_rules-end -->
|
||||
<!-- usage_rules:elixir-start -->
|
||||
## usage_rules:elixir usage
|
||||
# Elixir Core Usage Rules
|
||||
|
||||
## Pattern Matching
|
||||
- Use pattern matching over conditional logic when possible
|
||||
- Prefer to match on function heads instead of using `if`/`else` or `case` in function bodies
|
||||
- `%{}` matches ANY map, not just empty maps. Use `map_size(map) == 0` guard to check for truly empty maps
|
||||
|
||||
## Error Handling
|
||||
- Use `{:ok, result}` and `{:error, reason}` tuples for operations that can fail
|
||||
- Avoid raising exceptions for control flow
|
||||
- Use `with` for chaining operations that return `{:ok, _}` or `{:error, _}`
|
||||
|
||||
## Common Mistakes to Avoid
|
||||
- Elixir has no `return` statement, nor early returns. The last expression in a block is always returned.
|
||||
- Don't use `Enum` functions on large collections when `Stream` is more appropriate
|
||||
- Avoid nested `case` statements - refactor to a single `case`, `with` or separate functions
|
||||
- Don't use `String.to_atom/1` on user input (memory leak risk)
|
||||
- Lists and enumerables cannot be indexed with brackets. Use pattern matching or `Enum` functions
|
||||
- Prefer `Enum` functions like `Enum.reduce` over recursion
|
||||
- When recursion is necessary, prefer to use pattern matching in function heads for base case detection
|
||||
- Using the process dictionary is typically a sign of unidiomatic code
|
||||
- Only use macros if explicitly requested
|
||||
- There are many useful standard library functions, prefer to use them where possible
|
||||
|
||||
## Function Design
|
||||
- Use guard clauses: `when is_binary(name) and byte_size(name) > 0`
|
||||
- Prefer multiple function clauses over complex conditional logic
|
||||
- Name functions descriptively: `calculate_total_price/2` not `calc/2`
|
||||
- Predicate function names should not start with `is` and should end in a question mark.
|
||||
- Names like `is_thing` should be reserved for guards
|
||||
|
||||
## Data Structures
|
||||
- Use structs over maps when the shape is known: `defstruct [:name, :age]`
|
||||
- Prefer keyword lists for options: `[timeout: 5000, retries: 3]`
|
||||
- Use maps for dynamic key-value data
|
||||
- Prefer to prepend to lists `[new | list]` not `list ++ [new]`
|
||||
|
||||
## Mix Tasks
|
||||
|
||||
- Use `mix help` to list available mix tasks
|
||||
- Use `mix help task_name` to get docs for an individual task
|
||||
- Read the docs and options fully before using tasks
|
||||
|
||||
## Testing
|
||||
- Run tests in a specific file with `mix test test/my_test.exs` and a specific test with the line number `mix test path/to/test.exs:123`
|
||||
- Limit the number of failed tests with `mix test --max-failures n`
|
||||
- Use `@tag` to tag specific tests, and `mix test --only tag` to run only those tests
|
||||
- Use `assert_raise` for testing expected exceptions: `assert_raise ArgumentError, fn -> invalid_function() end`
|
||||
- Use `mix help test` to for full documentation on running tests
|
||||
|
||||
## Debugging
|
||||
|
||||
- Use `dbg/1` to print values while debugging. This will display the formatted value and other relevant information in the console.
|
||||
|
||||
<!-- usage_rules:elixir-end -->
|
||||
<!-- usage_rules:otp-start -->
|
||||
## usage_rules:otp usage
|
||||
# OTP Usage Rules
|
||||
|
||||
## GenServer Best Practices
|
||||
- Keep state simple and serializable
|
||||
- Handle all expected messages explicitly
|
||||
- Use `handle_continue/2` for post-init work
|
||||
- Implement proper cleanup in `terminate/2` when necessary
|
||||
|
||||
## Process Communication
|
||||
- Use `GenServer.call/3` for synchronous requests expecting replies
|
||||
- Use `GenServer.cast/2` for fire-and-forget messages.
|
||||
- When in doubt, use `call` over `cast`, to ensure back-pressure
|
||||
- Set appropriate timeouts for `call/3` operations
|
||||
|
||||
## Fault Tolerance
|
||||
- Set up processes such that they can handle crashing and being restarted by supervisors
|
||||
- Use `:max_restarts` and `:max_seconds` to prevent restart loops
|
||||
|
||||
## Task and Async
|
||||
- Use `Task.Supervisor` for better fault tolerance
|
||||
- Handle task failures with `Task.yield/2` or `Task.shutdown/2`
|
||||
- Set appropriate task timeouts
|
||||
- Use `Task.async_stream/3` for concurrent enumeration with back-pressure
|
||||
|
||||
<!-- usage_rules:otp-end -->
|
||||
<!-- phoenix:ecto-start -->
|
||||
## phoenix:ecto usage
|
||||
## Ecto Guidelines
|
||||
|
||||
- **Always** preload Ecto associations in queries when they'll be accessed in templates, ie a message that needs to reference the `message.user.email`
|
||||
- Remember `import Ecto.Query` and other supporting modules when you write `seeds.exs`
|
||||
- `Ecto.Schema` fields always use the `:string` type, even for `:text`, columns, ie: `field :name, :string`
|
||||
- `Ecto.Changeset.validate_number/2` **DOES NOT SUPPORT the `:allow_nil` option**. By default, Ecto validations only run if a change for the given field exists and the change value is not nil, so such as option is never needed
|
||||
- You **must** use `Ecto.Changeset.get_field(changeset, :field)` to access changeset fields
|
||||
- Fields which are set programatically, such as `user_id`, must not be listed in `cast` calls or similar for security purposes. Instead they must be explicitly set when creating the struct
|
||||
- **Always** invoke `mix ecto.gen.migration migration_name_using_underscores` when generating migration files, so the correct timestamp and conventions are applied
|
||||
|
||||
<!-- phoenix:ecto-end -->
|
||||
<!-- phoenix:elixir-start -->
|
||||
## phoenix:elixir usage
|
||||
## Elixir guidelines
|
||||
|
||||
- Elixir lists **do not support index based access via the access syntax**
|
||||
|
||||
**Never do this (invalid)**:
|
||||
|
||||
i = 0
|
||||
mylist = ["blue", "green"]
|
||||
mylist[i]
|
||||
|
||||
Instead, **always** use `Enum.at`, pattern matching, or `List` for index based list access, ie:
|
||||
|
||||
i = 0
|
||||
mylist = ["blue", "green"]
|
||||
Enum.at(mylist, i)
|
||||
|
||||
- Elixir variables are immutable, but can be rebound, so for block expressions like `if`, `case`, `cond`, etc
|
||||
you *must* bind the result of the expression to a variable if you want to use it and you CANNOT rebind the result inside the expression, ie:
|
||||
|
||||
# INVALID: we are rebinding inside the `if` and the result never gets assigned
|
||||
if connected?(socket) do
|
||||
socket = assign(socket, :val, val)
|
||||
end
|
||||
|
||||
# VALID: we rebind the result of the `if` to a new variable
|
||||
socket =
|
||||
if connected?(socket) do
|
||||
assign(socket, :val, val)
|
||||
end
|
||||
|
||||
- **Never** nest multiple modules in the same file as it can cause cyclic dependencies and compilation errors
|
||||
- **Never** use map access syntax (`changeset[:field]`) on structs as they do not implement the Access behaviour by default. For regular structs, you **must** access the fields directly, such as `my_struct.field` or use higher level APIs that are available on the struct if they exist, `Ecto.Changeset.get_field/2` for changesets
|
||||
- Elixir's standard library has everything necessary for date and time manipulation. Familiarize yourself with the common `Time`, `Date`, `DateTime`, and `Calendar` interfaces by accessing their documentation as necessary. **Never** install additional dependencies unless asked or for date/time parsing (which you can use the `date_time_parser` package)
|
||||
- Don't use `String.to_atom/1` on user input (memory leak risk)
|
||||
- Predicate function names should not start with `is_` and should end in a question mark. Names like `is_thing` should be reserved for guards
|
||||
- Elixir's builtin OTP primitives like `DynamicSupervisor` and `Registry`, require names in the child spec, such as `{DynamicSupervisor, name: MyApp.MyDynamicSup}`, then you can use `DynamicSupervisor.start_child(MyApp.MyDynamicSup, child_spec)`
|
||||
- Use `Task.async_stream(collection, callback, options)` for concurrent enumeration with back-pressure. The majority of times you will want to pass `timeout: :infinity` as option
|
||||
|
||||
## Mix guidelines
|
||||
|
||||
- Read the docs and options before using tasks (by using `mix help task_name`)
|
||||
- To debug test failures, run tests in a specific file with `mix test test/my_test.exs` or run all previously failed tests with `mix test --failed`
|
||||
- `mix deps.clean --all` is **almost never needed**. **Avoid** using it unless you have good reason
|
||||
|
||||
## Test guidelines
|
||||
|
||||
- **Always use `start_supervised!/1`** to start processes in tests as it guarantees cleanup between tests
|
||||
- **Avoid** `Process.sleep/1` and `Process.alive?/1` in tests
|
||||
- Instead of sleeping to wait for a process to finish, **always** use `Process.monitor/1` and assert on the DOWN message:
|
||||
|
||||
ref = Process.monitor(pid)
|
||||
assert_receive {:DOWN, ^ref, :process, ^pid, :normal}
|
||||
|
||||
- Instead of sleeping to synchronize before the next call, **always** use `_ = :sys.get_state/1` to ensure the process has handled prior messages
|
||||
|
||||
|
||||
<!-- phoenix:elixir-end -->
|
||||
<!-- phoenix:html-start -->
|
||||
## phoenix:html usage
|
||||
## Phoenix HTML guidelines
|
||||
|
||||
- Phoenix templates **always** use `~H` or .html.heex files (known as HEEx), **never** use `~E`
|
||||
- **Always** use the imported `Phoenix.Component.form/1` and `Phoenix.Component.inputs_for/1` function to build forms. **Never** use `Phoenix.HTML.form_for` or `Phoenix.HTML.inputs_for` as they are outdated
|
||||
- When building forms **always** use the already imported `Phoenix.Component.to_form/2` (`assign(socket, form: to_form(...))` and `<.form for={@form} id="msg-form">`), then access those forms in the template via `@form[:field]`
|
||||
- **Always** add unique DOM IDs to key elements (like forms, buttons, etc) when writing templates, these IDs can later be used in tests (`<.form for={@form} id="product-form">`)
|
||||
- For "app wide" template imports, you can import/alias into the `my_app_web.ex`'s `html_helpers` block, so they will be available to all LiveViews, LiveComponent's, and all modules that do `use MyAppWeb, :html` (replace "my_app" by the actual app name)
|
||||
|
||||
- Elixir supports `if/else` but **does NOT support `if/else if` or `if/elsif`**. **Never use `else if` or `elseif` in Elixir**, **always** use `cond` or `case` for multiple conditionals.
|
||||
|
||||
**Never do this (invalid)**:
|
||||
|
||||
<%= if condition do %>
|
||||
...
|
||||
<% else if other_condition %>
|
||||
...
|
||||
<% end %>
|
||||
|
||||
Instead **always** do this:
|
||||
|
||||
<%= cond do %>
|
||||
<% condition -> %>
|
||||
...
|
||||
<% condition2 -> %>
|
||||
...
|
||||
<% true -> %>
|
||||
...
|
||||
<% end %>
|
||||
|
||||
- HEEx require special tag annotation if you want to insert literal curly's like `{` or `}`. If you want to show a textual code snippet on the page in a `<pre>` or `<code>` block you *must* annotate the parent tag with `phx-no-curly-interpolation`:
|
||||
|
||||
<code phx-no-curly-interpolation>
|
||||
let obj = {key: "val"}
|
||||
</code>
|
||||
|
||||
Within `phx-no-curly-interpolation` annotated tags, you can use `{` and `}` without escaping them, and dynamic Elixir expressions can still be used with `<%= ... %>` syntax
|
||||
|
||||
- HEEx class attrs support lists, but you must **always** use list `[...]` syntax. You can use the class list syntax to conditionally add classes, **always do this for multiple class values**:
|
||||
|
||||
<a class={[
|
||||
"px-2 text-white",
|
||||
@some_flag && "py-5",
|
||||
if(@other_condition, do: "border-red-500", else: "border-blue-100"),
|
||||
...
|
||||
]}>Text</a>
|
||||
|
||||
and **always** wrap `if`'s inside `{...}` expressions with parens, like done above (`if(@other_condition, do: "...", else: "...")`)
|
||||
|
||||
and **never** do this, since it's invalid (note the missing `[` and `]`):
|
||||
|
||||
<a class={
|
||||
"px-2 text-white",
|
||||
@some_flag && "py-5"
|
||||
}> ...
|
||||
=> Raises compile syntax error on invalid HEEx attr syntax
|
||||
|
||||
- **Never** use `<% Enum.each %>` or non-for comprehensions for generating template content, instead **always** use `<%= for item <- @collection do %>`
|
||||
- HEEx HTML comments use `<%!-- comment --%>`. **Always** use the HEEx HTML comment syntax for template comments (`<%!-- comment --%>`)
|
||||
- HEEx allows interpolation via `{...}` and `<%= ... %>`, but the `<%= %>` **only** works within tag bodies. **Always** use the `{...}` syntax for interpolation within tag attributes, and for interpolation of values within tag bodies. **Always** interpolate block constructs (if, cond, case, for) within tag bodies using `<%= ... %>`.
|
||||
|
||||
**Always** do this:
|
||||
|
||||
<div id={@id}>
|
||||
{@my_assign}
|
||||
<%= if @some_block_condition do %>
|
||||
{@another_assign}
|
||||
<% end %>
|
||||
</div>
|
||||
|
||||
and **Never** do this – the program will terminate with a syntax error:
|
||||
|
||||
<%!-- THIS IS INVALID NEVER EVER DO THIS --%>
|
||||
<div id="<%= @invalid_interpolation %>">
|
||||
{if @invalid_block_construct do}
|
||||
{end}
|
||||
</div>
|
||||
|
||||
<!-- phoenix:html-end -->
|
||||
<!-- phoenix:liveview-start -->
|
||||
## phoenix:liveview usage
|
||||
## Phoenix LiveView guidelines
|
||||
|
||||
- **Never** use the deprecated `live_redirect` and `live_patch` functions, instead **always** use the `<.link navigate={href}>` and `<.link patch={href}>` in templates, and `push_navigate` and `push_patch` functions LiveViews
|
||||
- **Avoid LiveComponent's** unless you have a strong, specific need for them
|
||||
- LiveViews should be named like `AppWeb.WeatherLive`, with a `Live` suffix. When you go to add LiveView routes to the router, the default `:browser` scope is **already aliased** with the `AppWeb` module, so you can just do `live "/weather", WeatherLive`
|
||||
|
||||
### LiveView streams
|
||||
|
||||
- **Always** use LiveView streams for collections for assigning regular lists to avoid memory ballooning and runtime termination with the following operations:
|
||||
- basic append of N items - `stream(socket, :messages, [new_msg])`
|
||||
- resetting stream with new items - `stream(socket, :messages, [new_msg], reset: true)` (e.g. for filtering items)
|
||||
- prepend to stream - `stream(socket, :messages, [new_msg], at: -1)`
|
||||
- deleting items - `stream_delete(socket, :messages, msg)`
|
||||
|
||||
- When using the `stream/3` interfaces in the LiveView, the LiveView template must 1) always set `phx-update="stream"` on the parent element, with a DOM id on the parent element like `id="messages"` and 2) consume the `@streams.stream_name` collection and use the id as the DOM id for each child. For a call like `stream(socket, :messages, [new_msg])` in the LiveView, the template would be:
|
||||
|
||||
<div id="messages" phx-update="stream">
|
||||
<div :for={{id, msg} <- @streams.messages} id={id}>
|
||||
{msg.text}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
- LiveView streams are *not* enumerable, so you cannot use `Enum.filter/2` or `Enum.reject/2` on them. Instead, if you want to filter, prune, or refresh a list of items on the UI, you **must refetch the data and re-stream the entire stream collection, passing reset: true**:
|
||||
|
||||
def handle_event("filter", %{"filter" => filter}, socket) do
|
||||
# re-fetch the messages based on the filter
|
||||
messages = list_messages(filter)
|
||||
|
||||
{:noreply,
|
||||
socket
|
||||
|> assign(:messages_empty?, messages == [])
|
||||
# reset the stream with the new messages
|
||||
|> stream(:messages, messages, reset: true)}
|
||||
end
|
||||
|
||||
- LiveView streams *do not support counting or empty states*. If you need to display a count, you must track it using a separate assign. For empty states, you can use Tailwind classes:
|
||||
|
||||
<div id="tasks" phx-update="stream">
|
||||
<div class="hidden only:block">No tasks yet</div>
|
||||
<div :for={{id, task} <- @stream.tasks} id={id}>
|
||||
{task.name}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
The above only works if the empty state is the only HTML block alongside the stream for-comprehension.
|
||||
|
||||
- When updating an assign that should change content inside any streamed item(s), you MUST re-stream the items
|
||||
along with the updated assign:
|
||||
|
||||
def handle_event("edit_message", %{"message_id" => message_id}, socket) do
|
||||
message = Chat.get_message!(message_id)
|
||||
edit_form = to_form(Chat.change_message(message, %{content: message.content}))
|
||||
|
||||
# re-insert message so @editing_message_id toggle logic takes effect for that stream item
|
||||
{:noreply,
|
||||
socket
|
||||
|> stream_insert(:messages, message)
|
||||
|> assign(:editing_message_id, String.to_integer(message_id))
|
||||
|> assign(:edit_form, edit_form)}
|
||||
end
|
||||
|
||||
And in the template:
|
||||
|
||||
<div id="messages" phx-update="stream">
|
||||
<div :for={{id, message} <- @streams.messages} id={id} class="flex group">
|
||||
{message.username}
|
||||
<%= if @editing_message_id == message.id do %>
|
||||
<%!-- Edit mode --%>
|
||||
<.form for={@edit_form} id="edit-form-#{message.id}" phx-submit="save_edit">
|
||||
...
|
||||
</.form>
|
||||
<% end %>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
- **Never** use the deprecated `phx-update="append"` or `phx-update="prepend"` for collections
|
||||
|
||||
### LiveView JavaScript interop
|
||||
|
||||
- Remember anytime you use `phx-hook="MyHook"` and that JS hook manages its own DOM, you **must** also set the `phx-update="ignore"` attribute
|
||||
- **Always** provide an unique DOM id alongside `phx-hook` otherwise a compiler error will be raised
|
||||
|
||||
LiveView hooks come in two flavors, 1) colocated js hooks for "inline" scripts defined inside HEEx,
|
||||
and 2) external `phx-hook` annotations where JavaScript object literals are defined and passed to the `LiveSocket` constructor.
|
||||
|
||||
#### Inline colocated js hooks
|
||||
|
||||
**Never** write raw embedded `<script>` tags in heex as they are incompatible with LiveView.
|
||||
Instead, **always use a colocated js hook script tag (`:type={Phoenix.LiveView.ColocatedHook}`)
|
||||
when writing scripts inside the template**:
|
||||
|
||||
<input type="text" name="user[phone_number]" id="user-phone-number" phx-hook=".PhoneNumber" />
|
||||
<script :type={Phoenix.LiveView.ColocatedHook} name=".PhoneNumber">
|
||||
export default {
|
||||
mounted() {
|
||||
this.el.addEventListener("input", e => {
|
||||
let match = this.el.value.replace(/\D/g, "").match(/^(\d{3})(\d{3})(\d{4})$/)
|
||||
if(match) {
|
||||
this.el.value = `${match[1]}-${match[2]}-${match[3]}`
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
- colocated hooks are automatically integrated into the app.js bundle
|
||||
- colocated hooks names **MUST ALWAYS** start with a `.` prefix, i.e. `.PhoneNumber`
|
||||
|
||||
#### External phx-hook
|
||||
|
||||
External JS hooks (`<div id="myhook" phx-hook="MyHook">`) must be placed in `assets/js/` and passed to the
|
||||
LiveSocket constructor:
|
||||
|
||||
const MyHook = {
|
||||
mounted() { ... }
|
||||
}
|
||||
let liveSocket = new LiveSocket("/live", Socket, {
|
||||
hooks: { MyHook }
|
||||
});
|
||||
|
||||
#### Pushing events between client and server
|
||||
|
||||
Use LiveView's `push_event/3` when you need to push events/data to the client for a phx-hook to handle.
|
||||
**Always** return or rebind the socket on `push_event/3` when pushing events:
|
||||
|
||||
# re-bind socket so we maintain event state to be pushed
|
||||
socket = push_event(socket, "my_event", %{...})
|
||||
|
||||
# or return the modified socket directly:
|
||||
def handle_event("some_event", _, socket) do
|
||||
{:noreply, push_event(socket, "my_event", %{...})}
|
||||
end
|
||||
|
||||
Pushed events can then be picked up in a JS hook with `this.handleEvent`:
|
||||
|
||||
mounted() {
|
||||
this.handleEvent("my_event", data => console.log("from server:", data));
|
||||
}
|
||||
|
||||
Clients can also push an event to the server and receive a reply with `this.pushEvent`:
|
||||
|
||||
mounted() {
|
||||
this.el.addEventListener("click", e => {
|
||||
this.pushEvent("my_event", { one: 1 }, reply => console.log("got reply from server:", reply));
|
||||
})
|
||||
}
|
||||
|
||||
Where the server handled it via:
|
||||
|
||||
def handle_event("my_event", %{"one" => 1}, socket) do
|
||||
{:reply, %{two: 2}, socket}
|
||||
end
|
||||
|
||||
### LiveView tests
|
||||
|
||||
- `Phoenix.LiveViewTest` module and `LazyHTML` (included) for making your assertions
|
||||
- Form tests are driven by `Phoenix.LiveViewTest`'s `render_submit/2` and `render_change/2` functions
|
||||
- Come up with a step-by-step test plan that splits major test cases into small, isolated files. You may start with simpler tests that verify content exists, gradually add interaction tests
|
||||
- **Always reference the key element IDs you added in the LiveView templates in your tests** for `Phoenix.LiveViewTest` functions like `element/2`, `has_element/2`, selectors, etc
|
||||
- **Never** tests again raw HTML, **always** use `element/2`, `has_element/2`, and similar: `assert has_element?(view, "#my-form")`
|
||||
- Instead of relying on testing text content, which can change, favor testing for the presence of key elements
|
||||
- Focus on testing outcomes rather than implementation details
|
||||
- Be aware that `Phoenix.Component` functions like `<.form>` might produce different HTML than expected. Test against the output HTML structure, not your mental model of what you expect it to be
|
||||
- When facing test failures with element selectors, add debug statements to print the actual HTML, but use `LazyHTML` selectors to limit the output, ie:
|
||||
|
||||
html = render(view)
|
||||
document = LazyHTML.from_fragment(html)
|
||||
matches = LazyHTML.filter(document, "your-complex-selector")
|
||||
IO.inspect(matches, label: "Matches")
|
||||
|
||||
### Form handling
|
||||
|
||||
#### Creating a form from params
|
||||
|
||||
If you want to create a form based on `handle_event` params:
|
||||
|
||||
def handle_event("submitted", params, socket) do
|
||||
{:noreply, assign(socket, form: to_form(params))}
|
||||
end
|
||||
|
||||
When you pass a map to `to_form/1`, it assumes said map contains the form params, which are expected to have string keys.
|
||||
|
||||
You can also specify a name to nest the params:
|
||||
|
||||
def handle_event("submitted", %{"user" => user_params}, socket) do
|
||||
{:noreply, assign(socket, form: to_form(user_params, as: :user))}
|
||||
end
|
||||
|
||||
#### Creating a form from changesets
|
||||
|
||||
When using changesets, the underlying data, form params, and errors are retrieved from it. The `:as` option is automatically computed too. E.g. if you have a user schema:
|
||||
|
||||
defmodule MyApp.Users.User do
|
||||
use Ecto.Schema
|
||||
...
|
||||
end
|
||||
|
||||
And then you create a changeset that you pass to `to_form`:
|
||||
|
||||
%MyApp.Users.User{}
|
||||
|> Ecto.Changeset.change()
|
||||
|> to_form()
|
||||
|
||||
Once the form is submitted, the params will be available under `%{"user" => user_params}`.
|
||||
|
||||
In the template, the form form assign can be passed to the `<.form>` function component:
|
||||
|
||||
<.form for={@form} id="todo-form" phx-change="validate" phx-submit="save">
|
||||
<.input field={@form[:field]} type="text" />
|
||||
</.form>
|
||||
|
||||
Always give the form an explicit, unique DOM ID, like `id="todo-form"`.
|
||||
|
||||
#### Avoiding form errors
|
||||
|
||||
**Always** use a form assigned via `to_form/2` in the LiveView, and the `<.input>` component in the template. In the template **always access forms this**:
|
||||
|
||||
<%!-- ALWAYS do this (valid) --%>
|
||||
<.form for={@form} id="my-form">
|
||||
<.input field={@form[:field]} type="text" />
|
||||
</.form>
|
||||
|
||||
And **never** do this:
|
||||
|
||||
<%!-- NEVER do this (invalid) --%>
|
||||
<.form for={@changeset} id="my-form">
|
||||
<.input field={@changeset[:field]} type="text" />
|
||||
</.form>
|
||||
|
||||
- You are FORBIDDEN from accessing the changeset in the template as it will cause errors
|
||||
- **Never** use `<.form let={f} ...>` in the template, instead **always use `<.form for={@form} ...>`**, then drive all form references from the form assign as in `@form[:field]`. The UI should **always** be driven by a `to_form/2` assigned in the LiveView module that is derived from a changeset
|
||||
|
||||
<!-- phoenix:liveview-end -->
|
||||
<!-- phoenix:phoenix-start -->
|
||||
## phoenix:phoenix usage
|
||||
## Phoenix guidelines
|
||||
|
||||
- Remember Phoenix router `scope` blocks include an optional alias which is prefixed for all routes within the scope. **Always** be mindful of this when creating routes within a scope to avoid duplicate module prefixes.
|
||||
|
||||
- You **never** need to create your own `alias` for route definitions! The `scope` provides the alias, ie:
|
||||
|
||||
scope "/admin", AppWeb.Admin do
|
||||
pipe_through :browser
|
||||
|
||||
live "/users", UserLive, :index
|
||||
end
|
||||
|
||||
the UserLive route would point to the `AppWeb.Admin.UserLive` module
|
||||
|
||||
- `Phoenix.View` no longer is needed or included with Phoenix, don't use it
|
||||
|
||||
<!-- phoenix:phoenix-end -->
|
||||
<!-- fluxon-start -->
|
||||
## fluxon usage
|
||||
_Fluxon UI Components_
|
||||
|
||||
[fluxon usage rules](deps/fluxon/usage-rules.md)
|
||||
<!-- fluxon-end -->
|
||||
<!-- usage-rules-end -->
|
||||
@@ -1,79 +0,0 @@
|
||||
# Advanced Perceptually-Aware Color Extraction Algorithms
|
||||
|
||||
Modern image color extraction demands algorithms that understand visual perception, spatial context, and semantic importance rather than simple pixel frequency statistics. This comprehensive analysis covers state-of-the-art techniques specifically designed for extracting the perceptual "essence" of images like album covers, where artistic elements carry greater weight than raw pixel counts.
|
||||
|
||||
## Saliency-based attention algorithms prioritize visually important colors
|
||||
|
||||
**Graph-Based Visual Saliency (GBVS)** represents the most effective traditional approach for attention-guided color extraction. Unlike pixel-frequency methods, GBVS achieves **98% correlation with human attention patterns** by treating saliency as an equilibrium distribution of Markov chains on image graphs. The algorithm creates activation maps based on feature dissimilarity rather than simple center-surround differences, then normalizes these through random walks to produce probabilistic attention maps.
|
||||
|
||||
For practical implementation, GBVS can be integrated with color clustering through **saliency-weighted sampling**: instead of treating all pixels equally, pixels are sampled for k-means clustering proportionally to their saliency scores. This approach naturally emphasizes colors from visually important regions while de-emphasizing background areas.
|
||||
|
||||
Modern **transformer-based saliency models** like Vision Transformers provide even superior performance through global context modeling. These architectures capture long-range color relationships across entire images using self-attention mechanisms, avoiding the spatial limitations of traditional convolutional approaches. **CLIP-ViT models demonstrate particularly strong color understanding** when combined with natural language descriptions.
|
||||
|
||||
The key implementation strategy involves a multi-stage pipeline: generate saliency maps using GBVS or CNN-based models, apply attention-guided pixel sampling weighted by saliency scores, perform clustering in perceptual color spaces, and rank final palette colors by importance based on their saliency contributions.
|
||||
|
||||
## Perceptual color spaces align extraction with human vision
|
||||
|
||||
Traditional RGB-based color extraction fails because **equal RGB distances don't represent equal perceptual differences**. Perceptual color spaces like **CIELAB and CAM16-UCS** provide approximately uniform representations where numerical differences correspond to similar perceived color differences.
|
||||
|
||||
**CIELAB's superiority emerges from opponent color theory alignment** - the L* lightness channel, a* green-red axis, and b* blue-yellow axis match human visual system structure. Research shows the 'b' channel correlates most strongly with visual attention, making it particularly valuable for saliency-guided extraction.
|
||||
|
||||
**Advanced color difference metrics like CIEDE2000** dramatically improve clustering quality over simple Euclidean distance. For red-blue color comparisons, CIE2000 achieves perceptual accuracy scores of 52.9 versus 176.3 for basic RGB distance calculations. The algorithm incorporates hue rotation terms, neutral color compensation, and sophisticated weighting factors for lightness, chroma, and hue differences.
|
||||
|
||||
**Color appearance models like CAM16** add another sophistication layer by predicting how colors appear under different viewing conditions. These models include chromatic adaptation transforms (CAT16) that simulate human visual adaptation to different illuminants, ensuring consistent color extraction across varying lighting conditions.
|
||||
|
||||
Practical implementation involves converting RGB to CIELAB color space, applying CAT16 transforms for viewing condition normalization, using CIE2000 distance metrics in clustering algorithms, and weighting color contributions based on psychophysical principles like Weber-Fechner law applications.
|
||||
|
||||
## Spatial awareness through edge detection and region analysis
|
||||
|
||||
**Spatially-aware algorithms weight colors based on structural importance** rather than pure frequency. Edge-guided color weighting assigns higher importance to colors near strong boundaries, as these often represent perceptually significant object boundaries and artistic elements crucial in album cover design.
|
||||
|
||||
**SLIC superpixels provide optimal spatial coherence** by clustering pixels in combined spatial-color space [x, y, L, a, b]. The algorithm uses distance measures that balance color similarity with spatial proximity, creating regions that maintain both color consistency and spatial coherence. Each superpixel represents a meaningful color region whose contribution can be weighted by size, edge strength, and spatial position.
|
||||
|
||||
**Multi-scale analysis using Gaussian-Laplacian pyramids** captures color information at different hierarchical levels. Coarse scales identify major color themes while fine scales extract detailed variations. This approach proves particularly effective for complex artistic images where color significance varies across scales.
|
||||
|
||||
**Graph-based segmentation (Felzenszwalb-Huttenlocher)** provides adaptive region boundaries by representing images as weighted graphs where edge weights measure color dissimilarity between adjacent pixels. The algorithm merges regions based on internal homogeneity versus external differences, automatically adapting to image content complexity.
|
||||
|
||||
For album covers and artistic content, **figure-ground segmentation using GrabCut energy minimization** effectively separates foreground artistic elements from backgrounds. The algorithm combines Gaussian mixture models for color learning with spatial smoothness terms, enabling automatic identification of artistically important regions that should dominate the color palette.
|
||||
|
||||
## Deep learning enables semantic color understanding
|
||||
|
||||
**Modern CNN architectures like U-Net and ResNet** extract hierarchical color features that capture semantic relationships beyond simple clustering. CNN-based color palette extraction using pre-trained features achieves superior palette generation compared to pixel-based methods by understanding object-level color relationships.
|
||||
|
||||
**Vision transformers revolutionize color analysis** through global context modeling without spatial limitations. The self-attention mechanism captures long-range color dependencies across entire images, making them particularly effective for artistic content where color relationships span large spatial distances.
|
||||
|
||||
**Generative adversarial networks (GANs) trained on professional design databases** learn color harmony principles directly from human-curated content. Colormind's GAN architecture achieves **91%+ accuracy in user studies** by training on Adobe Color and Dribbble palettes, demonstrating how neural networks can internalize aesthetic color principles.
|
||||
|
||||
**Semantic segmentation models like DeepLab and Mask R-CNN** enable object-specific color analysis. These models identify semantically important regions (people, objects, text) and weight their color contributions more heavily than background areas, particularly valuable for album covers where specific elements carry artistic significance.
|
||||
|
||||
**End-to-end learned approaches** combine saliency detection, segmentation, and color extraction in unified frameworks. Models like SU2GE-Net use Swin TransformerV2 backbones for simultaneous saliency detection and color analysis, achieving state-of-the-art performance on complex artistic imagery.
|
||||
|
||||
## Industry implementations provide production-ready solutions
|
||||
|
||||
**Adobe's Creative Suite employs multi-modal color analysis** combining k-means clustering with Adobe Sensei AI for content-aware color matching. Their system uses multiple color spaces (sRGB, Adobe RGB, Lab, HSV) with neural networks for perceptual color space mapping and harmony suggestions based on color theory principles.
|
||||
|
||||
**Google's Vision API provides scalable cloud-based color extraction** using convolutional neural networks trained on massive image datasets. The service returns confidence scores and pixel fractions for each extracted color, enabling sophisticated weighting schemes for final palette generation.
|
||||
|
||||
**Apple's Vision Framework optimizes for on-device processing** using their Neural Engine for real-time color analysis. The system achieves sub-4ms processing times while maintaining privacy through local computation, making it ideal for interactive applications.
|
||||
|
||||
For practical implementation, **libvips integration offers 4-5x faster processing than ImageMagick** with comprehensive color space support and built-in color difference calculations (dE76, dE00, dECMC). The library's demand-driven processing architecture scales efficiently for production web applications.
|
||||
|
||||
## Implementation strategies for advanced color extraction
|
||||
|
||||
**Multi-stage processing pipelines** provide the most effective approach: begin with saliency detection using GBVS or transformer models, perform spatial segmentation using SLIC superpixels or semantic segmentation, extract regional color information weighted by saliency and spatial importance, cluster in perceptual color spaces using CIE2000 distances, and rank final colors by semantic and artistic importance.
|
||||
|
||||
**Performance optimization requires careful algorithmic choices**. For real-time applications, GBVS with OpenCV provides good balance of quality and speed. For highest quality results, transformer-based saliency with semantic segmentation yields superior artistic relevance. GPU acceleration enables real-time processing of high-resolution images through CUDA implementations.
|
||||
|
||||
**Color space selection impacts results significantly**. CIELAB provides good perceptual uniformity for most applications. CAM16-UCS offers superior performance when viewing conditions vary. OKLab represents the newest advancement with improved perceptual uniformity over CIELAB.
|
||||
|
||||
**Library integration strategies** depend on requirements: Google Vision API for cloud-scale processing, Apple Vision Framework for iOS/macOS applications, OpenCV with scikit-learn for custom implementations, and libvips for high-performance server applications.
|
||||
|
||||
## Future directions in perceptual color extraction
|
||||
|
||||
The field continues advancing through **multi-modal integration combining visual and semantic cues**. Vision-language models like CLIP demonstrate strong color understanding when guided by textual descriptions, enabling more sophisticated artistic color analysis.
|
||||
|
||||
**Personalized attention models** account for individual differences in color perception and aesthetic preferences. These systems adapt color extraction based on user feedback and demographic factors, particularly relevant for consumer applications.
|
||||
|
||||
**Foundation models for color analysis** leverage large-scale pre-training to understand color relationships across diverse domains. These models show promise for few-shot learning on specific artistic styles or cultural color preferences.
|
||||
|
||||
The convergence of attention-based methods, perceptual color science, spatial analysis, and deep learning creates a powerful toolkit for extracting truly meaningful color palettes from artistic content. These advanced techniques finally move beyond crude pixel counting to understand the visual and aesthetic essence that makes colors truly important in human perception.
|
||||
@@ -1,104 +0,0 @@
|
||||
# Record Management
|
||||
|
||||
- Import a record from a MusicBrainz release ID
|
||||
- Import a record from a MusicBrainz release group
|
||||
- Search records with filtering and pagination
|
||||
- Get random record from collection
|
||||
- Get latest purchased record
|
||||
- Refresh record metadata from MusicBrainz
|
||||
- Generate genres for a record using AI
|
||||
- Update cover art from MusicBrainz
|
||||
- Extract dominant colors from album artwork
|
||||
- Create, update, or delete records
|
||||
- Resize cover art to standard dimensions
|
||||
|
||||
# Collection Operations
|
||||
|
||||
- Add record to collection (mark as purchased)
|
||||
- Search collection with various filters
|
||||
- View collection statistics by format (CD, vinyl, etc.)
|
||||
- View collection statistics by type (album, EP, etc.)
|
||||
- Get top artists in collection by record count
|
||||
- Get top genres in collection by record count
|
||||
|
||||
# Wishlist Operations
|
||||
|
||||
- Add record to wishlist
|
||||
- Search wishlist records
|
||||
- Count total wishlist items
|
||||
- Check which releases are wishlisted
|
||||
|
||||
# Last.fm Integration
|
||||
|
||||
- Scrobble a release to Last.fm
|
||||
- Scrobble a specific disc/medium to Last.fm
|
||||
- Import listening history from Last.fm
|
||||
- Get top albums by time period (7/30/90/365 days)
|
||||
- Get top artists by time period
|
||||
- Get most recent scrobbled track for a record
|
||||
- Backfill historical Last.fm data
|
||||
- Refresh scrobbled tracks from Last.fm
|
||||
- Get Last.fm authentication session
|
||||
|
||||
# Search Operations
|
||||
|
||||
- Universal search across collection, wishlist, and artists
|
||||
- Search collection records
|
||||
- Search wishlist records
|
||||
- Search artists by name
|
||||
- Search records by barcode
|
||||
|
||||
# Artist Management
|
||||
|
||||
- Fetch artist information from MusicBrainz and Discogs
|
||||
- Get similar artists from Last.fm that are in collection
|
||||
- Download and cache artist images
|
||||
- View artist's records in collection
|
||||
- Clean up unused artist information
|
||||
|
||||
# Data Quality & Rules
|
||||
|
||||
- Create scrobble correction rules for album names
|
||||
- Create scrobble correction rules for artist names
|
||||
- Apply correction rules to scrobbled tracks
|
||||
- Count tracks affected by correction rules
|
||||
- Enable/disable correction rules
|
||||
|
||||
# Barcode Scanning
|
||||
|
||||
- Scan barcode and check collection/wishlist status
|
||||
- Import multiple scanned records in batch
|
||||
- Import single scanned record from barcode
|
||||
|
||||
# External Store Integration
|
||||
|
||||
- Create templates for online music stores
|
||||
- Generate purchase URLs from store templates
|
||||
- Manage enabled/disabled store templates
|
||||
|
||||
# Secrets Management
|
||||
|
||||
- Store encrypted API keys and credentials
|
||||
- Retrieve encrypted secrets by name
|
||||
|
||||
# Background Processing
|
||||
|
||||
- Queue color extraction jobs
|
||||
- Queue artist info fetching jobs
|
||||
- Queue cover art refresh jobs
|
||||
- Queue metadata refresh jobs
|
||||
- Queue artist image download jobs
|
||||
- Clean up unused artist information
|
||||
|
||||
# Statistics & Analytics
|
||||
|
||||
- View listening statistics by time period
|
||||
- Compare listening trends across multiple periods
|
||||
- Count records by format, type, artist, or genre
|
||||
- Get collection overview statistics
|
||||
|
||||
# Data Import/Export
|
||||
|
||||
- Import batch tracks from Last.fm API
|
||||
- Parse Obsidian music notes with frontmatter
|
||||
- Export listening data (via statistics views)
|
||||
@@ -1,186 +0,0 @@
|
||||
# Database Structure
|
||||
|
||||
<!--toc:start-->
|
||||
|
||||
- [Database Structure](#database-structure)
|
||||
- [Entity Relationship Diagram](#entity-relationship-diagram)
|
||||
- [Tables Description](#tables-description)
|
||||
- [Records](#records)
|
||||
- [Records Search Index](#records-search-index)
|
||||
- [Artist Infos](#artist-infos)
|
||||
- [Views](#views)
|
||||
- [Artist Records View](#artist-records-view)
|
||||
- [Triggers](#triggers)
|
||||
- [Indices](#indices)
|
||||
- [Notes](#notes)
|
||||
- [WHY ONE TABLE?](#why-one-table)
|
||||
<!--toc:end-->
|
||||
|
||||
This document describes the database structure of the Music Library application.
|
||||
|
||||
## Entity Relationship Diagram
|
||||
|
||||
```mermaid
|
||||
erDiagram
|
||||
RECORDS {
|
||||
uuid id PK
|
||||
enum type
|
||||
enum format
|
||||
string title
|
||||
string cover_url
|
||||
blob cover_data
|
||||
string cover_hash
|
||||
uuid musicbrainz_id
|
||||
map musicbrainz_data
|
||||
string[] genres
|
||||
string release_date
|
||||
datetime purchased_at
|
||||
string selected_release_id
|
||||
string[] release_ids
|
||||
string[] included_release_group_ids
|
||||
Artist[] artists
|
||||
datetime inserted_at
|
||||
datetime updated_at
|
||||
}
|
||||
|
||||
RECORDS_SEARCH_INDEX {
|
||||
uuid id PK
|
||||
enum type
|
||||
enum format
|
||||
string title
|
||||
string normalized_title
|
||||
Artist[] artists
|
||||
string normalized_artists
|
||||
string[] genres
|
||||
uuid musicbrainz_id
|
||||
string[] release_ids
|
||||
string[] included_release_group_ids
|
||||
string cover_hash
|
||||
datetime purchased_at
|
||||
string release_date
|
||||
}
|
||||
|
||||
ARTIST_INFOS {
|
||||
uuid id PK
|
||||
map musicbrainz_data
|
||||
map discogs_data
|
||||
blob image_data
|
||||
string image_data_hash
|
||||
integer image_data_width
|
||||
datetime inserted_at
|
||||
datetime updated_at
|
||||
}
|
||||
|
||||
ARTIST_RECORDS {
|
||||
uuid musicbrainz_id
|
||||
uuid record_id
|
||||
map artist
|
||||
}
|
||||
|
||||
RECORDS ||--o{ RECORDS_SEARCH_INDEX : "syncs via triggers"
|
||||
RECORDS ||--o{ ARTIST_INFOS : "references via musicbrainz_id"
|
||||
RECORDS ||--o{ ARTIST_RECORDS : "extracted via view"
|
||||
```
|
||||
|
||||
## Tables Description
|
||||
|
||||
### Records
|
||||
|
||||
The main table storing music records. Key features:
|
||||
|
||||
- Uses UUID as primary key
|
||||
- Stores basic record information (title, type [album, ep, live, compilation, single, other], format [cd, backup, vinyl, blu_ray, dvd, multi])
|
||||
- Includes MusicBrainz integration with IDs and additional data (musicbrainz_id, musicbrainz_data)
|
||||
- Stores cover image data and URLs (cover_url, cover_data, cover_hash)
|
||||
- Embeds artists data as an array of objects (each with musicbrainz_id, name, sort_name, disambiguation)
|
||||
- Includes timestamps for record keeping
|
||||
- Tracks purchase status via `purchased_at` field
|
||||
- Stores release information including multiple release IDs (`release_ids`), included release group IDs (`included_release_group_ids`), and a selected release ID (`selected_release_id`)
|
||||
- Maintains release date information (`release_date`)
|
||||
|
||||
### Records Search Index
|
||||
|
||||
A virtual FTS5 (Full Text Search) table that mirrors the records table for efficient searching:
|
||||
|
||||
- Automatically synced with the records table via triggers
|
||||
- Optimized for full-text search operations
|
||||
- Contains most fields from the records table, including:
|
||||
- id, type, format, title, normalized_title, artists, normalized_artists, genres, musicbrainz_id, release_ids, included_release_group_ids, cover_hash, purchased_at, release_date
|
||||
- `normalized_title` and `normalized_artists` are unaccented versions for improved search
|
||||
- Some fields are marked as UNINDEXED for efficiency
|
||||
|
||||
### Artist Infos
|
||||
|
||||
A table that stores additional artist information:
|
||||
|
||||
- Uses UUID as primary key
|
||||
- Stores MusicBrainz and Discogs data for artists
|
||||
- Maintains artist image data with dimensions and hash
|
||||
- Includes timestamps for record keeping
|
||||
|
||||
### Views
|
||||
|
||||
#### Artist Records View
|
||||
|
||||
A view that extracts artist information from the embedded JSON in the records table:
|
||||
|
||||
```sql
|
||||
CREATE VIEW artist_records AS
|
||||
SELECT json_each.value ->> '$.musicbrainz_id' AS musicbrainz_id,
|
||||
records.id AS record_id,
|
||||
json_each.value as artist
|
||||
FROM records,
|
||||
json_each(records.artists)
|
||||
```
|
||||
|
||||
This view is crucial for querying artist information as it:
|
||||
|
||||
- Extracts individual artists from the embedded JSON array in the records table
|
||||
- Provides a normalized view of the artist-record relationships
|
||||
- Makes it easier to query records by artist
|
||||
- Maintains the relationship between records and their artists without requiring a separate join table
|
||||
|
||||
### Triggers
|
||||
|
||||
The following triggers maintain the search index:
|
||||
|
||||
1. `records_after_insert`: Inserts new record data into search index after record creation
|
||||
2. `records_after_update`: Updates record data in search index after record updates
|
||||
|
||||
### Indices
|
||||
|
||||
The following indices are maintained for performance:
|
||||
|
||||
1. On `records`:
|
||||
- `type`
|
||||
- `format`
|
||||
- `title`
|
||||
- `musicbrainz_id`
|
||||
- `purchased_at`
|
||||
- `included_release_group_ids`
|
||||
- `release_ids`
|
||||
|
||||
## Notes
|
||||
|
||||
1. The database uses SQLite as the primary database.
|
||||
2. Artists data is embedded directly in the records table as an array of objects, not a separate table.
|
||||
3. The search index is implemented using SQLite's FTS5 extension for efficient full-text search capabilities, with normalized (unaccented) fields for better search.
|
||||
4. Where needed, queries use SQLite's `unicode` extension to filter/sort over UTF-8 data.
|
||||
5. The database supports both collection and wishlist functionality through the `purchased_at` field:
|
||||
- Records with `purchased_at IS NOT NULL` are in the collection
|
||||
- Records with `purchased_at IS NULL` are in the wishlist
|
||||
6. The schema uses `release_date` for clarity
|
||||
7. The `selected_release_id` field tracks the primary release for a record
|
||||
8. The `artist_infos` table stores additional artist metadata and images
|
||||
9. The `artist_records` view provides a normalized way to query artist-record relationships
|
||||
|
||||
## WHY ONE TABLE?
|
||||
|
||||
In traditional relational database design, you would split out artists into a
|
||||
separate table, and associate them with records via a join table. So why
|
||||
sticking with one table?
|
||||
|
||||
1. You only need to backup/export one table.
|
||||
2. Re-fetching data from MusicBrainz becomes trivial, as it just needs to update one field and everything else cascades accordingly.
|
||||
3. Traditional efficiency design constraints [do not apply to
|
||||
SQLite](https://www.sqlite.org/np1queryprob.html), so it makes it easier to experiment with alternative database designs.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,264 +0,0 @@
|
||||
╭─────────────────────────────────────────────────────────╮
|
||||
│ ✻ Welcome to Claude Code! │
|
||||
│ │
|
||||
│ /help for help, /status for your current setup │
|
||||
│ │
|
||||
│ cwd: /Users/cloud/github.com/cloud8421/music_library │
|
||||
╰─────────────────────────────────────────────────────────╯
|
||||
|
||||
> Can you optimize this query?
|
||||
|
||||
SELECT s0."scrobbled_at_uts", s0."musicbrainz_id", s0."title", s0."cover_url", s0."scrobbled_at_label", s0."artist", s0."album",
|
||||
s0."last_fm_data", s1."record_id", s2."record_id", s3."musicbrainz_id", coalesce(s1."cover_hash", s2."cover_hash") FROM
|
||||
"scrobbled_tracks" AS s0 LEFT OUTER JOIN (SELECT records.id AS "record_id", records.cover_hash AS "cover_hash", sf0."value" AS
|
||||
"release_id" FROM records, json_each(records.release_ids) AS sf0 WHERE (records.purchased_at IS NOT NULL)) AS s1 ON
|
||||
s1."release_id" = s0."album" ->> '$.musicbrainz_id' LEFT OUTER JOIN (SELECT records.id AS "record_id", records.cover_hash AS
|
||||
"cover_hash", sf0."value" AS "release_id" FROM records, json_each(records.release_ids) AS sf0 WHERE (records.purchased_at IS
|
||||
NULL)) AS s2 ON s2."release_id" = s0."album" ->> '$.musicbrainz_id' LEFT OUTER JOIN (SELECT DISTINCT sa0."musicbrainz_id" AS
|
||||
"musicbrainz_id", sa0."record_id" AS "record_id", sa0."artist" AS "artist" FROM "artist_records" AS sa0) AS s3 ON (s2."record_id"
|
||||
= s3."record_id") OR (s1."record_id" = s3."record_id") ORDER BY s0."scrobbled_at_uts" DESC LIMIT 100
|
||||
|
||||
⏺ I'll analyze this query to identify optimization opportunities. Let me first examine the current database schema and understand
|
||||
the query structure.
|
||||
⎿ * LastFm.Album at lib/last_fm/album.ex:1
|
||||
* LastFm.Artist at lib/last_fm/artist.ex:1
|
||||
* LastFm.Track at lib/last_fm/track.ex:1
|
||||
… +11 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Search(pattern: "scrobbled_tracks")
|
||||
⎿ Found 11 files (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/last_fm/track.ex)
|
||||
⎿ Read 95 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Read(priv/repo/migrations/20250530203701_create_scrobbled_tracks.exs)
|
||||
⎿ Read 19 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/music_library/records/record.ex)
|
||||
⎿ Read 276 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Now let me find where this query is being generated to understand the context:
|
||||
|
||||
⏺ Search(pattern: "FROM.*scrobbled_tracks.*AS s0")
|
||||
⎿ Found 0 files
|
||||
|
||||
⏺ Search(pattern: "scrobbled_tracks.*LEFT OUTER JOIN", output_mode: "content")
|
||||
⎿ Found 0 lines
|
||||
|
||||
⏺ Search(pattern: "json_each.*release_ids")
|
||||
⎿ Found 3 files (ctrl+o to expand)
|
||||
|
||||
⏺ Search(pattern: "scrobbled_tracks.*s0", output_mode: "content")
|
||||
⎿ Found 0 lines
|
||||
|
||||
⏺ Let me search in the last_fm.ex file where this query might be generated:
|
||||
|
||||
⏺ Read(lib/last_fm.ex)
|
||||
⎿ Read 87 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Let me search for more specific patterns to find where this query is generated:
|
||||
|
||||
⏺ Search(pattern: "purchased_at IS NOT NULL")
|
||||
⎿ Found 2 files (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/music_library/collection.ex)
|
||||
⎿ Read 127 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Search(pattern: "coalesce.*cover_hash")
|
||||
⎿ Found 1 file (ctrl+o to expand)
|
||||
|
||||
⏺ Read(lib/music_library/scrobble_activity.ex)
|
||||
⎿ Read 472 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Found it! The query comes from the recent_activity/2 function in MusicLibrary.ScrobbleActivity. Let me examine the database
|
||||
indexes to understand the current optimization opportunities:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name, sql FROM sqlite_master WHERE type='index' AND tbl_name IN
|
||||
('scrobbled_tracks', 'records') ORDER BY tbl_name, name")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name", "sql"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name, sql FROM sqlite_master WHERE type='index' ORDER BY name")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name", "sql"],
|
||||
… +7 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Let me check what indexes exist specifically for the tables we're working with:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('scrobbled_tracks')")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('records')")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name FROM sqlite_master WHERE type='table' ORDER BY name")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Let me check if the scrobbled_tracks table actually exists and examine its structure:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: ".tables")
|
||||
⎿ Error: Failed to execute query: %Exqlite.Error{message: "near \".\": syntax error", statement: ".tables"}
|
||||
|
||||
⏺ Bash(mix ecto.migrate)
|
||||
⎿ 21:39:52.643 [info] Migrations already up
|
||||
|
||||
21:39:52.646 [debug] Exqlite: loading extension `/Users/cloud/github.com/cloud8421/music_library/_build/dev/lib/music_library
|
||||
… +5 lines (ctrl+o to expand)
|
||||
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name FROM sqlite_master WHERE type='table' AND name LIKE '%scrobbled%' OR name
|
||||
LIKE '%records%'")
|
||||
⎿ %Exqlite.Result{command: :execute, columns: ["name"], rows: [], num_rows: 0}
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "SELECT name FROM sqlite_master WHERE type='table'", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["name"],
|
||||
… +28 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Now let me check the existing indexes:
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('scrobbled_tracks')", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_list('records')", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seq", "name", "unique", "origin", "partial"],
|
||||
… +18 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_info('scrobbled_tracks_scrobbled_at_uts_title_index')", repo:
|
||||
"MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seqno", "cid", "name"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ tidewave - execute_sql_query (MCP)(query: "PRAGMA index_info('records_release_ids_index')", repo: "MusicLibrary.Repo")
|
||||
⎿ %Exqlite.Result{
|
||||
command: :execute,
|
||||
columns: ["seqno", "cid", "name"],
|
||||
… +3 lines (ctrl+o to expand)
|
||||
|
||||
⏺ Query Optimization Analysis
|
||||
|
||||
The query from MusicLibrary.ScrobbleActivity.recent_activity/2 has several optimization opportunities:
|
||||
|
||||
Current Issues:
|
||||
|
||||
1. Complex subqueries with json_each(): The query uses json_each() on release_ids arrays for both collection and wishlist joins,
|
||||
which is expensive for each row.
|
||||
2. Multiple LEFT JOINs with subqueries: Three separate subqueries are executed and joined, creating a cartesian product that's
|
||||
then filtered.
|
||||
3. JSON extraction in WHERE clauses: Using fragment("? ->> '$.musicbrainz_id'", t.album) requires JSON parsing for every row.
|
||||
4. Missing indexes: No composite indexes for the JSON-based join conditions.
|
||||
|
||||
Recommended Optimizations:
|
||||
|
||||
1. Add Strategic Indexes
|
||||
|
||||
-- Index for album musicbrainz_id JSON extraction (most critical)
|
||||
CREATE INDEX idx_scrobbled_tracks_album_mbid
|
||||
ON scrobbled_tracks((album ->> '$.musicbrainz_id'));
|
||||
|
||||
-- Composite index for the main query
|
||||
CREATE INDEX idx_scrobbled_tracks_scrobbled_at_album_mbid
|
||||
ON scrobbled_tracks(scrobbled_at_uts DESC, (album ->> '$.musicbrainz_id'));
|
||||
|
||||
2. Rewrite the Query Structure
|
||||
|
||||
Instead of multiple subqueries, use CTEs and reduce the number of joins:
|
||||
|
||||
def recent_activity_optimized(timezone, limit \\ 100) do
|
||||
# Single CTE for all releases with their collection/wishlist status
|
||||
releases_cte = """
|
||||
WITH releases AS (
|
||||
SELECT
|
||||
r.id as record_id,
|
||||
r.cover_hash,
|
||||
r.purchased_at IS NOT NULL as is_collected,
|
||||
sf.value as release_id
|
||||
FROM records r, json_each(r.release_ids) sf
|
||||
)
|
||||
"""
|
||||
|
||||
query = from t in Track,
|
||||
left_join: r in fragment("""
|
||||
(SELECT record_id, cover_hash, release_id,
|
||||
CASE WHEN is_collected THEN record_id END as collected_record_id,
|
||||
CASE WHEN NOT is_collected THEN record_id END as wishlisted_record_id
|
||||
FROM releases)
|
||||
"""),
|
||||
on: r.release_id == fragment("? ->> '$.musicbrainz_id'", t.album),
|
||||
left_join: ar in ArtistRecord,
|
||||
on: ar.record_id == r.record_id,
|
||||
order_by: [desc: t.scrobbled_at_uts],
|
||||
limit: ^limit,
|
||||
select: %{
|
||||
track: t,
|
||||
collected_record_id: r.collected_record_id,
|
||||
wishlisted_record_id: r.wishlisted_record_id,
|
||||
artist_id: ar.musicbrainz_id,
|
||||
cover_hash: r.cover_hash
|
||||
}
|
||||
end
|
||||
|
||||
3. Alternative: Materialized View Approach
|
||||
|
||||
Create a materialized view that pre-joins release data:
|
||||
|
||||
CREATE VIEW release_lookup AS
|
||||
SELECT
|
||||
r.id as record_id,
|
||||
r.cover_hash,
|
||||
r.purchased_at IS NOT NULL as is_collected,
|
||||
sf.value as release_id
|
||||
FROM records r, json_each(r.release_ids) sf;
|
||||
|
||||
CREATE INDEX idx_release_lookup_release_id ON release_lookup(release_id);
|
||||
|
||||
4. Denormalization Option
|
||||
|
||||
Consider adding a release_musicbrainz_id field directly to scrobbled_tracks to avoid JSON extraction:
|
||||
|
||||
# In migration
|
||||
add :album_musicbrainz_id, :string
|
||||
create index(:scrobbled_tracks, [:album_musicbrainz_id])
|
||||
create index(:scrobbled_tracks, [:scrobbled_at_uts, :album_musicbrainz_id])
|
||||
|
||||
⏺ Immediate Impact Recommendations:
|
||||
|
||||
High Impact, Low Risk:
|
||||
1. Add the JSON extraction index: CREATE INDEX idx_scrobbled_tracks_album_mbid ON scrobbled_tracks((album ->>
|
||||
'$.musicbrainz_id'))
|
||||
2. Add composite index: CREATE INDEX idx_scrobbled_tracks_scrobbled_at_album_mbid ON scrobbled_tracks(scrobbled_at_uts DESC,
|
||||
(album ->> '$.musicbrainz_id'))
|
||||
|
||||
Medium Impact, Medium Risk:
|
||||
3. Rewrite query to use a single CTE instead of multiple subqueries
|
||||
4. Create the release_lookup view for better join performance
|
||||
|
||||
High Impact, Higher Risk:
|
||||
5. Denormalize by adding album_musicbrainz_id column to eliminate JSON extraction entirely
|
||||
|
||||
The most critical optimization is adding indexes for the JSON extraction operations, which should provide immediate performance
|
||||
improvements with no code changes required.
|
||||
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,546 +0,0 @@
|
||||
# Online Store Quick Check System - Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines the implementation of a system to quickly check for the presence of wishlisted records on configurable online record stores. The system will add an expandable section to record details pages with links to online stores, generated from user-configurable templates.
|
||||
|
||||
## Requirements Analysis
|
||||
|
||||
### Core Features
|
||||
|
||||
- **Expandable section on record details**: Collapsed by default, shows links to online stores
|
||||
- **Configurable store templates**: User-managed templates stored in database
|
||||
- **Template variables**: Support for record artist and title as search parameters
|
||||
- **User management interface**: Dedicated section for managing store templates
|
||||
|
||||
### Example Use Case
|
||||
|
||||
Create a template for Amazon UK search: `https://www.amazon.co.uk/s?k={artist}+{title}+vinyl`
|
||||
When viewing a record by "Pink Floyd" with title "The Wall", generate: `https://www.amazon.co.uk/s?k=Pink+Floyd+The+Wall+vinyl`
|
||||
|
||||
## Database Schema Design
|
||||
|
||||
### Online Store Templates Table
|
||||
|
||||
Following the application's established patterns:
|
||||
|
||||
```elixir
|
||||
# Migration: create_online_store_templates.exs
|
||||
defmodule MusicLibrary.Repo.Migrations.CreateOnlineStoreTemplates do
|
||||
use Ecto.Migration
|
||||
|
||||
def change do
|
||||
create table(:online_store_templates, primary_key: false) do
|
||||
add :id, :binary_id, primary_key: true
|
||||
add :name, :string, null: false
|
||||
add :description, :text
|
||||
add :url_template, :text, null: false
|
||||
add :enabled, :boolean, default: true, null: false
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
create index(:online_store_templates, [:enabled])
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Schema Module
|
||||
|
||||
```elixir
|
||||
# lib/music_library/online_store_templates/online_store_template.ex
|
||||
defmodule MusicLibrary.OnlineStoreTemplates.OnlineStoreTemplate do
|
||||
use Ecto.Schema
|
||||
import Ecto.Changeset
|
||||
|
||||
@primary_key {:id, :binary_id, autogenerate: true}
|
||||
@foreign_key_type :binary_id
|
||||
|
||||
schema "online_store_templates" do
|
||||
field :name, :string
|
||||
field :description, :string
|
||||
field :url_template, :string
|
||||
field :enabled, :boolean, default: true
|
||||
|
||||
timestamps(type: :utc_datetime)
|
||||
end
|
||||
|
||||
@doc false
|
||||
def changeset(template, attrs) do
|
||||
template
|
||||
|> cast(attrs, [:name, :description, :url_template, :enabled])
|
||||
|> validate_required([:name, :url_template])
|
||||
|> validate_length(:name, min: 1, max: 100)
|
||||
|> validate_length(:url_template, min: 1, max: 500)
|
||||
|> validate_url_template()
|
||||
end
|
||||
|
||||
defp validate_url_template(changeset) do
|
||||
if template = get_field(changeset, :url_template) do
|
||||
case URI.parse(template) do
|
||||
%URI{scheme: scheme} when scheme in ["http", "https"] -> changeset
|
||||
_ -> add_error(changeset, :url_template, "must be a valid HTTP or HTTPS URL")
|
||||
end
|
||||
else
|
||||
changeset
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
### Context Module
|
||||
|
||||
```elixir
|
||||
# lib/music_library/online_store_templates.ex
|
||||
defmodule MusicLibrary.OnlineStoreTemplates do
|
||||
@moduledoc """
|
||||
The OnlineStoreTemplates context.
|
||||
"""
|
||||
|
||||
import Ecto.Query, warn: false
|
||||
alias MusicLibrary.Repo
|
||||
alias MusicLibrary.OnlineStoreTemplates.OnlineStoreTemplate
|
||||
|
||||
@doc """
|
||||
Returns the list of enabled online store templates ordered by name.
|
||||
"""
|
||||
def list_enabled_templates do
|
||||
OnlineStoreTemplate
|
||||
|> where([t], t.enabled == true)
|
||||
|> order_by([t], [asc: t.name])
|
||||
|> Repo.all()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Returns the list of all online store templates for management.
|
||||
"""
|
||||
def list_templates do
|
||||
OnlineStoreTemplate
|
||||
|> order_by([t], [asc: t.name])
|
||||
|> Repo.all()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Gets a single online store template.
|
||||
"""
|
||||
def get_template!(id), do: Repo.get!(OnlineStoreTemplate, id)
|
||||
|
||||
@doc """
|
||||
Creates an online store template.
|
||||
"""
|
||||
def create_template(attrs \\ %{}) do
|
||||
%OnlineStoreTemplate{}
|
||||
|> OnlineStoreTemplate.changeset(attrs)
|
||||
|> Repo.insert()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Updates an online store template.
|
||||
"""
|
||||
def update_template(%OnlineStoreTemplate{} = template, attrs) do
|
||||
template
|
||||
|> OnlineStoreTemplate.changeset(attrs)
|
||||
|> Repo.update()
|
||||
end
|
||||
|
||||
@doc """
|
||||
Deletes an online store template.
|
||||
"""
|
||||
def delete_template(%OnlineStoreTemplate{} = template) do
|
||||
Repo.delete(template)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Returns an `%Ecto.Changeset{}` for tracking online store template changes.
|
||||
"""
|
||||
def change_template(%OnlineStoreTemplate{} = template, attrs \\ %{}) do
|
||||
OnlineStoreTemplate.changeset(template, attrs)
|
||||
end
|
||||
|
||||
@doc """
|
||||
Generates a URL from a template by replacing variables with record data.
|
||||
"""
|
||||
def generate_url(template, record) do
|
||||
artists_string = record.artists |> Enum.map(& &1.name) |> Enum.join(" ")
|
||||
|
||||
template.url_template
|
||||
|> String.replace("{artist}", URI.encode(artists_string))
|
||||
|> String.replace("{title}", URI.encode(record.title))
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## User Interface Design
|
||||
|
||||
### Management Interface
|
||||
|
||||
Following the Scrobble Rules pattern for consistency:
|
||||
|
||||
#### Route Addition
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/router.ex
|
||||
# Add to the authenticated scope:
|
||||
live "/online-store-templates", OnlineStoreTemplateLive.Index, :index
|
||||
live "/online-store-templates/new", OnlineStoreTemplateLive.Index, :new
|
||||
live "/online-store-templates/:id/edit", OnlineStoreTemplateLive.Index, :edit
|
||||
```
|
||||
|
||||
#### Navigation Integration
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/components/layouts/app.html.heex
|
||||
# Add to the "More" dropdown menu:
|
||||
<.dropdown_link href={~p"/online-store-templates"}>
|
||||
Online Store Templates
|
||||
</.dropdown_link>
|
||||
```
|
||||
|
||||
#### LiveView Index Page
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/live/online_store_template_live/index.ex
|
||||
defmodule MusicLibraryWeb.OnlineStoreTemplateLive.Index do
|
||||
use MusicLibraryWeb, :live_view
|
||||
|
||||
alias MusicLibrary.OnlineStoreTemplates
|
||||
alias MusicLibrary.OnlineStoreTemplates.OnlineStoreTemplate
|
||||
|
||||
@impl true
|
||||
def mount(_params, _session, socket) do
|
||||
{:ok, stream(socket, :templates, OnlineStoreTemplates.list_templates())}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_params(params, _url, socket) do
|
||||
{:noreply, apply_action(socket, socket.assigns.live_action, params)}
|
||||
end
|
||||
|
||||
defp apply_action(socket, :edit, %{"id" => id}) do
|
||||
socket
|
||||
|> assign(:page_title, "Edit Online Store Template")
|
||||
|> assign(:template, OnlineStoreTemplates.get_template!(id))
|
||||
end
|
||||
|
||||
defp apply_action(socket, :new, _params) do
|
||||
socket
|
||||
|> assign(:page_title, "New Online Store Template")
|
||||
|> assign(:template, %OnlineStoreTemplate{})
|
||||
end
|
||||
|
||||
defp apply_action(socket, :index, _params) do
|
||||
socket
|
||||
|> assign(:page_title, "Online Store Templates")
|
||||
|> assign(:template, nil)
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_info({MusicLibraryWeb.OnlineStoreTemplateLive.FormComponent, {:saved, template}}, socket) do
|
||||
{:noreply, stream_insert(socket, :templates, template)}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_event("delete", %{"id" => id}, socket) do
|
||||
template = OnlineStoreTemplates.get_template!(id)
|
||||
{:ok, _} = OnlineStoreTemplates.delete_template(template)
|
||||
|
||||
{:noreply, stream_delete(socket, :templates, template)}
|
||||
end
|
||||
|
||||
@impl true
|
||||
def handle_event("toggle-enabled", %{"id" => id}, socket) do
|
||||
template = OnlineStoreTemplates.get_template!(id)
|
||||
{:ok, updated_template} = OnlineStoreTemplates.update_template(template, %{enabled: !template.enabled})
|
||||
|
||||
{:noreply, stream_insert(socket, :templates, updated_template)}
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
#### Template Design
|
||||
|
||||
```heex
|
||||
<!-- lib/music_library_web/live/online_store_template_live/index.html.heex -->
|
||||
<div class="px-4 py-6 sm:px-6 lg:px-8">
|
||||
<div class="sm:flex sm:items-center">
|
||||
<div class="sm:flex-auto">
|
||||
<h1 class="text-base font-semibold text-gray-900 dark:text-white">
|
||||
Online Store Templates
|
||||
</h1>
|
||||
<p class="mt-2 text-sm text-gray-700 dark:text-gray-300">
|
||||
Manage templates for generating links to online record stores.
|
||||
</p>
|
||||
</div>
|
||||
<div class="mt-4 sm:ml-16 sm:mt-0 sm:flex-none">
|
||||
<.button patch={~p"/online-store-templates/new"}>
|
||||
Add Template
|
||||
</.button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="mt-8 flow-root">
|
||||
<div class="-mx-4 -my-2 overflow-x-auto sm:-mx-6 lg:-mx-8">
|
||||
<div class="inline-block min-w-full py-2 align-middle sm:px-6 lg:px-8">
|
||||
<ul role="list" class="divide-y divide-gray-200 dark:divide-gray-800" phx-update="stream" id="templates">
|
||||
<li
|
||||
:for={{id, template} <- @streams.templates}
|
||||
id={id}
|
||||
class="flex items-center justify-between gap-x-6 py-5"
|
||||
>
|
||||
<div class="min-w-0">
|
||||
<div class="flex items-start gap-x-3">
|
||||
<p class="text-sm font-semibold text-gray-900 dark:text-white">
|
||||
<%= template.name %>
|
||||
</p>
|
||||
<.badge :if={template.enabled} color="success" size="sm">
|
||||
Enabled
|
||||
</.badge>
|
||||
<.badge :if={!template.enabled} color="warning" size="sm">
|
||||
Disabled
|
||||
</.badge>
|
||||
</div>
|
||||
<div class="mt-1 flex items-center gap-x-2 text-xs text-gray-500 dark:text-gray-400">
|
||||
<p class="truncate"><%= template.url_template %></p>
|
||||
</div>
|
||||
<p :if={template.description} class="mt-1 text-xs text-gray-500 dark:text-gray-400">
|
||||
<%= template.description %>
|
||||
</p>
|
||||
</div>
|
||||
<div class="flex flex-none items-center gap-x-4">
|
||||
<.button
|
||||
phx-click="toggle-enabled"
|
||||
phx-value-id={template.id}
|
||||
color={if template.enabled, do: "warning", else: "success"}
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
>
|
||||
<%= if template.enabled, do: "Disable", else: "Enable" %>
|
||||
</.button>
|
||||
<.button
|
||||
patch={~p"/online-store-templates/#{template}/edit"}
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
>
|
||||
Edit
|
||||
</.button>
|
||||
<.button
|
||||
phx-click={JS.push("delete", value: %{id: template.id})}
|
||||
data-confirm="Are you sure?"
|
||||
color="danger"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
>
|
||||
Delete
|
||||
</.button>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<.modal :if={@live_action in [:new, :edit]} id="template-modal" show on_cancel={JS.patch(~p"/online-store-templates")}>
|
||||
<.live_component
|
||||
module={MusicLibraryWeb.OnlineStoreTemplateLive.FormComponent}
|
||||
id={@template.id || :new}
|
||||
title={@page_title}
|
||||
action={@live_action}
|
||||
template={@template}
|
||||
patch={~p"/online-store-templates"}
|
||||
/>
|
||||
</.modal>
|
||||
```
|
||||
|
||||
### Record Details Integration
|
||||
|
||||
Add to the wishlist show page template:
|
||||
|
||||
```heex
|
||||
<!-- lib/music_library_web/live/wishlist_live/show.html.heex -->
|
||||
<!-- Add this section after the existing record details -->
|
||||
|
||||
<div class="mt-8">
|
||||
<details class="px-4 text-zinc-700 hover:text-zinc-500 dark:text-zinc-400 dark:hover:text-zinc-300">
|
||||
<summary class="text-sm font-medium cursor-pointer">
|
||||
Check Online Stores
|
||||
</summary>
|
||||
<div class="mt-4 space-y-2">
|
||||
<div
|
||||
:for={template <- @online_store_templates}
|
||||
class="flex items-center justify-between py-2 px-3 bg-gray-50 dark:bg-gray-800 rounded-lg"
|
||||
>
|
||||
<span class="text-sm font-medium text-gray-900 dark:text-white">
|
||||
<%= template.name %>
|
||||
</span>
|
||||
<.button
|
||||
href={OnlineStoreTemplates.generate_url(template, @record)}
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
class="ml-2"
|
||||
>
|
||||
<.icon name="hero-arrow-top-right-on-square" class="h-4 w-4" />
|
||||
Search
|
||||
</.button>
|
||||
</div>
|
||||
</div>
|
||||
</details>
|
||||
</div>
|
||||
```
|
||||
|
||||
### LiveView Updates
|
||||
|
||||
Update the wishlist show LiveView to load templates:
|
||||
|
||||
```elixir
|
||||
# lib/music_library_web/live/wishlist_live/show.ex
|
||||
# Add to the mount function:
|
||||
online_store_templates = MusicLibrary.OnlineStoreTemplates.list_enabled_templates()
|
||||
|
||||
socket =
|
||||
socket
|
||||
|> assign(:record, record)
|
||||
|> assign(:online_store_templates, online_store_templates)
|
||||
|> assign(:page_title, record.title)
|
||||
```
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### Phase 1: Database Foundation
|
||||
|
||||
1. Create migration for `online_store_templates` table
|
||||
2. Create schema module with validation
|
||||
3. Create context module with CRUD operations
|
||||
4. Run migration and test basic functionality
|
||||
|
||||
### Phase 2: Management Interface
|
||||
|
||||
1. Create LiveView modules for template management
|
||||
2. Create form component for template editing
|
||||
3. Add routes to router
|
||||
4. Add navigation link to main layout
|
||||
5. Style templates using existing design patterns
|
||||
|
||||
### Phase 3: Record Details Integration
|
||||
|
||||
1. Update wishlist show LiveView to load templates
|
||||
2. Add expandable section to show template
|
||||
3. Implement URL generation logic
|
||||
4. Test template variable replacement
|
||||
|
||||
### Phase 4: Testing & Refinement
|
||||
|
||||
1. Add comprehensive tests for context functions
|
||||
2. Add LiveView tests for management interface
|
||||
3. Test URL generation with various record data
|
||||
4. Validate template functionality end-to-end
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### URL Template Validation
|
||||
|
||||
- Validate URL templates contain only HTTP/HTTPS schemes
|
||||
- Sanitize user input in templates
|
||||
- Encode record data properly when generating URLs
|
||||
|
||||
### Access Control
|
||||
|
||||
- Templates management requires authentication
|
||||
- Template visibility follows existing permission patterns
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- Context module functions (CRUD operations)
|
||||
- URL generation with various record data
|
||||
- Template validation logic
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- LiveView template management workflow
|
||||
- Template display on record details pages
|
||||
- URL generation end-to-end
|
||||
|
||||
### Example Test Cases
|
||||
|
||||
```elixir
|
||||
defmodule MusicLibrary.OnlineStoreTemplatesTest do
|
||||
use MusicLibrary.DataCase
|
||||
|
||||
alias MusicLibrary.OnlineStoreTemplates
|
||||
|
||||
describe "generate_url/2" do
|
||||
test "replaces artist and title variables correctly" do
|
||||
template = %OnlineStoreTemplate{
|
||||
url_template: "https://example.com/search?q={artist}+{title}"
|
||||
}
|
||||
|
||||
record = %Record{
|
||||
title: "Dark Side of the Moon",
|
||||
artists: [%Artist{name: "Pink Floyd"}]
|
||||
}
|
||||
|
||||
assert OnlineStoreTemplates.generate_url(template, record) ==
|
||||
"https://example.com/search?q=Pink+Floyd+Dark+Side+of+the+Moon"
|
||||
end
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
## Database Migration Commands
|
||||
|
||||
```bash
|
||||
# Create migration
|
||||
mix ecto.gen.migration create_online_store_templates
|
||||
|
||||
# Run migration
|
||||
mix ecto.migrate
|
||||
|
||||
# Rollback if needed
|
||||
mix ecto.rollback
|
||||
```
|
||||
|
||||
## Default Template Examples
|
||||
|
||||
Consider seeding the database with common store templates:
|
||||
|
||||
```elixir
|
||||
# priv/repo/seeds.exs additions
|
||||
alias MusicLibrary.OnlineStoreTemplates
|
||||
|
||||
templates = [
|
||||
%{
|
||||
name: "Amazon UK",
|
||||
description: "Search Amazon UK for vinyl records",
|
||||
url_template: "https://www.amazon.co.uk/s?k={artist}+{title}+vinyl&i=popular"
|
||||
},
|
||||
%{
|
||||
name: "Discogs",
|
||||
description: "Search Discogs marketplace",
|
||||
url_template: "https://www.discogs.com/search/?q={artist}+{title}&type=all"
|
||||
},
|
||||
%{
|
||||
name: "eBay",
|
||||
description: "Search eBay for records",
|
||||
url_template: "https://www.ebay.co.uk/sch/i.html?_nkw={artist}+{title}+vinyl"
|
||||
}
|
||||
]
|
||||
|
||||
Enum.each(templates, &OnlineStoreTemplates.create_template/1)
|
||||
```
|
||||
|
||||
## Conclusion
|
||||
|
||||
This implementation provides a clean, maintainable system for checking online stores that:
|
||||
|
||||
- Follows existing application patterns and conventions
|
||||
- Provides a user-friendly management interface
|
||||
- Integrates seamlessly with the existing record details pages
|
||||
- Supports extensibility for future enhancements
|
||||
- Maintains security and validation standards
|
||||
|
||||
The system is designed to be conservative, focusing on core functionality while following the established UI and database patterns throughout the application.
|
||||
|
||||
@@ -1,279 +0,0 @@
|
||||
# Scrobbled Tracks Data Transformation Rules - Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines the implementation of functionality to manage arbitrary rules for fixing data in the `scrobbled_tracks` table. The system will support two types of transformations:
|
||||
|
||||
1. **Album Rule**: Update the album MusicBrainz ID of all tracks belonging to a specific album (identified by album title)
|
||||
2. **Artist Rule**: Update the artist MusicBrainz ID of all tracks belonging to a specific artist (identified by artist name)
|
||||
|
||||
## Current State Analysis
|
||||
|
||||
### Existing Database Schema
|
||||
|
||||
- `scrobbled_tracks` table exists with the following structure:
|
||||
- `scrobbled_at_uts` (integer, primary key)
|
||||
- `musicbrainz_id` (string) - track MusicBrainz ID
|
||||
- `title` (string) - track title
|
||||
- `cover_url` (string)
|
||||
- `scrobbled_at_label` (string)
|
||||
- `artist` (map) - embedded artist data with `musicbrainz_id` and `name`
|
||||
- `album` (map) - embedded album data with `musicbrainz_id` and `title`
|
||||
- `last_fm_data` (map) - raw Last.fm API response
|
||||
|
||||
### Existing Schemas
|
||||
|
||||
- `LastFm.Track` - Schema for scrobbled tracks
|
||||
- `LastFm.Artist` - Embedded artist schema
|
||||
- `LastFm.Album` - Embedded album schema
|
||||
|
||||
### Navigation Structure
|
||||
|
||||
- Main navigation has Stats, Collection, Wishlist sections
|
||||
- "More" dropdown contains development tools and utilities
|
||||
- Uses Phoenix LiveView for interactive components
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### 1. Database Migration
|
||||
|
||||
**File**: `priv/repo/migrations/YYYYMMDDHHMMSS_create_scrobble_rules.exs`
|
||||
|
||||
Create `scrobble_rules` table with:
|
||||
|
||||
- `id` (primary key)
|
||||
- `type` (string) - "album" or "artist"
|
||||
- `match_value` (string) - album title or artist name to match
|
||||
- `target_musicbrainz_id` (string) - MusicBrainz ID to set
|
||||
- `enabled` (boolean) - whether rule is active
|
||||
- `description` (text) - optional description for the rule
|
||||
- `inserted_at` (timestamp)
|
||||
- `updated_at` (timestamp)
|
||||
|
||||
Indexes:
|
||||
|
||||
- `type, match_value` (compound index for efficient matching)
|
||||
- `enabled` (for filtering active rules)
|
||||
|
||||
### 2. Schema Module
|
||||
|
||||
**File**: `lib/music_library/scrobble_rules/scrobble_rule.ex`
|
||||
|
||||
Create Ecto schema with:
|
||||
|
||||
- Field definitions matching migration
|
||||
- Validations:
|
||||
- `type` must be "album" or "artist"
|
||||
- `match_value` required and non-empty
|
||||
- `target_musicbrainz_id` required and non-empty
|
||||
- `enabled` defaults to true
|
||||
- Changeset functions for create/update operations
|
||||
|
||||
### 3. Context Module
|
||||
|
||||
**File**: `lib/music_library/scrobble_rules.ex`
|
||||
|
||||
Implement context functions:
|
||||
|
||||
- `list_scrobble_rules(opts \\ [])` - list all rules with optional filtering
|
||||
- `get_scrobble_rule!(id)` - get specific rule by ID
|
||||
- `create_scrobble_rule(attrs)` - create new rule
|
||||
- `update_scrobble_rule(rule, attrs)` - update existing rule
|
||||
- `delete_scrobble_rule(rule)` - delete rule
|
||||
- `change_scrobble_rule(rule, attrs \\ %{})` - create changeset
|
||||
- `list_enabled_rules()` - get only enabled rules for worker
|
||||
- `apply_album_rule(rule)` - apply album transformation
|
||||
- `apply_artist_rule(rule)` - apply artist transformation
|
||||
- `apply_all_rules()` - apply all enabled rules
|
||||
|
||||
### 4. Navigation Update
|
||||
|
||||
**File**: `lib/music_library_web/components/layouts/app.html.heex`
|
||||
|
||||
Add new navigation link in the "More" dropdown:
|
||||
|
||||
- Icon: `hero-adjustments-horizontal`
|
||||
- Label: "Scrobble Rules"
|
||||
- Route: `/scrobble-rules`
|
||||
- Position: After "Oban" link, before separator
|
||||
|
||||
### 5. Router Configuration
|
||||
|
||||
**File**: `lib/music_library_web/router.ex`
|
||||
|
||||
Add new LiveView routes in the authenticated scope:
|
||||
|
||||
- `/scrobble-rules` - index page
|
||||
- `/scrobble-rules/new` - create new rule
|
||||
- `/scrobble-rules/:id/edit` - edit existing rule
|
||||
|
||||
### 6. LiveView Implementation
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobble_rules_live/index.ex`
|
||||
|
||||
Implement LiveView with:
|
||||
|
||||
- List view showing all rules with status indicators
|
||||
- Create/Edit forms with validation
|
||||
- Delete confirmation
|
||||
- Filter controls (by type, enabled/disabled)
|
||||
- Manual rule application triggers
|
||||
- Real-time updates via PubSub
|
||||
|
||||
**Template**: `lib/music_library_web/live/scrobble_rules_live/index.html.heex`
|
||||
|
||||
- Table layout with rule details
|
||||
- Action buttons (Edit, Delete, Apply)
|
||||
- Form components for create/edit
|
||||
- Status indicators for enabled/disabled rules
|
||||
|
||||
### 7. Form Components
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobble_rules_live/form_component.ex`
|
||||
|
||||
Create reusable form component:
|
||||
|
||||
- Type selection (album/artist)
|
||||
- Match value input with validation
|
||||
- Target MusicBrainz ID input
|
||||
- Description textarea
|
||||
- Enable/disable toggle
|
||||
- Save/Cancel actions
|
||||
|
||||
Where possible, use the components available at <https://docs.fluxonui.com/1.1.4/overview.html>.
|
||||
|
||||
### 8. Periodic Worker
|
||||
|
||||
**File**: `lib/music_library/scrobble_rules/worker.ex`
|
||||
|
||||
Implement Oban worker:
|
||||
|
||||
- Scheduled to run periodically (e.g., every 30 minutes)
|
||||
- Fetches all enabled rules
|
||||
- Applies each rule in sequence
|
||||
- Logs application results
|
||||
- Handles errors gracefully
|
||||
- Tracks last application time
|
||||
|
||||
### 9. Testing Strategy
|
||||
|
||||
#### Unit Tests
|
||||
|
||||
**File**: `test/music_library/scrobble_rules_test.exs`
|
||||
|
||||
- Test all context functions
|
||||
- Test rule application logic
|
||||
- Test edge cases and error handling
|
||||
|
||||
**File**: `test/music_library/scrobble_rules/scrobble_rule_test.exs`
|
||||
|
||||
- Test schema validations
|
||||
- Test changeset functions
|
||||
- Test constraint validations
|
||||
|
||||
#### Integration Tests
|
||||
|
||||
**File**: `test/music_library_web/live/scrobble_rules_live_test.exs`
|
||||
|
||||
- Test LiveView interactions
|
||||
- Test form submissions
|
||||
- Test real-time updates
|
||||
- Test navigation and routing
|
||||
|
||||
#### Worker Tests
|
||||
|
||||
**File**: `test/music_library/scrobble_rules/worker_test.exs`
|
||||
|
||||
- Test job execution
|
||||
- Test rule application
|
||||
- Test error handling
|
||||
- Test scheduling
|
||||
|
||||
### 11. Additional Considerations
|
||||
|
||||
#### Data Integrity
|
||||
|
||||
- Ensure atomic operations when applying rules
|
||||
- Validate MusicBrainz IDs before applying
|
||||
- Log all transformations for audit trail
|
||||
|
||||
#### Performance
|
||||
|
||||
- Index optimization for rule matching
|
||||
- Batch processing for large datasets
|
||||
- Rate limiting for external API calls
|
||||
|
||||
#### User Experience
|
||||
|
||||
- Clear feedback on rule application
|
||||
- Preview mode to show what would change
|
||||
- Bulk operations for managing multiple rules
|
||||
- Export/import functionality for rule sets
|
||||
|
||||
#### Security
|
||||
|
||||
- Input validation and sanitization
|
||||
- Rate limiting on rule creation
|
||||
- Audit logging for all changes
|
||||
|
||||
## Database Schema Summary
|
||||
|
||||
```sql
|
||||
-- scrobble_rules table
|
||||
CREATE TABLE scrobble_rules (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
type TEXT NOT NULL CHECK (type IN ('album', 'artist')),
|
||||
match_value TEXT NOT NULL,
|
||||
target_musicbrainz_id TEXT NOT NULL,
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
description TEXT,
|
||||
inserted_at TIMESTAMP NOT NULL,
|
||||
updated_at TIMESTAMP NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX idx_scrobble_rules_type_match ON scrobble_rules(type, match_value);
|
||||
CREATE INDEX idx_scrobble_rules_enabled ON scrobble_rules(enabled);
|
||||
```
|
||||
|
||||
## File Structure Summary
|
||||
|
||||
```
|
||||
lib/music_library/
|
||||
├── scrobble_rules/
|
||||
│ ├── scrobble_rule.ex # Schema
|
||||
│ └── worker.ex # Oban worker
|
||||
└── scrobble_rules.ex # Context
|
||||
|
||||
lib/music_library_web/
|
||||
└── live/
|
||||
└── scrobble_rules_live/
|
||||
├── index.ex # LiveView
|
||||
├── index.html.heex # Template
|
||||
└── form_component.ex # Form component
|
||||
|
||||
priv/repo/migrations/
|
||||
└── YYYYMMDDHHMMSS_create_scrobble_rules.exs
|
||||
|
||||
test/
|
||||
├── music_library/
|
||||
│ ├── scrobble_rules_test.exs
|
||||
│ └── scrobble_rules/
|
||||
│ ├── scrobble_rule_test.exs
|
||||
│ └── worker_test.exs
|
||||
└── music_library_web/
|
||||
└── live/
|
||||
└── scrobble_rules_live_test.exs
|
||||
```
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. Database migration and schema
|
||||
2. Context module with basic CRUD operations
|
||||
3. LiveView interface for rule management
|
||||
4. Navigation and routing updates
|
||||
5. Periodic worker implementation
|
||||
6. Comprehensive testing
|
||||
7. Documentation and deployment
|
||||
|
||||
This plan ensures a robust, maintainable solution that integrates seamlessly with the existing Music Library application architecture.
|
||||
|
||||
@@ -1,278 +0,0 @@
|
||||
# Scrobbled Tracks Management UI Plan
|
||||
|
||||
## Overview
|
||||
|
||||
Build a complete UI for managing scrobbled tracks with index, edit, and delete functionality. The interface will be accessible through the "More" menu in the top navigation and will follow existing patterns from collection and wishlist management.
|
||||
|
||||
## 1. Database & Context Layer
|
||||
|
||||
### Existing Schema
|
||||
|
||||
The `LastFm.Track` schema (`lib/last_fm/track.ex`) already exists with:
|
||||
|
||||
- Primary key: `scrobbled_at_uts` (Unix timestamp)
|
||||
- Fields: `musicbrainz_id`, `title`, `cover_url`, `scrobbled_at_label`
|
||||
- Embedded schemas: `artist` and `album`
|
||||
- Raw `last_fm_data` map
|
||||
|
||||
### Context Functions Needed
|
||||
|
||||
Add to `LastFm` module (`lib/last_fm.ex`):
|
||||
|
||||
- `list_tracks/1` - Paginated track listing with search support
|
||||
- `get_track!/1` - Individual track retrieval by `scrobbled_at_uts`
|
||||
- `update_track/2` - Track metadata updates
|
||||
- `delete_track/1` - Track deletion
|
||||
- `search_tracks_count/1` - Count for pagination totals
|
||||
|
||||
## 2. LiveView Components
|
||||
|
||||
### Index LiveView
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobbled_tracks_live/index.ex`
|
||||
|
||||
**Pattern**: Follow `CollectionLive.Index` structure
|
||||
|
||||
**Features**:
|
||||
|
||||
- Paginated list (200 items per page)
|
||||
- Search functionality by track title/artist name
|
||||
- Stream-based updates for performance (`phx-update="stream"`)
|
||||
- Delete confirmation modals
|
||||
- Edit links to individual track editing
|
||||
|
||||
**Default Parameters**:
|
||||
|
||||
```elixir
|
||||
@default_tracks_list_params %{
|
||||
query: "",
|
||||
page: 1,
|
||||
page_size: 200,
|
||||
order: :scrobbled_at
|
||||
}
|
||||
```
|
||||
|
||||
### Edit Form Component
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobbled_tracks_live/form_component.ex`
|
||||
|
||||
**Pattern**: Follow existing form components (e.g., `ScrobbleRulesLive.FormComponent`)
|
||||
|
||||
**Editable Fields**:
|
||||
|
||||
- Track title
|
||||
- Artist name
|
||||
- Album title
|
||||
- Scrobbled date/time
|
||||
- Cover URL (optional)
|
||||
|
||||
## 3. Templates & UI Design
|
||||
|
||||
### Index Template
|
||||
|
||||
**File**: `lib/music_library_web/live/scrobbled_tracks_live/index.html.heex`
|
||||
|
||||
**Layout**: Responsive card/list layout similar to collection
|
||||
|
||||
**Components**:
|
||||
|
||||
- Search bar at top with debounced input
|
||||
- Track cards displaying:
|
||||
- Cover image (with fallback for missing covers)
|
||||
- Track title
|
||||
- Artist name
|
||||
- Album title
|
||||
- Scrobbled date/time
|
||||
- Edit/Delete action buttons
|
||||
- Pagination component at bottom using `MusicLibraryWeb.Components.Pagination`
|
||||
- Empty state message when no tracks found
|
||||
|
||||
### Form Component Template
|
||||
|
||||
**Features**:
|
||||
|
||||
- Form inputs for editable track metadata
|
||||
- Client-side and server-side validation
|
||||
- Cancel/Save buttons with proper form handling
|
||||
- Error display for validation failures
|
||||
|
||||
## 4. Routing & Navigation
|
||||
|
||||
### Router Updates
|
||||
|
||||
**File**: `lib/music_library_web/router.ex`
|
||||
|
||||
Add to the logged-in scope around line 65:
|
||||
|
||||
```elixir
|
||||
live "/scrobbled-tracks", ScrobbledTracksLive.Index, :index
|
||||
live "/scrobbled-tracks/:scrobbled_at_uts/edit", ScrobbledTracksLive.Index, :edit
|
||||
```
|
||||
|
||||
### Navigation Integration
|
||||
|
||||
**File**: `lib/music_library_web/components/layouts/app.html.heex`
|
||||
|
||||
Add to the "More" dropdown menu (after line 58):
|
||||
|
||||
```elixir
|
||||
<.dropdown_link href={~p"/scrobbled-tracks"}>
|
||||
<.icon name="hero-musical-note" class="h-4 w-4 mr-2" aria-hidden="true" data-slot="icon" />
|
||||
{gettext("Scrobbled Tracks")}
|
||||
</.dropdown_link>
|
||||
```
|
||||
|
||||
## 5. Implementation Details
|
||||
|
||||
### Pagination Integration
|
||||
|
||||
- **Pattern**: Follow `CollectionLive.Index` pagination approach
|
||||
- **Component**: Reuse `MusicLibraryWeb.Components.Pagination`
|
||||
- **Parameters**: Support `query`, `page`, `page_size`, and `order`
|
||||
- **URL Structure**: `/scrobbled-tracks?page=2&query=artist&page_size=50`
|
||||
|
||||
### Stream Implementation
|
||||
|
||||
- Use `stream(:tracks, tracks)` for efficient rendering of large track lists
|
||||
- Implement `stream_delete(socket, :tracks, track)` for track deletion
|
||||
- Handle pagination with `reset: true` for filtered results
|
||||
- Use unique DOM IDs based on `scrobbled_at_uts`
|
||||
|
||||
### Search Functionality
|
||||
|
||||
- Search across track title, artist name, and album title
|
||||
- Use database LIKE queries for basic string matching
|
||||
- Debounced search input with LiveView `phx-change` events
|
||||
- Reset to page 1 when search query changes
|
||||
|
||||
### Delete Confirmation
|
||||
|
||||
- Use modal overlay for delete confirmation
|
||||
- Show track details (title, artist, date) in confirmation modal
|
||||
- Implement optimistic UI updates with `stream_delete`
|
||||
- Handle deletion errors gracefully
|
||||
|
||||
### Error Handling
|
||||
|
||||
- Validate track existence before edit operations
|
||||
- Handle database constraint violations
|
||||
- Display user-friendly error messages
|
||||
- Graceful fallbacks for missing track data
|
||||
|
||||
## 6. File Structure
|
||||
|
||||
```
|
||||
lib/music_library_web/live/scrobbled_tracks_live/
|
||||
├── index.ex # Main LiveView controller
|
||||
├── index.html.heex # Index template with track listing
|
||||
└── form_component.ex # Edit form component
|
||||
|
||||
docs/plans/
|
||||
└── scrobbled-tracks-crud.md # This plan document
|
||||
```
|
||||
|
||||
## 7. Implementation Steps
|
||||
|
||||
1. **Create context functions** in `MusicLibrary.ScrobbleActivity` module for CRUD operations
|
||||
2. **Implement pagination logic** following existing `CollectionLive.Index` patterns
|
||||
3. **Create LiveView** with stream-based track listing and search
|
||||
4. **Add search functionality** with query parameter handling and debouncing
|
||||
5. **Implement edit form** with validation and error handling
|
||||
6. **Add delete functionality** with confirmation modal
|
||||
7. **Update navigation** to include "Scrobbled Tracks" link in More dropdown
|
||||
8. **Add routing** for index and edit actions
|
||||
9. **Style components** using Tailwind CSS and Fluxon UI components
|
||||
10. **Add comprehensive tests** following existing test patterns
|
||||
|
||||
## 8. Technical Considerations
|
||||
|
||||
### Primary Key Handling
|
||||
|
||||
- Use `scrobbled_at_uts` as unique identifier in routes and database queries
|
||||
- Handle potential timestamp collisions gracefully
|
||||
- Consider compound keys if needed for uniqueness
|
||||
|
||||
### Performance Optimization
|
||||
|
||||
- Use LiveView streams for efficient rendering of large lists
|
||||
- Implement database indexes on searchable fields
|
||||
- Paginate results to avoid memory issues
|
||||
- Use debounced search to reduce database load
|
||||
|
||||
### Data Integrity
|
||||
|
||||
- Validate required fields (track title, artist name)
|
||||
- Handle embedded schema updates (artist, album data)
|
||||
- Preserve Last.fm API data integrity
|
||||
- Consider soft deletes vs hard deletes
|
||||
|
||||
### User Experience
|
||||
|
||||
- Consistent styling with existing collection/wishlist pages
|
||||
- Responsive design for mobile/tablet viewing
|
||||
- Loading states during pagination and search
|
||||
- Clear feedback for user actions (edit, delete)
|
||||
- Accessible keyboard navigation
|
||||
|
||||
### Security & Permissions
|
||||
|
||||
- Ensure proper authentication via `:logged_in` pipeline
|
||||
- Validate user permissions for track modifications
|
||||
- Sanitize input data to prevent XSS
|
||||
- Use Phoenix CSRF protection
|
||||
|
||||
## 9. Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
- Context function tests for CRUD operations
|
||||
- LiveView event handling tests
|
||||
- Form validation tests
|
||||
- Pagination logic tests
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- Full user flow tests (search, edit, delete)
|
||||
- Navigation integration tests
|
||||
- Error handling scenarios
|
||||
- Responsive layout tests
|
||||
|
||||
### Test Files
|
||||
|
||||
```
|
||||
test/music_library_web/live/scrobbled_tracks_live_test.exs
|
||||
test/last_fm_test.exs (extend existing)
|
||||
test/support/fixtures/scrobbled_tracks_fixtures.ex
|
||||
```
|
||||
|
||||
## 10. Localization
|
||||
|
||||
Add gettext entries for:
|
||||
|
||||
- Page titles and headings
|
||||
- Form labels and placeholders
|
||||
- Error messages
|
||||
- Action button text
|
||||
- Empty state messages
|
||||
- Navigation menu item
|
||||
|
||||
## 11. Future Enhancements
|
||||
|
||||
### Potential Extensions
|
||||
|
||||
- Bulk operations (delete multiple tracks)
|
||||
- Export functionality (CSV, JSON)
|
||||
- Advanced filtering (date ranges, specific artists)
|
||||
- Scrobble statistics and analytics
|
||||
- Integration with music streaming services
|
||||
- Track matching with collection records
|
||||
|
||||
### Performance Improvements
|
||||
|
||||
- Virtual scrolling for very large lists
|
||||
- Background track synchronization
|
||||
- Caching for frequently accessed data
|
||||
- Search index optimization
|
||||
|
||||
This plan ensures consistency with existing codebase patterns while providing a complete, user-friendly interface for scrobbled tracks management.
|
||||
|
||||
@@ -1,345 +0,0 @@
|
||||
# Universal Search Modal Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
This plan outlines the implementation of a universal search modal that provides unified search across Records (Collection), Records (Wishlist), and Artists. The modal will be accessible via a search icon in the top navigation bar and a keyboard shortcut.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Core Features
|
||||
- Modal accessible via:
|
||||
- Search icon in top navigation bar
|
||||
- Keyboard shortcut (Ctrl/Cmd + K)
|
||||
- Search across three entity types:
|
||||
1. Records in Collection
|
||||
2. Records in Wishlist
|
||||
3. Artists
|
||||
- Real-time search as user types
|
||||
- Keyboard navigation support
|
||||
- Mobile-responsive design
|
||||
|
||||
### Search Functionality
|
||||
- Leverage existing FTS5 search infrastructure
|
||||
- Support existing search syntax (artist:, album:, genre:, etc.)
|
||||
- Show search results grouped by entity type
|
||||
- Limit results per category (e.g., 5 per type)
|
||||
- Click-to-navigate to full results or individual records
|
||||
|
||||
## Technical Architecture
|
||||
|
||||
### 1. Modal Component Structure
|
||||
|
||||
**New Components:**
|
||||
- `UniversalSearchModal` - Main modal component
|
||||
- `SearchResultGroup` - Groups results by type (Collection, Wishlist, Artists)
|
||||
- `SearchResultItem` - Individual result display
|
||||
- `SearchInput` - Enhanced search input with keyboard shortcuts
|
||||
|
||||
**Component Location:**
|
||||
```
|
||||
lib/music_library_web/components/
|
||||
├── search_components.ex # New universal search components
|
||||
└── layouts/app.html.heex # Add search icon to navigation
|
||||
```
|
||||
|
||||
### 2. LiveView Integration
|
||||
|
||||
**New LiveView Module:**
|
||||
```
|
||||
lib/music_library_web/live/universal_search_live/
|
||||
├── index.ex # Main search LiveView
|
||||
└── index.html.heex # Search modal template
|
||||
```
|
||||
|
||||
**Integration Points:**
|
||||
- Mount as child LiveView in main layout
|
||||
- Handle modal open/close state
|
||||
- Manage search query and results
|
||||
- Handle keyboard navigation
|
||||
|
||||
### 3. Backend Search Implementation
|
||||
|
||||
**New Context Functions:**
|
||||
```elixir
|
||||
# In lib/music_library/search.ex (new module)
|
||||
def universal_search(query, opts \\ [])
|
||||
def search_collection(query, limit \\ 5)
|
||||
def search_wishlist(query, limit \\ 5)
|
||||
def search_artists(query, limit \\ 5)
|
||||
```
|
||||
|
||||
**Search Strategy:**
|
||||
- Reuse existing `Records.search_collection/2` and `Records.search_wishlist/2`
|
||||
- Extend `Artists.search/2` or create new artist search function
|
||||
- Combine results with type metadata
|
||||
- Implement result limiting and pagination
|
||||
|
||||
### 4. Database Considerations
|
||||
|
||||
**Leverage Existing Infrastructure:**
|
||||
- Use existing `records_search_index` FTS5 table
|
||||
- Utilize existing search parser for tagged queries
|
||||
- Consider artist search optimization (may need artist search index)
|
||||
|
||||
**Artist Search Enhancement:**
|
||||
- Current artist search may need optimization
|
||||
- Consider adding FTS5 index for artist_infos table if performance issues
|
||||
- Leverage existing artist_records view for artist-record relationships
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### 1. Modal Behavior
|
||||
|
||||
**Desktop:**
|
||||
- Modal opens centered on screen
|
||||
- Size: `max-w-2xl` (responsive)
|
||||
- Overlay with backdrop blur
|
||||
- Esc key to close, click outside to close
|
||||
|
||||
**Mobile:**
|
||||
- Full-screen modal on mobile (`sm:max-w-2xl max-w-full`)
|
||||
- Touch-friendly result items
|
||||
- Swipe down to close (if feasible)
|
||||
- Virtual keyboard considerations
|
||||
|
||||
### 2. Search Interface
|
||||
|
||||
**Search Input:**
|
||||
- Placeholder: "Search records and artists..."
|
||||
- Auto-focus when modal opens
|
||||
- Debounced search (300ms delay)
|
||||
- Clear button when text present
|
||||
- Loading indicator during search
|
||||
|
||||
**Search Results:**
|
||||
```
|
||||
┌─────────────────────────────────────┐
|
||||
│ Search: "radiohead ok computer" │
|
||||
├─────────────────────────────────────┤
|
||||
│ COLLECTION (3 results) │
|
||||
│ • OK Computer - Radiohead │
|
||||
│ • Kid A - Radiohead │
|
||||
│ • In Rainbows - Radiohead │
|
||||
│ View all 12 collection results → │
|
||||
├─────────────────────────────────────┤
|
||||
│ WISHLIST (1 result) │
|
||||
│ • Hail to the Thief - Radiohead │
|
||||
│ View all 1 wishlist results → │
|
||||
├─────────────────────────────────────┤
|
||||
│ ARTISTS (1 result) │
|
||||
│ • Radiohead │
|
||||
│ View artist page → │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 3. Keyboard Navigation
|
||||
|
||||
**Keyboard Shortcuts:**
|
||||
- `Ctrl/Cmd + K` - Open modal
|
||||
- `Esc` - Close modal
|
||||
- `↑/↓` - Navigate results
|
||||
- `Enter` - Select highlighted result
|
||||
- `Tab` - Move between sections
|
||||
|
||||
**Implementation:**
|
||||
- Use Phoenix LiveView's `phx-window-keydown` for global shortcuts
|
||||
- Implement focus management with JavaScript hooks
|
||||
- Maintain accessibility standards (ARIA labels, focus indicators)
|
||||
|
||||
### 4. Result Navigation
|
||||
|
||||
**Click Actions:**
|
||||
- Record results → Navigate to record show page
|
||||
- Artist results → Navigate to artist page
|
||||
- "View all X results" → Navigate to respective list page with search applied
|
||||
|
||||
**URL Strategy:**
|
||||
- Don't change URL for modal open/close
|
||||
- When navigating to results, apply search query to destination page
|
||||
- Use existing search parameter patterns
|
||||
|
||||
## Database Schema Impact
|
||||
|
||||
**No Schema Changes Required:**
|
||||
- Leverage existing tables and search infrastructure
|
||||
- May add database indexes for performance if needed
|
||||
|
||||
**Performance Considerations:**
|
||||
- Monitor query performance with combined searches
|
||||
- Consider query result caching for common searches
|
||||
- Implement search result limits to prevent slow queries
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### 1. Unit Tests
|
||||
|
||||
**Search Logic Tests:**
|
||||
```elixir
|
||||
# test/music_library/search_test.exs
|
||||
describe "universal_search/2" do
|
||||
test "searches across all entity types"
|
||||
test "limits results per category"
|
||||
test "handles empty queries"
|
||||
test "handles special characters"
|
||||
test "respects search syntax (artist:, album:, etc.)"
|
||||
end
|
||||
```
|
||||
|
||||
### 2. Integration Tests
|
||||
|
||||
**LiveView Tests:**
|
||||
```elixir
|
||||
# test/music_library_web/live/universal_search_live_test.exs
|
||||
describe "UniversalSearchLive" do
|
||||
test "opens modal on keyboard shortcut"
|
||||
test "searches as user types"
|
||||
test "displays results grouped by type"
|
||||
test "navigates to selected results"
|
||||
test "handles keyboard navigation"
|
||||
test "closes modal on escape"
|
||||
end
|
||||
```
|
||||
|
||||
### 3. Component Tests
|
||||
|
||||
**Search Components:**
|
||||
```elixir
|
||||
# test/music_library_web/components/search_components_test.exs
|
||||
describe "SearchResultGroup" do
|
||||
test "renders results with proper grouping"
|
||||
test "shows 'view all' link when more results available"
|
||||
test "handles empty result sets"
|
||||
end
|
||||
```
|
||||
|
||||
### 4. JavaScript/Hook Tests
|
||||
|
||||
**Browser Tests:**
|
||||
- Test keyboard shortcuts work globally
|
||||
- Test focus management
|
||||
- Test modal open/close behavior
|
||||
- Test mobile touch interactions
|
||||
|
||||
## Mobile Considerations
|
||||
|
||||
### 1. Layout Adaptations
|
||||
|
||||
**Mobile Modal:**
|
||||
- Full-screen on small devices
|
||||
- Reduced padding and margins
|
||||
- Larger touch targets for results
|
||||
- Simplified result display (fewer details)
|
||||
|
||||
**Touch Interactions:**
|
||||
- Swipe down to close modal
|
||||
- Tap to select results
|
||||
- Pull-to-refresh for search results
|
||||
|
||||
### 2. Performance
|
||||
|
||||
**Mobile Optimization:**
|
||||
- Shorter debounce delays on mobile
|
||||
- Reduced result limits on mobile
|
||||
- Lazy loading for large result sets
|
||||
- Optimize image loading for album covers
|
||||
|
||||
### 3. Virtual Keyboard
|
||||
|
||||
**iOS/Android Considerations:**
|
||||
- Modal positioning with virtual keyboard
|
||||
- Proper input focus management
|
||||
- Prevent zoom on input focus
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Core Modal Infrastructure
|
||||
1. Create basic modal component
|
||||
2. Add search icon to navigation
|
||||
3. Implement modal open/close
|
||||
4. Add keyboard shortcut support
|
||||
|
||||
### Phase 2: Search Backend
|
||||
1. Create universal search context
|
||||
2. Implement combined search functions
|
||||
3. Add result limiting and grouping
|
||||
4. Optimize query performance
|
||||
|
||||
### Phase 3: Search Interface
|
||||
1. Implement search input with debouncing
|
||||
2. Add search result display
|
||||
3. Implement keyboard navigation
|
||||
4. Add result selection and navigation
|
||||
|
||||
### Phase 4: Mobile & Polish
|
||||
1. Mobile layout optimizations
|
||||
2. Touch interaction improvements
|
||||
3. Performance optimizations
|
||||
4. Accessibility enhancements
|
||||
|
||||
### Phase 5: Testing & Documentation
|
||||
1. Comprehensive test coverage
|
||||
2. Performance testing
|
||||
3. Mobile device testing
|
||||
4. Documentation updates
|
||||
|
||||
## Success Metrics
|
||||
|
||||
**Functionality:**
|
||||
- Modal opens/closes reliably
|
||||
- Search results are accurate and fast
|
||||
- Keyboard navigation works smoothly
|
||||
- Mobile experience is touch-friendly
|
||||
|
||||
**Performance:**
|
||||
- Search queries complete in <500ms
|
||||
- Modal opens in <100ms
|
||||
- No janky animations or interactions
|
||||
- Efficient database queries
|
||||
|
||||
**Usability:**
|
||||
- Easy to discover and use
|
||||
- Intuitive keyboard shortcuts
|
||||
- Clear result presentation
|
||||
- Seamless navigation to results
|
||||
|
||||
## Risks and Mitigation
|
||||
|
||||
**Performance Risks:**
|
||||
- Complex queries may be slow
|
||||
- **Mitigation:** Implement query limits, caching, and monitoring
|
||||
|
||||
**Mobile Experience:**
|
||||
- Small screen constraints
|
||||
- **Mitigation:** Responsive design, touch-friendly interfaces
|
||||
|
||||
**Accessibility:**
|
||||
- Keyboard navigation complexity
|
||||
- **Mitigation:** Follow ARIA guidelines, comprehensive testing
|
||||
|
||||
**Search Accuracy:**
|
||||
- Users may expect Google-like search
|
||||
- **Mitigation:** Clear documentation of search syntax, good defaults
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
**Advanced Features:**
|
||||
- Search history and suggestions
|
||||
- Saved searches
|
||||
- Search filters and facets
|
||||
- Search result previews
|
||||
|
||||
**Integration:**
|
||||
- Integration with barcode scanning
|
||||
- Voice search capabilities
|
||||
- External search (MusicBrainz, Last.fm)
|
||||
|
||||
**Analytics:**
|
||||
- Search usage analytics
|
||||
- Popular search terms
|
||||
- Search success rates
|
||||
|
||||
## Conclusion
|
||||
|
||||
This universal search modal will significantly improve the user experience by providing quick access to all content types from any page. The implementation leverages existing infrastructure while adding a modern, accessible search interface that works well on both desktop and mobile devices.
|
||||
|
||||
The phased approach ensures a solid foundation while allowing for iterative improvements based on user feedback and usage patterns.
|
||||
@@ -1,219 +0,0 @@
|
||||
# Scrobble Rule Application Optimization
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the optimization made to the scrobble rule application system to improve performance when applying multiple rules.
|
||||
|
||||
## Problem
|
||||
|
||||
Previously, when applying all enabled scrobble rules, each rule was applied independently with a separate database UPDATE query. With N rules, this resulted in N separate database operations, which became inefficient as the number of rules grew.
|
||||
|
||||
### Previous Implementation
|
||||
|
||||
```elixir
|
||||
def apply_all_rules do
|
||||
list_enabled_rules()
|
||||
|> Enum.map(fn rule ->
|
||||
case apply_rule(rule) do
|
||||
{:ok, count} -> {:ok, {rule.type, rule.match_value, count}}
|
||||
{:error, reason} -> {:error, {rule.type, rule.match_value, reason}}
|
||||
end
|
||||
end)
|
||||
end
|
||||
```
|
||||
|
||||
This approach meant:
|
||||
- 10 rules = 10 separate UPDATE queries
|
||||
- 100 rules = 100 separate UPDATE queries
|
||||
- Each query had to scan the entire `scrobbled_tracks` table
|
||||
|
||||
## Solution
|
||||
|
||||
The optimization groups rules by type (album/artist) and applies all rules of each type in a single database query using SQLite's CASE statement.
|
||||
|
||||
### New Implementation
|
||||
|
||||
```elixir
|
||||
def apply_all_rules do
|
||||
enabled_rules = list_enabled_rules()
|
||||
|
||||
# Group rules by type
|
||||
{album_rules, artist_rules} =
|
||||
Enum.split_with(enabled_rules, fn rule -> rule.type == :album end)
|
||||
|
||||
# Apply all album rules in one query
|
||||
apply_all_album_rules(album_rules)
|
||||
|
||||
# Apply all artist rules in one query
|
||||
apply_all_artist_rules(artist_rules)
|
||||
end
|
||||
```
|
||||
|
||||
This approach means:
|
||||
- 10 album rules + 10 artist rules = 2 total UPDATE queries (one for albums, one for artists)
|
||||
- 100 album rules + 100 artist rules = 2 total UPDATE queries
|
||||
- Each query still scans the table once, but updates all matching records in a single pass
|
||||
|
||||
## Technical Details
|
||||
|
||||
### SQL Generation
|
||||
|
||||
The optimization dynamically builds a CASE statement for all rules of the same type:
|
||||
|
||||
**Example for 2 album rules:**
|
||||
|
||||
```sql
|
||||
UPDATE scrobbled_tracks
|
||||
SET album = CASE
|
||||
WHEN json_extract(album, '$.title') = 'Dark Side of the Moon'
|
||||
THEN json_set(album, '$.musicbrainz_id', '12345678-1234-1234-1234-123456789012')
|
||||
WHEN json_extract(album, '$.title') = 'Wish You Were Here'
|
||||
THEN json_set(album, '$.musicbrainz_id', 'abcdef12-3456-7890-abcd-ef1234567890')
|
||||
ELSE album
|
||||
END
|
||||
WHERE json_extract(album, '$.title') IN ('Dark Side of the Moon', 'Wish You Were Here')
|
||||
```
|
||||
|
||||
The WHERE clause uses IN to filter only tracks that match any of the rules, avoiding unnecessary updates.
|
||||
|
||||
### Functions Added
|
||||
|
||||
#### 1. `apply_all_album_rules/1`
|
||||
Applies all album rules in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_album_rules([ScrobbleRule.t()]) :: {:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
#### 2. `apply_all_artist_rules/1`
|
||||
Applies all artist rules in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_artist_rules([ScrobbleRule.t()]) :: {:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
#### 3. `apply_all_album_rules/2`
|
||||
Applies all album rules to a specific set of tracks in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_album_rules([ScrobbleRule.t()], [Track.t()]) ::
|
||||
{:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
#### 4. `apply_all_artist_rules/2`
|
||||
Applies all artist rules to a specific set of tracks in a single query.
|
||||
|
||||
```elixir
|
||||
@spec apply_all_artist_rules([ScrobbleRule.t()], [Track.t()]) ::
|
||||
{:ok, non_neg_integer()} | {:error, any()}
|
||||
```
|
||||
|
||||
### Use Cases
|
||||
|
||||
The optimization handles two main use cases:
|
||||
|
||||
#### 1. Periodic Full Application
|
||||
The `ApplyScrobbleRules` Oban worker runs every 30 minutes and applies all rules to all tracks:
|
||||
|
||||
```elixir
|
||||
ScrobbleRules.apply_all_rules()
|
||||
```
|
||||
|
||||
#### 2. Incremental Application on New Tracks
|
||||
When new tracks are inserted via `LastFm.Feed.update/1`, rules are applied to just those tracks:
|
||||
|
||||
```elixir
|
||||
tracks
|
||||
|> ScrobbleRules.apply_all_rules()
|
||||
```
|
||||
|
||||
Both use cases now benefit from batched rule application.
|
||||
|
||||
## Performance Impact
|
||||
|
||||
### Expected Improvements
|
||||
|
||||
For a database with:
|
||||
- 10,000 scrobbled tracks
|
||||
- 20 enabled rules (10 album + 10 artist)
|
||||
|
||||
**Before:**
|
||||
- 20 UPDATE queries
|
||||
- Each query scans ~10,000 rows
|
||||
- Total: ~200,000 row scans
|
||||
|
||||
**After:**
|
||||
- 2 UPDATE queries (1 for albums, 1 for artists)
|
||||
- Each query scans ~10,000 rows
|
||||
- Total: ~20,000 row scans
|
||||
|
||||
**Result:** ~10x reduction in database operations for this scenario.
|
||||
|
||||
The benefit scales with the number of rules:
|
||||
- 5 rules: ~2.5x improvement
|
||||
- 10 rules: ~5x improvement
|
||||
- 20 rules: ~10x improvement
|
||||
- 100 rules: ~50x improvement
|
||||
|
||||
## Trade-offs
|
||||
|
||||
### Return Value Semantics
|
||||
|
||||
With the batched approach, all rules of the same type report the same aggregate count (total rows updated) rather than individual per-rule counts:
|
||||
|
||||
```elixir
|
||||
# Before: Each rule gets its own count
|
||||
[
|
||||
{:ok, {:album, "Album 1", 5}}, # 5 tracks updated
|
||||
{:ok, {:album, "Album 2", 3}}, # 3 tracks updated
|
||||
]
|
||||
|
||||
# After: Both rules report the total updated
|
||||
[
|
||||
{:ok, {:album, "Album 1", 8}}, # 8 total tracks updated
|
||||
{:ok, {:album, "Album 2", 8}}, # 8 total tracks updated
|
||||
]
|
||||
```
|
||||
|
||||
This is an acceptable trade-off because:
|
||||
1. The logging still shows how many rules were applied
|
||||
2. The total tracks updated is more meaningful for understanding impact
|
||||
3. Getting individual counts would require separate queries, negating the optimization
|
||||
|
||||
### Backward Compatibility
|
||||
|
||||
The API remains fully backward compatible:
|
||||
- All existing functions (`apply_rule/1`, `apply_album_rule/1`, etc.) still work
|
||||
- Return value format is unchanged: `{:ok, {type, match_value, count}}`
|
||||
- Error handling is unchanged
|
||||
- Tests continue to pass
|
||||
|
||||
## Testing
|
||||
|
||||
New tests were added to verify the optimization:
|
||||
|
||||
1. `apply_all_album_rules/1` applies multiple album rules in one query
|
||||
2. `apply_all_artist_rules/1` applies multiple artist rules in one query
|
||||
3. Empty list handling returns `{:ok, 0}`
|
||||
4. `apply_all_rules/0` correctly batches rules by type
|
||||
5. All existing tests continue to pass
|
||||
|
||||
## Future Considerations
|
||||
|
||||
### Potential Enhancements
|
||||
|
||||
1. **Parallel Execution**: Album and artist rule applications could run in parallel since they're independent
|
||||
2. **Query Optimization**: Could add indexes on `json_extract(album, '$.title')` and `json_extract(artist, '$.name')` if performance is still a concern
|
||||
3. **Metrics**: Add telemetry events to track batch sizes and execution times
|
||||
4. **Rule Ordering**: If rule priority matters in the future, the CASE statement naturally handles this (first match wins)
|
||||
|
||||
### Scalability
|
||||
|
||||
The current approach scales well up to ~100 rules per type. Beyond that, consider:
|
||||
- SQL query length limits (SQLite's default limit is 1MB, which allows ~10,000 rules)
|
||||
- Transaction size and memory usage
|
||||
- Potential chunking if needed (apply rules in batches of N)
|
||||
|
||||
## Conclusion
|
||||
|
||||
This optimization significantly improves the performance of scrobble rule application by reducing database operations from O(N) to O(1) per rule type, where N is the number of rules. The implementation maintains backward compatibility while providing substantial performance benefits that scale with the number of rules.
|
||||
@@ -31,10 +31,6 @@ defmodule MusicLibraryWeb.Endpoint do
|
||||
gzip: not code_reloading?,
|
||||
only: MusicLibraryWeb.static_paths()
|
||||
|
||||
if Code.ensure_loaded?(Tidewave) do
|
||||
plug Tidewave
|
||||
end
|
||||
|
||||
# Code reloading can be explicitly enabled under the
|
||||
# :code_reloader configuration of your endpoint.
|
||||
if code_reloading? do
|
||||
|
||||
@@ -76,15 +76,12 @@ defmodule MusicLibrary.MixProject do
|
||||
{:live_toast, "~> 0.8.0"},
|
||||
|
||||
# Dev tooling
|
||||
{:igniter, "~> 0.6", only: [:dev, :test]},
|
||||
{:usage_rules, "~> 0.1", only: [:dev]},
|
||||
{:phoenix_live_reload, "~> 1.2", only: :dev},
|
||||
{:esbuild, "~> 0.8", runtime: Mix.env() == :dev},
|
||||
{:tailwind, "~> 0.3", runtime: Mix.env() == :dev},
|
||||
{:credo, "~> 1.7", only: [:dev, :test], runtime: false},
|
||||
{:quokka, "~> 2.6", only: [:dev, :test], runtime: false},
|
||||
{:live_debugger, "~> 0.5.0", only: :dev},
|
||||
{:tidewave, "~> 0.1", only: :dev},
|
||||
|
||||
# HTTP Clients
|
||||
{:finch, "~> 0.20.0"},
|
||||
@@ -144,8 +141,7 @@ defmodule MusicLibrary.MixProject do
|
||||
"ecto.setup",
|
||||
"assets.setup",
|
||||
"cmd npm ci --prefix assets",
|
||||
"assets.build",
|
||||
"usage_rules.sync AGENTS.md --all --inline usage_rules:all --link-to-folder deps --remove-missing --yes"
|
||||
"assets.build"
|
||||
],
|
||||
"ecto.setup": ["ecto.create", "ecto.migrate"],
|
||||
"ecto.reset": ["ecto.drop", "ecto.setup"],
|
||||
|
||||
@@ -23,10 +23,8 @@
|
||||
"flagmojis": {:hex, :flagmojis, "1.0.0", "d2b411d9d2c150180103a4d605ee394cabc23c2b6ddf7a0011a9a55a23575281", [:mix], [], "hexpm", "9d9916575cf9dcbe6f0199b410994bec9d19d47971f70089eb7f210f823cd293"},
|
||||
"fluxon": {:hex, :fluxon, "2.3.1", "af66ad3a45ca802d983fe4483acc4c2a23b48c1703d743ca9e8d00191370dc1d", [:mix], [{:phoenix_live_view, ">= 1.0.0", [hex: :phoenix_live_view, repo: "hexpm", optional: false]}], "fluxon", "dcc91bcbce390e6633a576e0338ca46d4abc1f291f785337e8f62cbb87d1be49"},
|
||||
"gettext": {:hex, :gettext, "1.0.2", "5457e1fd3f4abe47b0e13ff85086aabae760497a3497909b8473e0acee57673b", [:mix], [{:expo, "~> 0.5.1 or ~> 1.0", [hex: :expo, repo: "hexpm", optional: false]}], "hexpm", "eab805501886802071ad290714515c8c4a17196ea76e5afc9d06ca85fb1bfeb3"},
|
||||
"glob_ex": {:hex, :glob_ex, "0.1.11", "cb50d3f1ef53f6ca04d6252c7fde09fd7a1cf63387714fe96f340a1349e62c93", [:mix], [], "hexpm", "342729363056e3145e61766b416769984c329e4378f1d558b63e341020525de4"},
|
||||
"heroicons": {:git, "https://github.com/tailwindlabs/heroicons.git", "0435d4ca364a608cc75e2f8683d374e55abbae26", [tag: "v2.2.0", sparse: "optimized", depth: 1]},
|
||||
"hpax": {:hex, :hpax, "1.0.3", "ed67ef51ad4df91e75cc6a1494f851850c0bd98ebc0be6e81b026e765ee535aa", [:mix], [], "hexpm", "8eab6e1cfa8d5918c2ce4ba43588e894af35dbd8e91e6e55c817bca5847df34a"},
|
||||
"igniter": {:hex, :igniter, "0.7.0", "6848714fa5afa14258c82924a57af9364745316241a409435cf39cbe11e3ae80", [:mix], [{:glob_ex, "~> 0.1.7", [hex: :glob_ex, repo: "hexpm", optional: false]}, {:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: false]}, {:owl, "~> 0.11", [hex: :owl, repo: "hexpm", optional: false]}, {:phx_new, "~> 1.7", [hex: :phx_new, repo: "hexpm", optional: true]}, {:req, "~> 0.5", [hex: :req, repo: "hexpm", optional: false]}, {:rewrite, ">= 1.1.1 and < 2.0.0-0", [hex: :rewrite, repo: "hexpm", optional: false]}, {:sourceror, "~> 1.4", [hex: :sourceror, repo: "hexpm", optional: false]}, {:spitfire, ">= 0.1.3 and < 1.0.0-0", [hex: :spitfire, repo: "hexpm", optional: false]}], "hexpm", "1e7254780dbf4b44c9eccd6d86d47aa961efc298d7f520c24acb0258c8e90ba9"},
|
||||
"jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"},
|
||||
"lazy_html": {:hex, :lazy_html, "0.1.8", "677a8642e644eef8de98f3040e2520d42d0f0f8bd6c5cd49db36504e34dffe91", [:make, :mix], [{:cc_precompiler, "~> 0.1", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.9.0", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:fine, "~> 0.1.0", [hex: :fine, repo: "hexpm", optional: false]}], "hexpm", "0d8167d930b704feb94b41414ca7f5779dff9bca7fcf619fcef18de138f08736"},
|
||||
"live_debugger": {:hex, :live_debugger, "0.5.0", "95e0f7727d61010f7e9053923fb2a9416904a7533c2dfb36120e7684cba4c0af", [:mix], [{:igniter, ">= 0.5.40 and < 1.0.0-0", [hex: :igniter, repo: "hexpm", optional: true]}, {:phoenix, "~> 1.7", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_live_view, "~> 0.20.8 or ~> 1.0", [hex: :phoenix_live_view, repo: "hexpm", optional: false]}], "hexpm", "73ebe95118d22aa402675f677abd731cb16b136d1b6ae5f4010441fb50753b14"},
|
||||
@@ -40,7 +38,6 @@
|
||||
"oban": {:hex, :oban, "2.20.2", "f23313d83b578305cafa825a036cad84e7e2d61549ecbece3a2e6526d347cc3b", [:mix], [{:ecto_sql, "~> 3.10", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:ecto_sqlite3, "~> 0.9", [hex: :ecto_sqlite3, repo: "hexpm", optional: true]}, {:igniter, "~> 0.5", [hex: :igniter, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:myxql, "~> 0.7", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.20", [hex: :postgrex, repo: "hexpm", optional: true]}, {:telemetry, "~> 1.3", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "523365ef0217781c061d15f496e3200a5f1b43e08b1a27c34799ef8bfe95815f"},
|
||||
"oban_met": {:hex, :oban_met, "1.0.5", "bb633ab06448dab2ef9194f6688d33b3d07fc3f2ad793a1a08f4dfbb2cc9fe50", [:mix], [{:oban, "~> 2.19", [hex: :oban, repo: "hexpm", optional: false]}], "hexpm", "64664d50805bbfd3903aeada1f3c39634652a87844797ee400b0bcc95a28f5ea"},
|
||||
"oban_web": {:hex, :oban_web, "2.11.6", "53933cb4253c4d9f1098ee311c06f07935259f0e564dcf2d66bae4cc98e317fe", [:mix], [{:jason, "~> 1.2", [hex: :jason, repo: "hexpm", optional: false]}, {:oban, "~> 2.19", [hex: :oban, repo: "hexpm", optional: false]}, {:oban_met, "~> 1.0", [hex: :oban_met, repo: "hexpm", optional: false]}, {:phoenix, "~> 1.7", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 3.3 or ~> 4.0", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:phoenix_live_view, "~> 1.0", [hex: :phoenix_live_view, repo: "hexpm", optional: false]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}], "hexpm", "576d94b705688c313694c2c114ca21aa0f8f2ad1b9ca45c052c5ba316d3e8d10"},
|
||||
"owl": {:hex, :owl, "0.13.0", "26010e066d5992774268f3163506972ddac0a7e77bfe57fa42a250f24d6b876e", [:mix], [{:ucwidth, "~> 0.2", [hex: :ucwidth, repo: "hexpm", optional: true]}], "hexpm", "59bf9d11ce37a4db98f57cb68fbfd61593bf419ec4ed302852b6683d3d2f7475"},
|
||||
"phoenix": {:hex, :phoenix, "1.8.3", "49ac5e485083cb1495a905e47eb554277bdd9c65ccb4fc5100306b350151aa95", [:mix], [{:bandit, "~> 1.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 2.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:phoenix_template, "~> 1.0", [hex: :phoenix_template, repo: "hexpm", optional: false]}, {:phoenix_view, "~> 2.0", [hex: :phoenix_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.7", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:plug_crypto, "~> 1.2 or ~> 2.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:websock_adapter, "~> 0.5.3", [hex: :websock_adapter, repo: "hexpm", optional: false]}], "hexpm", "36169f95cc2e155b78be93d9590acc3f462f1e5438db06e6248613f27c80caec"},
|
||||
"phoenix_ecto": {:hex, :phoenix_ecto, "4.7.0", "75c4b9dfb3efdc42aec2bd5f8bccd978aca0651dbcbc7a3f362ea5d9d43153c6", [:mix], [{:ecto, "~> 3.5", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.14.2 or ~> 3.0 or ~> 4.1", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.16 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}], "hexpm", "1d75011e4254cb4ddf823e81823a9629559a1be93b4321a6a5f11a5306fbf4cc"},
|
||||
"phoenix_html": {:hex, :phoenix_html, "4.3.0", "d3577a5df4b6954cd7890c84d955c470b5310bb49647f0a114a6eeecc850f7ad", [:mix], [], "hexpm", "3eaa290a78bab0f075f791a46a981bbe769d94bc776869f4f3063a14f30497ad"},
|
||||
@@ -56,20 +53,14 @@
|
||||
"quokka": {:hex, :quokka, "2.11.2", "2856118154425f18547720d997199be54febed771a740ba3c988a17762328287", [:mix], [{:credo, "~> 1.7", [hex: :credo, repo: "hexpm", optional: false]}], "hexpm", "8208f5d814007cb35a2eb278462464d083fca8c463f62517ab94eef982f181cc"},
|
||||
"recon": {:hex, :recon, "2.5.6", "9052588e83bfedfd9b72e1034532aee2a5369d9d9343b61aeb7fbce761010741", [:mix, :rebar3], [], "hexpm", "96c6799792d735cc0f0fd0f86267e9d351e63339cbe03df9d162010cefc26bb0"},
|
||||
"req": {:hex, :req, "0.5.16", "99ba6a36b014458e52a8b9a0543bfa752cb0344b2a9d756651db1281d4ba4450", [:mix], [{:brotli, "~> 0.3.1", [hex: :brotli, repo: "hexpm", optional: true]}, {:ezstd, "~> 1.0", [hex: :ezstd, repo: "hexpm", optional: true]}, {:finch, "~> 0.17", [hex: :finch, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mime, "~> 2.0.6 or ~> 2.1", [hex: :mime, repo: "hexpm", optional: false]}, {:nimble_csv, "~> 1.0", [hex: :nimble_csv, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: true]}], "hexpm", "974a7a27982b9b791df84e8f6687d21483795882a7840e8309abdbe08bb06f09"},
|
||||
"rewrite": {:hex, :rewrite, "1.2.0", "80220eb14010e175b67c939397e1a8cdaa2c32db6e2e0a9d5e23e45c0414ce21", [:mix], [{:glob_ex, "~> 0.1", [hex: :glob_ex, repo: "hexpm", optional: false]}, {:sourceror, "~> 1.0", [hex: :sourceror, repo: "hexpm", optional: false]}, {:text_diff, "~> 0.1", [hex: :text_diff, repo: "hexpm", optional: false]}], "hexpm", "a1cd702bbb9d51613ab21091f04a386d750fc6f4516b81900df082d78b2d8c50"},
|
||||
"sentry": {:hex, :sentry, "11.0.4", "60371c96cefd247e0fc98840bba2648f64f19aa0b8db8e938f5a98421f55b619", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: true]}, {:igniter, "~> 0.5", [hex: :igniter, repo: "hexpm", optional: true]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: true]}, {:nimble_options, "~> 1.0", [hex: :nimble_options, repo: "hexpm", optional: false]}, {:nimble_ownership, "~> 0.3.0 or ~> 1.0", [hex: :nimble_ownership, repo: "hexpm", optional: false]}, {:opentelemetry, ">= 0.0.0", [hex: :opentelemetry, repo: "hexpm", optional: true]}, {:opentelemetry_api, ">= 0.0.0", [hex: :opentelemetry_api, repo: "hexpm", optional: true]}, {:opentelemetry_exporter, ">= 0.0.0", [hex: :opentelemetry_exporter, repo: "hexpm", optional: true]}, {:opentelemetry_semantic_conventions, ">= 0.0.0", [hex: :opentelemetry_semantic_conventions, repo: "hexpm", optional: true]}, {:phoenix, "~> 1.6", [hex: :phoenix, repo: "hexpm", optional: true]}, {:phoenix_live_view, "~> 0.20 or ~> 1.0", [hex: :phoenix_live_view, repo: "hexpm", optional: true]}, {:plug, "~> 1.6", [hex: :plug, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm", "feaafc284dc204c82aadaddc884227aeaa3480decb274d30e184b9d41a700c66"},
|
||||
"sourceror": {:hex, :sourceror, "1.10.0", "38397dedbbc286966ec48c7af13e228b171332be1ad731974438c77791945ce9", [:mix], [], "hexpm", "29dbdfc92e04569c9d8e6efdc422fc1d815f4bd0055dc7c51b8800fb75c4b3f1"},
|
||||
"spitfire": {:hex, :spitfire, "0.2.1", "29e154873f05444669c7453d3d931820822cbca5170e88f0f8faa1de74a79b47", [:mix], [], "hexpm", "6eeed75054a38341b2e1814d41bb0a250564092358de2669fdb57ff88141d91b"},
|
||||
"table_rex": {:hex, :table_rex, "4.1.0", "fbaa8b1ce154c9772012bf445bfb86b587430fb96f3b12022d3f35ee4a68c918", [:mix], [], "hexpm", "95932701df195d43bc2d1c6531178fc8338aa8f38c80f098504d529c43bc2601"},
|
||||
"tailwind": {:hex, :tailwind, "0.4.1", "e7bcc222fe96a1e55f948e76d13dd84a1a7653fb051d2a167135db3b4b08d3e9", [:mix], [], "hexpm", "6249d4f9819052911120dbdbe9e532e6bd64ea23476056adb7f730aa25c220d1"},
|
||||
"telemetry": {:hex, :telemetry, "1.3.0", "fedebbae410d715cf8e7062c96a1ef32ec22e764197f70cda73d82778d61e7a2", [:rebar3], [], "hexpm", "7015fc8919dbe63764f4b4b87a95b7c0996bd539e0d499be6ec9d7f3875b79e6"},
|
||||
"telemetry_metrics": {:hex, :telemetry_metrics, "1.1.0", "5bd5f3b5637e0abea0426b947e3ce5dd304f8b3bc6617039e2b5a008adc02f8f", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "e7b79e8ddfde70adb6db8a6623d1778ec66401f366e9a8f5dd0955c56bc8ce67"},
|
||||
"telemetry_poller": {:hex, :telemetry_poller, "1.3.0", "d5c46420126b5ac2d72bc6580fb4f537d35e851cc0f8dbd571acf6d6e10f5ec7", [:rebar3], [{:telemetry, "~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "51f18bed7128544a50f75897db9974436ea9bfba560420b646af27a9a9b35211"},
|
||||
"text_diff": {:hex, :text_diff, "0.1.0", "1caf3175e11a53a9a139bc9339bd607c47b9e376b073d4571c031913317fecaa", [:mix], [], "hexpm", "d1ffaaecab338e49357b6daa82e435f877e0649041ace7755583a0ea3362dbd7"},
|
||||
"thousand_island": {:hex, :thousand_island, "1.4.3", "2158209580f633be38d43ec4e3ce0a01079592b9657afff9080d5d8ca149a3af", [:mix], [{:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "6e4ce09b0fd761a58594d02814d40f77daff460c48a7354a15ab353bb998ea0b"},
|
||||
"tidewave": {:hex, :tidewave, "0.5.3", "1378aefa93dbf887c2df60842be4cf312c57fdf99dbf91c5227cd4344050876e", [:mix], [{:circular_buffer, "~> 0.4 or ~> 1.0", [hex: :circular_buffer, repo: "hexpm", optional: false]}, {:igniter, "~> 0.6", [hex: :igniter, repo: "hexpm", optional: true]}, {:jason, "~> 1.4", [hex: :jason, repo: "hexpm", optional: false]}, {:phoenix_live_reload, ">= 1.6.1", [hex: :phoenix_live_reload, repo: "hexpm", optional: true]}, {:plug, "~> 1.17", [hex: :plug, repo: "hexpm", optional: false]}, {:req, "~> 0.5", [hex: :req, repo: "hexpm", optional: false]}], "hexpm", "605a1b912b7a8b56498077b3426be96b7129c4ac06d166311d408dccd0e5e0d3"},
|
||||
"time_zone_info": {:hex, :time_zone_info, "0.7.9", "1fb22e9186e844617959e4d5a8c0fe43c41fc04dca43b4cec8d84e31d0589099", [:mix], [{:castore, "~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:mint, "~> 1.0", [hex: :mint, repo: "hexpm", optional: true]}, {:nimble_parsec, "~> 1.3", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm", "ebd961d10d9db31e35e7b52e6b8c25f9db70c582f318bb5b0fb32f9d90c57244"},
|
||||
"usage_rules": {:hex, :usage_rules, "0.1.26", "19d38c8b9b5c35434eae44f7e4554caeb5f08037a1d45a6b059a9782543ac22e", [:mix], [{:igniter, ">= 0.6.6 and < 1.0.0-0", [hex: :igniter, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:req, "~> 0.5", [hex: :req, repo: "hexpm", optional: false]}], "hexpm", "9f0d203aa288e1b48318929066778ec26fc423fd51f08518c5b47f58ad5caca9"},
|
||||
"vix": {:hex, :vix, "0.35.0", "f6319b715e3b072e53eba456a21af5f2ff010a7a7b19b884600ea98a0609b18c", [:make, :mix], [{:cc_precompiler, "~> 0.1.4 or ~> 0.2", [hex: :cc_precompiler, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.7.3 or ~> 0.8", [hex: :elixir_make, repo: "hexpm", optional: false]}, {:kino, "~> 0.7", [hex: :kino, repo: "hexpm", optional: true]}], "hexpm", "a3e80067a89d0631b6cf2b93594e03c1b303a2c7cddbbdd28040750d521984e5"},
|
||||
"websock": {:hex, :websock, "0.5.3", "2f69a6ebe810328555b6fe5c831a851f485e303a7c8ce6c5f675abeb20ebdadc", [:mix], [], "hexpm", "6105453d7fac22c712ad66fab1d45abdf049868f253cf719b625151460b8b453"},
|
||||
"websock_adapter": {:hex, :websock_adapter, "0.5.9", "43dc3ba6d89ef5dec5b1d0a39698436a1e856d000d84bf31a3149862b01a287f", [:mix], [{:bandit, ">= 0.6.0", [hex: :bandit, repo: "hexpm", optional: true]}, {:plug, "~> 1.14", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 2.6", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "5534d5c9adad3c18a0f58a9371220d75a803bf0b9a3d87e6fe072faaeed76a08"},
|
||||
|
||||
@@ -21,8 +21,6 @@ debug_msg "Updating mix dependencies"
|
||||
|
||||
mix deps.update --all
|
||||
|
||||
mix usage_rules.sync AGENTS.md --all --inline usage_rules:all --link-to-folder deps --remove-missing --yes
|
||||
|
||||
debug_msg "Updating gh extensions"
|
||||
|
||||
gh extension upgrade gfm-preview
|
||||
|
||||
Reference in New Issue
Block a user