ML-156: initial research
This commit is contained in:
Claudio Ortolina
+250
@@ -0,0 +1,250 @@
|
||||
---
|
||||
id: doc-1
|
||||
title: 'ML-156 Research: Alternatives for reducing collection chat token usage'
|
||||
type: other
|
||||
created_date: '2026-05-02 16:12'
|
||||
---
|
||||
# ML-156 Research: Alternatives for reducing collection chat token usage
|
||||
|
||||
Research document for [ML-156 - Explore alternatives to reduce token usage when providing collection context to LLM for collection chat](backlog://task/ML-156).
|
||||
|
||||
---
|
||||
|
||||
## Current state
|
||||
|
||||
**Token flow per new collection chat:**
|
||||
1. `Collection.collection_summary/0` runs on mount via `start_async`
|
||||
2. Loads ALL records: `from(r in Record, where: not is_nil(r.purchased_at), order_by: [order_alphabetically()], select: ^essential_fields())`
|
||||
3. Groups by `musicbrainz_id`, formats each group as `"Artist - Title (year, formats) [genre1, genre2]"`
|
||||
4. Builds stats header: `"# Stats: N releases, M artists\nGenres: ...\nFormats: ...\nEras: ..."`
|
||||
5. Returns `{stats + "\n\n" + catalog, group_count}`
|
||||
6. Stored in `@collection_summary` assign on the LiveView
|
||||
7. Passed to Chat component as `chat_context={@collection_summary}`
|
||||
8. When user sends first message, `do_send_message` calls `chat_module.stream_response(messages, chat_context, callback)`
|
||||
9. `CollectionChat.stream_response/3` calls `build_instructions(summary, record_count)`
|
||||
10. `build_instructions/2` calls `Prompt.build/2` which wraps the summary in identity + approach templates
|
||||
11. Full instructions string is sent as the `instructions` field in the OpenAI Responses API request
|
||||
|
||||
**Token estimate per catalog entry:** ~15-20 tokens (e.g., "Radiohead - OK Computer (1997, cd/vinyl) [alternative rock, art rock]\n")
|
||||
**For 500 releases:** ~9,000 input tokens for catalog alone + ~100 tokens for stats + ~500 tokens for prompt template = ~9,600 tokens
|
||||
|
||||
**Key files:**
|
||||
- `lib/music_library/collection.ex:203-235` — `collection_summary/0` (loads and formats all records)
|
||||
- `lib/music_library/chats/collection_chat.ex:18-31` — `build_instructions/2` (embeds summary in prompt)
|
||||
- `lib/music_library/chats/prompt.ex` — `Prompt.build/2` (wraps in identity + approach)
|
||||
- `lib/open_ai/api.ex:54-73` — `chat_stream/6` (sends to Responses API with `tools: [%{type: "web_search_preview"}]`)
|
||||
- `lib/open_ai/api.ex:158-178` — `decode_responses_event/2` (SSE parser, currently only handles `response.output_text.delta`, `response.failed`, generic `response.*`)
|
||||
- `lib/music_library_web/components/chat.ex:196-234` — `do_send_message/2` (dispatches to `chat_module.stream_response`)
|
||||
- `lib/music_library_web/live/collection_live/index.ex:226-233,246-254,304-305` — Chat component mount and summary async loading
|
||||
|
||||
### Streaming architecture constraints
|
||||
|
||||
The Chat component dispatches streaming to a `Task.Supervisor` child:
|
||||
```elixir
|
||||
Task.Supervisor.start_child(MusicLibrary.TaskSupervisor, fn ->
|
||||
case chat_module.stream_response(stream_messages, chat_context, fn chunk ->
|
||||
LiveView.send_update(parent_pid, __MODULE__, id: component_id, chunk: chunk)
|
||||
end) do
|
||||
:ok -> LiveView.send_update(parent_pid, __MODULE__, id: component_id, done: true)
|
||||
{:error, reason} -> ...send error update...
|
||||
end
|
||||
end)
|
||||
```
|
||||
|
||||
The callback sends `[chunk: chunk]` updates; `update/2` in the Chat component accumulates text via `MDEx.Document.put_markdown`. The response is rendered as streaming markdown.
|
||||
|
||||
---
|
||||
|
||||
## Alternatives
|
||||
|
||||
### Alternative A: Stats-only instructions (no catalog)
|
||||
|
||||
**Description:** Remove the catalog lines from instructions. Keep only the aggregated stats (release count, artist count, top genres, formats, eras).
|
||||
|
||||
**Token savings:** ~9,000 → ~100 input tokens (~99% reduction)
|
||||
|
||||
**Pros:**
|
||||
- Simplest possible change; < 10 lines of code
|
||||
- No architectural changes needed
|
||||
- No streaming infrastructure changes
|
||||
- The LLM can still answer statistical questions ("what's my most common genre?", "how many jazz records do I have?")
|
||||
|
||||
**Cons:**
|
||||
- LLM loses ability to answer specific questions ("do I have Kid A?", "which Radiohead albums do I own?", "show me my 90s electronic albums")
|
||||
- User experience degrades for record-specific queries
|
||||
- The LLM will hallucinate or say "I don't have access to your specific collection" frequently
|
||||
|
||||
**Impact on code:**
|
||||
1. `CollectionChat.build_instructions/2` — remove `#{collection_summary}` interpolation, keep only stats
|
||||
2. `Collection.collection_summary/0` — could be simplified to return only stats (or keep as-is, the function is also tested independently)
|
||||
|
||||
---
|
||||
|
||||
### Alternative B: Function calling (tool-based search)
|
||||
|
||||
**Description:** Provide the LLM with a `search_collection` function tool. The LLM calls this tool when it needs to look up specific records in the user's collection. The instructions only include aggregated stats.
|
||||
|
||||
**Token savings:** ~9,000 → ~100 input tokens base + tool call overhead + tool results (~200-500 tokens when actually searching)
|
||||
|
||||
**How it works:**
|
||||
1. Add a tool definition to the Responses API request:
|
||||
```json
|
||||
{
|
||||
"type": "function",
|
||||
"name": "search_collection",
|
||||
"description": "Search the user's music collection by artist name, album title, genre, format, or any combination",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"query": {"type": "string", "description": "Search query"}
|
||||
},
|
||||
"required": ["query"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. The SSE streaming flow changes: instead of simple text-delta → done, we need to handle:
|
||||
- `response.function_call_arguments.delta` / `response.function_call_arguments.done`
|
||||
- Build the function call from accumulated deltas
|
||||
- Execute `Collection.search_records(query)` locally
|
||||
- Submit the function result back to the Responses API (second request)
|
||||
- Continue streaming the text response
|
||||
|
||||
3. The Chat component's streaming architecture needs to handle this multi-turn flow.
|
||||
|
||||
**Pros:**
|
||||
- Maximal token efficiency — only pay for what's actually needed
|
||||
- LLM can answer arbitrary specific questions with real data
|
||||
- Scales to any collection size
|
||||
- The same tool pattern could be reused for artist chat, record chat, etc.
|
||||
- Leverages the existing `tools` infrastructure in the Responses API request
|
||||
|
||||
**Cons:**
|
||||
- **Significantly more complex** — requires:
|
||||
- Changes to `OpenAI.API.chat_stream/6` to support function calls in streaming mode
|
||||
- Changes to `decode_responses_event/2` to handle function call SSE events
|
||||
- A function call execution loop (model calls function → execute → submit result → model responds)
|
||||
- Changes to the Chat component's `do_send_message/2` to orchestrate multi-turn tool use
|
||||
- The streaming Task process becomes stateful (needs to handle the submit-then-continue loop)
|
||||
- Tool results still consume tokens (but only the matching records, not the entire catalog)
|
||||
- User may experience a brief pause while the tool executes (mitigated by streaming the tool call status)
|
||||
- More error states to handle (function execution failures, parse errors)
|
||||
|
||||
**Implementation scope:**
|
||||
1. **Tool definition module** — New module ~30 lines defining the OpenAI function tool schema
|
||||
2. **Function executor** — New module or function in `CollectionChat` that executes the tool call (~20 lines)
|
||||
3. **Streaming changes in `OpenAI.API`** — New `chat_stream_with_tools/7` or modify `chat_stream/6` to accept tools and handle function call events (~80 lines)
|
||||
4. **SSE event handling** — Add cases to `decode_responses_event/2` for function call events (~30 lines)
|
||||
5. **Chat component orchestration** — Modify `do_send_message/2` or the Task function to handle tool call loops (~50 lines)
|
||||
6. **`CollectionChat.build_instructions/2`** — Remove catalog, keep stats, add tool usage guidance (~10 lines)
|
||||
|
||||
**Estimated lines of change:** ~200-300 lines across 6-8 files
|
||||
|
||||
---
|
||||
|
||||
### Alternative C: Cached LLM-generated summary
|
||||
|
||||
**Description:** Generate a concise, human-readable summary of the collection using an LLM call (e.g., "Your collection spans from 1967 to 2024 with a focus on progressive rock, featuring 3 albums by Radiohead, 2 by Pink Floyd..."). Store this summary in a `Chat` record or asset, and regenerate it when the collection changes (record added/removed/edited).
|
||||
|
||||
**Token savings:** ~9,000 → ~400 tokens (summary text + stats)
|
||||
|
||||
**Pros:**
|
||||
- Good middle ground — compact but informative
|
||||
- The LLM has a narrative understanding of the collection
|
||||
- No streaming architecture changes needed
|
||||
- Single one-time cost to generate (amortized across many chats)
|
||||
|
||||
**Cons:**
|
||||
- Still doesn't give the LLM ability to answer specific queries ("do I have Kid A?")
|
||||
- Summary can go stale if not regenerated on collection changes
|
||||
- Requires an initial LLM call to generate the summary (token cost + latency)
|
||||
- When to regenerate is tricky — every record add/edit/delete?
|
||||
- The summary is only as good as the LLM's compression; important details may be lost
|
||||
- Adds a new background job / async concern
|
||||
|
||||
**Implementation scope:**
|
||||
1. Store the cached summary (new DB field on a canonical collection `Chat` record, or a new schema)
|
||||
2. A function/worker to generate the LLM summary (calls OpenAI, stores result)
|
||||
3. Invalidation triggers (PubSub on record add/edit/delete, regenerate async)
|
||||
4. Changes to `CollectionChat.build_instructions/2` to use the cached summary
|
||||
5. Fallback to stats-only if cache is stale/missing
|
||||
|
||||
---
|
||||
|
||||
### Alternative D: Hybrid — stats in instructions + tool for catalog search
|
||||
|
||||
**Description:** Combine Alternative A (stats always included) with Alternative B (function calling for specific queries). The instructions include aggregated stats and guidance to use the `search_collection` tool when the user asks about specific records.
|
||||
|
||||
**Token savings:** ~9,000 → ~100 tokens base + tool results (0-500 tokens per use)
|
||||
|
||||
**Pros:**
|
||||
- Best of both worlds: statistical awareness always available, specific lookup on demand
|
||||
- Token-efficient — base cost is minimal
|
||||
- LLM knows to reach for the tool when appropriate
|
||||
|
||||
**Cons:**
|
||||
- Same implementation complexity as Alternative B for the tool infrastructure
|
||||
- Slightly more prompt engineering to ensure the model uses the tool appropriately
|
||||
|
||||
**Implementation scope:** Same as Alternative B, plus refined prompt instructions.
|
||||
|
||||
---
|
||||
|
||||
### Alternative E: OpenAI `file_search` tool (recommended)
|
||||
|
||||
Upload the collection catalog as a file to OpenAI, create a vector store, and use the Responses API's built-in `file_search` tool. OpenAI automatically performs semantic search over the file and includes relevant results inline.
|
||||
|
||||
**Token savings:** ~9,000 → ~100 tokens base + retrieval overhead (~200-500 when model searches)
|
||||
|
||||
**How it works:**
|
||||
1. Format collection catalog as text (same as current `collection_summary/0` output)
|
||||
2. Upload to OpenAI: `POST /v1/files` with `purpose: "assistants"` → `file_id`
|
||||
3. Create vector store: `POST /v1/vector_stores` → `vector_store_id`
|
||||
4. Attach file: `POST /v1/vector_stores/{id}/files` → OpenAI indexes it
|
||||
5. In `chat_stream`, add `%{type: "file_search", vector_store_ids: [store_id]}` to tools
|
||||
6. Model searches file when needed — results appear inline, same as web_search_preview today
|
||||
|
||||
**Critical difference from Alt B:** No SSE event handling changes, no orchestration loop, no custom tool execution. `file_search` works exactly like `web_search_preview` (already in use) — OpenAI handles retrieval automatically.
|
||||
|
||||
**Pros:**
|
||||
- ~150 lines vs ~250 for Alt B
|
||||
- No changes to `decode_responses_event/2` or Chat streaming loop
|
||||
- Semantic search (finds "upbeat 80s rock" not just keywords)
|
||||
- Files only uploaded on collection change (amortized cost)
|
||||
- Reusable for artist bios, notes, etc.
|
||||
|
||||
**Cons:**
|
||||
- File can go stale if not updated on collection change
|
||||
- 2-3 new API endpoints needed (Files API, Vector Stores API)
|
||||
- Vector store indexing is async (may need brief poll)
|
||||
- First chat after deploy has upload+index latency
|
||||
- File storage cost at OpenAI (negligible for text)
|
||||
- Search quality depends on OpenAI's embedding model, not directly controllable
|
||||
|
||||
---
|
||||
|
||||
## Comparison
|
||||
|
||||
| Criterion | A (stats-only) | B (custom func) | C (cached) | E (file_search) |
|
||||
|---|---|---|---|---|
|
||||
| Implementation complexity | ★☆☆☆☆ | ★★★★☆ | ★★★☆☆ | ★★☆☆☆ |
|
||||
| Response quality | ★★☆☆☆ | ★★★★★ | ★★★☆☆ | ★★★★★ |
|
||||
| Token efficiency | ★★★★★ | ★★★★★ | ★★★★☆ | ★★★★★ |
|
||||
| Infrastructure risk | ★★★★★ | ★★★☆☆ | ★★★★☆ | ★★★★☆ |
|
||||
| Reusability | ★☆☆☆☆ | ★★★★☆ | ★★☆☆☆ | ★★★☆☆ |
|
||||
|
||||
---
|
||||
|
||||
## Why Alternative E over B
|
||||
|
||||
Alternative E (`file_search`) was chosen over Alternative B (custom function calling) for these reasons:
|
||||
|
||||
1. **Same response quality with half the code** (~150 lines vs ~250 lines). Both approaches let the LLM answer arbitrary specific questions with real collection data.
|
||||
|
||||
2. **No streaming infrastructure changes.** Alternative B requires modifying `decode_responses_event/2`, the SSE event loop, and `do_send_message/2` to handle function call orchestration. Alternative E simply adds a `file_search` tool entry alongside the existing `web_search_preview` — no new SSE event types to parse, no orchestration loop, no stateful streaming.
|
||||
|
||||
3. **Lower risk.** The core streaming code path is untouched. The new code is additive (new API endpoints, new `FileStore` module) rather than modifying the shared streaming infrastructure that serves all three chat types (record, artist, collection).
|
||||
|
||||
4. **Better search quality.** OpenAI's semantic search is likely superior to FTS5 for natural language queries like "upbeat 80s rock with synths".
|
||||
|
||||
5. **Alternative B would be the right choice if** the application needed parameterized queries (artist+format+year range), real-time data that changes mid-chat, or complex database filtering. For a static-ish catalog that changes infrequently, `file_search` is the pragmatic choice.
|
||||
+231
@@ -0,0 +1,231 @@
|
||||
---
|
||||
id: ML-156
|
||||
title: >-
|
||||
Explore alternatives to reduce token usage when providing collection context
|
||||
to LLM for collection chat
|
||||
status: To Do
|
||||
assignee: []
|
||||
created_date: '2026-05-02 16:02'
|
||||
updated_date: '2026-05-02 16:13'
|
||||
labels:
|
||||
- chat
|
||||
- collection
|
||||
- openai
|
||||
- token-optimization
|
||||
dependencies: []
|
||||
references:
|
||||
- 'backlog://document/doc-1'
|
||||
documentation:
|
||||
- lib/music_library/chats/collection_chat.ex
|
||||
- lib/music_library/collection.ex
|
||||
- lib/music_library/chats/prompt.ex
|
||||
- lib/music_library/chats/stream_provider.ex
|
||||
- lib/open_ai/api.ex
|
||||
- lib/music_library_web/components/chat.ex
|
||||
- lib/music_library_web/live/collection_live/index.ex
|
||||
priority: high
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
<!-- SECTION:DESCRIPTION:BEGIN -->
|
||||
Currently, every new collection chat sends the ENTIRE collection catalog (all records formatted as "Artist - Title (year, format) [genres]") plus aggregated stats as the `instructions` parameter to the OpenAI Responses API. For a collection of 500+ records, this burns ~9,000+ input tokens on EVERY new chat start — regardless of what the user asks.
|
||||
|
||||
The goal of this task is to analyze alternatives, pick the best one, and implement it. The selected approach should:
|
||||
- Significantly reduce per-chat token usage
|
||||
- Preserve or improve the quality of LLM responses about the collection
|
||||
- Not require architectural overhauls beyond the chat/streaming layer
|
||||
|
||||
The `Collection.collection_summary/0` function loads ALL records from the DB, formats them, and returns `{summary, count}`. This is computed asynchronously in `CollectionLive.Index.mount/3` and passed to the Chat component as `chat_context`. `CollectionChat.build_instructions/2` then embeds the full summary into the instructions string sent to OpenAI.
|
||||
<!-- SECTION:DESCRIPTION:END -->
|
||||
|
||||
## Acceptance Criteria
|
||||
<!-- AC:BEGIN -->
|
||||
- [ ] #1 `CollectionChat.build_instructions/2` no longer interpolates the full collection catalog into the instructions sent to OpenAI — only aggregated stats and a record count are included
|
||||
- [ ] #2 A `file_search` tool with the collection's vector store ID is included in every collection chat request to the OpenAI Responses API
|
||||
- [ ] #3 A `CollectionChat.FileStore` module manages the collection file lifecycle: upload to OpenAI Files API, vector store creation, and file-to-store attachment, persisting IDs via `Secrets`
|
||||
- [ ] #4 When a record is added, edited, or deleted, the collection file at OpenAI is refreshed (async, non-blocking) so the LLM always searches up-to-date data
|
||||
- [ ] #5 If the file/vector store is unavailable (upload failed, first deploy), the chat falls back to stats-only instructions without errors
|
||||
- [ ] #6 Existing record and artist chats continue to work without changes (no regression in streaming)
|
||||
- [ ] #7 Tests cover: Files API endpoints, file upload/create/refresh lifecycle, `file_search` tool inclusion in chat requests, fallback when file store is unavailable, empty collection edge case
|
||||
- [ ] #8 Per-chat input tokens for the collection chat instructions are O(1) relative to collection size (the file is searched by OpenAI on demand, not embedded in instructions)
|
||||
<!-- AC:END -->
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
<!-- SECTION:PLAN:BEGIN -->
|
||||
## Approach: OpenAI `file_search` tool
|
||||
|
||||
Upload the collection catalog as a file to OpenAI, create a vector store, and use the Responses API's built-in `file_search` tool. OpenAI automatically performs semantic search over the file and includes relevant results inline in the response stream — no SSE event handling changes, no orchestration loop.
|
||||
|
||||
Token savings: ~9,000 → ~100 tokens per chat (99% reduction). File search results consume ~200-500 tokens only when the model actually searches.
|
||||
|
||||
Research and alternative analysis: see [ML-156 Research document](backlog://document/doc-1).
|
||||
|
||||
---
|
||||
|
||||
### Phase 1: OpenAI API extensions
|
||||
|
||||
Add to `OpenAI.API` (following existing `new_request/1` pattern with `Req.RateLimiter` on `:open_ai` bucket):
|
||||
|
||||
- `upload_file(file_content, config)` — `POST /v1/files` with multipart body, `purpose: "assistants"`
|
||||
- `create_vector_store(name, config)` — `POST /v1/vector_stores`
|
||||
- `add_file_to_vector_store(store_id, file_id, config)` — `POST /v1/vector_stores/{id}/files`
|
||||
- `delete_file(file_id, config)` — `DELETE /v1/files/{id}` (cleanup on re-upload)
|
||||
|
||||
~60 lines.
|
||||
|
||||
### Phase 2: File management module
|
||||
|
||||
Create `MusicLibrary.Chats.CollectionChat.FileStore`:
|
||||
|
||||
```
|
||||
defmodule MusicLibrary.Chats.CollectionChat.FileStore do
|
||||
@moduledoc """
|
||||
Manages the collection catalog file lifecycle at OpenAI.
|
||||
Persists file_id and vector_store_id via Secrets.
|
||||
"""
|
||||
|
||||
@spec ensure_uploaded() :: {:ok, String.t()} | {:error, term()}
|
||||
def ensure_uploaded do
|
||||
# Check Secrets for existing file_id + vector_store_id
|
||||
# If missing, call Collection.collection_summary/0
|
||||
# Upload to OpenAI, create vector store, attach file
|
||||
# Persist IDs via Secrets.store/2
|
||||
# Return {:ok, vector_store_id}
|
||||
end
|
||||
|
||||
@spec refresh() :: :ok | {:error, term()}
|
||||
def refresh do
|
||||
# Delete old file from OpenAI (if exists)
|
||||
# Regenerate summary, upload new file
|
||||
# Attach to existing vector store (re-indexes automatically)
|
||||
end
|
||||
|
||||
@spec get_vector_store_id() :: {:ok, String.t()} | {:error, :not_uploaded}
|
||||
def get_vector_store_id do
|
||||
# Read vector_store_id from Secrets
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
- `ensure_uploaded/0` — lazy init on first chat; idempotent
|
||||
- `refresh/0` — called when records are added/edited/deleted (async, non-blocking)
|
||||
- `get_vector_store_id/0` — reads from `Secrets`; returns error if never uploaded
|
||||
|
||||
~50 lines.
|
||||
|
||||
### Phase 3: Chat streaming changes
|
||||
|
||||
**`OpenAI.chat_stream/2`** — add optional `vector_store_ids` option:
|
||||
|
||||
```elixir
|
||||
def chat_stream(messages, opts) do
|
||||
model = Keyword.get(opts, :model, "gpt-4.1")
|
||||
vector_store_ids = Keyword.get(opts, :vector_store_ids, [])
|
||||
|
||||
tools = [%{type: "web_search_preview"}]
|
||||
tools = if vector_store_ids != [],
|
||||
do: [%{type: "file_search", vector_store_ids: vector_store_ids} | tools],
|
||||
else: tools
|
||||
|
||||
# ... rest unchanged
|
||||
end
|
||||
```
|
||||
|
||||
**`OpenAI.API.chat_stream/6`** — accept tools as parameter instead of hardcoding (or add `tools` parameter):
|
||||
|
||||
```elixir
|
||||
def chat_stream(messages, instructions, model, temperature, config, cb, tools \\ nil) do
|
||||
tools = tools || [%{type: "web_search_preview"}]
|
||||
# ... use tools in json body
|
||||
end
|
||||
```
|
||||
|
||||
**`CollectionChat.stream_response/3`:**
|
||||
|
||||
```elixir
|
||||
def stream_response(messages, {_summary, record_count}, callback) do
|
||||
instructions = build_instructions(record_count)
|
||||
|
||||
vector_store_opts = case FileStore.get_vector_store_id() do
|
||||
{:ok, id} -> [vector_store_ids: [id]]
|
||||
{:error, _} -> [] # fall back to stats-only
|
||||
end
|
||||
|
||||
OpenAI.chat_stream(messages, [
|
||||
on_chunk: callback,
|
||||
instructions: instructions,
|
||||
model: "gpt-5.1"
|
||||
] ++ vector_store_opts)
|
||||
end
|
||||
```
|
||||
|
||||
~15 lines.
|
||||
|
||||
### Phase 4: Prompt changes
|
||||
|
||||
Update `CollectionChat.build_instructions/2`:
|
||||
|
||||
```elixir
|
||||
defp build_instructions(record_count) do
|
||||
Prompt.build("""
|
||||
Answer questions about the user's music collection.
|
||||
The collection contains #{record_count} records.
|
||||
Use file search to find specific records when the user asks about \
|
||||
artists, albums, genres, or formats in their collection.
|
||||
|
||||
# Mentioning artists/albums
|
||||
|
||||
**IF YOU MENTION AN ARTIST NAME OR ALBUM NAME, wrap it in "[[name]]", \
|
||||
for example "[[Steven Wilson]]"
|
||||
""")
|
||||
end
|
||||
```
|
||||
|
||||
Key changes:
|
||||
- Remove `#{collection_summary}` interpolation entirely
|
||||
- Remove `Collection catalog:` section
|
||||
- Remove `Use the provided collection catalog as your primary reference`
|
||||
- Add guidance to use file search for specific record lookup
|
||||
- `stream_response/3` signature changes from `{summary, count}` to just `count` (summary only used for stats, which we no longer pass)
|
||||
|
||||
~10 lines.
|
||||
|
||||
### Phase 5: Trigger refresh on collection changes
|
||||
|
||||
In `CollectionLive.Index`, handle record add/edit/delete events:
|
||||
|
||||
```elixir
|
||||
# In handle_info for RecordForm saved / AddRecord imported / delete:
|
||||
def handle_info({MusicLibraryWeb.Components.RecordForm, {:saved, _record}}, socket) do
|
||||
Task.start(&MusicLibrary.Chats.CollectionChat.FileStore.refresh/0)
|
||||
IndexActions.handle_record_saved(socket)
|
||||
end
|
||||
```
|
||||
|
||||
Or use the existing PubSub topic `"records:#{id}"` to trigger refresh from a central place. Debounce rapid changes (multiple quick adds) with a short timer.
|
||||
|
||||
~20 lines.
|
||||
|
||||
### Phase 6: Testing
|
||||
|
||||
- `test/open_ai/api_test.exs` — test `upload_file`, `create_vector_store`, `add_file_to_vector_store`, `delete_file` endpoints via `Req.Test` stubs
|
||||
- `test/music_library/chats/collection_chat/file_store_test.exs` — test `ensure_uploaded` (first-time upload), `refresh` (re-upload), `get_vector_store_id` (missing, present), and `Secrets` persistence
|
||||
- `test/music_library/chats/collection_chat_test.exs` — verify instructions no longer contain catalog; verify `file_search` tool included when vector store is available; verify fallback to stats-only when unavailable
|
||||
- `test/music_library/collection_test.exs` — existing `collection_summary/0` tests continue to pass unchanged
|
||||
|
||||
Estimated total: ~150 lines across 5-6 files. Low risk — no changes to core streaming infrastructure.
|
||||
<!-- SECTION:PLAN:END -->
|
||||
|
||||
## Definition of Done
|
||||
<!-- DOD:BEGIN -->
|
||||
- [ ] #1 All new and modified modules have @moduledoc
|
||||
- [ ] #2 All public functions have @spec and @doc
|
||||
- [ ] #3 Mix compile --warnings-as-errors passes
|
||||
- [ ] #4 mix test passes with no failures
|
||||
- [ ] #5 mix format --check-formatted passes
|
||||
- [ ] #6 mix credo passes
|
||||
- [ ] #7 Documentation updated (architecture.md if new modules/schemas added)
|
||||
- [ ] #8 Commit subject references ML-156
|
||||
<!-- DOD:END -->
|
||||
Reference in New Issue
Block a user