music_library/.agents/skills/external-api-integration/SKILL.md

name, description

name

description

external-api-integration

Use this skill when adding, modifying, extending, or debugging ANY external API integration (MusicBrainz, Last.fm, Discogs, Wikipedia, OpenAI, BraveSearch, Mailgun). Use PROACTIVELY when the user mentions "API", "integration", "HTTP client", "rate limit", "Req", "MusicBrainz", "Last.fm", "Discogs", "Wikipedia", "OpenAI", "Brave", "webhook", "external service", "third-party API", "error response", or asks about API patterns. Also use when adding new API calls to existing integrations or creating worker modules for API operations.

External API Integration

Project-specific patterns for external API integrations. Every integration follows a consistent three-module architecture with rate limiting, structured error handling, and test stubbing.

Checklist Before Adding or Modifying an API Integration

1. Does an ErrorResponse module exist? Every API MUST have a <API>.API.ErrorResponse module implementing MusicLibrary.ErrorResponse behaviour.

2. Is rate limiting configured? Every API uses Req.RateLimiter with a per-API interval. Check the architecture doc for existing intervals.

3. Are test stubs in place? All HTTP calls must be stubbed via config/test.exs using Req.Test. Use fixture modules for response data.

4. Does the worker use ErrorHandler? Workers that call APIs must route errors through MusicLibrary.Worker.ErrorHandler.to_oban_result/1.

Three-Module Architecture

Every external API integration follows this pattern:

lib/music_library/<service>/
├── <service>.ex           # Facade: public API, delegates to API module
├── api.ex                 # API: Req HTTP client, rate-limited
├── api/
│   ├── config.ex          # Config: NimbleOptions schema, reads from app env
│   ├── error_response.ex  # ErrorResponse: implements MusicLibrary.ErrorResponse behaviour
│   └── fixtures.ex        # Fixtures: test response data (for API-specific fixtures)

Facade Module (<Service>.ex)

The public interface. No HTTP logic here — delegates to the API module:

defmodule MusicLibrary.LastFm do
  @spec get_recent_tracks(keyword()) :: {:ok, [LastFm.Track.t()]} | {:error, term()}
  def get_recent_tracks(opts \\ []) do
    LastFm.API.get_recent_tracks(opts)
  end
end

API Module (api.ex)

Req-based HTTP client. Uses Req.RateLimiter via the request pipeline:

defmodule MusicLibrary.LastFm.API do
  @spec get_recent_tracks(keyword()) :: {:ok, [LastFm.Track.t()]} | {:error, term()}
  def get_recent_tracks(opts) do
    config = LastFm.API.Config.new!(opts)

    Req.new(
      base_url: "https://ws.audioscrobbler.com/2.0",
      params: [api_key: config.api_key, format: "json"],
      plugins: [
        {Req.RateLimiter, id: :last_fm}
      ]
    )
    |> Req.get(url: "/", params: [method: "user.getRecentTracks", user: config.user])
    |> case do
      {:ok, %{status: 200, body: body}} -> {:ok, parse_tracks(body)}
      {:ok, %{status: status, body: body}} -> {:error, LastFm.API.ErrorResponse.new(status, body)}
      {:error, error} -> {:error, error}
    end
  end
end

Key patterns:

• Config goes first — build it from opts, validate with NimbleOptions
• Rate limiter as Req plugin — {Req.RateLimiter, id: :service_name}
• Non-200 status codes are wrapped in ErrorResponse structs, not raw maps
• Req-level errors (timeouts, DNS) are passed through as-is
• Successful responses are parsed into domain structs

Config Module (api/config.ex)

NimbleOptions schema, reads from application env:

defmodule MusicLibrary.LastFm.API.Config do
  @schema [
    api_key: [type: :string, required: true],
    shared_secret: [type: :string, required: false],
    user: [type: :string, required: false]
  ]

  def new!(opts) do
    env = Application.get_env(:music_library, :last_fm, [])

    opts
    |> Keyword.merge(env)
    |> NimbleOptions.validate!(@schema)
  end
end

ErrorResponse Behaviour

Every API must have an ErrorResponse module implementing:

defmodule MusicLibrary.ErrorResponse do
  @type t :: struct()
  @callback retryable?(t()) :: boolean()
  @callback retry_delay_seconds(t()) :: non_neg_integer() | nil
end

Example implementation:

defmodule MusicLibrary.LastFm.API.ErrorResponse do
  defstruct [:status, :body, :kind]

  @behaviour MusicLibrary.ErrorResponse

  def new(status, body) do
    %__MODULE__{
      status: status,
      body: body,
      kind: MusicLibrary.HttpError.kind(status)  # baseline mapping
    }
  end

  @impl true
  def retryable?(%{kind: :rate_limit}), do: true    # 429
  def retryable?(%{kind: :server_error}), do: true   # 5xx
  def retryable?(_), do: false

  @impl true
  def retry_delay_seconds(%{status: 429, body: %{"error" => _, "message" => msg}}) do
    MusicLibrary.RetryDelay.parse(msg)  # parse Retry-After header
  end

  def retry_delay_seconds(_), do: nil
end

HTTP Error Kind Mapping (MusicLibrary.HttpError)

Baseline mapping used by all APIs:

HTTP status	Kind
429	:rate_limit
5xx	:server_error
Timeout	:timeout
401/403	:auth_error
404	:not_found
4xx	:client_error
Other	:unknown

Per-API overrides: Some APIs have quirks that override the baseline:

• MusicBrainz: HTTP 503 is their rate-limit signal (not a server error)
• OpenAI: HTTP 429 with body code: "insufficient_quota" is :auth_error (permanent), not :rate_limit

Override in the ErrorResponse module by pattern-matching on status + body before falling through to MusicLibrary.HttpError.kind/1.

Rate Limiting

Intervals (from architecture doc)

API	Interval
MusicBrainz	1000 ms
Last.fm	500 ms
Discogs	2000 ms
Wikipedia	1000 ms
BraveSearch	1000 ms
OpenAI	250 ms

Req.RateLimiter

ETS-backed, per-API enforcement via Req request step. Configured per-request:

Req.new(plugins: [{Req.RateLimiter, id: :music_brainz}])

The :id atom corresponds to the Req.RateLimiter ETS table entry. In tests, use Req.RateLimiter.Clock behaviour with SystemClock implementation.

Worker Error Handling

Workers that call APIs follow this flow:

defmodule MusicLibrary.Worker.FetchArtistInfo do
  use Oban.Worker

  @impl true
  def perform(%{args: %{"musicbrainz_id" => mbid}}) do
    case Artists.fetch_info(mbid) do
      {:ok, _artist} ->
        :ok

      {:error, :no_english_wikipedia} ->
        # App-layer permanent failures — match first
        {:cancel, :no_english_wikipedia}

      {:error, reason} ->
        # Forward to ErrorHandler for structured HTTP error handling
        MusicLibrary.Worker.ErrorHandler.to_oban_result(reason)
    end
  end
end

ErrorHandler.to_oban_result/1:

• %ErrorResponse{retryable?: true} → {:snooze, seconds} (Oban snooze + retry)
• %ErrorResponse{retryable?: false} → {:cancel, reason} (permanent)
• Other errors → {:error, reason} (Oban default retry)

Worker Return Values

Return	Meaning	Oban Behaviour
:ok	Success	Job completed
{:error, reason}	Transient failure	Default retry with backoff
{:snooze, seconds}	Retryable with delay	Retry after seconds
{:cancel, reason}	Permanent failure	Job cancelled, never retried

Never use {:discard, reason} — it's deprecated. Use {:cancel, reason}.

Test Stubs (Req.Test)

All API HTTP calls are stubbed in config/test.exs:

# config/test.exs
config :req, plug: {
  Req.Test,
  json_plug: &YourApp.JsonPlug.json/1
}

Each API's fixture module provides realistic response data. When adding a new API call:

1. Add stub in config/test.exs matching the request pattern
2. Use fixture data from the appropriate fixtures module
3. Cover error cases: rate limit (429), auth error (401/403), not found (404), server error (5xx)

# Example: stubbing Last.fm with fixture
Req.Test.stub(LastFm.API, fn conn ->
  case conn.method do
    "GET" -> Req.Test.json(conn, LastFm.Fixtures.RecentTracks.get())
  end
end)

Available API Fixture Modules

Module	API
MusicBrainz.ReleaseFixtures	MusicBrainz release data
MusicBrainz.ReleaseGroupFixtures	MusicBrainz release group data
MusicBrainz.ArtistFixtures	MusicBrainz artist data
Discogs.ArtistFixtures	Discogs artist profiles
LastFm.ArtistFixtures	Last.fm artist info
LastFm.Fixtures.RecentTracks	Last.fm scrobble history
Wikipedia.Fixtures	Wikipedia extracts

Adding a New API Integration

Step-by-step:

1. Create directory lib/music_library/<service>/
2. Create Config (api/config.ex) — NimbleOptions schema
3. Create ErrorResponse (api/error_response.ex) — implement behaviour, handle quirks
4. Create API (api.ex) — Req client with rate limiter plugin
5. Create Facade (<service>.ex) — public functions, delegates to API
6. Register rate limiter in config/config.exs with interval
7. Add test stubs in config/test.exs
8. Create fixture module in test/support/fixtures/
9. Update architecture doc — add to External API Integrations table
10. Add worker (if needed) — thin wrapper with ErrorHandler