feat: move the whole metadata/search stack from iTunes to Deezer (1.3)

Apple's iTunes /search returns 403 from server/datacenter IPs, which had broken global
search, charts, album/artist/single downloads and the metadata-editor autofill in
production. Everything now sources metadata from Deezer (free, no key, more requests,
returns explicit versions + real artist images + durations). Web + Android share one
source, so an artist resolves to one consistent name everywhere. Deezer results are
mapped to the legacy "ItunesItem" shape, so the UI + download flow are unchanged.

Backend:
- New backend/lib/deezer.js: lookupTrackMeta, fetchAlbumTracks, searchAlbumTracks,
  fetchArtistAlbums (+ pure pickTrackMatch/mapAlbumTrack for tests).
- metadata.js drops the iTunes lookups; novasonic-server download paths (track, album,
  artist, playlist enrich, spotify artwork) + autofill now use Deezer.
- Artist download uses Deezer discography (real albums only), not the iTunes album search.
- Editor autofill endpoint /api/itunes/lookup -> /api/meta/lookup; removed the dead
  iTunes /search proxy and the Apple charts RSS proxy.
- backend/test/deezer.test.js: unit tests for numbering, mixed-artist album-artist
  stability, explicit flags, the Japanese-title match guard (npm test; RUN_LIVE=1 for live).

Web:
- api.js search/lookups already on Deezer; charts now use Deezer /chart. Artist cards
  show the real Deezer photo. Visible "iTunes"/"Apple Music" labels -> "Deezer"
  (search badge, charts badge, preview bar, editor button, artist heading). SW ns-v51.

Android:
- New DeezerModels.kt with Dz* models + mappers to ItunesItem/ItunesRssEntry.
- ItunesClient is Deezer-backed (search = tracks+albums+artists in parallel; album/artist
  lookups; /chart charts), same method signatures. Visible strings -> Deezer.

Docs: README, CLAUDE.md and CHANGELOG (1.3) updated; Platform Parity Matrix reflects the
shared Deezer source + Android non-local artist page.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
HoneyandClaude Opus 4.8 committed 2026-07-31 12:48:33 +02:00
1 parent a65613b5a1
commit 1e69831dd8
19 files changed
+628 -350

No files matched your search

+27 -17
View File
@@ -20,14 +20,15 @@ plus a **Node.js/Express backend** that downloads tracks via yt-dlp and proxies
## Layout
- `src/` — web PWA (vanilla ES modules). State in `src/state.js` (the `S` object);
Subsonic/iTunes API in `src/api.js`; views in `src/views/`, UI in `src/ui/`.
Subsonic/Deezer API in `src/api.js`; views in `src/views/`, UI in `src/ui/`.
- **Client-side router** (`src/views/navigation.js`, History API): `navigate()` pushStates `{nsIdx}`
only on genuine new navigation; `popstate` applies the target entry's `nsIdx` for **both** back and
forward. Never `pushState` inside the `popstate`/back path — it wipes the browser's forward stack
(that was the "forward doesn't work" bug). In-app back buttons call `history.back()` (one path).
- `backend/novasonic-server.js` — Express API (downloads, metadata edit/delete, events/ABS proxy).
- `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`,
`metadata.js` (iTunes lookup, file resolution helpers), `config.js`, `jobs.js`, `romaji.js`.
`deezer.js` (Deezer metadata: track/album/artist lookups + matching),
`metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`.
- `android/` — native Kotlin app (separate; backend-dependent web features like the
metadata editor are **not** in the Android app).
@@ -46,12 +47,19 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
## Download pipeline (hard-won knowledge)
- Downloads are **yt-dlp only** (the old squid.wtf/Qobuz backend was removed).
- **iTunes store = US everywhere** (`api.js` search, `metadata.js` `lookupItunesTrackMeta` /
`searchItunesAlbumTracks`). Was JP-first, which localized Western artists to katakana ("DJスネイク")
on some tracks and kept kanji on others → the *same* artist resolved differently between downloads.
US-only gives one consistent name per artist and halves the download's iTunes calls. Titles come
from the source, so a Japanese title the user picked stays Japanese; only iTunes-derived fields
(album\_artist, genre, year, artwork) follow the US store.
- **Metadata source = Deezer, not iTunes** (as of 1.3). Apple's iTunes `/search` returns **403 from
server/datacenter IPs**, which broke search + downloads + charts + editor autofill in production.
All metadata now comes from Deezer (`backend/lib/deezer.js` server-side; `_deezerBase()` in
`src/api.js`; Android `ItunesClient` is Deezer-backed). Free, no key, more requests, returns
explicit versions + real artist images + durations. Deezer generally gives one consistent name per
artist, so the old JP/US-store inconsistency (Western artists as katakana "DJスネイク" on some tracks,
kanji on others) is gone. Titles come from the source, so a Japanese title the user picked stays
Japanese; only Deezer-derived fields (album\_artist, genre, year, artwork) follow Deezer. The
Deezer results are mapped to the legacy "ItunesItem" shape, so the UI + download flow are unchanged.
`lookupTrackMeta`/`fetchAlbumTracks`/`searchAlbumTracks`/`fetchArtistAlbums` replace the old
`lookupItunesTrackMeta`/`fetchItunesTracks`/`searchItunesAlbumTracks`. Editor autofill endpoint:
`/api/meta/lookup` (was `/api/itunes/lookup`). Matching logic (`pickTrackMatch`) is unit-tested in
`backend/test/deezer.test.js` (`npm test`).
- **File naming:** `{NN - }Artist - Title.ext` with the release suffix stripped from the
title (no `- Single`/`- EP`). The **album folder keeps the full album name**. Original names
are used for folder, filename, and all main tags (artist/title/album/album\_artist) — even
@@ -74,8 +82,8 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
- **Size cap:** `--max-filesize 30M` on every yt-dlp call; Mittelweg steps also cap at 2×
expected duration. Prevents saving a 28-min video (52 MB) when the wrong result slips through.
- **Leading aesthetic characters:** artists often style their name with leading punctuation
(`.diedlonely`, `†…`). `metadata.js` strips leading non-alphanumerics (`stripLeadSym`) before the
iTunes lookup and in `firstWord`. Without it the lookup misses, falls through to a title-only
(`.diedlonely`, `†…`). `deezer.js` strips leading non-alphanumerics (`stripLeadSym`) before the
Deezer lookup and in `firstWord`. Without it the lookup misses, falls through to a title-only
match, and returns an unrelated same-titled track — whose wrong duration then *also* misleads the
YouTube duration filter (observed: a 96 s track resolved to a 176 s slowed version). The same trim
runs client-side in Android's `ExploreViewModel.search()`.
@@ -85,10 +93,11 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
so without this every track on a soundtrack searched the album-artist blob + its channel filter and
matched nothing (e.g. "Strategy" = TWICE, "Love, Maybe" = MeloMance failed). The `album_artist` tag
still uses `albumArtist`, so album grouping is unaffected.
- **Album/artist download `album_artist`** comes from the iTunes tracks' `collectionArtistName`
(album-level, identical across the album), derived in `novasonic-server.js` — **not** the request's
`artist`, which can be the lead single's performer (a KPop soundtrack got tagged `album_artist=TWICE`
from track 1 "TAKEDOWN"). Falls back to the request artist only when iTunes omits it.
- **Album/artist download `album_artist`** comes from the Deezer tracks' `collectionArtistName`
(= the album-level `album.artist`, identical across the album), derived in `novasonic-server.js` —
**not** the request's `artist`, which can be the lead single's performer (a KPop soundtrack got
tagged `album_artist=TWICE` from track 1 "TAKEDOWN"). Falls back to the request artist only when
Deezer omits it.
- **Always write an `album` tag.** `ytdlp.js` defaults it to `Singles` when no album is known.
Previously no tag was written at all, so Navidrome showed its own "Singles" fallback while the
metadata editor showed "[Unknown Album]" — the two never matched and file resolution failed.
@@ -103,7 +112,7 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
or a stale stem. `metadata.js` rebases the path and falls back to recursive/stem matching
(tolerant of release-suffix mismatch). A ghost Navidrome entry needs a full library rescan.
- **Album splitting:** prevented by consistent `album_artist` tag. All tracks in a download
share the same `mainArtist` value (= iTunes `collectionArtistName` when available, else
share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else
`albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`.
## Android offline system
@@ -138,8 +147,9 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
`HttpClient` for *every cover art request* — this was the cause of visible scroll/tab-switch lag.
- **Star/favorite is optimistic and global** via `data/StarState.kt`. Toggle through it so every icon
(song rows, NowPlaying, context sheet) flips instantly and rolls back if the API call fails.
- **iTunes artists have no artwork.** The Search API returns no `artworkUrl100` for artist entities;
borrow the cover of one of their albums (`entity=album&limit=1`) — otherwise you get grey circles.
- **Deezer artists carry a real photo** (`picture_*`), so artist search cards show it directly — no
album-cover borrowing needed. (The old iTunes artist entities had no `artworkUrl100`, which is why
the `entity=album&limit=1` album-cover fallback exists; it now only runs if a picture is missing.)
## UI / references