feat: move the whole metadata/search stack from iTunes to Deezer (1.3)
Apple's iTunes /search returns 403 from server/datacenter IPs, which had broken global search, charts, album/artist/single downloads and the metadata-editor autofill in production. Everything now sources metadata from Deezer (free, no key, more requests, returns explicit versions + real artist images + durations). Web + Android share one source, so an artist resolves to one consistent name everywhere. Deezer results are mapped to the legacy "ItunesItem" shape, so the UI + download flow are unchanged. Backend: - New backend/lib/deezer.js: lookupTrackMeta, fetchAlbumTracks, searchAlbumTracks, fetchArtistAlbums (+ pure pickTrackMatch/mapAlbumTrack for tests). - metadata.js drops the iTunes lookups; novasonic-server download paths (track, album, artist, playlist enrich, spotify artwork) + autofill now use Deezer. - Artist download uses Deezer discography (real albums only), not the iTunes album search. - Editor autofill endpoint /api/itunes/lookup -> /api/meta/lookup; removed the dead iTunes /search proxy and the Apple charts RSS proxy. - backend/test/deezer.test.js: unit tests for numbering, mixed-artist album-artist stability, explicit flags, the Japanese-title match guard (npm test; RUN_LIVE=1 for live). Web: - api.js search/lookups already on Deezer; charts now use Deezer /chart. Artist cards show the real Deezer photo. Visible "iTunes"/"Apple Music" labels -> "Deezer" (search badge, charts badge, preview bar, editor button, artist heading). SW ns-v51. Android: - New DeezerModels.kt with Dz* models + mappers to ItunesItem/ItunesRssEntry. - ItunesClient is Deezer-backed (search = tracks+albums+artists in parallel; album/artist lookups; /chart charts), same method signatures. Visible strings -> Deezer. Docs: README, CLAUDE.md and CHANGELOG (1.3) updated; Platform Parity Matrix reflects the shared Deezer source + Android non-local artist page. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
a65613b5a1
commit
1e69831dd8
19 files changed
+628
-350
No files matched your search
@@ -20,14 +20,15 @@ plus a **Node.js/Express backend** that downloads tracks via yt-dlp and proxies
|
||||
## Layout
|
||||
|
||||
- `src/` — web PWA (vanilla ES modules). State in `src/state.js` (the `S` object);
|
||||
Subsonic/iTunes API in `src/api.js`; views in `src/views/`, UI in `src/ui/`.
|
||||
Subsonic/Deezer API in `src/api.js`; views in `src/views/`, UI in `src/ui/`.
|
||||
- **Client-side router** (`src/views/navigation.js`, History API): `navigate()` pushStates `{nsIdx}`
|
||||
only on genuine new navigation; `popstate` applies the target entry's `nsIdx` for **both** back and
|
||||
forward. Never `pushState` inside the `popstate`/back path — it wipes the browser's forward stack
|
||||
(that was the "forward doesn't work" bug). In-app back buttons call `history.back()` (one path).
|
||||
- `backend/novasonic-server.js` — Express API (downloads, metadata edit/delete, events/ABS proxy).
|
||||
- `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`,
|
||||
`metadata.js` (iTunes lookup, file resolution helpers), `config.js`, `jobs.js`, `romaji.js`.
|
||||
`deezer.js` (Deezer metadata: track/album/artist lookups + matching),
|
||||
`metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`.
|
||||
- `android/` — native Kotlin app (separate; backend-dependent web features like the
|
||||
metadata editor are **not** in the Android app).
|
||||
|
||||
@@ -46,12 +47,19 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
|
||||
## Download pipeline (hard-won knowledge)
|
||||
|
||||
- Downloads are **yt-dlp only** (the old squid.wtf/Qobuz backend was removed).
|
||||
- **iTunes store = US everywhere** (`api.js` search, `metadata.js` `lookupItunesTrackMeta` /
|
||||
`searchItunesAlbumTracks`). Was JP-first, which localized Western artists to katakana ("DJスネイク")
|
||||
on some tracks and kept kanji on others → the *same* artist resolved differently between downloads.
|
||||
US-only gives one consistent name per artist and halves the download's iTunes calls. Titles come
|
||||
from the source, so a Japanese title the user picked stays Japanese; only iTunes-derived fields
|
||||
(album\_artist, genre, year, artwork) follow the US store.
|
||||
- **Metadata source = Deezer, not iTunes** (as of 1.3). Apple's iTunes `/search` returns **403 from
|
||||
server/datacenter IPs**, which broke search + downloads + charts + editor autofill in production.
|
||||
All metadata now comes from Deezer (`backend/lib/deezer.js` server-side; `_deezerBase()` in
|
||||
`src/api.js`; Android `ItunesClient` is Deezer-backed). Free, no key, more requests, returns
|
||||
explicit versions + real artist images + durations. Deezer generally gives one consistent name per
|
||||
artist, so the old JP/US-store inconsistency (Western artists as katakana "DJスネイク" on some tracks,
|
||||
kanji on others) is gone. Titles come from the source, so a Japanese title the user picked stays
|
||||
Japanese; only Deezer-derived fields (album\_artist, genre, year, artwork) follow Deezer. The
|
||||
Deezer results are mapped to the legacy "ItunesItem" shape, so the UI + download flow are unchanged.
|
||||
`lookupTrackMeta`/`fetchAlbumTracks`/`searchAlbumTracks`/`fetchArtistAlbums` replace the old
|
||||
`lookupItunesTrackMeta`/`fetchItunesTracks`/`searchItunesAlbumTracks`. Editor autofill endpoint:
|
||||
`/api/meta/lookup` (was `/api/itunes/lookup`). Matching logic (`pickTrackMatch`) is unit-tested in
|
||||
`backend/test/deezer.test.js` (`npm test`).
|
||||
- **File naming:** `{NN - }Artist - Title.ext` with the release suffix stripped from the
|
||||
title (no `- Single`/`- EP`). The **album folder keeps the full album name**. Original names
|
||||
are used for folder, filename, and all main tags (artist/title/album/album\_artist) — even
|
||||
@@ -74,8 +82,8 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
|
||||
- **Size cap:** `--max-filesize 30M` on every yt-dlp call; Mittelweg steps also cap at 2×
|
||||
expected duration. Prevents saving a 28-min video (52 MB) when the wrong result slips through.
|
||||
- **Leading aesthetic characters:** artists often style their name with leading punctuation
|
||||
(`.diedlonely`, `†…`). `metadata.js` strips leading non-alphanumerics (`stripLeadSym`) before the
|
||||
iTunes lookup and in `firstWord`. Without it the lookup misses, falls through to a title-only
|
||||
(`.diedlonely`, `†…`). `deezer.js` strips leading non-alphanumerics (`stripLeadSym`) before the
|
||||
Deezer lookup and in `firstWord`. Without it the lookup misses, falls through to a title-only
|
||||
match, and returns an unrelated same-titled track — whose wrong duration then *also* misleads the
|
||||
YouTube duration filter (observed: a 96 s track resolved to a 176 s slowed version). The same trim
|
||||
runs client-side in Android's `ExploreViewModel.search()`.
|
||||
@@ -85,10 +93,11 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
|
||||
so without this every track on a soundtrack searched the album-artist blob + its channel filter and
|
||||
matched nothing (e.g. "Strategy" = TWICE, "Love, Maybe" = MeloMance failed). The `album_artist` tag
|
||||
still uses `albumArtist`, so album grouping is unaffected.
|
||||
- **Album/artist download `album_artist`** comes from the iTunes tracks' `collectionArtistName`
|
||||
(album-level, identical across the album), derived in `novasonic-server.js` — **not** the request's
|
||||
`artist`, which can be the lead single's performer (a KPop soundtrack got tagged `album_artist=TWICE`
|
||||
from track 1 "TAKEDOWN"). Falls back to the request artist only when iTunes omits it.
|
||||
- **Album/artist download `album_artist`** comes from the Deezer tracks' `collectionArtistName`
|
||||
(= the album-level `album.artist`, identical across the album), derived in `novasonic-server.js` —
|
||||
**not** the request's `artist`, which can be the lead single's performer (a KPop soundtrack got
|
||||
tagged `album_artist=TWICE` from track 1 "TAKEDOWN"). Falls back to the request artist only when
|
||||
Deezer omits it.
|
||||
- **Always write an `album` tag.** `ytdlp.js` defaults it to `Singles` when no album is known.
|
||||
Previously no tag was written at all, so Navidrome showed its own "Singles" fallback while the
|
||||
metadata editor showed "[Unknown Album]" — the two never matched and file resolution failed.
|
||||
@@ -103,7 +112,7 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
|
||||
or a stale stem. `metadata.js` rebases the path and falls back to recursive/stem matching
|
||||
(tolerant of release-suffix mismatch). A ghost Navidrome entry needs a full library rescan.
|
||||
- **Album splitting:** prevented by consistent `album_artist` tag. All tracks in a download
|
||||
share the same `mainArtist` value (= iTunes `collectionArtistName` when available, else
|
||||
share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else
|
||||
`albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`.
|
||||
|
||||
## Android offline system
|
||||
@@ -138,8 +147,9 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
|
||||
`HttpClient` for *every cover art request* — this was the cause of visible scroll/tab-switch lag.
|
||||
- **Star/favorite is optimistic and global** via `data/StarState.kt`. Toggle through it so every icon
|
||||
(song rows, NowPlaying, context sheet) flips instantly and rolls back if the API call fails.
|
||||
- **iTunes artists have no artwork.** The Search API returns no `artworkUrl100` for artist entities;
|
||||
borrow the cover of one of their albums (`entity=album&limit=1`) — otherwise you get grey circles.
|
||||
- **Deezer artists carry a real photo** (`picture_*`), so artist search cards show it directly — no
|
||||
album-cover borrowing needed. (The old iTunes artist entities had no `artworkUrl100`, which is why
|
||||
the `entity=album&limit=1` album-cover fallback exists; it now only runs if a picture is missing.)
|
||||
|
||||
## UI / references
|
||||
|
||||
|
||||
Reference in new issue
Block a user