diff --git a/CHANGELOG.md b/CHANGELOG.md index 63bdf1e..02e6d4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,8 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an | Multi-artist navigation | ✅ | ✅ | | Album detail page | ✅ | ✅ | | Artist page (Navidrome) | ✅ | ✅ | -| Artist page (iTunes fallback) | ✅ | ❌ | +| Artist page (Deezer fallback) | ✅ | ✅ | +| Global search source | Deezer | Deezer | | Concert events (Ticketmaster) | ✅ | ✅ | | Genre browsing | ✅ | ✅ | | Global search | ✅ | ✅ | @@ -55,6 +56,29 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an --- +## [1.3] — 2026-07-31 + +Big change: **the entire metadata/search stack moved from Apple's iTunes to Deezer** — on web, backend downloads *and* Android. Apple's iTunes `/search` returns **403 from server/datacenter IPs**, which had broken global search, charts, artist/album/single downloads and the metadata-editor autofill in production. Deezer is free, needs no key, allows more requests, and returns explicit versions + real artist images + track durations. Web and Android now share the **same** source, so an artist resolves to one consistent name everywhere. + +### Changed +- **Web + Android + Backend** — Global search, album/artist lookups, artist top tracks and the "Neu & Beliebt" charts all run on **Deezer** (`api.deezer.com`). Results are mapped to the existing item shape, so the UI is unchanged; artist cards now show real photos (iTunes artist entities had none) +- **Backend (download)** — Single-track, album and **artist** downloads, plus the metadata-editor autofill, now enrich from Deezer: + - `backend/lib/deezer.js` — new module: `lookupTrackMeta`, `fetchAlbumTracks`, `searchAlbumTracks`, `fetchArtistAlbums` + - Album/artist `album_artist` still comes from the album-level artist (Deezer `album.artist`), keeping soundtracks from splitting + - Genre comes from the Deezer album (one album-level genre); per-track artist is preserved for mixed-artist albums + - The autofill endpoint moved `/api/itunes/lookup` → **`/api/meta/lookup`** +- **Web + Android** — All user-visible "iTunes" / "Apple Music" labels are now "Deezer" (search badge, charts badge, preview bar, editor autofill button, artist "populär" heading) +- **Backend** — Removed the dead iTunes `/search` proxy and the Apple charts RSS proxy; the Deezer proxy (`/api/deezer/*`) now also serves charts and the editor lookup + +### Added +- **Backend** — Unit tests for the Deezer mapping/matching logic (`backend/test/deezer.test.js`, run with `npm test`), incl. per-track numbering, mixed-artist album-artist stability, explicit flags and the Japanese-title match guard. A live smoke test runs with `RUN_LIVE=1` + +### Notes +- Deploy: `git pull`, then rebuild the frontend (`docker compose up -d --build novasonic-frontend`) and restart the backend (`docker compose restart novasonic-backend`). Service worker bumped to `ns-v51` +- Titles a user picked stay as chosen; only derived fields (album, album-artist, genre, year, artwork) follow Deezer + +--- + ## [1.2] — 2026-07-25 Follow-up to 1.1: web history navigation, correct download tagging for mixed-artist albums, and UI polish. diff --git a/CLAUDE.md b/CLAUDE.md index 0cba2b4..b4e4196 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -20,14 +20,15 @@ plus a **Node.js/Express backend** that downloads tracks via yt-dlp and proxies ## Layout - `src/` — web PWA (vanilla ES modules). State in `src/state.js` (the `S` object); - Subsonic/iTunes API in `src/api.js`; views in `src/views/`, UI in `src/ui/`. + Subsonic/Deezer API in `src/api.js`; views in `src/views/`, UI in `src/ui/`. - **Client-side router** (`src/views/navigation.js`, History API): `navigate()` pushStates `{nsIdx}` only on genuine new navigation; `popstate` applies the target entry's `nsIdx` for **both** back and forward. Never `pushState` inside the `popstate`/back path — it wipes the browser's forward stack (that was the "forward doesn't work" bug). In-app back buttons call `history.back()` (one path). - `backend/novasonic-server.js` — Express API (downloads, metadata edit/delete, events/ABS proxy). - `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`, - `metadata.js` (iTunes lookup, file resolution helpers), `config.js`, `jobs.js`, `romaji.js`. + `deezer.js` (Deezer metadata: track/album/artist lookups + matching), + `metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`. - `android/` — native Kotlin app (separate; backend-dependent web features like the metadata editor are **not** in the Android app). @@ -46,12 +47,19 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a ## Download pipeline (hard-won knowledge) - Downloads are **yt-dlp only** (the old squid.wtf/Qobuz backend was removed). -- **iTunes store = US everywhere** (`api.js` search, `metadata.js` `lookupItunesTrackMeta` / - `searchItunesAlbumTracks`). Was JP-first, which localized Western artists to katakana ("DJスネイク") - on some tracks and kept kanji on others → the *same* artist resolved differently between downloads. - US-only gives one consistent name per artist and halves the download's iTunes calls. Titles come - from the source, so a Japanese title the user picked stays Japanese; only iTunes-derived fields - (album\_artist, genre, year, artwork) follow the US store. +- **Metadata source = Deezer, not iTunes** (as of 1.3). Apple's iTunes `/search` returns **403 from + server/datacenter IPs**, which broke search + downloads + charts + editor autofill in production. + All metadata now comes from Deezer (`backend/lib/deezer.js` server-side; `_deezerBase()` in + `src/api.js`; Android `ItunesClient` is Deezer-backed). Free, no key, more requests, returns + explicit versions + real artist images + durations. Deezer generally gives one consistent name per + artist, so the old JP/US-store inconsistency (Western artists as katakana "DJスネイク" on some tracks, + kanji on others) is gone. Titles come from the source, so a Japanese title the user picked stays + Japanese; only Deezer-derived fields (album\_artist, genre, year, artwork) follow Deezer. The + Deezer results are mapped to the legacy "ItunesItem" shape, so the UI + download flow are unchanged. + `lookupTrackMeta`/`fetchAlbumTracks`/`searchAlbumTracks`/`fetchArtistAlbums` replace the old + `lookupItunesTrackMeta`/`fetchItunesTracks`/`searchItunesAlbumTracks`. Editor autofill endpoint: + `/api/meta/lookup` (was `/api/itunes/lookup`). Matching logic (`pickTrackMatch`) is unit-tested in + `backend/test/deezer.test.js` (`npm test`). - **File naming:** `{NN - }Artist - Title.ext` with the release suffix stripped from the title (no `- Single`/`- EP`). The **album folder keeps the full album name**. Original names are used for folder, filename, and all main tags (artist/title/album/album\_artist) — even @@ -74,8 +82,8 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a - **Size cap:** `--max-filesize 30M` on every yt-dlp call; Mittelweg steps also cap at 2× expected duration. Prevents saving a 28-min video (52 MB) when the wrong result slips through. - **Leading aesthetic characters:** artists often style their name with leading punctuation - (`.diedlonely`, `†…`). `metadata.js` strips leading non-alphanumerics (`stripLeadSym`) before the - iTunes lookup and in `firstWord`. Without it the lookup misses, falls through to a title-only + (`.diedlonely`, `†…`). `deezer.js` strips leading non-alphanumerics (`stripLeadSym`) before the + Deezer lookup and in `firstWord`. Without it the lookup misses, falls through to a title-only match, and returns an unrelated same-titled track — whose wrong duration then *also* misleads the YouTube duration filter (observed: a 96 s track resolved to a 176 s slowed version). The same trim runs client-side in Android's `ExploreViewModel.search()`. @@ -85,10 +93,11 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a so without this every track on a soundtrack searched the album-artist blob + its channel filter and matched nothing (e.g. "Strategy" = TWICE, "Love, Maybe" = MeloMance failed). The `album_artist` tag still uses `albumArtist`, so album grouping is unaffected. -- **Album/artist download `album_artist`** comes from the iTunes tracks' `collectionArtistName` - (album-level, identical across the album), derived in `novasonic-server.js` — **not** the request's - `artist`, which can be the lead single's performer (a KPop soundtrack got tagged `album_artist=TWICE` - from track 1 "TAKEDOWN"). Falls back to the request artist only when iTunes omits it. +- **Album/artist download `album_artist`** comes from the Deezer tracks' `collectionArtistName` + (= the album-level `album.artist`, identical across the album), derived in `novasonic-server.js` — + **not** the request's `artist`, which can be the lead single's performer (a KPop soundtrack got + tagged `album_artist=TWICE` from track 1 "TAKEDOWN"). Falls back to the request artist only when + Deezer omits it. - **Always write an `album` tag.** `ytdlp.js` defaults it to `Singles` when no album is known. Previously no tag was written at all, so Navidrome showed its own "Singles" fallback while the metadata editor showed "[Unknown Album]" — the two never matched and file resolution failed. @@ -103,7 +112,7 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a or a stale stem. `metadata.js` rebases the path and falls back to recursive/stem matching (tolerant of release-suffix mismatch). A ghost Navidrome entry needs a full library rescan. - **Album splitting:** prevented by consistent `album_artist` tag. All tracks in a download - share the same `mainArtist` value (= iTunes `collectionArtistName` when available, else + share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else `albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`. ## Android offline system @@ -138,8 +147,9 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a `HttpClient` for *every cover art request* — this was the cause of visible scroll/tab-switch lag. - **Star/favorite is optimistic and global** via `data/StarState.kt`. Toggle through it so every icon (song rows, NowPlaying, context sheet) flips instantly and rolls back if the API call fails. -- **iTunes artists have no artwork.** The Search API returns no `artworkUrl100` for artist entities; - borrow the cover of one of their albums (`entity=album&limit=1`) — otherwise you get grey circles. +- **Deezer artists carry a real photo** (`picture_*`), so artist search cards show it directly — no + album-cover borrowing needed. (The old iTunes artist entities had no `artworkUrl100`, which is why + the `entity=album&limit=1` album-cover fallback exists; it now only runs if a picture is missing.) ## UI / references diff --git a/README.md b/README.md index 12935ae..dce77c9 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,9 @@ Built with Vite + vanilla ES modules (web) and Jetpack Compose + Media3/ExoPlaye - **Library** — Stream from Navidrome / Subsonic; browse artists, albums, playlists, genres, starred, and random; search; star + context menu on every row. - **Playback** — Queue with shuffle, repeat, and radio mode (endless similar songs); gapless-style preload; lock-screen / notification controls; Navidrome scrobbling. - **Lyrics** — Synced and plain lyrics via lrclib.net, auto-fetched and cached per session. -- **Artist pages** — Hero card, Last.fm bio / similar / top songs, Ticketmaster events, and iTunes cross-reference showing owned vs. missing tracks. +- **Artist pages** — Hero card, Last.fm bio / similar / top songs, Ticketmaster events, and a Deezer cross-reference showing owned vs. missing tracks. - **Playlists** — Create, rename, delete, add/remove tracks, and upload a cover (Web + Android). -- **Importer & downloads** — Paste a Spotify / YouTube / YouTube Music / SoundCloud link to download via yt-dlp, with iTunes metadata enrichment and duplicate detection. +- **Importer & downloads** — Paste a Spotify / YouTube / YouTube Music / SoundCloud link to download via yt-dlp, with Deezer metadata enrichment and duplicate detection. - **Metadata editor** (web) — Edit tags + cover art; files are renamed/moved and Navidrome is rescanned on save. - **Offline** (Android) — Download albums, playlists, or single songs to private storage; an Offline tab to play and manage them (works without a server connection); downloaded tracks play from disk on every screen. - **Audiobookshelf** — Browse and stream audiobooks with chapter navigation, speed control, sleep timer, and progress sync. @@ -51,20 +51,21 @@ NovaSonic/ │ │ └── settings.js # Settings page │ ├── state.js # Shared mutable state (S object) │ ├── utils.js # Helpers (md5, esc, formatDur, toast) -│ ├── api.js # Subsonic API, iTunes API, cover art +│ ├── api.js # Subsonic API, Deezer API, cover art │ └── main.js # App init, audio wiring, window globals ├── backend/ │ ├── novasonic-server.js # Express API: downloads, metadata, events proxy, ABS proxy │ └── lib/ │ ├── config.js # Env var → config object │ ├── jobs.js # In-memory download job store -│ ├── metadata.js # iTunes lookup, artwork, filename helpers +│ ├── deezer.js # Deezer metadata: track/album/artist lookups +│ ├── metadata.js # Artwork, filename helpers, duplicate detection │ ├── ytdlp.js # yt-dlp download + ffmpeg metadata embedding │ ├── downloader.js # Unified downloadTrack / downloadAlbumTracks │ └── romaji.js # Japanese → romaji search index ├── android/ # Native Android app (Jetpack Compose + Media3) │ └── app/src/main/kotlin/com/novasonic/app/ -│ ├── api/ # SubsonicClient, AbsClient, LrcLibClient, ItunesClient +│ ├── api/ # SubsonicClient, AbsClient, LrcLibClient, ItunesClient (Deezer-backed) │ ├── data/ # PrefsStore (DataStore), MusicRepository, StarState │ ├── offline/ # Room DB, DAO, OfflineRepository, DownloadWorker (offline downloads) │ ├── player/ # PlayerController, MusicService (Media3) diff --git a/android/app/src/main/kotlin/com/novasonic/app/api/ItunesClient.kt b/android/app/src/main/kotlin/com/novasonic/app/api/ItunesClient.kt index 45351ee..46b8070 100644 --- a/android/app/src/main/kotlin/com/novasonic/app/api/ItunesClient.kt +++ b/android/app/src/main/kotlin/com/novasonic/app/api/ItunesClient.kt @@ -8,64 +8,81 @@ import io.ktor.client.plugins.contentnegotiation.* import io.ktor.client.request.* import io.ktor.http.ContentType import io.ktor.serialization.kotlinx.json.* +import kotlinx.coroutines.async +import kotlinx.coroutines.coroutineScope import kotlinx.serialization.json.Json +/** + * Metadata/search client. Backed by **Deezer** (api.deezer.com) — Apple's iTunes /search + * returns 403 from datacenter IPs, and Deezer keeps the phone + web PWA on the *same* source + * (identical naming + explicit versions). Deezer responses are mapped to the legacy ItunesItem + * shape (see DeezerModels.kt) so the screens/view models did not change. The public method names + * are kept for the same reason. + */ object ItunesClient { + private const val BASE = "https://api.deezer.com" + private val http = HttpClient(Android) { install(ContentNegotiation) { - // iTunes returns text/javascript instead of application/json — accept any content type json(Json { ignoreUnknownKeys = true; coerceInputValues = true }, contentType = ContentType.Any) } } + /** + * Combined search. Deezer splits by type, so `entity` "album" hits /search/album; anything else + * runs tracks + albums + artists in parallel and merges them (the view model filters by wrapperType). + */ suspend fun search( query: String, entity: String = "musicTrack,album,musicArtist", limit: Int = 30, country: String = "us", - ): ItunesSearchResponse = http.get("https://itunes.apple.com/search") { - parameter("term", query) - parameter("media", "music") - parameter("entity", entity) - parameter("limit", limit) - parameter("country", country) - }.body() + ): ItunesSearchResponse { + if (entity == "album") { + val albums: DzList = http.get("$BASE/search/album") { + parameter("q", query); parameter("limit", limit) + }.body() + return ItunesSearchResponse(results = albums.data.map { it.toItunesItem() }) + } + return coroutineScope { + val tracksD = async { runCatching { http.get("$BASE/search") { parameter("q", query); parameter("limit", limit) }.body>().data }.getOrDefault(emptyList()) } + val albumsD = async { runCatching { http.get("$BASE/search/album") { parameter("q", query); parameter("limit", 12) }.body>().data }.getOrDefault(emptyList()) } + val artistsD = async { runCatching { http.get("$BASE/search/artist") { parameter("q", query); parameter("limit", 8) }.body>().data }.getOrDefault(emptyList()) } + val results = artistsD.await().map { it.toItunesItem() } + + albumsD.await().map { it.toItunesItem() } + + tracksD.await().map { it.toItunesItem() } + ItunesSearchResponse(results = results) + } + } + /** All tracks of a Deezer album, in the ItunesItem song shape. */ suspend fun lookupAlbum(collectionId: Long): List { - val resp: ItunesSearchResponse = http.get("https://itunes.apple.com/lookup") { - parameter("id", collectionId) - parameter("entity", "song") - }.body() - // First result is the album itself; rest are tracks - return resp.results.filter { it.kind == "song" } + val album: DzAlbum = http.get("$BASE/album/$collectionId").body() + return album.tracks?.data.orEmpty().mapIndexed { i, t -> t.toAlbumTrackItem(album, i) } } - /** All albums for an iTunes artist */ + /** A Deezer artist's real albums (not singles/EPs). */ suspend fun lookupArtistAlbums(artistId: Long): List { - val resp: ItunesSearchResponse = http.get("https://itunes.apple.com/lookup") { - parameter("id", artistId) - parameter("entity", "album") - parameter("limit", 200) + val albums: DzList = http.get("$BASE/artist/$artistId/albums") { + parameter("limit", 200) }.body() - return resp.results.filter { it.wrapperType == "collection" } + return albums.data.filter { it.record_type == "album" }.map { it.toItunesItem() } } - /** Worldwide top tracks for an iTunes artist */ + /** Top tracks for a Deezer artist. */ suspend fun lookupArtistTopTracks(artistId: Long, limit: Int = 10): List { - val resp: ItunesSearchResponse = http.get("https://itunes.apple.com/lookup") { - parameter("id", artistId) - parameter("entity", "song") // "musicTrack" only works on /search — lookup needs "song" - parameter("limit", limit + 1) - parameter("explicit", "Yes") + val tracks: DzList = http.get("$BASE/artist/$artistId/top") { + parameter("limit", limit) }.body() - return resp.results.filter { it.wrapperType == "track" && it.kind == "song" }.take(limit) + return tracks.data.map { it.toItunesItem() } } - /** Top albums RSS feed for a given country */ + /** "Neu & Beliebt" — Deezer's global album chart, mapped to the RSS-entry shape. */ suspend fun topAlbums(country: String = "us", limit: Int = 25): List { - val resp: ItunesRssResponse = - http.get("https://itunes.apple.com/$country/rss/topalbums/limit=$limit/json").body() - return resp.feed?.entry ?: emptyList() + val albums: DzList = http.get("$BASE/chart/0/albums") { + parameter("limit", limit) + }.body() + return albums.data.map { it.toRssEntry() } } } diff --git a/android/app/src/main/kotlin/com/novasonic/app/api/model/DeezerModels.kt b/android/app/src/main/kotlin/com/novasonic/app/api/model/DeezerModels.kt new file mode 100644 index 0000000..a0ee4b4 --- /dev/null +++ b/android/app/src/main/kotlin/com/novasonic/app/api/model/DeezerModels.kt @@ -0,0 +1,129 @@ +package com.novasonic.app.api.model + +import kotlinx.serialization.Serializable + +// ─── DEEZER API MODELS ──────────────────────────────────────────────────────── +// Apple's iTunes /search returns 403 from server IPs, so the Explore/search source is +// Deezer (free, no key, returns explicit flags + artist images + durations). Deezer JSON is +// mapped to the existing ItunesItem/ItunesRssEntry shapes so the screens + view models are +// unchanged. See ItunesClient for the endpoint calls. + +@Serializable +data class DzList(val data: List = emptyList()) + +@Serializable +data class DzArtist( + val id: Long? = null, + val name: String? = null, + val picture_medium: String? = null, + val picture_big: String? = null, + val picture_xl: String? = null, +) + +@Serializable +data class DzGenre(val name: String? = null) + +@Serializable +data class DzGenres(val data: List = emptyList()) + +@Serializable +data class DzAlbum( + val id: Long? = null, + val title: String? = null, + val cover_medium: String? = null, + val cover_big: String? = null, + val cover_xl: String? = null, + val release_date: String? = null, + val nb_tracks: Int? = null, + val record_type: String? = null, + val genres: DzGenres? = null, + val artist: DzArtist? = null, + val tracks: DzList? = null, + val position: Int? = null, +) + +@Serializable +data class DzTrack( + val id: Long? = null, + val title: String? = null, + val duration: Long? = null, + val track_position: Int? = null, + val disk_number: Int? = null, + val explicit_lyrics: Boolean = false, + val preview: String? = null, + val artist: DzArtist? = null, + val album: DzAlbum? = null, +) + +// ─── MAPPERS to the legacy ItunesItem shape ─────────────────────────────────── + +private fun DzAlbum.coverXl(): String? = cover_xl ?: cover_big ?: cover_medium +private fun DzArtist.pictureXl(): String? = picture_xl ?: picture_big ?: picture_medium + +/** A Deezer search-result track → ItunesItem (album cover for artwork). */ +fun DzTrack.toItunesItem(): ItunesItem = ItunesItem( + wrapperType = "track", + kind = "song", + trackId = id, + artistId = artist?.id, + collectionId = album?.id, + artistName = artist?.name, + collectionName = album?.title, + trackName = title, + artworkUrl100 = album?.coverXl() ?: artist?.pictureXl(), + previewUrl = preview, + trackTimeMillis = duration?.let { it * 1000 }, + trackNumber = track_position, +) + +/** A Deezer album → ItunesItem collection. */ +fun DzAlbum.toItunesItem(): ItunesItem = ItunesItem( + wrapperType = "collection", + kind = "album", + collectionId = id, + artistId = artist?.id, + artistName = artist?.name, + collectionName = title, + artworkUrl100 = coverXl(), + trackCount = nb_tracks, + releaseDate = release_date, + primaryGenreName = genres?.data?.firstOrNull()?.name, +) + +/** A Deezer artist → ItunesItem artist (carries a real picture, unlike iTunes). */ +fun DzArtist.toItunesItem(): ItunesItem = ItunesItem( + wrapperType = "artist", + artistId = id, + artistName = name, + artworkUrl100 = pictureXl(), +) + +/** + * A Deezer album track (from /album/{id}) → ItunesItem song. The embedded tracks carry no + * track_position, so the album ordering supplies the number; genre/release come from the album. + */ +fun DzTrack.toAlbumTrackItem(album: DzAlbum, index: Int): ItunesItem = ItunesItem( + wrapperType = "track", + kind = "song", + trackId = id, + artistId = artist?.id ?: album.artist?.id, + collectionId = album.id, + artistName = artist?.name ?: album.artist?.name, + collectionName = album.title, + trackName = title, + artworkUrl100 = album.coverXl(), + previewUrl = preview, + trackTimeMillis = duration?.let { it * 1000 }, + trackNumber = track_position ?: (index + 1), + trackCount = album.nb_tracks, + releaseDate = album.release_date, + primaryGenreName = album.genres?.data?.firstOrNull()?.name, +) + +/** A Deezer chart album → ItunesRssEntry (keeps the charts UI unchanged). */ +fun DzAlbum.toRssEntry(): ItunesRssEntry = ItunesRssEntry( + name = ItunesLabel(title), + artist = ItunesLabel(artist?.name), + image = listOfNotNull(coverXl()?.let { ItunesLabel(it) }), + id = ItunesId(attributes = ItunesIdAttributes(imId = id?.toString())), +) diff --git a/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ArtistDetailScreen.kt b/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ArtistDetailScreen.kt index 46901a7..be7e211 100644 --- a/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ArtistDetailScreen.kt +++ b/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ArtistDetailScreen.kt @@ -364,7 +364,7 @@ fun ArtistDetailScreen( color = NsAccent, modifier = Modifier.size(16.dp), strokeWidth = 2.dp, ) Spacer(Modifier.width(10.dp)) - Text("iTunes Daten werden geladen…", fontFamily = Outfit, + Text("Deezer Daten werden geladen…", fontFamily = Outfit, fontSize = 12.sp, color = NsOnSurface2) } } diff --git a/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ItunesArtistScreen.kt b/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ItunesArtistScreen.kt index 7a693c7..8bd3000 100644 --- a/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ItunesArtistScreen.kt +++ b/android/app/src/main/kotlin/com/novasonic/app/ui/screen/ItunesArtistScreen.kt @@ -231,7 +231,7 @@ fun ItunesArtistScreen( // iTunes top tracks if (artistState.iTracks.isNotEmpty()) { - item { ArtistSectionHeader("Weltweit populär (iTunes)") } + item { ArtistSectionHeader("Weltweit populär (Deezer)") } itemsIndexed(artistState.iTracks, key = { _, t -> t.trackId ?: 0L }) { _, track -> ItunesArtistTrackRow( track = track, diff --git a/backend/lib/deezer.js b/backend/lib/deezer.js new file mode 100644 index 0000000..d377147 --- /dev/null +++ b/backend/lib/deezer.js @@ -0,0 +1,163 @@ +const https = require('https'); + +// ─── DEEZER METADATA SOURCE ─────────────────────────────────────────────────── +// Apple's iTunes /search returns 403 from server/datacenter IPs, so all server-side +// metadata (single-track enrichment, album track lists, artist discography, the +// metadata-editor autofill) comes from Deezer. Free, no key, ~50 req/5s, returns +// explicit flags + artist images + durations. Results are mapped to the same +// "ItunesItem" shape the download pipeline (downloadAlbumTracks / downloadTrack) +// already consumes, so nothing downstream had to change. + +const DZ = 'https://api.deezer.com'; + +function dzGet(subpath) { + return new Promise((resolve, reject) => { + https.get(`${DZ}/${subpath}`, { headers: { 'User-Agent': 'NovaSonic/1.0' } }, res => { + let d = ''; + res.on('data', c => d += c); + res.on('end', () => { try { resolve(JSON.parse(d) || {}); } catch { reject(new Error('Deezer parse error')); } }); + }).on('error', reject); + }); +} + +// ─── matching helpers (local copies to avoid a circular require with metadata.js) ─ +function normalizeTitle(s) { + return (s || '') + .toLowerCase().trim() + .replace(/\s+-\s+([^-]+)$/, ' ($1)') + .replace(/[\[((][^\]))]*[\]))]/g, '') + .replace(/[,.'!?]/g, '') + .replace(/\s+/g, ' ') + .trim(); +} +const stripLeadSym = s => (s || '').replace(/^[^\p{L}\p{N}]+/u, '').trim(); +const firstWord = s => stripLeadSym((s || '').toLowerCase()).split(/[\s,;/]/)[0]; +const yearOf = d => { const y = d ? new Date(d).getFullYear() : NaN; return Number.isFinite(y) ? y : undefined; }; +// Deezer covers come in fixed sizes; cover_xl is 1000x1000 (matches the iTunes hi-res artwork). +const coverXl = a => a?.cover_xl || a?.cover_big || a?.cover_medium || a?.cover || ''; + +// Map one album track (from /album/{id} .tracks.data — these carry no track_position, +// so the album's own ordering supplies the number) to the iTunes track shape. +function mapAlbumTrack(t, al, index) { + return { + wrapperType: 'track', + trackId: t.id, + trackName: t.title, + artistName: t.artist?.name || al.artist?.name || '', // per-track artist (mixed albums) + collectionName: al.title, + collectionArtistName: al.artist?.name || '', // album-level artist (stable) + collectionType: al.record_type === 'compilation' ? 'Compilation' : 'Album', + artworkUrl100: coverXl(al), + releaseDate: al.release_date || undefined, + primaryGenreName: al.genres?.data?.[0]?.name || undefined, + trackNumber: index + 1, + trackCount: al.nb_tracks || undefined, + trackTimeMillis: t.duration ? t.duration * 1000 : undefined, + trackExplicitness: t.explicit_lyrics ? 'explicit' : 'notExplicit', + previewUrl: t.preview || '', + }; +} + +// Pick the best track from a Deezer /search result list for (artist, title). Pure + exported +// so the matching rules are unit-testable. Strict = same title AND same first artist word; +// falls back to a title-only match, but never for Japanese titles (same title, different artist +// is common for anime/OST). Returns the matched Deezer track or null. +function pickTrackMatch(results, artist, title) { + if (!Array.isArray(results) || !results.length) return null; + const normT = normalizeTitle(title); + const normA = stripLeadSym((artist || '').split(/[;,&]/)[0].trim()).toLowerCase().trim(); + const strict = results.find(t => normalizeTitle(t.title) === normT && firstWord(t.artist?.name) === firstWord(normA)); + const JP_RE = /[぀-ヿ㐀-䶿一-鿿]/; + const isJp = JP_RE.test(normT) || JP_RE.test(normA); + const titleOnly = results.find(t => normalizeTitle(t.title) === normT); + return strict || (!isJp ? titleOnly || null : null); +} + +// Single-track metadata — the Deezer equivalent of the old iTunes lookupTrackMeta. +// Fills album/year/trackNum/genre/duration/artwork when the caller only has artist+title. +// Returns {} when nothing convincing matches (callers treat that as "no enrichment"). +async function lookupTrackMeta(artist, title) { + try { + const baseTitle = title.replace(/\s*[-–]\s*\S[^-]*$/, '').trim() || title; + const searchTitle = baseTitle !== title ? baseTitle : title; + const searchArtist = stripLeadSym((artist || '').split(/[;,&]/)[0].trim()); + const q = encodeURIComponent(`${searchArtist} ${searchTitle}`); + const results = (await dzGet(`search?q=${q}&limit=10`)).data || []; + if (!results.length) return {}; + + const match = pickTrackMatch(results, artist, title); + if (!match) return {}; + + // The search result lacks genre / track number / release date — the album endpoint carries them. + const al = match.album?.id ? await dzGet(`album/${match.album.id}`).catch(() => ({})) : {}; + // track_position + disk_number live on the full /track record, not on the search hit. + const tr = match.id ? await dzGet(`track/${match.id}`).catch(() => ({})) : {}; + + return { + album: match.album?.title || al.title || '', + albumArtist: al.artist?.name || undefined, + trackName: match.title || undefined, + year: yearOf(tr.release_date || al.release_date), + trackNum: tr.track_position || undefined, + totalTracks: al.nb_tracks || undefined, + discNum: tr.disk_number || undefined, + discTotal: undefined, + genre: al.genres?.data?.[0]?.name || undefined, + artworkUrl: coverXl(al) || coverXl(match.album) || '', + durationMs: (tr.duration || match.duration) ? (tr.duration || match.duration) * 1000 : undefined, + metaArtistName: match.artist?.name || undefined, + }; + } catch { return {}; } +} + +// All tracks of a Deezer album id, in the legacy ItunesItem track shape. +async function fetchAlbumTracks(albumId) { + try { + const al = await dzGet(`album/${albumId}`); + return (al.tracks?.data || []).map((t, i) => mapAlbumTrack(t, al, i)); + } catch { return []; } +} + +// Album fallback when no album id is known: search Deezer for the album, then its tracks. +async function searchAlbumTracks(artist, album) { + try { + const q = encodeURIComponent(`${artist} ${album}`); + const results = (await dzGet(`search/album?q=${q}&limit=5`)).data || []; + const lc = s => (s || '').toLowerCase(); + const match = results.find(r => + lc(r.title).includes(lc(album)) || lc(r.artist?.name).includes(lc(artist)) + ) || results[0]; + return match ? await fetchAlbumTracks(match.id) : []; + } catch { return []; } +} + +// Artist discography for the artist-download: search the artist, take real albums +// (record_type "album", not singles/EPs), return {collectionId, collectionName, artworkUrl100}. +async function fetchArtistAlbums(artistName, limit = 10) { + try { + const artists = (await dzGet(`search/artist?q=${encodeURIComponent(artistName)}&limit=1`)).data || []; + if (!artists.length) return []; + const albums = (await dzGet(`artist/${artists[0].id}/albums?limit=100`)).data || []; + return albums + .filter(a => a.record_type === 'album') + .slice(0, limit) + .map(a => ({ + collectionId: a.id, + collectionName: a.title, + artistName: artists[0].name, + artworkUrl100: coverXl(a), + })); + } catch { return []; } +} + +module.exports = { + dzGet, + lookupTrackMeta, + fetchAlbumTracks, + searchAlbumTracks, + fetchArtistAlbums, + // exported for unit tests + mapAlbumTrack, + pickTrackMatch, + normalizeTitle, +}; diff --git a/backend/lib/metadata.js b/backend/lib/metadata.js index 0aea128..ae0993d 100644 --- a/backend/lib/metadata.js +++ b/backend/lib/metadata.js @@ -17,7 +17,7 @@ function navidromeArtist(name) { .replace(/,\s*/g, ';'); } -// Download hi-res iTunes artwork (1000x1000) to cover.jpg, return path or null +// Download hi-res cover artwork (1000x1000) to cover.jpg, return path or null async function downloadArtwork(url, dir) { if (!url) return null; const dest = path.join(dir, 'cover.jpg'); @@ -112,133 +112,11 @@ async function fetchLyrics(artist, title, album) { } catch { return null; } } -// Normalize a track title for comparison: unify variant suffixes so that -// "don't rush - slowed" and "don't rush (slowed)" are treated as equal. -function normalizeTitle(s) { - return (s || '') - .toLowerCase() - .trim() - // convert "title - variant" → "title (variant)" so both forms compare equal - .replace(/\s+-\s+([^-]+)$/, ' ($1)') - // strip all remaining parens/brackets content for the base comparison - .replace(/[\[((][^\]))]*[\]))]/g, '') - // strip punctuation so "Mr. Lawrence" == "Mr Lawrence" and comma variants match - .replace(/[,.'!?]/g, '') - .replace(/\s+/g, ' ') - .trim(); -} - -// iTunes JP store returns localized genre names — map them to English equivalents -const JP_GENRE_MAP = { - 'アニメ': 'Anime', 'イージーリスニング': 'Easy Listening', 'ポップ': 'Pop', - 'J-Pop': 'J-Pop', 'ロック': 'Rock', 'クラシック': 'Classical', 'ジャズ': 'Jazz', - 'エレクトロニック': 'Electronic', 'R&B/ソウル': 'R&B/Soul', 'ヒップホップ/ラップ': 'Hip-Hop/Rap', - 'オルタナティブ': 'Alternative', 'サウンドトラック': 'Soundtrack', 'ボーカル': 'Vocal', - 'インストゥルメンタル': 'Instrumental', 'ワールド': 'World', 'フォーク': 'Folk', - 'カントリー': 'Country', 'レゲエ': 'Reggae', 'ソウル': 'Soul', 'ブルース': 'Blues', - 'メタル': 'Metal', 'パンク': 'Punk', 'ダンス': 'Dance', 'コメディ': 'Comedy', - 'スピーチ': 'Speech', 'キッズ/ファミリー': 'Kids/Family', -}; -function toEnglishGenre(g) { return JP_GENRE_MAP[g] || g; } - -// iTunes track lookup — fills in album, year, trackNum, totalTracks, genre when missing -async function lookupItunesTrackMeta(artist, title) { - try { - // Search with base title (strip variant suffix) for better iTunes hit rate — - // e.g. "don't rush - slowed" → search "don't rush" so iTunes finds the single - const baseTitle = title.replace(/\s*[-–]\s*\S[^-]*$/, '').trim() || title; - const searchTitle = baseTitle !== title ? baseTitle : title; - // Strip leading aesthetic punctuation many artists use (".diedlonely", "†", "✝", "_x") - // so the iTunes search actually surfaces them instead of a same-titled track by someone else. - const stripLeadSym = s => (s || '').replace(/^[^\p{L}\p{N}]+/u, '').trim(); - const searchArtist = stripLeadSym(artist.split(/[;,&]/)[0].trim()); - const fetchItunes = (country) => new Promise((resolve, reject) => { - const url = `https://itunes.apple.com/search?term=${encodeURIComponent(searchArtist + ' ' + searchTitle)}&media=music&entity=musicTrack&limit=10&country=${country}&explicit=Yes`; - https.get(url, res => { - let d = ''; res.on('data', c => d += c); - res.on('end', () => { try { resolve(JSON.parse(d).results || []); } catch { reject(new Error('parse')); } }); - }).on('error', reject); - }); - const normT = normalizeTitle(title); - const normA = (searchArtist || '').toLowerCase().trim(); - // Split by space OR comma/semicolon/slash so "VXLLAIN, iGRES" → "vxllain" not "vxllain," - const firstWord = s => stripLeadSym((s || '').toLowerCase()).split(/[\s,;/]/)[0]; - const strictMatch = results => results.find(t => - normalizeTitle(t.trackName) === normT && firstWord(t.artistName) === firstWord(normA) - ); - const titleOnlyMatch = results => results.find(t => normalizeTitle(t.trackName) === normT); - - // US store only — one consistent naming source (English/romaji) across search + download, so the - // same artist always resolves the same way instead of "sometimes Japanese, sometimes romaji". A - // single request instead of JP+US also halves the download's iTunes calls (helps the rate limit). - const usResults = await fetchItunes('us'); - const JP_RE = /[぀-ヿ㐀-䶿一-鿿]/; - const isJpSearch = JP_RE.test(normT) || JP_RE.test(normA); - // titleOnlyMatch (no artist check) is too permissive when the title is Japanese — common anime/ - // OST titles exist under different artists — so only use it as a fallback for non-Japanese titles. - const match = strictMatch(usResults) || (!isJpSearch && titleOnlyMatch(usResults)); - if (!match) return {}; - const naming = match; - const rawGenre = match.primaryGenreName; - const genre = rawGenre ? toEnglishGenre(rawGenre) : undefined; - return { - album: naming.collectionName || '', - albumArtist: naming.collectionArtistName || undefined, - trackName: naming.trackName || undefined, - year: match.releaseDate ? new Date(match.releaseDate).getFullYear() : undefined, - trackNum: match.trackNumber || undefined, - totalTracks: match.trackCount || undefined, - discNum: match.discNumber || undefined, - discTotal: match.discCount || undefined, - genre: genre, - artworkUrl: (naming.artworkUrl100 || match.artworkUrl100 || '').replace('100x100bb', '1000x1000bb'), - durationMs: match.trackTimeMillis || undefined, - itunesArtistName: naming.artistName || undefined, - }; - } catch { return {}; } -} - -// Fetch individual tracks for an iTunes collectionId -async function fetchItunesTracks(collectionId) { - return new Promise((resolve, reject) => { - const url = `https://itunes.apple.com/lookup?id=${collectionId}&entity=song&explicit=Yes`; - https.get(url, res => { - let data = ''; - res.on('data', d => data += d); - res.on('end', () => { - try { - const parsed = JSON.parse(data); - resolve((parsed.results || []).filter(r => r.wrapperType === 'track')); - } catch { reject(new Error('iTunes API parse error')); } - }); - }).on('error', reject); - }); -} - -// Search iTunes for album tracks by artist + album name (fallback when no collectionId) -async function searchItunesAlbumTracks(artist, album) { - return new Promise(resolve => { - const q = encodeURIComponent(`${artist} ${album}`); - https.get(`https://itunes.apple.com/search?term=${q}&media=music&entity=album&limit=5&country=us&explicit=Yes`, res => { - let d = ''; - res.on('data', c => d += c); - res.on('end', () => { - try { - const results = JSON.parse(d).results || []; - const match = results.find(r => - r.collectionName?.toLowerCase().includes(album.toLowerCase()) || - r.artistName?.toLowerCase().includes(artist.toLowerCase()) - ); - if (match) fetchItunesTracks(match.collectionId).then(resolve).catch(() => resolve([])); - else resolve([]); - } catch { resolve([]); } - }); - }).on('error', () => resolve([])); - }); -} +// Track/album/artist metadata now comes from Deezer (backend/lib/deezer.js) — Apple's +// iTunes /search returns 403 from server IPs. See that module for lookupTrackMeta, +// fetchAlbumTracks, searchAlbumTracks and fetchArtistAlbums. module.exports = { sanitize, navidromeArtist, downloadArtwork, findExistingTrack, - lookupItunesTrackMeta, fetchItunesTracks, searchItunesAlbumTracks, }; diff --git a/backend/novasonic-server.js b/backend/novasonic-server.js index 50006d3..08ccf32 100644 --- a/backend/novasonic-server.js +++ b/backend/novasonic-server.js @@ -17,7 +17,8 @@ const crypto = require('crypto'); const CONFIG = require('./lib/config'); const { jobs, updateJob, log } = require('./lib/jobs'); -const { sanitize, lookupItunesTrackMeta, fetchItunesTracks, searchItunesAlbumTracks, findExistingTrack } = require('./lib/metadata'); +const { sanitize, findExistingTrack } = require('./lib/metadata'); +const dz = require('./lib/deezer'); const { downloadTrack, downloadAlbumTracks } = require('./lib/downloader'); const { buildRomajiIndex, clearRomajiCache, isTokenizerReady, fetchNdSongs, romajiForStorage } = require('./lib/romaji'); @@ -274,7 +275,7 @@ app.post('/api/spotify/track-artwork', async (req, res) => { const batch = tracks.slice(i, i + CONC); const batchRes = await Promise.all(batch.map(async t => { const searchArtist = (t.artistName || '').split(/\s*[,/•]\s*/)[0].trim(); - const meta = await lookupItunesTrackMeta(searchArtist, t.trackName || ''); + const meta = await dz.lookupTrackMeta(searchArtist, t.trackName || ''); return { idx: t.idx, artworkUrl: meta.artworkUrl || t.existingArt || '' }; })); results.push(...batchRes); @@ -282,51 +283,36 @@ app.post('/api/spotify/track-artwork', async (req, res) => { res.json({ results }); }); -// ─── ITUNES METADATA LOOKUP (must be before the proxy route) ───────────────── -app.get('/api/itunes/lookup', async (req, res, next) => { +// ─── METADATA-EDITOR AUTOFILL ──────────────────────────────────────────────── +// Fills the editor from Deezer given artist + title (replaces the old iTunes autofill). +app.get('/api/meta/lookup', async (req, res) => { const { artist, title } = req.query; - // ID-based lookups (?id=...) are not for track meta — let them fall through to the proxy. - if (!artist || !title) return next(); + if (!artist || !title) return res.status(400).json({ error: 'artist and title required' }); try { - const meta = await lookupItunesTrackMeta(artist, title); + const meta = await dz.lookupTrackMeta(artist, title); if (!meta.album && !meta.genre && !meta.year) return res.json({ found: false }); res.json({ found: true, - title: meta.trackName || undefined, - artist: meta.itunesArtistName || undefined, - albumArtist: meta.albumArtist || undefined, - album: meta.album || undefined, - year: meta.year || undefined, - trackNum: meta.trackNum || undefined, - totalTracks: meta.totalTracks || undefined, - discNum: meta.discNum || undefined, - genre: meta.genre || undefined, - coverUrl: meta.artworkUrl || undefined, + title: meta.trackName || undefined, + artist: meta.metaArtistName || undefined, + albumArtist: meta.albumArtist || undefined, + album: meta.album || undefined, + year: meta.year || undefined, + trackNum: meta.trackNum || undefined, + totalTracks: meta.totalTracks || undefined, + discNum: meta.discNum || undefined, + genre: meta.genre || undefined, + coverUrl: meta.artworkUrl || undefined, }); } catch (e) { - console.error('[itunes/lookup]', e.message); + console.error('[meta/lookup]', e.message); res.status(500).json({ error: e.message }); } }); -// ─── ITUNES PROXY ───────────────────────────────────────────────────────────── -// Browsers can't call itunes.apple.com directly (no CORS headers). Proxy it here. -// Note: /api/itunes/lookup is handled above — proxy only covers /search and id-based /lookup. -app.get('/api/itunes/:endpoint', (req, res) => { - const allowed = new Set(['search', 'lookup']); - if (!allowed.has(req.params.endpoint)) return res.status(404).json({ error: 'not found' }); - const qs = new URLSearchParams(req.query).toString(); - const target = `https://itunes.apple.com/${req.params.endpoint}${qs ? '?' + qs : ''}`; - https.get(target, { headers: { 'User-Agent': 'NovaSonic/1.0' } }, upstream => { - res.setHeader('Content-Type', 'application/json'); - res.status(upstream.statusCode || 200); - upstream.pipe(res); - }).on('error', e => res.status(502).json({ error: e.message })); -}); - -// Deezer API proxy — the search source (Apple's /search returns 403 from server IPs). Deezer is -// free, needs no key, and returns explicit versions + artist images + track durations. Covers -// /search, /search/album, /search/artist, /album/{id}, /album/{id}/tracks, /artist/{id}/albums, … +// Deezer API proxy — the metadata source for search, charts, album/artist lookups (Apple's /search +// returns 403 from server IPs). Free, no key, returns explicit versions + artist images + durations. +// Covers /search, /search/album, /search/artist, /album/{id}, /artist/{id}/albums, /chart/…, etc. // Regex route (not '/api/deezer/*') — a bare "*" is an invalid path pattern in Express 5 and // crashes the server at startup, taking every endpoint down with it. app.get(/^\/api\/deezer\/(.+)/, (req, res) => { @@ -340,19 +326,6 @@ app.get(/^\/api\/deezer\/(.+)/, (req, res) => { }).on('error', e => res.status(502).json({ error: e.message })); }); -// Apple "most-played" charts RSS — the browser can't fetch rss.applemarketingtools.com directly -// (no CORS header), so proxy it. kind = albums | songs; country defaults to de. -app.get('/api/itunes-charts/:kind', (req, res) => { - const kind = req.params.kind === 'songs' ? 'songs' : 'albums'; - const country = String(req.query.country || 'de').replace(/[^a-z]/gi, '').slice(0, 2).toLowerCase() || 'de'; - const target = `https://rss.applemarketingtools.com/api/v2/${country}/music/most-played/50/${kind}.json`; - https.get(target, { headers: { 'User-Agent': 'NovaSonic/1.0' } }, upstream => { - res.setHeader('Content-Type', 'application/json'); - res.status(upstream.statusCode || 200); - upstream.pipe(res); - }).on('error', e => res.status(502).json({ error: e.message })); -}); - app.get('/api/events/:artist', (req, res) => { const key = CONFIG.ticketmasterKey; if (!key) { console.warn('[events] TICKETMASTER_API_KEY not set'); return res.json([]); } @@ -545,27 +518,27 @@ app.post('/api/youtube/resolve', async (req, res) => { artworkUrl: entries[0]?.thumbnail || '', tracks }); } - // Single video — enrich with iTunes + // Single video — enrich with Deezer const e = entries[0]; const rawArtist = _cleanYtArtist(e.uploader || e.channel || e.creator || ''); const title = e.title || ''; const sourceUrl = e.webpage_url || url; - let itunes = {}; - try { itunes = await lookupItunesTrackMeta(rawArtist.split(',')[0].trim(), title); } catch {} + let meta = {}; + try { meta = await dz.lookupTrackMeta(rawArtist.split(',')[0].trim(), title); } catch {} - const artist = itunes.artist || rawArtist; - const artworkUrl = itunes.artworkUrl || e.thumbnail || ''; + const artist = meta.artist || rawArtist; + const artworkUrl = meta.artworkUrl || e.thumbnail || ''; return res.json({ type: 'track', name: title, artist, artworkUrl, tracks: [{ trackName: title, artistName: artist, artworkUrl: artworkUrl, - collectionName: itunes.album || '', - releaseDate: itunes.year ? String(itunes.year) : '', - trackNumber: itunes.trackNum || undefined, - durationMs: itunes.durationMs || (e.duration ? Math.round(e.duration * 1000) : undefined), - genre: itunes.genre || '', + collectionName: meta.album || '', + releaseDate: meta.year ? String(meta.year) : '', + trackNumber: meta.trackNum || undefined, + durationMs: meta.durationMs || (e.duration ? Math.round(e.duration * 1000) : undefined), + genre: meta.genre || '', sourceUrl, }], }); @@ -609,19 +582,19 @@ app.post('/api/download', async (req, res) => { }; const searchArtist = artist.split(',')[0].trim(); if (!meta.album) { - log(jobId, '📋 Suche Album-Metadaten von iTunes…'); - const extra = await lookupItunesTrackMeta(searchArtist, title); + log(jobId, '📋 Suche Album-Metadaten von Deezer…'); + const extra = await dz.lookupTrackMeta(searchArtist, title); if (extra.album) log(jobId, `📋 Album: "${extra.album}" (${extra.year || '?'})`); meta = { ...meta, ...extra, artworkUrl: spArt || extra.artworkUrl || '', genre: meta.genre || extra.genre || undefined }; } else { log(jobId, `📋 Album: "${meta.album}"`); - // Always fetch iTunes when any field is missing — durationMs is required for the yt-dlp + // Always fetch Deezer when any field is missing — durationMs is required for the yt-dlp // duration filter; without it any unrelated video that matches the title text gets downloaded. if (!meta.genre || !meta.totalTracks || !meta.durationMs || !meta.trackNum || !meta.year) { - // Strip release-type suffixes before iTunes lookup so "recognize - Single" → "recognize" + // Strip release-type suffixes before the Deezer lookup so "recognize - Single" → "recognize" const RELEASE_SUFFIX_RE = /\s*[-–]\s*(?:(?:official\s+)?(?:music\s+)?(?:single|ep|lp|album|ost|soundtrack|remaster(?:ed)?|deluxe(?:\s+edition)?|bonus\s+tracks?|b-?sides?|version|edit)\b.*|(?:20|19)\d{2})\s*$/i; - const itunesTitle = title.replace(RELEASE_SUFFIX_RE, '').trim() || title; - const genreMeta = await lookupItunesTrackMeta(searchArtist, itunesTitle).catch(() => ({})); + const lookupTitle = title.replace(RELEASE_SUFFIX_RE, '').trim() || title; + const genreMeta = await dz.lookupTrackMeta(searchArtist, lookupTitle).catch(() => ({})); if (!meta.genre && genreMeta.genre) { meta.genre = genreMeta.genre; log(jobId, `📋 Genre: "${genreMeta.genre}"`); } if (!meta.year && genreMeta.year) { meta.year = genreMeta.year; log(jobId, `📋 Jahr: ${genreMeta.year}`); } if (!meta.totalTracks && genreMeta.totalTracks) { meta.totalTracks = genreMeta.totalTracks; } @@ -631,7 +604,7 @@ app.post('/api/download', async (req, res) => { if (!meta.trackNum && genreMeta.trackNum) { meta.trackNum = genreMeta.trackNum; } } } - const finalArtist = meta.itunesArtistName || artist; + const finalArtist = meta.metaArtistName || artist; const result = await downloadTrack({ jobId, artist: finalArtist, rawArtist: artist, title, ...meta, sourceUrl: sourceUrl || sp.sourceUrl || undefined }); if (result?.skipped) { updateJob(jobId, { status: 'done', progress: 100, message: 'Bereits vorhanden – übersprungen', skipped: true }); @@ -643,26 +616,26 @@ app.post('/api/download', async (req, res) => { } else if (type === 'album') { // Prefer a track list supplied in the request body (Deezer albums send their tracks directly, - // pre-mapped to the iTunes track shape). Fall back to the iTunes lookup for legacy callers. + // pre-mapped to the ItunesItem shape). Fall back to a Deezer lookup for legacy callers. let tracks = Array.isArray(req.body.tracks) ? req.body.tracks : []; if (tracks.length) log(jobId, `📋 ${tracks.length} Tracks aus Anfrage (Deezer)`); if (!tracks.length && collectionId) { - log(jobId, '📋 Lade Track-Liste von iTunes…'); - tracks = await fetchItunesTracks(collectionId); + log(jobId, '📋 Lade Track-Liste von Deezer…'); + tracks = await dz.fetchAlbumTracks(collectionId); } if (!tracks.length) { - log(jobId, '🔍 Suche Album auf iTunes…'); - tracks = await searchItunesAlbumTracks(artist, album || title); + log(jobId, '🔍 Suche Album auf Deezer…'); + tracks = await dz.searchAlbumTracks(artist, album || title); } if (!tracks.length) { - throw new Error(`Keine einzelnen Tracks für "${album || title}" gefunden. Bitte über iTunes Global-Suche herunterladen.`); + throw new Error(`Keine einzelnen Tracks für "${album || title}" gefunden. Bitte über die Global-Suche herunterladen.`); } - // Album artist: prefer iTunes' album-level collectionArtistName (identical across every track) + // Album artist: prefer the album-level collectionArtistName (identical across every track) // over the request's `artist`, which can be a single track's performer — e.g. the lead single // "TAKEDOWN" is by TWICE, so `artist` arrives as "TWICE" and would tag the whole soundtrack's // album_artist as TWICE. Keeps album_artist consistent so Navidrome doesn't split the album. const albumArtistName = tracks.map(t => t.collectionArtistName).find(Boolean) || artist; - if (albumArtistName !== artist) log(jobId, `👥 Album-Künstler: "${albumArtistName}" (iTunes)`); + if (albumArtistName !== artist) log(jobId, `👥 Album-Künstler: "${albumArtistName}" (Deezer)`); const { done: dlDone, skipped: dlSkipped } = await downloadAlbumTracks({ jobId, artist: albumArtistName, albumTitle: album || title, tracks, artworkUrl }); const allSkipped = dlSkipped === tracks.length; const someSkipped = dlSkipped > 0 && !allSkipped; @@ -675,19 +648,14 @@ app.post('/api/download', async (req, res) => { } else if (type === 'artist') { log(jobId, `🎤 Künstler-Download: ${artist}`); - const artistData = await new Promise((resolve, reject) => { - https.get(`https://itunes.apple.com/search?term=${encodeURIComponent(artist)}&media=music&entity=album&limit=10&country=de`, res => { - let d = ''; res.on('data', c => d += c); - res.on('end', () => { try { resolve(JSON.parse(d).results || []); } catch { reject(new Error('parse')); } }); - }).on('error', reject); - }); - const artistAlbums = artistData.filter(a => a.artistName?.toLowerCase() === artist.toLowerCase()).slice(0, 10); + // Deezer discography: search the artist, take their real albums (not singles/EPs). + const artistAlbums = await dz.fetchArtistAlbums(artist, 10).catch(() => []); log(jobId, `📀 ${artistAlbums.length} Alben gefunden`); let done = 0; for (const alb of artistAlbums) { - const tracks = await fetchItunesTracks(alb.collectionId).catch(() => []); + const tracks = await dz.fetchAlbumTracks(alb.collectionId).catch(() => []); if (tracks.length) { - // Same as the album path: album_artist from iTunes' album-level artist, so a collaboration + // Same as the album path: album_artist from the album-level artist, so a collaboration // or soundtrack in the artist's discography isn't tagged with the searched artist. const albArtist = tracks.map(t => t.collectionArtistName).find(Boolean) || alb.artistName || artist; await downloadAlbumTracks({ jobId, artist: albArtist, albumTitle: alb.collectionName, tracks, artworkUrl: alb.artworkUrl100 }); @@ -708,46 +676,46 @@ app.post('/api/download', async (req, res) => { const batchResults = await Promise.allSettled(batch.map(async track => { const trackTitle = track.trackName || ''; const trackArtist = track.artistName || artist || ''; - // Strip release-type suffixes before iTunes lookup ("Song - Single" → "Song") + // Strip release-type suffixes before the Deezer lookup ("Song - Single" → "Song") const RELEASE_SUFFIX_RE = /\s*[-–]\s*(?:(?:official\s+)?(?:music\s+)?(?:single|ep|lp|album|ost|soundtrack|remaster(?:ed)?|deluxe(?:\s+edition)?|bonus\s+tracks?|b-?sides?|version|edit)\b.*|(?:20|19)\d{2})\s*$/i; - const itunesTitle = trackTitle.replace(RELEASE_SUFFIX_RE, '').trim() || trackTitle; - // If the frontend already enriched this track with iTunes data (collectionName + + const lookupTitle = trackTitle.replace(RELEASE_SUFFIX_RE, '').trim() || trackTitle; + // If the frontend already enriched this track with Deezer data (collectionName + // trackNumber + durationMs all present), skip the re-lookup to avoid a different - // iTunes result overriding the already-correct enriched metadata. + // Deezer result overriding the already-correct enriched metadata. const alreadyEnriched = !!(track.collectionName && track.trackNumber && track.durationMs); const searchArtist = trackArtist.split(',')[0].trim(); - let itunes = {}; + let meta = {}; if (alreadyEnriched) { log(jobId, `📋 Frontend-Metadaten: "${trackArtist} – ${trackTitle}" (${track.collectionName})`); // Spotify enrichment copies album/trackNum/duration but not genre/totalTracks/disc — fetch those - const genreMeta = await lookupItunesTrackMeta(searchArtist, itunesTitle).catch(() => ({})); - itunes = { + const genreMeta = await dz.lookupTrackMeta(searchArtist, lookupTitle).catch(() => ({})); + meta = { genre: genreMeta.genre, totalTracks: genreMeta.totalTracks, discNum: genreMeta.discNum, discTotal: genreMeta.discTotal, }; } else { - log(jobId, `📋 iTunes: "${trackArtist} – ${trackTitle}"`); - itunes = await lookupItunesTrackMeta(searchArtist, itunesTitle); + log(jobId, `📋 Deezer: "${trackArtist} – ${trackTitle}"`); + meta = await dz.lookupTrackMeta(searchArtist, lookupTitle); } - // Prefer iTunes-resolved name (from server lookup or frontend enrichment) for sort tags / folders. + // Prefer the resolved name (from server lookup or frontend enrichment) for sort tags / folders. // trackArtist stays as rawArtist for main tags and YTM search. - const resolvedArtist = itunes.itunesArtistName || track.itunesArtistName || trackArtist; + const resolvedArtist = meta.metaArtistName || track.metaArtistName || trackArtist; return downloadTrack({ jobId, - artist: resolvedArtist, // iTunes English name → used for sort tags + folder names + artist: resolvedArtist, // resolved (Deezer) name → used for sort tags + folder names rawArtist: trackArtist, // original Spotify/Japanese name → used for main tag + YTM search albumArtist: resolvedArtist, title: trackTitle, - album: itunes.album || track.collectionName || track.album || '', - year: itunes.year || (track.releaseDate ? new Date(track.releaseDate).getFullYear() : undefined) || track.year, - trackNum: itunes.trackNum || track.trackNumber || undefined, - totalTracks: itunes.totalTracks || undefined, - genre: itunes.genre || track.primaryGenreName || track.genre || undefined, - artworkUrl: itunes.artworkUrl || track.artworkUrl100 || track.artworkUrl || artworkUrl || '', + album: meta.album || track.collectionName || track.album || '', + year: meta.year || (track.releaseDate ? new Date(track.releaseDate).getFullYear() : undefined) || track.year, + trackNum: meta.trackNum || track.trackNumber || undefined, + totalTracks: meta.totalTracks || undefined, + genre: meta.genre || track.primaryGenreName || track.genre || undefined, + artworkUrl: meta.artworkUrl || track.artworkUrl100 || track.artworkUrl || artworkUrl || '', explicit: track.trackExplicitness === 'explicit', - durationMs: itunes.durationMs || track.durationMs || undefined, + durationMs: meta.durationMs || track.durationMs || undefined, sourceUrl: track.sourceUrl || undefined, }); })); diff --git a/backend/package.json b/backend/package.json index d312528..6aea490 100644 --- a/backend/package.json +++ b/backend/package.json @@ -1,6 +1,9 @@ { "name": "novasonic-backend", "private": true, + "scripts": { + "test": "node --test" + }, "dependencies": { "express": "^4.18.0", "kuromoji": "^0.1.2" diff --git a/backend/test/deezer.test.js b/backend/test/deezer.test.js new file mode 100644 index 0000000..09456ee --- /dev/null +++ b/backend/test/deezer.test.js @@ -0,0 +1,109 @@ +'use strict'; +// Unit tests for the Deezer metadata mapping/matching (backend/lib/deezer.js). +// Pure logic only — no network. Run: `node --test` from backend/ (or `npm test`). +// A live smoke test against api.deezer.com is included but only runs with RUN_LIVE=1. + +const test = require('node:test'); +const assert = require('node:assert/strict'); +const dz = require('../lib/deezer'); + +// ── Fixtures: the exact Deezer shapes the mappers consume ───────────────────── +const ALBUM = { + id: 302127, + title: 'Discovery', + cover_xl: 'https://img/cover_xl.jpg', + cover_big: 'https://img/cover_big.jpg', + release_date: '2001-03-07', + nb_tracks: 14, + record_type: 'album', + genres: { data: [{ id: 113, name: 'Dance' }] }, + artist: { id: 27, name: 'Daft Punk' }, +}; +const TRACK = { + id: 3135556, + title: 'Harder, Better, Faster, Stronger', + duration: 224, + explicit_lyrics: false, + preview: 'https://img/preview.mp3', + artist: { id: 27, name: 'Daft Punk' }, +}; + +test('mapAlbumTrack: numbers by album order and inherits album-level fields', () => { + const item = dz.mapAlbumTrack(TRACK, ALBUM, 3); // 4th track + assert.equal(item.wrapperType, 'track'); + assert.equal(item.kind, undefined); // shape has no kind here; downloadAlbumTracks doesn't read it + assert.equal(item.trackNumber, 4); // index + 1 (Deezer album tracks carry no track_position) + assert.equal(item.trackName, 'Harder, Better, Faster, Stronger'); + assert.equal(item.collectionName, 'Discovery'); + assert.equal(item.collectionArtistName, 'Daft Punk'); + assert.equal(item.primaryGenreName, 'Dance'); + assert.equal(item.releaseDate, '2001-03-07'); + assert.equal(item.trackCount, 14); + assert.equal(item.trackTimeMillis, 224000); // seconds → ms + assert.equal(item.trackExplicitness, 'notExplicit'); + assert.equal(item.artworkUrl100, 'https://img/cover_xl.jpg'); +}); + +test('mapAlbumTrack: per-track artist wins over the album artist (mixed-artist albums)', () => { + const feat = { ...TRACK, artist: { id: 99, name: 'MeloMance' } }; + const item = dz.mapAlbumTrack(feat, ALBUM, 0); + assert.equal(item.artistName, 'MeloMance'); // drives the per-track YTM search + assert.equal(item.collectionArtistName, 'Daft Punk'); // album_artist stays stable → no album split +}); + +test('mapAlbumTrack: explicit flag maps through', () => { + const item = dz.mapAlbumTrack({ ...TRACK, explicit_lyrics: true }, ALBUM, 0); + assert.equal(item.trackExplicitness, 'explicit'); +}); + +test('pickTrackMatch: strict match on artist + title', () => { + const results = [ + { title: 'Some Other Song', artist: { name: 'Daft Punk' } }, + { title: 'Harder, Better, Faster, Stronger', artist: { name: 'Daft Punk' } }, + ]; + const m = dz.pickTrackMatch(results, 'Daft Punk', 'Harder, Better, Faster, Stronger'); + assert.equal(m, results[1]); +}); + +test('pickTrackMatch: title-only fallback for non-Japanese when artist differs', () => { + const results = [{ title: 'Levitating', artist: { name: 'Some Cover Act' } }]; + const m = dz.pickTrackMatch(results, 'Dua Lipa', 'Levitating'); + assert.equal(m, results[0]); // ASCII title → title-only fallback allowed +}); + +test('pickTrackMatch: NO title-only fallback for Japanese titles', () => { + const results = [{ title: '残酷な天使のテーゼ', artist: { name: 'Someone Else' } }]; + const m = dz.pickTrackMatch(results, '高橋洋子', '残酷な天使のテーゼ'); + assert.equal(m, null); // JP title + wrong artist must not match a same-titled cover +}); + +test('pickTrackMatch: Japanese title DOES match when the artist matches', () => { + const results = [{ title: '残酷な天使のテーゼ', artist: { name: '高橋洋子' } }]; + const m = dz.pickTrackMatch(results, '高橋洋子', '残酷な天使のテーゼ'); + assert.equal(m, results[0]); +}); + +test('pickTrackMatch: leading aesthetic punctuation is ignored on the artist', () => { + const results = [{ title: 'nostalgia', artist: { name: '.diedlonely' } }]; + const m = dz.pickTrackMatch(results, '.diedlonely', 'nostalgia'); + assert.equal(m, results[0]); +}); + +test('pickTrackMatch: empty / no match returns null', () => { + assert.equal(dz.pickTrackMatch([], 'x', 'y'), null); + assert.equal(dz.pickTrackMatch([{ title: 'A', artist: { name: 'B' } }], 'C', 'D'), null); +}); + +test('normalizeTitle: unifies variant suffixes and punctuation', () => { + assert.equal(dz.normalizeTitle("don't rush - slowed"), dz.normalizeTitle("don't rush (slowed)")); + assert.equal(dz.normalizeTitle('Mr. Lawrence'), dz.normalizeTitle('Mr Lawrence')); +}); + +// ── Live smoke test (opt-in): confirms the real Deezer shape still maps ──────── +test('LIVE: lookupTrackMeta returns enriched fields', { skip: process.env.RUN_LIVE !== '1' }, async () => { + const meta = await dz.lookupTrackMeta('Daft Punk', 'Harder, Better, Faster, Stronger'); + assert.ok(meta.album, 'album present'); + assert.ok(meta.genre, 'genre present'); + assert.ok(meta.year, 'year present'); + assert.ok(meta.durationMs > 0, 'duration present'); +}); diff --git a/index.html b/index.html index 79210cc..ed34da8 100644 --- a/index.html +++ b/index.html @@ -172,7 +172,7 @@
💿
-
iTunes
+
Deezer
—
@@ -333,7 +333,7 @@