Drei Luecken derselben Familie, die dazu fuehrten, dass dieselben Songs ein
zweites Mal geladen wurden:
- Kuenstlernamen wurden zeichengenau verglichen: "Cécile Corbel" und "Cecile
Corbel" galten als zwei Personen. Welche Schreibweise ankommt, haengt an der
Quelle (Deezer vs. Spotify). Der Vergleich normalisiert jetzt Akzente weg —
nur der Vergleich, Tags und Dateinamen behalten das Original.
- Typografische Apostrophe: "Arrietty’s Song" (U+2019) und "Arrietty's Song"
fielen auseinander. Fiel beim Testen auf, betrifft viele Soundtracks.
- findExistingTrack durchsucht nur den Ordner des uebergebenen Kuenstlers. Bei
Soundtracks weicht der Album- vom Titel-Kuenstler ab, der Song landet je nach
Ladeweg unter dem einen oder anderen — geprueft wurde jeweils der falsche.
Weicht der Titel-Kuenstler ab, wird nun auch dessen Ordner geprueft (~0,6 ms
bei 28.801 Dateien, gegenueber 3-9 s pro Download).
Sieben neue Tests zielen gezielt auf Fehlalarme: aehnliche Namen (Cecil Taylor
vs. Cécile Corbel), gleicher Titel bei verschiedenen Kuenstlern, Remix gegen
Basistitel, fremder Titel im Soundtrack-Ordner — alle bleiben getrennt.
Gesamte Suite: 42 Tests gruen.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
getYtmVideoUrls wertete jedes /browse/UC in der Ausgabe als "yt-dlp hat einen
Artist-Channel aufgeklappt" und gab [] zurueck — samt der bereits gesammelten
Song-URLs. Bei "DAT ADAM UFO" stand der Kanal auf Platz 9 und riss sieben gute
Treffer mit, darunter den offiziellen Art Track auf Platz 1; uebrig blieb der
YouTube-Fallback mit einem Live-Mitschnitt. An 8 echten Suchen gemessen feuerte
das in 7 Faellen, Schritt 1 der Kaskade war also praktisch tot.
Mit --flat-playlist kann gar nichts aufgeklappt werden, ein Kanal ist EINE Zeile.
pickYtmVideoUrls() behaelt daher alle watch?v=-Zeilen und ueberspringt browse/UC
(Kanal) wie browse/MPREb_ (Album). Ueber 20k Zufallsausgaben geprueft: die neue
Funktion ist eine Obermenge der alten, kann also nur Kandidaten hinzufuegen.
Drei echte Downloads liefen end-to-end sauber durch (Dauer sekundengenau auf
Deezer, korrekte Tags), alle aus Schritt 1 statt aus dem Fallback.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
is_live meint nur "laeuft gerade als Stream" — eine Konzert-AUFNAHME kam
ungehindert durch und landete als Albumtrack in der Bibliothek ("Dat Adam - Ufo
Video Days 2015 Live" auf dem echten Kuenstlerkanal, Dauer im Fenster). Der
Variantenfilter bekommt daher eine live-Regel. Sie schaltet sich ab, wenn der
gesuchte Titel selbst live enthaelt; die Pruefung nutzt \blive\b statt includes,
sonst wuerde "deliver" die Regel aushebeln. Drei Tests dazu.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Folgekorrektur zu 02b5905: "… - 2020 Remaster" steht auf der Veroeffentlichung,
aber nie im YouTube-Titel — als Pflicht-Keyword haette die Zahl den richtigen
Upload abgelehnt. Jahreszahlen neben einem Qualifier (remaster/version/edition/
anniversary/mono/stereo) fallen daher wieder raus; ein Titel, der selbst eine
Jahreszahl ist ("1999", "Berlin 2020"), behaelt sie. Gegen 29 Beispieltitel
gemessen verhalten sich 24 exakt wie vorher.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"700 Main Street" lud "777 Main Street" eines fremden Artists: buildTitleKeywords
behielt nur die zwei laengsten Woerter (Street, Main), die unterscheidende 700
fiel als zu kurz raus — und Schritt 3 der Kaskade prueft den Kuenstler-Channel
nicht mehr. Zahlen werden jetzt immer als \b<n>\b gefordert. Dazu eine kleine
Abkuerzungstabelle (street/st, saint/st, mount/mt, doctor/dr, versus/vs,
number/no), denn der korrekte Art Track heisst "700 Main st" und scheiterte
bisher an jedem Schritt. Tests in backend/test/ytdlp.test.js.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
None of it got downloads working on the flagged datacenter IP (web_music media
403, then 429 rate-limiting), so revert the whole lot to keep things clean:
- config.js: remove potProviderUrl + ytdlpProxy
- ytdlp.js: remove potArgs/proxyArgs; player_client back to web_music,default
- novasonic-server.js: remove pot/proxy args from importer resolve
- docker-compose.yml: remove the bgutil-provider service + backend env/plugin/depends_on
- CHANGELOG: drop the [2.2] section
Kept the one genuine fix: a failed direct-link download now reports failure
(moved its note into [2.1]).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PO token provider works (web_music gets a token) but web_music's media URLs
still 403 from a datacenter IP. Prefer the default player clients for the
DOWNLOAD (their URLs go through with a PO token), keeping web_music only as a
fallback for Art-Track extraction; the YTM search still leads with web_music.
Also add an optional YTDLP_PROXY (--proxy) — the durable fix if the IP stays
flagged (403/429) even with a PO token. Empty → unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the cookies-file stopgap (needs a throwaway account + manual refresh)
with a self-hosted PO token provider — the cookie-less way past YouTube's
datacenter-IP block (bot check + media 403).
- docker-compose: new bgutil-provider service (brainicism/bgutil-ytdlp-pot-provider)
+ install the bgutil-ytdlp-pot-provider plugin in the backend + POT_PROVIDER_URL
env + depends_on.
- ytdlp.js / server.js: when POT_PROVIDER_URL is set, pass
--extractor-args youtubepot-bgutilhttp:base_url=... to the download cascade, the
YTM search probe and the importer resolve. Empty → omitted (unchanged behaviour).
- Removed the cookiesFile config + --cookies wiring so there's no dead code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The direct-URL (sourceUrl) path returned { success: true } even when no audio
file was produced (e.g. YouTube 403 / bot check), so the job was marked
"done/Fertig!" and web + Android both said the track was added. It now returns
{ failed: true } → the server sets status 'error' with a clear message, which
both frontends already render as a failure. Search-cascade path was already
correct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
YouTube now returns "Sign in to confirm you're not a bot" for yt-dlp from
datacenter IPs, blocking downloads, search and link-resolve. When a cookies file
is present (YTDLP_COOKIES, default /data/youtube-cookies.txt) every yt-dlp call
passes --cookies so it authenticates. Checked at call time (drop the file in, no
code change); absent → unchanged behaviour. Applied to the download cascade, the
YTM search probe, and the importer resolve.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The strict cascade steps required the YouTube channel name to contain the artist, so
official uploads on label channels (BANGTANTV, HYBE LABELS, …) were rejected and the
no-channel fallback grabbed a fan reupload instead (observed: BTS "Pied Piper").
Add yt-dlp's music `artist` field as an OR alternative to the channel check in the
strict steps (two --match-filter args = OR). A metadata dump of "BTS Pied Piper"
confirmed only the official upload has artist=BTS (fan/live/lyric reuploads: NA), and
it's the #1 result — so it now passes strict step 2. channel_is_verified was rejected
as a signal: lyric videos and fan MVs also run on verified channels.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two-part fix for "[Unknown Album]" vs "Singles" mismatch that broke the
metadata editor (File not found):
- Download: always write an album tag, defaulting to "Singles" so it matches
the on-disk folder (album || 'Singles'). No-album tracks no longer show as
"[Unknown Album]" while living in a "Singles" folder. (Future downloads.)
- Editor resolve: Subsonic getSong returns a *virtual* tag-built path (e.g.
"[Unknown Album]/Title.mp3" with raw "|"), not the physical file ("Singles/
Artist - Title_YTPMV.mp3"). Add findExistingTrack as a final fallback, and
make its normalizer map the sanitized chars (/ \ : * ? " < > | and _) to a
space so a title tag matches the sanitized filename. (Fixes existing tracks.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Japanese Art Tracks on YTM/YouTube are often romanized ("Souiunatsu"
for "そういう夏") — the Japanese keyword filter in --match-filter always
rejected the correct video. Channel + duration are sufficient quality
gates for JP tracks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Album words ("KAGUYA Music", "OST" etc.) appended to the search query
caused wrong or no results — "Kei Haneoka 確信 KAGUYA Music" would miss
tracks the plain "Kei Haneoka 確信" query finds immediately.
Artist + cleanTitle is sufficient for both JP and Western tracks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Japanese tracks stored with original names in folder/filename/tags; romaji only in sort tags
- findExistingTrack: case-insensitive, searches all album folders under artist, strips release suffix
- --max-filesize 30M + 2x duration cap on all yt-dlp calls
- iTunes collectionArtistName captured as album_artist; fixes compilation album splitting
- album_artist tag uses mainArtist (not rawArtist) so all tracks in one album are consistent
- No-duration cascade: steps 5-8 (title-only/no-filter) removed — fail cleanly like with-duration
- Autofill iTunes: fills title, artist, albumArtist, discNum (was album/genre/year/track/total only)
- Autofill MusicBrainz: fills albumArtist, discNum (was missing both)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a track's duration is known but nothing matches the duration window, the
cascade no longer falls back to blind title-only/no-filter grabs (which pulled
same-titled videos by other uploaders, e.g. a "recognize" fan clip). Instead it
relaxes only the duration constraint while still REQUIRING the artist channel +
title. If nothing matches, the download produces no file and the job now fails
cleanly with "Nicht eindeutig gefunden — bitte Direktlink im Importer nutzen"
(previously it falsely reported success with no file).
Single, album and playlist paths all surface the not-found state.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-add the ¶ms= Songs-shelf filter (the documented way to song-search
YouTube Music) so results exclude videos/playlists/clips. Verified neutral
for the ambiguous-artist case (yt-dlp's search can't surface that track
regardless) but cleaner for normal tracks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The filename used the raw title, so "recognize - Single" leaked into the
filename ("01 - MINA - recognize - Single.mp3"). Now the cleaned title drives
both the filename and the embedded title tag, giving "01 - MINA - recognize.mp3"
(Track - Artist - Title). The album folder keeps the full album name.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ¶ms= Songs-tab filter re-ranked YTM results toward a same-named
artist's discography and excluded the official track (qNffDItRUeo) entirely,
even logged-out. The default blended search (what the browser shows) ranks
the exact track first. Removing the param so yt-dlp gets the same result set.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"MINA - recognize" skewed YTM ranking toward a same-named artist's full
discography, so the actual track (qNffDItRUeo) never appeared in results.
"MINA recognize" — matching how users type in the YTM search box — ranks
the exact official track first.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The YTM search (getYtmVideoUrls) ran without the web_music client, so yt-dlp
parsed the search page as a generic YouTube search and returned unrelated
videos instead of the native Songs shelf — the actual artist track (e.g.
qNffDItRUeo for MINA - recognize) never appeared in results. Apply the same
player_client=web_music,default arg to the search step.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
yt-dlp's android_vr player client returns "This video is not available" for
many YouTube Music Art Track / topic uploads (e.g. MINA - recognize) that
play fine in the browser. Switch to player_client=web_music with default
fallback so these tracks become downloadable, fixing the cascade falling
through to wrong YouTube videos.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
searchTitle (e.g. "recognize") is now passed to embedMetadata instead of
the raw title ("recognize - Single"), so the ID3 title tag matches the
actual song name. Album tag stays unchanged ("recognize - Single").
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
[youtube:tab] in stderr fires for all YTM search page processing, not just
artist channel expansion, causing channelExpanded=true for every search and
returning [] — bypassing all YTM cascade steps entirely. Fix: only check
stdout for /browse/UC (actual channel URL in results). Remove ytmsearch5:
scheme which is not supported by this yt-dlp version. Revert pool to 10.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ytmsearch5 (yt-dlp native YTM search) added as step 2 — returns different
result ordering than the manual YTM URL, recovering songs that the URL
approach ranks too low. YTM pool expanded from 10 → 20 results to reduce
fallthrough to YouTube.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
spawnYtdlp now always resolves so the hasAudio() cascade continues when a
single YTM result is geo-blocked or unavailable. Error is still logged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
YTM search used `title` directly, so "recognize - Single" was searched on
YouTube Music instead of "recognize" — causing no YTM results and falling
through to the less reliable YouTube search cascade.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Matches spotdl behaviour; still covers YouTube intros/outros while
reducing false positives from similarly-lengthed wrong songs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Handles all Spotify trailing suffixes: Single, EP, Remaster, Deluxe Edition,
Bonus Tracks, B-Sides, bare years (2024), Official Music Single, etc.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- "recognize - Single" title → search uses "recognize" (strips "- Single/EP/Album/…")
- Adds Single/EP/Remaster/Deluxe/… to titleKws STOPWORDS so they never become
mandatory title-match filters (no YouTube video title contains "Single")
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Western tracks now use artist + title only (matches spotdl behaviour).
Album suffix was causing noise for short English titles like singles
where album name = title + "Single".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Album "recognize - Single" produced query "MINA - recognize recognize Single"
because "recognize" appeared in both title and album name. Now filters album
words already present in the title, plus generic words (Single, EP, Remaster…).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Five separate root causes fixed:
1. buildTitleKeywords (ytdlp.js): For Japanese titles \w strips all
kana/kanji leaving no keywords. Now uses the first whitespace-
delimited segment as a literal regex keyword so --match-filter
title ~= requires the YTM video to contain the Japanese text.
2. Tokenizer wait (ytdlp.js): If kuromoji hasn't finished loading when
a Japanese download starts, romajiForStorage returns the original
Japanese text unchanged — producing Japanese folder/filenames and
skipping sort tags. Now waits up to 30 s for the tokenizer.
3. titleOnlyMatch guard (metadata.js): iTunes titleOnlyMatch (no artist
check) was used as a fallback for all searches. For Japanese
titles/artists this matched wrong US songs with identical titles
(common in anime OSTs). Now titleOnlyMatch is skipped when the
search title or artist contains Japanese characters.
4. squid.js rawArtist: rawArtist was not in squid's signature and
never forwarded to embedMetadata, so Qobuz-downloaded J-Pop tracks
stored the iTunes-resolved English name as artist instead of the
Japanese original. Fixed by accepting and forwarding rawArtist.
5. squid.js romaji paths: Folder and filename used raw artist/title
with no romaji conversion, creating Japanese-named paths
inconsistent with the yt-dlp path. Now uses romajiForStorage same
as ytdlp.js.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When iTunes enrichment returns a jingle/preview as the first result (e.g.
trackTimeMillis=9000), the yt-dlp duration filter becomes impossible
(duration > 30 & duration < 24), all filtered steps fail, and the final
unfiltered fallback downloads whatever YouTube returns first — often a
short clip.
Fix 1: spotify.js — only copy iTunes trackTimeMillis to durationMs when
it is > 30 seconds (genuine song, not a jingle or preview).
Fix 2: ytdlp.js — only enter the duration-filtered search branch when
durationMs > 30 000ms, so a corrupted short value silently falls back to
the unfiltered search path which still finds the correct full song.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
albumQuerySuffix and related consts (escRe, JP_CHARS, variantFilter, titleKws,
artistChannelPart) were declared after the ytmQuery/query lines that reference them,
causing ReferenceError on every download. Moves the block to before its first use.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Skip channel filter when artistFirstWord is non-ASCII (Japanese kanji/kana):
YouTube channels are named in romaji/English, not kanji — the filter was
rejecting all valid results and causing 6+ extra cascade steps
- Add albumQuerySuffix: when title is Japanese (no keyword match) OR has only
one romaji keyword, include the album's first two English words in the search
query so YTM ranks the official OST/album version above fan covers
Fixes: 照井順政 channel filter rejecting all YTM results
Fixes: "Souiunatsu" matching covers with romaji subtitle instead of official OST
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Downloads now store original Japanese in TITLE/ARTIST/ALBUM tags
so Apple Music, iTunes, YTM etc. all see the authentic original text
- Sort tags (TITLESORT/ARTISTSORT/ALBUMSORT) written with romaji for
apps that use them for sorting/display
- iTunes-resolved English artist name used for ARTISTSORT and folder names
when available (more accurate than kuromoji for known artists)
- rawArtist parameter added to downloadTrack so YTM search query uses
the original Japanese channel name (fixes wrong song downloads)
- spotify.js: artistName preserved as original; itunesArtistName stored
separately so the Spotify importer shows Japanese titles
- romaji index now includes titleRomaji/artistRomaji/albumRomaji fields
- Frontend: showRomaji setting (Settings > Bibliothek) — when on, player
bar, now-playing, queue, and song lists all show romaji via index lookup
- refreshDisplayMode() updates all live UI when the toggle is saved
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ytdlp.js + novasonic-server.js: apply romajiForStorage to album tag
and folder/rename paths (was only applied to artist/title before)
- ui.js: _createNdPlaylist falls back to romaji index for Japanese titles
since Navidrome search3 doesn't index Japanese text properly
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add romajiForStorage() to romaji.js — smart word spacing + Title Case
- ytdlp.js: apply to folder names, filenames, and embedded tags
- novasonic-server.js: apply in edit-metadata endpoint and Spotify resolve
- spotify.js: update track artist/name from iTunes canonical English names
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
"Arriez & King Dif" was used as-is for the folder; now splits on , ; &
so only the first artist is used: "Arriez".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Naming convention (download + edit-metadata):
Before: {01 - }Title.ext
After: {01 - }Artist1, Artist2 - Title.ext
- ytdlp.js: builds fileBase with artist prefix; findExistingTrack checks
both new and old format so existing files aren't re-downloaded
- metadata.js: findExistingTrack accepts optional safeArtist param
- novasonic-server.js: file rename on title edit now includes artist
(falls back to Navidrome's stored artist if not provided in body)
Song list subtitle: hide album name when album === title (singles where
the album is named after the track appear to duplicate the title)
Player bar: removed .player-title:hover underline/colour effect
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes LRCLIB lookup, .lrc sidecar file writing, and lyrics/uslt
metadata embedding from all downloads.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SC tracks: use artworkUrl/album/genre fields (not iTunes artworkUrl100/collectionName/primaryGenreName)
- Playlist handler: same SC field fallbacks for artwork, album, genre, year
- ytdlp.js: convert & to , in album_artist tag so Navidrome albumartist.Split works
- Importer: clear both previews on any URL input change, not just on clear
- edit-metadata: rename album folder on disk when album name changes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New src/soundcloud.js: resolves SoundCloud URLs via yt-dlp --dump-single-json,
renders track/playlist preview with per-track download buttons
- Backend POST /api/soundcloud/resolve: extracts metadata (title, artist, genre,
artwork, duration) directly from SoundCloud — no iTunes lookup needed
- ytdlp.js: sourceUrl param — if set, downloads directly from URL instead of
ytsearch cascade (used for SoundCloud tracks)
- Spotify importer page renamed to Song Importer, now contains both
Spotify and SoundCloud sections in one view
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each tryYtmSearch() call made a fresh HTTP request, so YouTube could
return different result orderings between steps. The correct video
(right title, wrong channel name) appeared in step 1 but was gone by
step 3 — a worse video with matching keywords slipped through instead.
Now fetch 10 YTM URLs once upfront and filter the same list with
progressively relaxed rules. Also saves 2-4 extra YTM requests per
download.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
HoneyandClaude Opus 4.8