revert: drop the YouTube cookies/PO-token/proxy workarounds (back to pre-saga state)

None of it got downloads working on the flagged datacenter IP (web_music media
403, then 429 rate-limiting), so revert the whole lot to keep things clean:
- config.js: remove potProviderUrl + ytdlpProxy
- ytdlp.js: remove potArgs/proxyArgs; player_client back to web_music,default
- novasonic-server.js: remove pot/proxy args from importer resolve
- docker-compose.yml: remove the bgutil-provider service + backend env/plugin/depends_on
- CHANGELOG: drop the [2.2] section

Kept the one genuine fix: a failed direct-link download now reports failure
(moved its note into [2.1]).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
HoneyandClaude Opus 4.8 committed 2026-08-17 20:24:13 +02:00
1 parent 31d4a72e62
commit 635ba8d353
5 files changed
+8 -68

No files matched your search

+3 -22
View File
@@ -64,28 +64,6 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an
--- ---
## [2.2] — 2026-08-14
Keeps downloads working after YouTube began blocking datacenter IPs, and fixes a related
false-success in the UI.
### Added
- **PO token provider (bgutil)** — YouTube now 403s media downloads from datacenter IPs without a
**PO ("Proof of Origin") token** that yt-dlp can't compute itself (seen first as "Sign in to
confirm you're not a bot", then `HTTP 403` on the audio stream). A self-hosted **`bgutil-provider`**
container (`brainicism/bgutil-ytdlp-pot-provider`) mints the tokens and the matching
`bgutil-ytdlp-pot-provider` plugin (installed in the backend) hands them to yt-dlp — **no cookies or
account required**. Wired into the download cascade, the YTM search probe and the importer resolve,
guarded by `POT_PROVIDER_URL` (empty → yt-dlp runs exactly as before). Deploy: `docker compose up -d`.
### Fixed
- **A failed direct-link download reported success** — the direct-URL path returned `success` even
when no audio was produced (e.g. the YouTube 403), so the job showed "Fertig!" and the UI (web +
Android) said the track was added. It now reports failure with a clear message ("Download
fehlgeschlagen — YouTube hat den Abruf blockiert (403)").
---
## [2.1] — 2026-08-14 ## [2.1] — 2026-08-14
Builds on the 2.0 listening archive with a **head-to-head "Musik-Match"**, brings the **artist pin** Builds on the 2.0 listening archive with a **head-to-head "Musik-Match"**, brings the **artist pin**
@@ -110,6 +88,9 @@ to Android at parity with web, and fixes a batch of same-name-artist and player
as avatar chips, matching the Android app (previously plain text). as avatar chips, matching the Android app (previously plain text).
### Fixed ### Fixed
- **A failed direct-link download reported success** — the direct-URL path returned `success` even
when no audio was produced (e.g. a YouTube 403), so the job showed "Fertig!" and the UI (web +
Android) said the track was added. It now reports failure with a clear message.
- **Artist pin — search & similar artists (Web + Android)** — a pinned artist now shows the correct - **Artist pin — search & similar artists (Web + Android)** — a pinned artist now shows the correct
Deezer photo in library search (not the wrong same-name person's), and its "Similar Artists" use Deezer photo in library search (not the wrong same-name person's), and its "Similar Artists" use
Deezer's **related** for the pinned artist instead of Last.fm's list (which was keyed to the Deezer's **related** for the pinned artist instead of Last.fm's list (which was keyed to the
-8
View File
@@ -8,14 +8,6 @@ const CONFIG = {
maxParallel: parseInt(process.env.MAX_PARALLEL || '2', 10), maxParallel: parseInt(process.env.MAX_PARALLEL || '2', 10),
downloadBackend: (process.env.DOWNLOAD_BACKEND || 'ytdlp').toLowerCase(), downloadBackend: (process.env.DOWNLOAD_BACKEND || 'ytdlp').toLowerCase(),
ticketmasterKey: process.env.TICKETMASTER_API_KEY || '', ticketmasterKey: process.env.TICKETMASTER_API_KEY || '',
// YouTube 403s media downloads from datacenter IPs without a PO ("Proof of Origin") token, which
// yt-dlp can't compute itself. The bgutil-ytdlp-pot-provider plugin fetches tokens from a small
// provider service; set its URL here (e.g. http://bgutil-provider:4416). Empty → no PO token used
// (yt-dlp runs exactly as before). See docker-compose.yml.
potProviderUrl: process.env.POT_PROVIDER_URL || '',
// Optional proxy for yt-dlp (e.g. a residential/rotating proxy) — the durable fix when the
// datacenter IP stays flagged (403/429) even with a PO token. Empty → yt-dlp runs directly.
ytdlpProxy: process.env.YTDLP_PROXY || '',
}; };
module.exports = CONFIG; module.exports = CONFIG;
+4 -21
View File
@@ -134,18 +134,6 @@ async function downloadTrack({ jobId, artist, rawArtist, albumArtist, title, alb
// YTM ranking toward a same-named artist's full discography. // YTM ranking toward a same-named artist's full discography.
const query = `${ytmArtist} ${searchTitle}`; const query = `${ytmArtist} ${searchTitle}`;
// YouTube 403s media downloads from datacenter IPs without a PO token. When a provider URL is
// configured, point the bgutil-ytdlp-pot-provider plugin at it so yt-dlp fetches a token. Empty
// config → no arg → unchanged behaviour.
const potArgs = () =>
CONFIG.potProviderUrl
? ['--extractor-args', `youtubepot-bgutilhttp:base_url=${CONFIG.potProviderUrl}`]
: [];
// Optional proxy — the durable fix when a datacenter IP stays flagged (403/429) even with a PO
// token. Route yt-dlp through a residential/rotating proxy via YTDLP_PROXY. Empty → not used.
const proxyArgs = () => (CONFIG.ytdlpProxy ? ['--proxy', CONFIG.ytdlpProxy] : []);
const baseArgs = [ const baseArgs = [
'--extract-audio', '--extract-audio',
'--audio-format', CONFIG.audioFmt, '--audio-format', CONFIG.audioFmt,
@@ -156,13 +144,10 @@ async function downloadTrack({ jobId, artist, rawArtist, albumArtist, title, alb
'--newline', '--newline',
// Hard upper size limit — prevents saving a wrong 28-min video as a ~52 MB mp3 // Hard upper size limit — prevents saving a wrong 28-min video as a ~52 MB mp3
'--max-filesize', '30M', '--max-filesize', '30M',
// For the DOWNLOAD, prefer the default clients (their media URLs go through with a PO token from // Use the web_music player client (falling back to default) so YouTube Music
// a datacenter IP) and keep web_music only as a fallback for Art Tracks that only it can extract. // "Art Track" / topic uploads are reachable. The android_vr client returns
// web_music's own media URLs 403 from datacenter IPs even with a valid PO token. (The YTM *search* // "This video is not available" for many YTM tracks that play fine in the browser.
// in getYtmVideoUrls still leads with web_music, which is where Art-Track discovery matters.) '--extractor-args', 'youtube:player_client=web_music,default',
'--extractor-args', 'youtube:player_client=default,web_music',
...potArgs(),
...proxyArgs(),
]; ];
const spawnYtdlp = (args) => new Promise((resolve, reject) => { const spawnYtdlp = (args) => new Promise((resolve, reject) => {
@@ -235,8 +220,6 @@ async function downloadTrack({ jobId, artist, rawArtist, albumArtist, title, alb
// generic YouTube search and returns unrelated videos instead of the // generic YouTube search and returns unrelated videos instead of the
// native Songs shelf (where the actual artist track lives). // native Songs shelf (where the actual artist track lives).
'--extractor-args', 'youtube:player_client=web_music,default', '--extractor-args', 'youtube:player_client=web_music,default',
...potArgs(),
...proxyArgs(),
], { env: { ...process.env, PYTHONUNBUFFERED: '1' } }); ], { env: { ...process.env, PYTHONUNBUFFERED: '1' } });
proc.stdout.on('data', d => { proc.stdout.on('data', d => {
const text = d.toString(); const text = d.toString();
-3
View File
@@ -533,9 +533,6 @@ app.post('/api/youtube/resolve', async (req, res) => {
execFile(CONFIG.ytdlpBin, [ execFile(CONFIG.ytdlpBin, [
'--dump-json', '--flat-playlist', '--no-warnings', '--dump-json', '--flat-playlist', '--no-warnings',
'--playlist-items', '1-50', '--playlist-items', '1-50',
// PO token provider (if configured) to get past YouTube's datacenter-IP block. Empty → omitted.
...(CONFIG.potProviderUrl ? ['--extractor-args', `youtubepot-bgutilhttp:base_url=${CONFIG.potProviderUrl}`] : []),
...(CONFIG.ytdlpProxy ? ['--proxy', CONFIG.ytdlpProxy] : []),
url, url,
], { timeout: 45000, maxBuffer: 20 * 1024 * 1024 }, (err, out, stderr) => { ], { timeout: 45000, maxBuffer: 20 * 1024 * 1024 }, (err, out, stderr) => {
if (err && !out?.trim()) return reject(new Error(stderr?.trim() || err.message)); if (err && !out?.trim()) return reject(new Error(stderr?.trim() || err.message));
+1 -14
View File
@@ -33,28 +33,15 @@ services:
- MAX_PARALLEL=${MAX_PARALLEL:-2} - MAX_PARALLEL=${MAX_PARALLEL:-2}
- DOWNLOAD_BACKEND=${DOWNLOAD_BACKEND:-ytdlp} - DOWNLOAD_BACKEND=${DOWNLOAD_BACKEND:-ytdlp}
- TICKETMASTER_API_KEY=${TICKETMASTER_API_KEY} - TICKETMASTER_API_KEY=${TICKETMASTER_API_KEY}
# PO token provider (bgutil) — lets yt-dlp download from datacenter IPs without cookies/account.
- POT_PROVIDER_URL=http://bgutil-provider:4416
depends_on:
- bgutil-provider
command: > command: >
sh -c " sh -c "
apt-get update && apt-get install -y --no-install-recommends python3 python3-pip ffmpeg curl git && rm -rf /var/lib/apt/lists/* && apt-get update && apt-get install -y --no-install-recommends python3 python3-pip ffmpeg curl git && rm -rf /var/lib/apt/lists/* &&
curl -sL https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp && chmod a+rx /usr/local/bin/yt-dlp && curl -sL https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o /usr/local/bin/yt-dlp && chmod a+rx /usr/local/bin/yt-dlp &&
pip install --break-system-packages spotapi 'git+https://github.com/TzurSoffer/spotify-downloader' bgutil-ytdlp-pot-provider && pip install --break-system-packages spotapi 'git+https://github.com/TzurSoffer/spotify-downloader' &&
mkdir -p /deps && cd /deps && npm install express kuromoji && mkdir -p /deps && cd /deps && npm install express kuromoji &&
cd /app && NODE_PATH=/deps/node_modules node novasonic-server.js cd /app && NODE_PATH=/deps/node_modules node novasonic-server.js
" "
# ── PO token provider (bgutil) — mints YouTube "Proof of Origin" tokens so yt-dlp can download
# from a datacenter IP without cookies or an account. Paired with the bgutil-ytdlp-pot-provider
# plugin installed in the backend above. Port 4416 is internal only (no reverse proxy needed). ──
bgutil-provider:
image: brainicism/bgutil-ytdlp-pot-provider:latest
container_name: bgutil-provider
restart: unless-stopped
init: true
# ── Scrobble service (isolated timestamped play-log → stats + Wrapped) ──────── # ── Scrobble service (isolated timestamped play-log → stats + Wrapped) ────────
novasonic-scrobble: novasonic-scrobble:
image: node:22-slim image: node:22-slim