fix(backend): Zahlen im Titel sind Pflicht-Keywords, Abkuerzungen erlaubt
"700 Main Street" lud "777 Main Street" eines fremden Artists: buildTitleKeywords behielt nur die zwei laengsten Woerter (Street, Main), die unterscheidende 700 fiel als zu kurz raus — und Schritt 3 der Kaskade prueft den Kuenstler-Channel nicht mehr. Zahlen werden jetzt immer als \b<n>\b gefordert. Dazu eine kleine Abkuerzungstabelle (street/st, saint/st, mount/mt, doctor/dr, versus/vs, number/no), denn der korrekte Art Track heisst "700 Main st" und scheiterte bisher an jedem Schritt. Tests in backend/test/ytdlp.test.js. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9f0d4c3153
commit
02b590505a
4 files changed
+75
-4
No files matched your search
@@ -64,6 +64,20 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **Downloader grabbed a same-shaped title with a different number** — searching DAT ADAM's
|
||||||
|
"700 Main Street" saved **"777 Main Street" by an unrelated artist**. `buildTitleKeywords` required
|
||||||
|
only the two longest words (Street, Main); "700" was dropped for being short, and the cascade step
|
||||||
|
that widens to the top 10 no longer checks the artist channel, so nothing else caught it. Numbers
|
||||||
|
are now always required keywords. In the same pass, a small abbreviation map (street/st, saint/st,
|
||||||
|
mount/mt, doctor/dr, versus/vs, number/no) lets a candidate spelled "700 Main St" match a release
|
||||||
|
titled "700 Main Street" — that mismatch had rejected the correct official Art Track in every step,
|
||||||
|
which is why the search fell through to the wrong video in the first place.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [2.3] — 2026-09-03
|
## [2.3] — 2026-09-03
|
||||||
|
|
||||||
A deployment-and-delivery release: production **pulls pre-built images** instead of compiling on the
|
A deployment-and-delivery release: production **pulls pre-built images** instead of compiling on the
|
||||||
|
|||||||
@@ -93,6 +93,14 @@ On the deploy host the music lives at `/opt/navidrome/music`.
|
|||||||
matches, download **nothing** and fail cleanly with "Nicht eindeutig gefunden — bitte
|
matches, download **nothing** and fail cleanly with "Nicht eindeutig gefunden — bitte
|
||||||
Direktlink im Importer nutzen". Never fall back to a blind title-only/no-filter grab — it
|
Direktlink im Importer nutzen". Never fall back to a blind title-only/no-filter grab — it
|
||||||
pulled same-titled covers by other uploaders, and wrong Piano Echoes for Japanese tracks.
|
pulled same-titled covers by other uploaders, and wrong Piano Echoes for Japanese tracks.
|
||||||
|
- **Numbers in a title are always required keywords.** `buildTitleKeywords` kept only the two
|
||||||
|
LONGEST words, and numbers are short — so "700 Main Street" was filtered on Street+Main only and
|
||||||
|
the cascade happily saved **"777 Main Street" by a different artist** (step 3 drops the channel
|
||||||
|
check, so nothing else caught it). Numeric tokens are now emitted as `\b<n>\b` in addition to the
|
||||||
|
two longest words. Same fix carries a small abbreviation map (street/st, saint/st, mount/mt,
|
||||||
|
doctor/dr, versus/vs, number/no): uploaders write "700 Main St" where the release says "Street",
|
||||||
|
which had rejected the correct official Art Track in every step. Unit-tested in
|
||||||
|
`backend/test/ytdlp.test.js`.
|
||||||
- **Size cap:** `--max-filesize 30M` on every yt-dlp call; Mittelweg steps also cap at 2×
|
- **Size cap:** `--max-filesize 30M` on every yt-dlp call; Mittelweg steps also cap at 2×
|
||||||
expected duration. Prevents saving a 28-min video (52 MB) when the wrong result slips through.
|
expected duration. Prevents saving a 28-min video (52 MB) when the wrong result slips through.
|
||||||
- **Leading aesthetic characters:** artists often style their name with leading punctuation
|
- **Leading aesthetic characters:** artists often style their name with leading punctuation
|
||||||
|
|||||||
+18
-4
@@ -26,9 +26,23 @@ function buildTitleKeywords(title) {
|
|||||||
.replace(/[^\w\s]/g, ' ')
|
.replace(/[^\w\s]/g, ' ')
|
||||||
.split(/\s+/)
|
.split(/\s+/)
|
||||||
.filter(w => w.length >= 3 && !VARIANT_WORDS.has(w.toLowerCase()) && !STOPWORDS.has(w.toLowerCase()));
|
.filter(w => w.length >= 3 && !VARIANT_WORDS.has(w.toLowerCase()) && !STOPWORDS.has(w.toLowerCase()));
|
||||||
const top2 = words.sort((a, b) => b.length - a.length).slice(0, 2);
|
// Numbers are the most distinctive token a title has and they are usually SHORT, so the
|
||||||
const toPattern = w => (w.length >= 4 && w.endsWith('s')) ? w + '?' : w;
|
// longest-two rule threw them away: "700 Main Street" kept only Street+Main and happily matched
|
||||||
const result = top2.map(toPattern);
|
// "777 Main Street" by a different artist. Numbers are therefore always required, on top of the
|
||||||
|
// two longest words.
|
||||||
|
const numbers = words.filter(w => /^\d+$/.test(w));
|
||||||
|
const wordsNoNum = words.filter(w => !/^\d+$/.test(w));
|
||||||
|
const top2 = wordsNoNum.sort((a, b) => b.length - a.length).slice(0, 2);
|
||||||
|
// Uploaders abbreviate the same word the release spells out ("700 Main St" vs "700 Main Street"),
|
||||||
|
// which rejected the official Art Track in every cascade step. Accept either spelling; \b keeps
|
||||||
|
// the short form from matching inside another word.
|
||||||
|
const ABBREV = { street: 'st', saint: 'st', mount: 'mt', doctor: 'dr', versus: 'vs', number: 'no' };
|
||||||
|
const toPattern = w => {
|
||||||
|
const abbr = ABBREV[w.toLowerCase()];
|
||||||
|
if (abbr) return `\\b(?:${w}|${abbr})\\b`;
|
||||||
|
return (w.length >= 4 && w.endsWith('s')) ? w + '?' : w;
|
||||||
|
};
|
||||||
|
const result = [...numbers.map(n => `\\b${n}\\b`), ...top2.map(toPattern)];
|
||||||
const qualifier = lower.includes('mix') ? MIX_QUALIFIERS.find(q => lower.includes(q)) : null;
|
const qualifier = lower.includes('mix') ? MIX_QUALIFIERS.find(q => lower.includes(q)) : null;
|
||||||
if (qualifier && !top2.some(w => w.toLowerCase() === qualifier)) result.push(qualifier);
|
if (qualifier && !top2.some(w => w.toLowerCase() === qualifier)) result.push(qualifier);
|
||||||
return result;
|
return result;
|
||||||
@@ -424,4 +438,4 @@ async function embedMetadata({ jobId, audioFile, dir, trackPrefix, safeTitle, ra
|
|||||||
return { success: true, dir };
|
return { success: true, dir };
|
||||||
}
|
}
|
||||||
|
|
||||||
module.exports = { downloadTrack, embedMetadata };
|
module.exports = { downloadTrack, embedMetadata, buildTitleKeywords };
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
const test = require('node:test');
|
||||||
|
const assert = require('node:assert');
|
||||||
|
const { buildTitleKeywords } = require('../lib/ytdlp.js');
|
||||||
|
|
||||||
|
// The keywords become `title ~= (?i)<kw>` match-filters in yt-dlp, i.e. every keyword must match
|
||||||
|
// the candidate title case-insensitively. This mirrors that check.
|
||||||
|
const accepts = (title, candidate) =>
|
||||||
|
buildTitleKeywords(title).every(kw => new RegExp(kw, 'i').test(candidate));
|
||||||
|
|
||||||
|
test('a number in the title is always required, however short', () => {
|
||||||
|
const kws = buildTitleKeywords('700 Main Street');
|
||||||
|
assert.ok(kws.some(k => k.includes('700')), `700 fehlt in ${JSON.stringify(kws)}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('rejects a same-shaped title with a different number', () => {
|
||||||
|
// The real regression: "777 Main Street" (Release - Topic) was downloaded for DAT ADAM's
|
||||||
|
// "700 Main Street", because only the two longest words (Street, Main) were required.
|
||||||
|
assert.equal(accepts('700 Main Street', '777 Main Street'), false);
|
||||||
|
assert.equal(accepts('700 Main Street', '700 Main Street'), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('accepts the abbreviated spelling uploaders use', () => {
|
||||||
|
assert.equal(accepts('700 Main Street', '700 Main st (cr-Version)'), true);
|
||||||
|
assert.equal(accepts('700 Main Street', 'DAT ADAM - 700 Main St [FULL SONG]'), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the abbreviation does not match inside another word', () => {
|
||||||
|
assert.equal(accepts('700 Main Street', '700 Main Stuff'), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('titles without numbers keep the previous two-longest-words behaviour', () => {
|
||||||
|
assert.deepEqual(buildTitleKeywords('Bohemian Rhapsody'), ['Bohemian', 'Rhapsody']);
|
||||||
|
assert.equal(accepts('Bohemian Rhapsody', 'Queen - Bohemian Rhapsody (Official Video)'), true);
|
||||||
|
assert.equal(accepts('Bohemian Rhapsody', 'Queen - Another One Bites The Dust'), false);
|
||||||
|
});
|
||||||
Reference in new issue
Block a user