docs: v2.0 — document the scrobble / listening-stats system

- CHANGELOG: new [2.0] entry (own timestamped play-log service, stats view on
  web + android with cover art, remix-dedup fix, user-select UI fix, node 20->22);
  Platform Parity Matrix gains play-log / stats / stats-cover rows (+ Wrapped 🔜).
- README: Listening stats feature, scrobble/ in the project tree, SCROBBLE_URL
  env + scrobble container table, port 3710 + SSO-exclusion note in setup.
- CLAUDE.md: new 'Listening stats / scrobble system' section (isolation rationale,
  node:sqlite gotchas, ping.view multi-user auth, capture point, cover resolution,
  SSO deploy gotcha).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
HoneyandClaude Opus 4.8 committed 2026-08-07 19:33:46 +02:00
1 parent 906b3b48cb
commit 561014c83f
3 files changed
+93 -3

No files matched your search

+35
View File
@@ -54,11 +54,46 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an
| Offline playback | ❌ | ✅ | | Offline playback | ❌ | ✅ |
| Offline detail / manage downloads | ❌ | ✅ | | Offline detail / manage downloads | ❌ | ✅ |
| Player bar on all screens | ✅ | ✅ | | Player bar on all screens | ✅ | ✅ |
| Timestamped play-log (own scrobble) | ✅ | ✅ |
| Listening statistics view | ✅ | ✅ |
| Stats cover art (artist/album) | ✅ | ✅ |
| Yearly Wrapped | 🔜 | 🔜 |
| Romaji / Japanese sort | ✅ | ❌ | | Romaji / Japanese sort | ✅ | ❌ |
| Settings (Navidrome, ABS, etc.) | ✅ | ✅ | | Settings (Navidrome, ABS, etc.) | ✅ | ✅ |
--- ---
## [2.0] — 2026-08-07
Big change: **NovaSonic gets its own self-hosted, multi-user listening archive.** Navidrome only stores a play *count* and the *last* play date — no timestamped history — so per-day/month/year stats and a yearly "Wrapped" were impossible. Koito and Spindle (the off-the-shelf options) are effectively single-user. Since NovaSonic *is* the player, it now records every counted play into an **isolated scrobble service** with its own database, and shows the results in a **statistics view on web and Android**. This runs **in addition to** the existing Navidrome/Last.fm scrobble — it never replaces it.
### Added
- **New `novasonic-scrobble` service** — a small, isolated container (`node:22-slim` + built-in `node:sqlite`, own volume) that owns the play-log. Kept separate from the download backend on purpose: the backend is heavy (yt-dlp/ffmpeg) and restarts regularly, while an archive needs uptime + data integrity. Routes:
- `POST /scrobble/plays` — batch ingest, **idempotent by uuid** (safe offline-retry)
- `GET /scrobble/history` — raw chronological log (the archive)
- `GET /scrobble/stats` — totals, top artists/albums/tracks, day/month/year series, 24h listening clock (on-the-fly `GROUP BY strftime`, local-tz bucketing)
- `DELETE /scrobble/plays?user=` — purge one user's rows (test cleanup)
- `GET /scrobble/health` — liveness
- **Multi-user**: every request carries the same Subsonic token creds the app already uses; the service verifies them via Navidrome `ping.view` (cached), so the stored `user` is real, not spoofable. No new login.
- **Play capture (Web + Android)** — fires at the *existing* counted-play moment (respects the user's `scrobbleMode`, consistent with Last.fm), with an **offline-safe queue** (localStorage on web, a JSON file on Android) flushed as an idempotent batch on start and when connectivity returns.
- **Statistics view (Web + Android)** — opened from Settings ("📊 Meine Statistiken"): totals tiles (**Plays** + listening time), a day/month/year play series, top artists/tracks/albums, and a 24h listening clock, with period tabs (Alles / Dieses Jahr / Letzte 30 Tage).
- **Cover art in stats (Web + Android)** — top artists show a round artist image, tracks/albums a square cover, resolved client-side from Navidrome (the scrobble service stays cover-agnostic).
- **`SCROBBLE_URL`** environment variable (frontend) + a **Scrobble URL** field with a test button in Settings on both platforms.
### Changed
- **Backend** — base image bumped `node:20-slim` → **`node:22-slim`** (Node 20 reaches end-of-life April 2026; enables the built-in `node:sqlite`).
### Fixed
- **Downloads** — remix / live / acoustic / instrumental tracks were skipped as duplicates of the base track: duplicate detection stripped *all* bracketed content, collapsing "Song (Remix)" → "Song". Version-marker brackets are now preserved, so a distinct recording downloads (noise brackets like `(feat. X)` / `(Official Video)` are still stripped). Verified against Deezer + unit-tested both directions.
- **UI** — dragging the mouse near a field edge (Global search, importer) started selecting the surrounding titles/labels instead of feeling like an app. `user-select` is now disabled on the `#app` shell; real text fields and lyrics stay selectable.
### Notes
- The scrobble subdomain **must be excluded from any SSO / forward-auth gateway** (Authelia/Authentik/etc.), exactly like the download backend — it self-authenticates via `ping.view`, and a gateway in front 302-redirects the browser's API calls to the login portal (breaks `fetch`).
- Deploy: add `SCROBBLE_URL` to `.env`, `docker compose up -d novasonic-scrobble`, and route port **3710** through your reverse proxy.
- Planned next: a yearly **Wrapped** (`/scrobble/wrapped` + story page) and an optional **ListenBrainz relay** — not in this release.
---
## [1.3] — 2026-07-31 ## [1.3] — 2026-07-31
Big change: **the entire metadata/search stack moved from Apple's iTunes to Deezer** — on web, backend downloads *and* Android. Apple's iTunes `/search` returns **403 from server/datacenter IPs**, which had broken global search, charts, artist/album/single downloads and the metadata-editor autofill in production. Deezer is free, needs no key, allows more requests, and returns explicit versions + real artist images + track durations. Web and Android now share the **same** source, so an artist resolves to one consistent name everywhere. Big change: **the entire metadata/search stack moved from Apple's iTunes to Deezer** — on web, backend downloads *and* Android. Apple's iTunes `/search` returns **403 from server/datacenter IPs**, which had broken global search, charts, artist/album/single downloads and the metadata-editor autofill in production. Deezer is free, needs no key, allows more requests, and returns explicit versions + real artist images + track durations. Web and Android now share the **same** source, so an artist resolves to one consistent name everywhere.
+39
View File
@@ -29,6 +29,7 @@ plus a **Node.js/Express backend** that downloads tracks via yt-dlp and proxies
- `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`, - `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`,
`deezer.js` (Deezer metadata: track/album/artist lookups + matching), `deezer.js` (Deezer metadata: track/album/artist lookups + matching),
`metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`. `metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`.
- `scrobble/scrobble-server.js` — **separate** listening-stats service (own container + DB), see below.
- `android/` — native Kotlin app (separate; backend-dependent web features like the - `android/` — native Kotlin app (separate; backend-dependent web features like the
metadata editor are **not** in the Android app). metadata editor are **not** in the Android app).
@@ -115,6 +116,44 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else
`albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`. `albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`.
## Listening stats / scrobble system (v2.0)
- **Why separate:** Navidrome stores only a play *count* + *last* play date (no timestamps), so
per-day/month/year stats and a "Wrapped" are impossible from it. NovaSonic *is* the player, so it
records every counted play into its **own** timestamped, multi-user log.
- **Own container `novasonic-scrobble`** (`scrobble/scrobble-server.js`) — deliberately isolated from
the download backend: the backend is heavy (yt-dlp/ffmpeg) and restarts often; an archive needs
uptime + data integrity. Runs on `node:22-slim` with the **built-in `node:sqlite`** (run with
`--experimental-sqlite`; no native build). DB `/data/novasonic-plays.db` in its own volume.
- **Routes:** `POST /scrobble/plays` (batch, **idempotent by uuid**), `GET /scrobble/history`,
`GET /scrobble/stats` (totals, top artists/albums/tracks, day/month/year series, 24h clock),
`DELETE /scrobble/plays?user=` (test cleanup), `GET /scrobble/health`.
- **Multi-user auth:** every request carries the same Subsonic token creds (`u`, `t`, `s`) the app
already uses; the service verifies them via Navidrome **`ping.view`** (cached 5 min) → the stored
`user` is real, not spoofable. `node:sqlite` **throws on extra named params**, so each query gets
exactly the params its SQL references. `from` defaults to all-time (do **not** route `0` through
`_toEpochSec`, which maps `0`→now).
- **Denormalized on purpose:** title/artist/album are stored per play so the archive survives even if
the song is later deleted from Navidrome. Deleting a song does **not** touch the play-log.
- **Capture point (Web + Android):** `logPlay()` fires at the **existing** counted-scrobble moment
(respects the user's `scrobbleMode`, consistent with Last.fm) — **not** a separate 50%/4-min rule.
It runs **in addition to** the Navidrome/Last.fm scrobble, never replacing it. Web:
`src/player/scrobbleLog.js` (localStorage queue) + `subsonicAuthParams()` in `api.js`. Android:
`data/PlayLogClient.kt` (JSON-file queue, **one reused Ktor client**) + `repo.logPlay/flushPlayLog`
+ hook in `PlayerController.scheduleScrobble`. Both queue offline and flush as an **idempotent**
batch (server dedups by uuid) on start / when connectivity returns.
- **Stats view:** Web `src/ui/stats.js` (overlay from Settings), Android `ui/screen/StatsScreen.kt` +
`StatsViewModel.kt` (from a Settings button). **Cover art is resolved client-side** from Navidrome
(`coverUrl`/`coverArtUrl` via `artistId`/`albumId`/`songId`) — the scrobble service stays
cover-agnostic; the stats endpoint just returns the IDs.
- **Config:** `SCROBBLE_URL` env → `window.NS_SCROBBLE_URL` (entrypoint) → `S.cfg.scrobbleUrl` (web) /
`scrobbleUrl` pref (Android), mirroring `backendUrl`. Port **3710**.
- **Deploy gotcha:** the scrobble subdomain **must be excluded from any SSO / forward-auth gateway**
(like the download backend) — it self-authenticates via `ping.view`, and a gateway in front
302-redirects the browser's API calls to the login portal (breaks `fetch` → "nicht erreichbar").
- **Planned:** `/scrobble/wrapped` (yearly story) + optional ListenBrainz relay. See
`docs/scrobble-plan.md`.
## Android offline system ## Android offline system
- **Storage:** `context.filesDir/offline/audio/{songId}.{ext}` and `.../covers/{coverArtId}.jpg` — private internal storage, not accessible via file manager. - **Storage:** `context.filesDir/offline/audio/{songId}.{ext}` and `.../covers/{coverArtId}.jpg` — private internal storage, not accessible via file manager.
+19 -3
View File
@@ -15,6 +15,7 @@ Built with Vite + vanilla ES modules (web) and Jetpack Compose + Media3/ExoPlaye
- **Importer & downloads** — Paste a Spotify / YouTube / YouTube Music / SoundCloud link to download via yt-dlp, with Deezer metadata enrichment and duplicate detection. - **Importer & downloads** — Paste a Spotify / YouTube / YouTube Music / SoundCloud link to download via yt-dlp, with Deezer metadata enrichment and duplicate detection.
- **Metadata editor** (web) — Edit tags + cover art; files are renamed/moved and Navidrome is rescanned on save. - **Metadata editor** (web) — Edit tags + cover art; files are renamed/moved and Navidrome is rescanned on save.
- **Offline** (Android) — Download albums, playlists, or single songs to private storage; an Offline tab to play and manage them (works without a server connection); downloaded tracks play from disk on every screen. - **Offline** (Android) — Download albums, playlists, or single songs to private storage; an Offline tab to play and manage them (works without a server connection); downloaded tracks play from disk on every screen.
- **Listening stats** — A separate, self-hosted scrobble service records **timestamped plays** (multi-user; Navidrome only keeps a play *count*). A statistics view (Web + Android, from Settings) shows totals, top artists/tracks/albums with cover art, a day/month/year play series, and a 24h listening clock. Runs *in addition to* Navidrome/Last.fm scrobbling.
- **Audiobookshelf** — Browse and stream audiobooks with chapter navigation, speed control, sleep timer, and progress sync. - **Audiobookshelf** — Browse and stream audiobooks with chapter navigation, speed control, sleep timer, and progress sync.
Web is an installable PWA (themes, German UI); Android is native Jetpack Compose + Media3/ExoPlayer with a mini player, media/ABS notifications, and Android Auto. See [CHANGELOG.md](CHANGELOG.md) for the full web/Android parity matrix. Web is an installable PWA (themes, German UI); Android is native Jetpack Compose + Media3/ExoPlayer with a mini player, media/ABS notifications, and Android Auto. See [CHANGELOG.md](CHANGELOG.md) for the full web/Android parity matrix.
@@ -28,7 +29,8 @@ NovaSonic/
├── src/ ├── src/
│ ├── player/ │ ├── player/
│ │ ├── player.js # Playback, queue, transport, media session, preload │ │ ├── player.js # Playback, queue, transport, media session, preload
│ │ └── nowplaying.js # Now Playing overlay + synced lyrics │ │ ├── nowplaying.js # Now Playing overlay + synced lyrics
│ │ └── scrobbleLog.js # Play-log capture + offline queue → scrobble service
│ ├── views/ │ ├── views/
│ │ ├── detail.js # Artist / album detail pages │ │ ├── detail.js # Artist / album detail pages
│ │ ├── views.js # Navigation, page loaders, search │ │ ├── views.js # Navigation, page loaders, search
@@ -48,6 +50,7 @@ NovaSonic/
│ │ ├── metadata.js # Metadata editor │ │ ├── metadata.js # Metadata editor
│ │ ├── modals.js # Song info, events modals │ │ ├── modals.js # Song info, events modals
│ │ ├── notify.js # Toast notifications │ │ ├── notify.js # Toast notifications
│ │ ├── stats.js # Listening statistics view (overlay)
│ │ └── settings.js # Settings page │ │ └── settings.js # Settings page
│ ├── state.js # Shared mutable state (S object) │ ├── state.js # Shared mutable state (S object)
│ ├── utils.js # Helpers (md5, esc, formatDur, toast) │ ├── utils.js # Helpers (md5, esc, formatDur, toast)
@@ -63,6 +66,8 @@ NovaSonic/
│ ├── ytdlp.js # yt-dlp download + ffmpeg metadata embedding │ ├── ytdlp.js # yt-dlp download + ffmpeg metadata embedding
│ ├── downloader.js # Unified downloadTrack / downloadAlbumTracks │ ├── downloader.js # Unified downloadTrack / downloadAlbumTracks
│ └── romaji.js # Japanese → romaji search index │ └── romaji.js # Japanese → romaji search index
├── scrobble/ # Isolated listening-stats service (own container + DB)
│ └── scrobble-server.js # node:sqlite play-log: ingest / history / stats / health
├── android/ # Native Android app (Jetpack Compose + Media3) ├── android/ # Native Android app (Jetpack Compose + Media3)
│ └── app/src/main/kotlin/com/novasonic/app/ │ └── app/src/main/kotlin/com/novasonic/app/
│ ├── api/ # SubsonicClient, AbsClient, LrcLibClient, ItunesClient (Deezer-backed) │ ├── api/ # SubsonicClient, AbsClient, LrcLibClient, ItunesClient (Deezer-backed)
@@ -84,7 +89,7 @@ NovaSonic/
├── Dockerfile ├── Dockerfile
├── docker-compose.yml ├── docker-compose.yml
├── nginx.conf ├── nginx.conf
└── docker-entrypoint.sh # Injects NAVIDROME_URL / BACKEND_URL at runtime └── docker-entrypoint.sh # Injects NAVIDROME_URL / BACKEND_URL / SCROBBLE_URL at runtime
``` ```
--- ---
@@ -104,6 +109,7 @@ Edit `.env`:
```env ```env
NAVIDROME_URL=https://music.yourdomain.com NAVIDROME_URL=https://music.yourdomain.com
BACKEND_URL=https://novasonic-api.yourdomain.com BACKEND_URL=https://novasonic-api.yourdomain.com
SCROBBLE_URL=https://novasonic-scrobble.yourdomain.com # listening-stats service
MUSIC_PATH=/srv/navidrome/music MUSIC_PATH=/srv/navidrome/music
@@ -120,7 +126,7 @@ TICKETMASTER_API_KEY= # optional — free key from developer.ticketmaster.com
docker compose up -d docker compose up -d
``` ```
Frontend on port 8080, backend on port 3700. Route both through your reverse proxy. Frontend on port 8080, backend on port 3700, scrobble service on port 3710. Route each through your reverse proxy. **Important:** the scrobble subdomain must *not* sit behind an SSO / forward-auth gateway — it authenticates itself via Navidrome (`ping.view`), and a gateway in front redirects the app's API calls to a login page.
### 3. First launch ### 3. First launch
@@ -128,6 +134,7 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro
2. Optionally add a Last.fm API key in Settings for bios and recommendations 2. Optionally add a Last.fm API key in Settings for bios and recommendations
3. Optionally connect Audiobookshelf in Settings 3. Optionally connect Audiobookshelf in Settings
4. Optionally add the NovaSonic backend URL in Settings to enable downloads 4. Optionally add the NovaSonic backend URL in Settings to enable downloads
5. Optionally add the NovaSonic scrobble URL in Settings to record listening stats
--- ---
@@ -139,6 +146,7 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro
|---|---| |---|---|
| `NAVIDROME_URL` | Pre-fills the Navidrome server URL on the login screen | | `NAVIDROME_URL` | Pre-fills the Navidrome server URL on the login screen |
| `BACKEND_URL` | Pre-fills the download backend URL on the login screen | | `BACKEND_URL` | Pre-fills the download backend URL on the login screen |
| `SCROBBLE_URL` | Pre-fills the listening-stats (scrobble) service URL on the login screen |
### Backend container ### Backend container
@@ -151,6 +159,14 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro
| `MAX_PARALLEL` | `2` | Max concurrent download jobs | | `MAX_PARALLEL` | `2` | Max concurrent download jobs |
| `TICKETMASTER_API_KEY` | — | Free key from [developer.ticketmaster.com](https://developer.ticketmaster.com) | | `TICKETMASTER_API_KEY` | — | Free key from [developer.ticketmaster.com](https://developer.ticketmaster.com) |
### Scrobble container
| Variable | Default | Description |
|---|---|---|
| `PORT` | `3710` | Port the scrobble service listens on |
| `DATA_PATH` | `/data` | Where the play-log SQLite DB is stored (own volume) |
| `NAVIDROME_URL` | — | Used to verify user credentials via `ping.view` (multi-user auth) |
--- ---
## Development ## Development