docs: v2.0 — document the scrobble / listening-stats system
- CHANGELOG: new [2.0] entry (own timestamped play-log service, stats view on web + android with cover art, remix-dedup fix, user-select UI fix, node 20->22); Platform Parity Matrix gains play-log / stats / stats-cover rows (+ Wrapped 🔜). - README: Listening stats feature, scrobble/ in the project tree, SCROBBLE_URL env + scrobble container table, port 3710 + SSO-exclusion note in setup. - CLAUDE.md: new 'Listening stats / scrobble system' section (isolation rationale, node:sqlite gotchas, ping.view multi-user auth, capture point, cover resolution, SSO deploy gotcha). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
906b3b48cb
commit
561014c83f
3 files changed
+93
-3
No files matched your search
@@ -54,11 +54,46 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an
|
||||
| Offline playback | ❌ | ✅ |
|
||||
| Offline detail / manage downloads | ❌ | ✅ |
|
||||
| Player bar on all screens | ✅ | ✅ |
|
||||
| Timestamped play-log (own scrobble) | ✅ | ✅ |
|
||||
| Listening statistics view | ✅ | ✅ |
|
||||
| Stats cover art (artist/album) | ✅ | ✅ |
|
||||
| Yearly Wrapped | 🔜 | 🔜 |
|
||||
| Romaji / Japanese sort | ✅ | ❌ |
|
||||
| Settings (Navidrome, ABS, etc.) | ✅ | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## [2.0] — 2026-08-07
|
||||
|
||||
Big change: **NovaSonic gets its own self-hosted, multi-user listening archive.** Navidrome only stores a play *count* and the *last* play date — no timestamped history — so per-day/month/year stats and a yearly "Wrapped" were impossible. Koito and Spindle (the off-the-shelf options) are effectively single-user. Since NovaSonic *is* the player, it now records every counted play into an **isolated scrobble service** with its own database, and shows the results in a **statistics view on web and Android**. This runs **in addition to** the existing Navidrome/Last.fm scrobble — it never replaces it.
|
||||
|
||||
### Added
|
||||
- **New `novasonic-scrobble` service** — a small, isolated container (`node:22-slim` + built-in `node:sqlite`, own volume) that owns the play-log. Kept separate from the download backend on purpose: the backend is heavy (yt-dlp/ffmpeg) and restarts regularly, while an archive needs uptime + data integrity. Routes:
|
||||
- `POST /scrobble/plays` — batch ingest, **idempotent by uuid** (safe offline-retry)
|
||||
- `GET /scrobble/history` — raw chronological log (the archive)
|
||||
- `GET /scrobble/stats` — totals, top artists/albums/tracks, day/month/year series, 24h listening clock (on-the-fly `GROUP BY strftime`, local-tz bucketing)
|
||||
- `DELETE /scrobble/plays?user=` — purge one user's rows (test cleanup)
|
||||
- `GET /scrobble/health` — liveness
|
||||
- **Multi-user**: every request carries the same Subsonic token creds the app already uses; the service verifies them via Navidrome `ping.view` (cached), so the stored `user` is real, not spoofable. No new login.
|
||||
- **Play capture (Web + Android)** — fires at the *existing* counted-play moment (respects the user's `scrobbleMode`, consistent with Last.fm), with an **offline-safe queue** (localStorage on web, a JSON file on Android) flushed as an idempotent batch on start and when connectivity returns.
|
||||
- **Statistics view (Web + Android)** — opened from Settings ("📊 Meine Statistiken"): totals tiles (**Plays** + listening time), a day/month/year play series, top artists/tracks/albums, and a 24h listening clock, with period tabs (Alles / Dieses Jahr / Letzte 30 Tage).
|
||||
- **Cover art in stats (Web + Android)** — top artists show a round artist image, tracks/albums a square cover, resolved client-side from Navidrome (the scrobble service stays cover-agnostic).
|
||||
- **`SCROBBLE_URL`** environment variable (frontend) + a **Scrobble URL** field with a test button in Settings on both platforms.
|
||||
|
||||
### Changed
|
||||
- **Backend** — base image bumped `node:20-slim` → **`node:22-slim`** (Node 20 reaches end-of-life April 2026; enables the built-in `node:sqlite`).
|
||||
|
||||
### Fixed
|
||||
- **Downloads** — remix / live / acoustic / instrumental tracks were skipped as duplicates of the base track: duplicate detection stripped *all* bracketed content, collapsing "Song (Remix)" → "Song". Version-marker brackets are now preserved, so a distinct recording downloads (noise brackets like `(feat. X)` / `(Official Video)` are still stripped). Verified against Deezer + unit-tested both directions.
|
||||
- **UI** — dragging the mouse near a field edge (Global search, importer) started selecting the surrounding titles/labels instead of feeling like an app. `user-select` is now disabled on the `#app` shell; real text fields and lyrics stay selectable.
|
||||
|
||||
### Notes
|
||||
- The scrobble subdomain **must be excluded from any SSO / forward-auth gateway** (Authelia/Authentik/etc.), exactly like the download backend — it self-authenticates via `ping.view`, and a gateway in front 302-redirects the browser's API calls to the login portal (breaks `fetch`).
|
||||
- Deploy: add `SCROBBLE_URL` to `.env`, `docker compose up -d novasonic-scrobble`, and route port **3710** through your reverse proxy.
|
||||
- Planned next: a yearly **Wrapped** (`/scrobble/wrapped` + story page) and an optional **ListenBrainz relay** — not in this release.
|
||||
|
||||
---
|
||||
|
||||
## [1.3] — 2026-07-31
|
||||
|
||||
Big change: **the entire metadata/search stack moved from Apple's iTunes to Deezer** — on web, backend downloads *and* Android. Apple's iTunes `/search` returns **403 from server/datacenter IPs**, which had broken global search, charts, artist/album/single downloads and the metadata-editor autofill in production. Deezer is free, needs no key, allows more requests, and returns explicit versions + real artist images + track durations. Web and Android now share the **same** source, so an artist resolves to one consistent name everywhere.
|
||||
|
||||
@@ -29,6 +29,7 @@ plus a **Node.js/Express backend** that downloads tracks via yt-dlp and proxies
|
||||
- `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`,
|
||||
`deezer.js` (Deezer metadata: track/album/artist lookups + matching),
|
||||
`metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`.
|
||||
- `scrobble/scrobble-server.js` — **separate** listening-stats service (own container + DB), see below.
|
||||
- `android/` — native Kotlin app (separate; backend-dependent web features like the
|
||||
metadata editor are **not** in the Android app).
|
||||
|
||||
@@ -115,6 +116,44 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a
|
||||
share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else
|
||||
`albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`.
|
||||
|
||||
## Listening stats / scrobble system (v2.0)
|
||||
|
||||
- **Why separate:** Navidrome stores only a play *count* + *last* play date (no timestamps), so
|
||||
per-day/month/year stats and a "Wrapped" are impossible from it. NovaSonic *is* the player, so it
|
||||
records every counted play into its **own** timestamped, multi-user log.
|
||||
- **Own container `novasonic-scrobble`** (`scrobble/scrobble-server.js`) — deliberately isolated from
|
||||
the download backend: the backend is heavy (yt-dlp/ffmpeg) and restarts often; an archive needs
|
||||
uptime + data integrity. Runs on `node:22-slim` with the **built-in `node:sqlite`** (run with
|
||||
`--experimental-sqlite`; no native build). DB `/data/novasonic-plays.db` in its own volume.
|
||||
- **Routes:** `POST /scrobble/plays` (batch, **idempotent by uuid**), `GET /scrobble/history`,
|
||||
`GET /scrobble/stats` (totals, top artists/albums/tracks, day/month/year series, 24h clock),
|
||||
`DELETE /scrobble/plays?user=` (test cleanup), `GET /scrobble/health`.
|
||||
- **Multi-user auth:** every request carries the same Subsonic token creds (`u`, `t`, `s`) the app
|
||||
already uses; the service verifies them via Navidrome **`ping.view`** (cached 5 min) → the stored
|
||||
`user` is real, not spoofable. `node:sqlite` **throws on extra named params**, so each query gets
|
||||
exactly the params its SQL references. `from` defaults to all-time (do **not** route `0` through
|
||||
`_toEpochSec`, which maps `0`→now).
|
||||
- **Denormalized on purpose:** title/artist/album are stored per play so the archive survives even if
|
||||
the song is later deleted from Navidrome. Deleting a song does **not** touch the play-log.
|
||||
- **Capture point (Web + Android):** `logPlay()` fires at the **existing** counted-scrobble moment
|
||||
(respects the user's `scrobbleMode`, consistent with Last.fm) — **not** a separate 50%/4-min rule.
|
||||
It runs **in addition to** the Navidrome/Last.fm scrobble, never replacing it. Web:
|
||||
`src/player/scrobbleLog.js` (localStorage queue) + `subsonicAuthParams()` in `api.js`. Android:
|
||||
`data/PlayLogClient.kt` (JSON-file queue, **one reused Ktor client**) + `repo.logPlay/flushPlayLog`
|
||||
+ hook in `PlayerController.scheduleScrobble`. Both queue offline and flush as an **idempotent**
|
||||
batch (server dedups by uuid) on start / when connectivity returns.
|
||||
- **Stats view:** Web `src/ui/stats.js` (overlay from Settings), Android `ui/screen/StatsScreen.kt` +
|
||||
`StatsViewModel.kt` (from a Settings button). **Cover art is resolved client-side** from Navidrome
|
||||
(`coverUrl`/`coverArtUrl` via `artistId`/`albumId`/`songId`) — the scrobble service stays
|
||||
cover-agnostic; the stats endpoint just returns the IDs.
|
||||
- **Config:** `SCROBBLE_URL` env → `window.NS_SCROBBLE_URL` (entrypoint) → `S.cfg.scrobbleUrl` (web) /
|
||||
`scrobbleUrl` pref (Android), mirroring `backendUrl`. Port **3710**.
|
||||
- **Deploy gotcha:** the scrobble subdomain **must be excluded from any SSO / forward-auth gateway**
|
||||
(like the download backend) — it self-authenticates via `ping.view`, and a gateway in front
|
||||
302-redirects the browser's API calls to the login portal (breaks `fetch` → "nicht erreichbar").
|
||||
- **Planned:** `/scrobble/wrapped` (yearly story) + optional ListenBrainz relay. See
|
||||
`docs/scrobble-plan.md`.
|
||||
|
||||
## Android offline system
|
||||
|
||||
- **Storage:** `context.filesDir/offline/audio/{songId}.{ext}` and `.../covers/{coverArtId}.jpg` — private internal storage, not accessible via file manager.
|
||||
|
||||
@@ -15,6 +15,7 @@ Built with Vite + vanilla ES modules (web) and Jetpack Compose + Media3/ExoPlaye
|
||||
- **Importer & downloads** — Paste a Spotify / YouTube / YouTube Music / SoundCloud link to download via yt-dlp, with Deezer metadata enrichment and duplicate detection.
|
||||
- **Metadata editor** (web) — Edit tags + cover art; files are renamed/moved and Navidrome is rescanned on save.
|
||||
- **Offline** (Android) — Download albums, playlists, or single songs to private storage; an Offline tab to play and manage them (works without a server connection); downloaded tracks play from disk on every screen.
|
||||
- **Listening stats** — A separate, self-hosted scrobble service records **timestamped plays** (multi-user; Navidrome only keeps a play *count*). A statistics view (Web + Android, from Settings) shows totals, top artists/tracks/albums with cover art, a day/month/year play series, and a 24h listening clock. Runs *in addition to* Navidrome/Last.fm scrobbling.
|
||||
- **Audiobookshelf** — Browse and stream audiobooks with chapter navigation, speed control, sleep timer, and progress sync.
|
||||
|
||||
Web is an installable PWA (themes, German UI); Android is native Jetpack Compose + Media3/ExoPlayer with a mini player, media/ABS notifications, and Android Auto. See [CHANGELOG.md](CHANGELOG.md) for the full web/Android parity matrix.
|
||||
@@ -28,7 +29,8 @@ NovaSonic/
|
||||
├── src/
|
||||
│ ├── player/
|
||||
│ │ ├── player.js # Playback, queue, transport, media session, preload
|
||||
│ │ └── nowplaying.js # Now Playing overlay + synced lyrics
|
||||
│ │ ├── nowplaying.js # Now Playing overlay + synced lyrics
|
||||
│ │ └── scrobbleLog.js # Play-log capture + offline queue → scrobble service
|
||||
│ ├── views/
|
||||
│ │ ├── detail.js # Artist / album detail pages
|
||||
│ │ ├── views.js # Navigation, page loaders, search
|
||||
@@ -48,6 +50,7 @@ NovaSonic/
|
||||
│ │ ├── metadata.js # Metadata editor
|
||||
│ │ ├── modals.js # Song info, events modals
|
||||
│ │ ├── notify.js # Toast notifications
|
||||
│ │ ├── stats.js # Listening statistics view (overlay)
|
||||
│ │ └── settings.js # Settings page
|
||||
│ ├── state.js # Shared mutable state (S object)
|
||||
│ ├── utils.js # Helpers (md5, esc, formatDur, toast)
|
||||
@@ -63,6 +66,8 @@ NovaSonic/
|
||||
│ ├── ytdlp.js # yt-dlp download + ffmpeg metadata embedding
|
||||
│ ├── downloader.js # Unified downloadTrack / downloadAlbumTracks
|
||||
│ └── romaji.js # Japanese → romaji search index
|
||||
├── scrobble/ # Isolated listening-stats service (own container + DB)
|
||||
│ └── scrobble-server.js # node:sqlite play-log: ingest / history / stats / health
|
||||
├── android/ # Native Android app (Jetpack Compose + Media3)
|
||||
│ └── app/src/main/kotlin/com/novasonic/app/
|
||||
│ ├── api/ # SubsonicClient, AbsClient, LrcLibClient, ItunesClient (Deezer-backed)
|
||||
@@ -84,7 +89,7 @@ NovaSonic/
|
||||
├── Dockerfile
|
||||
├── docker-compose.yml
|
||||
├── nginx.conf
|
||||
└── docker-entrypoint.sh # Injects NAVIDROME_URL / BACKEND_URL at runtime
|
||||
└── docker-entrypoint.sh # Injects NAVIDROME_URL / BACKEND_URL / SCROBBLE_URL at runtime
|
||||
```
|
||||
|
||||
---
|
||||
@@ -104,6 +109,7 @@ Edit `.env`:
|
||||
```env
|
||||
NAVIDROME_URL=https://music.yourdomain.com
|
||||
BACKEND_URL=https://novasonic-api.yourdomain.com
|
||||
SCROBBLE_URL=https://novasonic-scrobble.yourdomain.com # listening-stats service
|
||||
|
||||
MUSIC_PATH=/srv/navidrome/music
|
||||
|
||||
@@ -120,7 +126,7 @@ TICKETMASTER_API_KEY= # optional — free key from developer.ticketmaster.com
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Frontend on port 8080, backend on port 3700. Route both through your reverse proxy.
|
||||
Frontend on port 8080, backend on port 3700, scrobble service on port 3710. Route each through your reverse proxy. **Important:** the scrobble subdomain must *not* sit behind an SSO / forward-auth gateway — it authenticates itself via Navidrome (`ping.view`), and a gateway in front redirects the app's API calls to a login page.
|
||||
|
||||
### 3. First launch
|
||||
|
||||
@@ -128,6 +134,7 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro
|
||||
2. Optionally add a Last.fm API key in Settings for bios and recommendations
|
||||
3. Optionally connect Audiobookshelf in Settings
|
||||
4. Optionally add the NovaSonic backend URL in Settings to enable downloads
|
||||
5. Optionally add the NovaSonic scrobble URL in Settings to record listening stats
|
||||
|
||||
---
|
||||
|
||||
@@ -139,6 +146,7 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro
|
||||
|---|---|
|
||||
| `NAVIDROME_URL` | Pre-fills the Navidrome server URL on the login screen |
|
||||
| `BACKEND_URL` | Pre-fills the download backend URL on the login screen |
|
||||
| `SCROBBLE_URL` | Pre-fills the listening-stats (scrobble) service URL on the login screen |
|
||||
|
||||
### Backend container
|
||||
|
||||
@@ -151,6 +159,14 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro
|
||||
| `MAX_PARALLEL` | `2` | Max concurrent download jobs |
|
||||
| `TICKETMASTER_API_KEY` | — | Free key from [developer.ticketmaster.com](https://developer.ticketmaster.com) |
|
||||
|
||||
### Scrobble container
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `PORT` | `3710` | Port the scrobble service listens on |
|
||||
| `DATA_PATH` | `/data` | Where the play-log SQLite DB is stored (own volume) |
|
||||
| `NAVIDROME_URL` | — | Used to verify user credentials via `ping.view` (multi-user auth) |
|
||||
|
||||
---
|
||||
|
||||
## Development
|
||||
|
||||
Reference in new issue
Block a user