diff --git a/CHANGELOG.md b/CHANGELOG.md index e7f6126..00dd835 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,11 +54,46 @@ Versioning: two-part `MAJOR.MINOR` — each release bumps the minor (features an | Offline playback | ❌ | ✅ | | Offline detail / manage downloads | ❌ | ✅ | | Player bar on all screens | ✅ | ✅ | +| Timestamped play-log (own scrobble) | ✅ | ✅ | +| Listening statistics view | ✅ | ✅ | +| Stats cover art (artist/album) | ✅ | ✅ | +| Yearly Wrapped | 🔜 | 🔜 | | Romaji / Japanese sort | ✅ | ❌ | | Settings (Navidrome, ABS, etc.) | ✅ | ✅ | --- +## [2.0] — 2026-08-07 + +Big change: **NovaSonic gets its own self-hosted, multi-user listening archive.** Navidrome only stores a play *count* and the *last* play date — no timestamped history — so per-day/month/year stats and a yearly "Wrapped" were impossible. Koito and Spindle (the off-the-shelf options) are effectively single-user. Since NovaSonic *is* the player, it now records every counted play into an **isolated scrobble service** with its own database, and shows the results in a **statistics view on web and Android**. This runs **in addition to** the existing Navidrome/Last.fm scrobble — it never replaces it. + +### Added +- **New `novasonic-scrobble` service** — a small, isolated container (`node:22-slim` + built-in `node:sqlite`, own volume) that owns the play-log. Kept separate from the download backend on purpose: the backend is heavy (yt-dlp/ffmpeg) and restarts regularly, while an archive needs uptime + data integrity. Routes: + - `POST /scrobble/plays` — batch ingest, **idempotent by uuid** (safe offline-retry) + - `GET /scrobble/history` — raw chronological log (the archive) + - `GET /scrobble/stats` — totals, top artists/albums/tracks, day/month/year series, 24h listening clock (on-the-fly `GROUP BY strftime`, local-tz bucketing) + - `DELETE /scrobble/plays?user=` — purge one user's rows (test cleanup) + - `GET /scrobble/health` — liveness + - **Multi-user**: every request carries the same Subsonic token creds the app already uses; the service verifies them via Navidrome `ping.view` (cached), so the stored `user` is real, not spoofable. No new login. +- **Play capture (Web + Android)** — fires at the *existing* counted-play moment (respects the user's `scrobbleMode`, consistent with Last.fm), with an **offline-safe queue** (localStorage on web, a JSON file on Android) flushed as an idempotent batch on start and when connectivity returns. +- **Statistics view (Web + Android)** — opened from Settings ("📊 Meine Statistiken"): totals tiles (**Plays** + listening time), a day/month/year play series, top artists/tracks/albums, and a 24h listening clock, with period tabs (Alles / Dieses Jahr / Letzte 30 Tage). +- **Cover art in stats (Web + Android)** — top artists show a round artist image, tracks/albums a square cover, resolved client-side from Navidrome (the scrobble service stays cover-agnostic). +- **`SCROBBLE_URL`** environment variable (frontend) + a **Scrobble URL** field with a test button in Settings on both platforms. + +### Changed +- **Backend** — base image bumped `node:20-slim` → **`node:22-slim`** (Node 20 reaches end-of-life April 2026; enables the built-in `node:sqlite`). + +### Fixed +- **Downloads** — remix / live / acoustic / instrumental tracks were skipped as duplicates of the base track: duplicate detection stripped *all* bracketed content, collapsing "Song (Remix)" → "Song". Version-marker brackets are now preserved, so a distinct recording downloads (noise brackets like `(feat. X)` / `(Official Video)` are still stripped). Verified against Deezer + unit-tested both directions. +- **UI** — dragging the mouse near a field edge (Global search, importer) started selecting the surrounding titles/labels instead of feeling like an app. `user-select` is now disabled on the `#app` shell; real text fields and lyrics stay selectable. + +### Notes +- The scrobble subdomain **must be excluded from any SSO / forward-auth gateway** (Authelia/Authentik/etc.), exactly like the download backend — it self-authenticates via `ping.view`, and a gateway in front 302-redirects the browser's API calls to the login portal (breaks `fetch`). +- Deploy: add `SCROBBLE_URL` to `.env`, `docker compose up -d novasonic-scrobble`, and route port **3710** through your reverse proxy. +- Planned next: a yearly **Wrapped** (`/scrobble/wrapped` + story page) and an optional **ListenBrainz relay** — not in this release. + +--- + ## [1.3] — 2026-07-31 Big change: **the entire metadata/search stack moved from Apple's iTunes to Deezer** — on web, backend downloads *and* Android. Apple's iTunes `/search` returns **403 from server/datacenter IPs**, which had broken global search, charts, artist/album/single downloads and the metadata-editor autofill in production. Deezer is free, needs no key, allows more requests, and returns explicit versions + real artist images + track durations. Web and Android now share the **same** source, so an artist resolves to one consistent name everywhere. diff --git a/CLAUDE.md b/CLAUDE.md index dc7b524..d7b2ff4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,6 +29,7 @@ plus a **Node.js/Express backend** that downloads tracks via yt-dlp and proxies - `backend/lib/` — `ytdlp.js` (download + cascade + ffmpeg tagging), `downloader.js`, `deezer.js` (Deezer metadata: track/album/artist lookups + matching), `metadata.js` (artwork, file resolution helpers, duplicate detection), `config.js`, `jobs.js`, `romaji.js`. +- `scrobble/scrobble-server.js` — **separate** listening-stats service (own container + DB), see below. - `android/` — native Kotlin app (separate; backend-dependent web features like the metadata editor are **not** in the Android app). @@ -115,6 +116,44 @@ Backend runs in Docker (see `docker-compose.yml`); on the deploy host it lives a share the same `mainArtist` value (= Deezer `collectionArtistName` when available, else `albumArtist` param, else track artist). Never use `rawArtist` for `album_artist`. +## Listening stats / scrobble system (v2.0) + +- **Why separate:** Navidrome stores only a play *count* + *last* play date (no timestamps), so + per-day/month/year stats and a "Wrapped" are impossible from it. NovaSonic *is* the player, so it + records every counted play into its **own** timestamped, multi-user log. +- **Own container `novasonic-scrobble`** (`scrobble/scrobble-server.js`) — deliberately isolated from + the download backend: the backend is heavy (yt-dlp/ffmpeg) and restarts often; an archive needs + uptime + data integrity. Runs on `node:22-slim` with the **built-in `node:sqlite`** (run with + `--experimental-sqlite`; no native build). DB `/data/novasonic-plays.db` in its own volume. +- **Routes:** `POST /scrobble/plays` (batch, **idempotent by uuid**), `GET /scrobble/history`, + `GET /scrobble/stats` (totals, top artists/albums/tracks, day/month/year series, 24h clock), + `DELETE /scrobble/plays?user=` (test cleanup), `GET /scrobble/health`. +- **Multi-user auth:** every request carries the same Subsonic token creds (`u`, `t`, `s`) the app + already uses; the service verifies them via Navidrome **`ping.view`** (cached 5 min) → the stored + `user` is real, not spoofable. `node:sqlite` **throws on extra named params**, so each query gets + exactly the params its SQL references. `from` defaults to all-time (do **not** route `0` through + `_toEpochSec`, which maps `0`→now). +- **Denormalized on purpose:** title/artist/album are stored per play so the archive survives even if + the song is later deleted from Navidrome. Deleting a song does **not** touch the play-log. +- **Capture point (Web + Android):** `logPlay()` fires at the **existing** counted-scrobble moment + (respects the user's `scrobbleMode`, consistent with Last.fm) — **not** a separate 50%/4-min rule. + It runs **in addition to** the Navidrome/Last.fm scrobble, never replacing it. Web: + `src/player/scrobbleLog.js` (localStorage queue) + `subsonicAuthParams()` in `api.js`. Android: + `data/PlayLogClient.kt` (JSON-file queue, **one reused Ktor client**) + `repo.logPlay/flushPlayLog` + + hook in `PlayerController.scheduleScrobble`. Both queue offline and flush as an **idempotent** + batch (server dedups by uuid) on start / when connectivity returns. +- **Stats view:** Web `src/ui/stats.js` (overlay from Settings), Android `ui/screen/StatsScreen.kt` + + `StatsViewModel.kt` (from a Settings button). **Cover art is resolved client-side** from Navidrome + (`coverUrl`/`coverArtUrl` via `artistId`/`albumId`/`songId`) — the scrobble service stays + cover-agnostic; the stats endpoint just returns the IDs. +- **Config:** `SCROBBLE_URL` env → `window.NS_SCROBBLE_URL` (entrypoint) → `S.cfg.scrobbleUrl` (web) / + `scrobbleUrl` pref (Android), mirroring `backendUrl`. Port **3710**. +- **Deploy gotcha:** the scrobble subdomain **must be excluded from any SSO / forward-auth gateway** + (like the download backend) — it self-authenticates via `ping.view`, and a gateway in front + 302-redirects the browser's API calls to the login portal (breaks `fetch` → "nicht erreichbar"). +- **Planned:** `/scrobble/wrapped` (yearly story) + optional ListenBrainz relay. See + `docs/scrobble-plan.md`. + ## Android offline system - **Storage:** `context.filesDir/offline/audio/{songId}.{ext}` and `.../covers/{coverArtId}.jpg` — private internal storage, not accessible via file manager. diff --git a/README.md b/README.md index dce77c9..dfbd662 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,7 @@ Built with Vite + vanilla ES modules (web) and Jetpack Compose + Media3/ExoPlaye - **Importer & downloads** — Paste a Spotify / YouTube / YouTube Music / SoundCloud link to download via yt-dlp, with Deezer metadata enrichment and duplicate detection. - **Metadata editor** (web) — Edit tags + cover art; files are renamed/moved and Navidrome is rescanned on save. - **Offline** (Android) — Download albums, playlists, or single songs to private storage; an Offline tab to play and manage them (works without a server connection); downloaded tracks play from disk on every screen. +- **Listening stats** — A separate, self-hosted scrobble service records **timestamped plays** (multi-user; Navidrome only keeps a play *count*). A statistics view (Web + Android, from Settings) shows totals, top artists/tracks/albums with cover art, a day/month/year play series, and a 24h listening clock. Runs *in addition to* Navidrome/Last.fm scrobbling. - **Audiobookshelf** — Browse and stream audiobooks with chapter navigation, speed control, sleep timer, and progress sync. Web is an installable PWA (themes, German UI); Android is native Jetpack Compose + Media3/ExoPlayer with a mini player, media/ABS notifications, and Android Auto. See [CHANGELOG.md](CHANGELOG.md) for the full web/Android parity matrix. @@ -28,7 +29,8 @@ NovaSonic/ ├── src/ │ ├── player/ │ │ ├── player.js # Playback, queue, transport, media session, preload -│ │ └── nowplaying.js # Now Playing overlay + synced lyrics +│ │ ├── nowplaying.js # Now Playing overlay + synced lyrics +│ │ └── scrobbleLog.js # Play-log capture + offline queue → scrobble service │ ├── views/ │ │ ├── detail.js # Artist / album detail pages │ │ ├── views.js # Navigation, page loaders, search @@ -48,6 +50,7 @@ NovaSonic/ │ │ ├── metadata.js # Metadata editor │ │ ├── modals.js # Song info, events modals │ │ ├── notify.js # Toast notifications +│ │ ├── stats.js # Listening statistics view (overlay) │ │ └── settings.js # Settings page │ ├── state.js # Shared mutable state (S object) │ ├── utils.js # Helpers (md5, esc, formatDur, toast) @@ -63,6 +66,8 @@ NovaSonic/ │ ├── ytdlp.js # yt-dlp download + ffmpeg metadata embedding │ ├── downloader.js # Unified downloadTrack / downloadAlbumTracks │ └── romaji.js # Japanese → romaji search index +├── scrobble/ # Isolated listening-stats service (own container + DB) +│ └── scrobble-server.js # node:sqlite play-log: ingest / history / stats / health ├── android/ # Native Android app (Jetpack Compose + Media3) │ └── app/src/main/kotlin/com/novasonic/app/ │ ├── api/ # SubsonicClient, AbsClient, LrcLibClient, ItunesClient (Deezer-backed) @@ -84,7 +89,7 @@ NovaSonic/ ├── Dockerfile ├── docker-compose.yml ├── nginx.conf -└── docker-entrypoint.sh # Injects NAVIDROME_URL / BACKEND_URL at runtime +└── docker-entrypoint.sh # Injects NAVIDROME_URL / BACKEND_URL / SCROBBLE_URL at runtime ``` --- @@ -104,6 +109,7 @@ Edit `.env`: ```env NAVIDROME_URL=https://music.yourdomain.com BACKEND_URL=https://novasonic-api.yourdomain.com +SCROBBLE_URL=https://novasonic-scrobble.yourdomain.com # listening-stats service MUSIC_PATH=/srv/navidrome/music @@ -120,7 +126,7 @@ TICKETMASTER_API_KEY= # optional — free key from developer.ticketmaster.com docker compose up -d ``` -Frontend on port 8080, backend on port 3700. Route both through your reverse proxy. +Frontend on port 8080, backend on port 3700, scrobble service on port 3710. Route each through your reverse proxy. **Important:** the scrobble subdomain must *not* sit behind an SSO / forward-auth gateway — it authenticates itself via Navidrome (`ping.view`), and a gateway in front redirects the app's API calls to a login page. ### 3. First launch @@ -128,6 +134,7 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro 2. Optionally add a Last.fm API key in Settings for bios and recommendations 3. Optionally connect Audiobookshelf in Settings 4. Optionally add the NovaSonic backend URL in Settings to enable downloads +5. Optionally add the NovaSonic scrobble URL in Settings to record listening stats --- @@ -139,6 +146,7 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro |---|---| | `NAVIDROME_URL` | Pre-fills the Navidrome server URL on the login screen | | `BACKEND_URL` | Pre-fills the download backend URL on the login screen | +| `SCROBBLE_URL` | Pre-fills the listening-stats (scrobble) service URL on the login screen | ### Backend container @@ -151,6 +159,14 @@ Frontend on port 8080, backend on port 3700. Route both through your reverse pro | `MAX_PARALLEL` | `2` | Max concurrent download jobs | | `TICKETMASTER_API_KEY` | — | Free key from [developer.ticketmaster.com](https://developer.ticketmaster.com) | +### Scrobble container + +| Variable | Default | Description | +|---|---|---| +| `PORT` | `3710` | Port the scrobble service listens on | +| `DATA_PATH` | `/data` | Where the play-log SQLite DB is stored (own volume) | +| `NAVIDROME_URL` | — | Used to verify user credentials via `ping.view` (multi-user auth) | + --- ## Development