Docker/InkNote
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
52bd2b8f3dd2aeee157d976434b6115018f8ef21
InkNote/readme.md
T
marcandClaude Opus 5 52bd2b8f3d Put the interface into English, and give the notebook one way in 
The interface had drifted into two languages — the frame in English, everything
added later in German — which is worse than either, since the same idea appeared
under two names depending on when that part was written. Sorting by title still
uses the German collation, so Übung 2 comes before Übung 10: that is about the
notes, not the chrome.

Two archive folders were renamed with it (bilder/ → images/, zeichnungen/ →
drawings/). The importer reads both spellings, and both the old and new marker
line for a drawing — archives exported before this are on disk, and an import
that quietly dropped their drawings would be the worst kind of tidy-up.

Making a note, uploading a PDF and importing Markdown were three buttons side by
side, two of them icons you had to hover to identify. They are the same act —
putting something new in this notebook — so they are now one + menu.

The search box moves out of the sidebar and into the note list, labelled with the
notebook it narrows. Nothing changed about what it does: it has always searched
only the notebook you are in, but from the sidebar it read as searching
everything.

A separator no longer carries a sentence explaining the order the sort menu
already names. Renaming joins removing on the right-click menu — a left-click on
the label started an edit, so the row could not be touched without falling into
one.

218 unit tests, 238 browser checks across eleven suites. Two of those caught real
breakage from this change rather than needing a translation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-11 02:17:22 +02:00

1012 lines
47 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ✏️ InkNote
A self-hosted note-taking application with rich text editing, freehand drawing, full-text search and encrypted sync — inspired by Samsung Notes.
---
## Features
### Core
- **Rich Text Notes** — Bold, italic, underline, headings, bullet/numbered lists, task lists, quotes, code blocks
- **Tables** — Insert, resize columns, add/remove rows and columns, merge cells, toggle header row
- **Note Links** — Type `@` to link to another note or notebook; every note shows what links back to it
- **Version History** — Past states of every note, restorable; restoring is itself reversible
- **Full-Text Search** — Searches note bodies, not just titles, and works offline
- **Image Attachments** — Pasted images are stored as deduplicated blobs, not inlined into the note
- **Freehand Drawing** — Pen, highlighter, eraser with adjustable thickness and color palette (20 colors, 7 pen sizes)
- **Undo/Redo** — Full history stack for drawing operations
- **Notebooks & Folders** — Create notebooks with custom gradient covers, organize notes by folder
- **Star/Favorite** — Quick access to important notes
- **PDF Reading & Annotation** — Import a PDF, read it, select its text, draw on it, pin notes to individual passages, write notes beside it, and export it with or without your marks
- **Export** — Any note, notebook or the whole account as Markdown or PDF, entirely in the browser
- **Import** — Markdown files, or a ZIP this app exported, come back as notes with their images
- **Sorting** — Each notebook remembers how its notes are ordered, including an arrangement made by hand
- **Dark & Light Mode** — Full theme support with refined color schemes
### Sync & Storage
- **Offline-First** — Works fully offline, syncs when connection returns
- **Conflict-Aware Sync** — A write from a device that missed another's changes keeps the state it replaces as a version, rather than burying it
- **WebSocket Real-Time Push** — Changes appear on other devices instantly
- **Encrypted at Rest** — AES-256-GCM for note content *and* uploaded files on the server
- **JWT Authentication** — Username + password accounts with bcrypt password hashing. No public
sign-up: an administrator creates the accounts
- **Vector Drawings** — Strokes are stored as vectors in one canonical page space, so a drawing looks identical on every screen and stays sharp at any pixel density
### Architecture
- **Self-Hosted Backend** — Node.js + Express + SQLite (or PostgreSQL)
- **Docker Ready** — docker-compose up and you're running
- **Client Sync Engine** — Portable JS library for any frontend (React Native, Flutter JS bridge, Electron, web)
---
## Project Structure
```
inknote/
├── docker-compose.yml # Deploy from the Gitea registry
├── docker-compose.dev.yml # Build from source for local work
├── .env.example # Copy to .env — Compose reads it from here
├── build.sh # Build & push images to Gitea
├── backend/
│ ├── server.js # Express API + WebSocket server
│ ├── sync-client.js # Client-side sync engine library
│ ├── package.json # Node dependencies
│ └── Dockerfile # Server container
├── frontend/
│ ├── src/
│ │ ├── App.jsx # Root: where the user is, which dialog is open
│ │ ├── types.ts # Shapes shared by server, IndexedDB and UI
│ │ ├── syncEngine.ts # Offline-first sync + API client
│ │ ├── offlineStore.ts # IndexedDB wrapper
│ │ ├── search.ts # Local full-text index
│ │ ├── hooks/ # Note data, search, layout, history, drag
│ │ ├── export/ # Markdown, ZIP and print-to-PDF
│ │ ├── ui/ # Sidebar, dialogs, icons, palette
│ │ ├── features/notes/ # List, editor pane, version history
│ │ ├── features/notebooks/ # Dashboard
│ │ ├── features/drawing/ # Canonical stroke model + render layer
│ │ └── editor/ # TipTap schema, toolbar, note links, attachments
│ ├── public/sw.js # PWA service worker
│ ├── nginx.conf # Static serving + API/WS proxy
│ └── dockerfile # Web container
├── e2e/ # Browser suites — run with ./e2e/run.sh
└── readme.md # This file
```
---
## Quick Start
### Where the data lives
Both compose files use **bind mounts**, not named volumes, so everything sits in
a plain directory on the Docker host and is picked up by whatever backs that
host up:
```
$INKNOTE_DATA_DIR/
├── db/ # inknote.db — notes, drawings, version history
└── uploads/ # image attachments, legacy raster drawings
```
Default is `./data` for development and `/srv/inknote` in production; override
with `INKNOTE_DATA_DIR`. It is a default, not an extra path — set it and the
default location is never touched.
**Create the directory before the first start.** Docker will happily create a
missing bind-mount source itself, but it creates it as `root:root`, and the
container runs as uid 1000 and then cannot write:
```bash
mkdir -p /srv/inknote/{db,uploads} && chown -R 1000:1000 /srv/inknote
```
The path has to start with `/` or `./`. Anything else Docker reads as the name
of a named volume, and a name containing a slash fails outright.
#### Keeping it under `/root`
`/root` is mode `0700`, so uid 1000 cannot pass through it — the server fails to
write no matter who owns the directory underneath. Two ways out:
```bash
# A) let uid 1000 traverse /root, without making it readable
mkdir -p /root/inknote/{db,uploads}
chown -R 1000:1000 /root/inknote
chmod o+x /root # traverse only — `ls /root` is still refused
# .env: INKNOTE_DATA_DIR=/root/inknote
# B) run the container as root instead — no chown, no chmod, but a break-out
# lands on root rather than on an unprivileged user
# .env: INKNOTE_DATA_DIR=/root/inknote
# INKNOTE_UID=0
# INKNOTE_GID=0
```
A is the better trade: it keeps the unprivileged container, which is the layer
that matters if the app is ever exploited. Neither is needed under `/srv` or
`/opt`, which is why those are the default.
> Back up the `.env` holding `ENCRYPTION_KEY` too. Note content is encrypted at
> rest, so a copy of the database without that key cannot be read.
### 1. Local development (Docker)
```bash
# Generate secrets once
printf 'JWT_SECRET=%s\nENCRYPTION_KEY=%s\n' "$(openssl rand -base64 32)" "$(openssl rand -hex 32)" > .env
# Build and run backend + nginx-served frontend
docker compose -f docker-compose.dev.yml up -d --build # → http://localhost:3000
# Same, plus a Vite dev server with hot reload
docker compose -f docker-compose.dev.yml --profile dev up -d --build # → http://localhost:5173
```
`docker-compose.dev.yml` builds from source and uses its own volumes, so it never
touches production data.
### 2. Docker Deployment
```bash
mkdir -p /srv/inknote/{db,uploads} && chown -R 1000:1000 /srv/inknote
cp .env.example .env # then fill it in, see below
docker login gitea.honeyy.duckdns.org
docker compose pull && docker compose up -d
docker compose logs inknote-server | head # says which account became admin
```
`.env` needs `JWT_SECRET`, `ENCRYPTION_KEY` and — on a fresh install —
`INKNOTE_ADMIN_USER` plus `INKNOTE_ADMIN_PASSWORD`, since there is no sign-up to
create the first account with. On an install that already has accounts,
`INKNOTE_ADMIN_USER` alone decides which of them is promoted; without it the
oldest one is, which is rarely the one you meant.
The web UI listens on port `3000`, the API only inside the compose network. If a
reverse proxy sits in front, bind the port to localhost (`"127.0.0.1:3000:80"`),
otherwise the app is reachable on the host's address without TLS.
---
## API Reference
### Authentication
| Method | Endpoint | Body | Response |
|--------|-----------------------|-------------------------------|---------------------|
| POST | `/api/auth/login` | `{username, password}` | `{token, user}` |
| GET | `/api/auth/me` | — | `{id, username, displayName, isAdmin}` |
| PUT | `/api/auth/username` | `{newUsername, password}` | updated user |
| PUT | `/api/auth/password` | `{currentPassword, newPassword}` | `{success}` |
**There is no registration endpoint.** This is a private instance: accounts are created by an
administrator, and nothing on the login screen suggests otherwise.
Usernames are 3–32 characters of letters, digits, `.`, `_` or `-`, and match case-insensitively.
Servers created before this carried e-mail logins; on first start they migrate automatically,
deriving each username from the local part of the address (logged to the console).
### Administration
| Method | Endpoint | Body | Response |
|--------|-----------------------------------|-----------------------------|----------|
| GET | `/api/admin/users` | — | user list with note counts |
| POST | `/api/admin/users` | `{username, password, displayName}` | created user |
| PUT | `/api/admin/users/:id/password` | `{newPassword}` | `{success}` |
| DELETE | `/api/admin/users/:id` | — | `{success}` |
All four require an account flagged `is_admin`; the server checks this itself, so hiding the menu
entry is presentation, not access control. An admin may not delete their own account.
**Where the first admin comes from.** `INKNOTE_ADMIN_USER` and `INKNOTE_ADMIN_PASSWORD` in `.env`:
| Database state | What happens at start |
| --- | --- |
| no users at all | the account is created from both values |
| users, but no admin | `INKNOTE_ADMIN_USER` picks which one is promoted; without it, the oldest |
| an admin exists | both are ignored |
So the variables can never reset an existing password or take rights away — on a running
instance the worst they can do is nothing. Set `INKNOTE_ADMIN_USER` *before* upgrading an older
installation if the oldest account is not the one that should hold the keys.
Deleting an account destroys its notes, notebooks, drawings and version history through
`ON DELETE CASCADE`; the files behind attachments are collected by the hourly orphan sweep. The
deleted person's session stops working immediately: tokens are signed rather than stored, so
`authMiddleware` checks on every request that the account still exists.
### Notebooks
| Method | Endpoint | Description |
|--------|-----------------------|--------------------------|
| GET | `/api/notebooks` | List all notebooks |
| POST | `/api/notebooks` | Create notebook |
| PUT | `/api/notebooks/:id` | Update notebook |
| DELETE | `/api/notebooks/:id` | Soft-delete notebook |
### Notes
| Method | Endpoint | Description |
|--------|---------------------------|-------------------------------|
| GET | `/api/notes` | List all notes (metadata) |
| GET | `/api/notes/:id` | Get note with decrypted content |
| POST | `/api/notes` | Create note |
| PUT | `/api/notes/:id` | Update note (`{content, links}`) |
| GET | `/api/notes/:id/backlinks`| Notes linking to this one |
| GET | `/api/links` | Every link, for backlink counts |
| DELETE | `/api/notes/:id` | Soft-delete note |
| PUT | `/api/notes/:id/drawing` | Save vector strokes |
### Sync
| Method | Endpoint | Description |
|--------|------------- |------------------------------------------|
### WebSocket
Connect to `ws://localhost:4000/ws`, then send:
```json
{"type": "auth", "token": "JWT_TOKEN", "deviceId": "device-123"}
```
Receive real-time events: `note_created`, `note_updated`, `note_deleted`, `notebook_created`, etc.
---
## Sync Engine (Client)
```javascript
import { SyncEngine } from './sync-client';
const sync = new SyncEngine('https://your-server.com', 'device-abc');
// Authenticate
await sync.authenticate('username', 'password');
// Listen for events
sync.on('status', (status) => console.log('Sync:', status));
sync.on('conflict', (conflict) => {
// Present both versions to user, let them choose
console.log('Conflict:', conflict);
});
sync.on('remote_change', (change) => {
// Update local state with remote changes
console.log('Remote change:', change);
});
// Start syncing (every 30s + WebSocket push)
sync.start();
// CRUD operations auto-queue for sync
await sync.createNote({ type: 'text', title: 'Hello', content: '<p>World</p>' });
await sync.updateNote('note-id', { title: 'Updated' });
await sync.deleteNote('note-id');
await sync.uploadPDF(fileBlob);
await sync.saveDrawing('note-id', strokeDoc);
```
---
## Working on the frontend
```bash
npm run test # Vitest — store, search and content migration
npm run typecheck # tsc --noEmit
npm run format # Prettier
npm run build # production bundle
```
TypeScript is enabled with `allowJs`, so `.js` and `.ts` coexist and modules can
be converted one at a time. The whole data layer is typed — `types`,
`offlineStore`, `search`, `editor/content` and `syncEngine` — because that is
where the bugs have been: every serious one so far has been a shape bug, or a
failure distinguished by comparing an error's `.message` to a magic string.
Failures now carry a typed `SyncError.code`; `.message` still holds the same
string, so the remaining JavaScript callers are unaffected.
The 218 unit tests concentrate on the same places, and several exist because the
bug they cover actually happened:
- a queue entry that fails must stay queued, with an attempt counter
- `fullSync` must not drop notes created offline
- `hasPendingWrites` must cover the debounce window, not just requests in flight
- v1 note bodies must migrate without losing task state or smuggling in markup
### Browser suites
The unit tests cover the data layer; these drive a real browser against a real
server, because the failures that hurt most in this app were never visible in a
unit test — a stale response overwriting fresh strokes, a note link surviving in
memory but not through a reload.
```bash
docker compose -f docker-compose.dev.yml up -d --build # must be running
./e2e/run.sh # all suites
./e2e/run.sh editor drawing # only these
```
| Suite | Checks | What it holds down |
| --- | --- | --- |
| `admin` | 15 | no sign-up anywhere; `/api/admin/*` refused for non-admins even when called directly |
| `editor` | 17 | tables and their right-click menu, task lists, note links, backlinks — and that each survives a reload |
| `search-attachments` | 12 | body-only matches, ranking, umlauts; images as blob URLs, not inline bytes |
| `history` | 10 | a large deletion is snapshotted, restoring works, restoring is itself undoable |
| `drawing` | 7 | strokes survive the poller, and match across four viewport widths |
| `pdf` | 85 | import, page proportions, lazy loading, and reading with the network off |
| `attachments-lifecycle` | 10 | a picture is kept while any version shows it, and freed once none does |
| `conflict` | 10 | a late write wins but the state it buries survives as a version |
| `export` | 21 | the downloaded bytes are real Markdown, ZIP and PNG, unpacked and read back |
| `organise` | 38 | a hand-made order survives a search; separators hold their place; a note exported and imported comes back whole |
| `offline` | 12 | a cold page load with the network off still opens notes and searches them |
Each suite creates a throwaway account through the admin API and deletes it
afterwards, so runs leave the database as they found it. They need an admin to
do that with — `marc`/`geheim123` by default, override with `INKNOTE_ADMIN` and
`INKNOTE_ADMIN_PASS`.
Screenshots land in `e2e/shots/`, which is worth a look when a check fails.
### How the frontend is arranged
`App.jsx` holds only *where the user is* — the open notebook, the open note, the
search box, whichever dialog is up — and hands each of the three panes what it
needs. Everything else lives behind a hook:
| Hook | Owns |
| --- | --- |
| `useNoteData` | notes, notebooks, bodies, every mutation, background sync |
| `useSearchIndex` | the local index and the background backfill that fills it |
| `useResponsive` | phone vs desktop, sidebar and which pane a phone shows |
| `useNavigationHistory` | mapping the current location onto browser history |
| `useNotebookDnd` | drag-to-reorder, shared by the sidebar, the dashboard and the note list |
| `usePwaInstall` | the browser's install offer |
`useNoteData` deliberately has no confirmation prompts in it: whether to ask
before deleting something is a question for the interface, not the store. The
panes (`Sidebar`, `Dashboard`, `NoteList`, `EditorPane`) take props and hold no
state of their own, so they can be read on their own.
---
## Interface language
English throughout. It had drifted into two languages — the frame in English,
everything added later in German — which is worse than either: the same idea
appeared under two names depending on when that part was written.
The one place German is deliberately kept is sorting by title, which compares
with the German collation so that *Übung 2* comes before *Übung 10* and umlauts
sort where a German reader expects. That is about the notes, not the chrome.
Two archive folders were renamed with the interface (`bilder/` → `images/`,
`zeichnungen/` → `drawings/`). **The importer still reads both**, along with the
old German marker line for a drawing — archives exported before the change are
on people's disks, and an import that quietly dropped their drawings would be the
worst kind of tidy-up.
---
## Note Content
The body of a note is a **TipTap/ProseMirror document** stored as JSON:
```json
{ "v": 2, "doc": { "type": "doc", "content": [ … ] } }
```
Tables, task lists and note links are real nodes rather than HTML that happened
to survive `document.execCommand`. Notes written before this were
`{ html, tasks }`; they are converted to a document the first time they are
opened and rewritten in the new format on the next save, so nothing needs
migrating up front.
### Note links and backlinks
Typing `@` opens a search over notes and notebooks and inserts a `noteLink`
node holding the target's id — so a link survives the target being renamed, and
the label is refreshed from the current title whenever the note is opened.
The server cannot read encrypted note bodies, so it cannot discover links
itself. The client sends the ids it references alongside the content, and the
server maintains a `note_links` index from them. That index is what powers
`GET /api/notes/:id/backlinks` and the "Linked from N notes" bar under the
editor.
---
## Full-Text Search
Search runs entirely on the client. The server stores note bodies encrypted and
deliberately keeps no plaintext index, so it cannot search them — and a
server-side index would undo the point of encrypting at rest.
Bodies are normally fetched only when a note is opened, which would leave a
fresh device with nothing to search. After login a background pass backfills the
missing ones into IndexedDB, slowly enough to stay out of the way, indexing each
as it lands. Searching therefore also works offline.
Matching is AND across terms, case- and diacritic-insensitive (`kuhlschrank`
finds *Kühlschrank*), with title matches ranked above body matches. Results show
a snippet around the first match.
> Bodies of notes edited on another device are refreshed when that note is
> opened, so search results for untouched notes can lag behind until then.
---
## Reading PDFs
An imported PDF becomes a note of type `pdf` and is shown by a viewer instead of
the text editor. Pages are laid out at their true proportions before anything is
drawn, so the scrollbar is honest from the first frame; a document mixing
portrait and landscape keeps both, rather than stretching the narrow pages.
Only pages near the viewport are rendered, and each keeps its own canvas — a PDF
page is expensive to draw and cheap to hold, so scrolling back up repaints
nothing. Canvases are sized in device pixels and scaled down by CSS, which is
what keeps text sharp on a high-density screen.
**pdf.js is not in the main bundle.** It is larger than the rest of the
application put together, so it sits behind a dynamic import: a session that
never opens a PDF never downloads it. The worker is bundled too, not fetched
from a CDN — a viewer that stops working during someone else's outage is not one
you can rely on offline.
The **legacy** pdf.js build is used deliberately. Version 6 calls
`Uint8Array.prototype.toHex`, which only very recent browsers have; anything
older fails to parse the document with a message that says nothing about why.
Bytes are read through the authenticated attachment route and cached in
IndexedDB, so a document opened once stays readable with no connection. Because
storage is content-addressed, a cached copy can never go stale.
> nginx needs a MIME type for `.mjs` — without one it serves the worker as
> `application/octet-stream` and the browser refuses to run it as a module.
---
## Exporting an annotated PDF
A PDF note offers three things rather than the note formats: the document with
your drawings written into it, the untouched original, or just the notes you
wrote beside it as Markdown.
pdf.js can only read, so producing a file needs a second library — `pdf-lib`,
loaded the same way, behind a dynamic import, so the bundle everyone downloads
carries no PDF writer for the sake of the people who never export one.
This is where the coordinate decision pays off. Strokes were stored in page
points from the start, which is the only space a PDF understands: nothing has to
be guessed back from screen pixels, and the stamping is a direct translation
apart from flipping the origin, which PDF puts at the bottom-left.
**The stored file is never modified.** `pdf-lib` builds a new document, so a
clean copy of the original stays available however often it has been drawn on —
checked by the suite, which re-fetches the stored bytes after exporting and
compares them.
Erased areas cannot come along. On screen an eraser removes ink by drawing in
`destination-out`; a PDF content stream has no equivalent, so those strokes are
dropped rather than stamped as visible grey scribbles where something was rubbed
out.
The suite renders both files and counts dark pixels — 1633 against 2557 — because
a larger file proves bytes were added, not that any ink can be seen.
---
## Notes beside a PDF
A PDF note is a note that happens to have a document attached, not a separate
kind of thing: what you write beside it goes into the note's ordinary `content`,
so it is encrypted, versioned, linkable and searchable like any other note.
The panel is composed from the editor pieces directly rather than reusing
`NoteEditor`, which carries a whole drawing mode with its own canvas and undo
stack — the PDF has its own drawing layer, and a second one hidden behind the
panel would be two places to draw with one of them invisible.
Side by side on a wide screen, two tabs on a narrow one. Both halves can be
hidden: reading the original without one's own marks in the way is a normal
thing to want, and so is writing without the document taking two thirds of the
screen.
**One trap this uncovered.** A PDF note has two sources of searchable text —
what the reader typed and what the document says — written at different moments
by different code. Both called `setEntry`, so whichever ran last was the only
half you could search: typing a note dropped the document's text, and re-reading
the document dropped the note. The index now keeps them apart and only replaces
the half it was given.
---
## Selecting and searching PDF text
pdf.js draws pages as pictures, so selecting means laying transparent spans over
them exactly where each word was painted. The layer carries the same scale as the
canvas — the two drift apart otherwise and a selection lands on the wrong words.
While drawing it stops taking pointer events entirely, or a stroke would come out
as a text selection.
The text is also pulled out and handed to the same index that holds the notes, so
one search box finds a sentence whether it was typed or imported. Extraction runs
*after* the document is on screen and yields between pages: it must never stand
between the reader and the first page.
**The extracted text is not sent to the server.** It is derivable from a file the
server already holds, so storing it again would cost space and a second thing to
encrypt for nothing gained. The price is that a PDF becomes searchable on a
device once that device has opened it — the same shape as the limitation notes
already have, and the honest trade for not duplicating data.
The minimal text-layer CSS is written out in `textLayer.css` rather than
importing `pdfjs-dist/web/pdf_viewer.css`, which carries the whole reference
viewer's toolbars and dialogs.
---
## Drawing on a PDF
Strokes are stored **per page, in that page's own coordinate space** — PDF
points at scale 1, exactly what pdf.js reports — rather than in the fixed
1200×1600 space a note's drawing uses. A PDF cannot use one space: every page
has its own size, and a stroke belongs to the page it was drawn on.
That choice is what keeps a mark on the word it was drawn over at any zoom, on
any screen, and it is the only form that can be stamped back into the file later,
where page points are the only coordinates that mean anything. The suite measures
it: zooming in moves a stroke by 0.001 of the page.
```jsonc
{ "v": 2, "pages": { "1": { "w": 595, "h": 842, "strokes": [...] } } }
```
Only pages that carry ink get an entry. Pen sizes are scaled from the note page
width, or the same number would draw twice as thick on an A4 page half as wide.
Reading and drawing are separate modes. While reading, the ink layer takes no
pointer events at all, so scrolling and selecting text reach the page beneath;
while drawing, a finger still scrolls and only a pen or mouse leaves a mark. The
drawing can be hidden without being erased, for reading the original clean.
---
## PDF storage
An imported PDF is stored as an attachment: encrypted, named by the hash of its
contents, and shared by any note holding the same file. The note points at it
through `notes.pdf_attachment_id`.
It goes up as a raw body rather than the base64 JSON images use — a scanned book
runs to tens of megabytes and base64 would add a third to that for nothing. The
bytes are checked for the `%PDF-` signature before anything is stored, so the
declared content type is never taken on trust.
Images and PDFs are limited separately, because a 25 MB picture is nearly always
a mistake while a 25 MB document is not: `INKNOTE_MAX_IMAGE_MB` (25) and
`INKNOTE_MAX_PDF_MB` (200). The limits are there so a runaway upload cannot fill
the disk, not to ration anything.
> An earlier `POST /api/notes/pdf` stored a *filesystem path* in the note, which
> is why it never survived a container rebuild. That route is gone.
---
## Encryption at rest
Note bodies and uploaded files are both encrypted with AES-256-GCM under
`ENCRYPTION_KEY`. Losing that key loses the notes — back it up with the database,
not next to it.
Files carry an 8-byte `INKENC01` header, so a file says for itself whether it is
encrypted. That is what lets the startup sweep run on every boot and encrypt only
what is still in the clear: files are shared between users by content hash, so a
flag on a database row could not have answered the question.
Two consequences worth knowing:
- **Deduplication still works.** Files are named by the hash of the *plaintext*,
so two identical uploads share one copy — encrypting each with a fresh IV would
otherwise produce two different files from the same picture.
- **Reads are not streamed.** GCM only proves a file is intact once the final
block is in, so an attachment is read whole and verified before anything is
sent. Half of a corrupt image is worse than an error.
Uploads are written to a temporary name and renamed into place. A rename is
atomic on one filesystem, so a crash mid-write leaves the old file or the new
one, never a half-written one that would fail its tag forever after.
---
## Freeing attachments
Take a picture out of a note and the file cannot simply be deleted: the versions
from before still show it, and restoring one would find a hole where the image
was. So an attachment is released only once **no current note and no kept
version** refers to it — which, with retention by count, is the moment the last
version holding it is pushed out by newer ones.
The server cannot look for itself: bodies are encrypted. So the client reports
which attachments a note uses on every save, exactly as it already does for note
links, and a version inherits whatever the note held at the moment it was
snapshotted — which is by definition what that version contains.
Two things make this safe rather than merely plausible:
- **Silence is not "uses nothing".** A note written before references existed has
no rows, which looks identical to a note with no pictures. Nothing is released
for a user while any of their notes has yet to report; the background prefetch
that fills the search index reports for them on its first pass.
- **An upload in flight has a row before the note that uses it does.** Attachments
younger than the grace period are left alone.
`POST /api/admin/housekeeping` runs the sweep on demand instead of waiting for
its timer — admin only, and it accepts `graceSeconds` to shorten that wait.
---
## Attachments
Images used to be inlined into the note body as base64 data URIs — a few
screenshots turned one note into megabytes that were re-encrypted and re-uploaded
on every keystroke. They are now separate blobs; the body only keeps an id.
- Stored on disk under the SHA-256 of their bytes, so the same image pasted into
ten notes is kept once
- Uploads are type-checked against the actual bytes, not the declared MIME type
- Downloads require the owner's token, so `<img>` cannot fetch them directly —
the client fetches with its token, caches the blob in IndexedDB and renders a
blob URL, which also makes images work offline
- Files with no database row are swept up daily, one hour after last write
| Method | Endpoint | Description |
|--------|-------------------------|------------------------------------|
| POST | `/api/attachments/pdf` | raw PDF body, `?filename=` | `{id, mime, size, deduplicated}` |
| POST | `/api/attachments` | Upload a base64 data URL |
| GET | `/api/attachments/:id` | Fetch the bytes (owner only) |
> Removing an image from a note does not yet delete its blob: the body is
> encrypted, so the server cannot tell what is still referenced. Unused blobs
> cost disk space, nothing else.
---
## Version History
Every write snapshots the state it is about to replace into `note_versions`,
encrypted the same way the note itself is.
Saves arrive about once a second while typing, so snapshots **coalesce**: within
a 5-minute window the existing newest version already represents "before this
editing session". Two cases bypass the window, because they are exactly the ones
worth undoing:
- a write that shrinks a note to under half its size (`large-deletion`)
- clearing a drawing (`drawing-cleared`)
Restoring snapshots the current state first (`before-restore`), so picking the
wrong version is not a one-way door. A restored body also rebuilds its entries
in the backlink index.
**Retention is by count, not by age.** The newest N versions of a note are kept
and trimmed on every write; nothing expires merely for being old.
There used to be a second rule dropping anything past 90 days, and it was a bad
one: the count already bounds how much a note can accumulate, so age only ever
deleted history that nothing had replaced — and it bit hardest exactly where
history is most wanted. A note left alone for a quarter of a year lost its whole
past for having done nothing at all. `VERSION_RETENTION_DAYS` still exists and
still works, but it is off unless set.
**How many is decided per notebook.** Edit a notebook to give it its own number;
leave the field empty and it follows the server's default. A reference notebook
can keep fifty versions of everything while a scratch one keeps three.
| Variable | Default | Meaning |
|----------------------------|---------|------------------------------------------|
| `MAX_VERSIONS_PER_NOTE` | `50` | Default for notebooks with no setting |
| `VERSION_RETENTION_DAYS` | *off* | Optional age limit; unset means never |
| `VERSION_COALESCE_MINUTES` | `5` | Window in which saves share one snapshot |
**Deleting a note** — moving one to the trash keeps its history, because the
note can still be restored. Deleting it permanently, or emptying the trash,
removes its versions along with it (`ON DELETE CASCADE`), leaving no orphaned
rows behind.
| Method | Endpoint | Description |
|--------|-------------------------------------------|----------------------------|
| GET | `/api/notes/:id/versions` | List (metadata only) |
| GET | `/api/notes/:id/versions/:versionId` | One version, decrypted |
| POST | `/api/notes/:id/versions/:versionId/restore` | Restore it |
> Version history protects against **overwritten** data. It does not protect
> against a lost disk, because it lives in the same database — that is what
> backing up the data directory is for.
---
## Sorting a notebook
Each notebook remembers how its notes are ordered, and remembers it on the
server rather than per device — the order is a property of the notebook, and
finding one sorted differently on the phone than on the desktop is its own small
annoyance. A reference book can stay alphabetical while a journal stays
newest-first.
| Mode | Order |
| --- | --- |
| `manual` | the user's own arrangement, dragged into place — notebooks only |
| `updated` | most recently changed first *(the default)* |
| `oldest` | least recently changed first |
| `created` | most recently created first |
| `title` | A–Z, case- and umlaut-aware, with numbers read as numbers |
**The manual order has to survive being looked away from.** Searching imposes
its own order — relevance — and switching back has to show exactly what was
arranged. That is only keepable if the arrangement lives in the notes and is
never re-derived from what is on screen, so two rules hold it up:
- `reindex` renumbers **every note in the notebook**, taken from the store, not
the list being displayed. Renumbering a filtered subset would give a hidden
note the same position as a visible one and quietly scramble the rest.
- Dragging is switched off while a search is narrowing the list, and the list
says so rather than silently ignoring the drag.
"All Notes" and "Starred" are not offered a manual order at all. They are
questions asked of every notebook at once, so there is nothing for an
arrangement to belong to and nowhere to keep it.
**Separators** are lines drawn between notes, optionally named, stored on the
notebook beside the sort mode. Renaming and removing are both right-click, which
is how everything else in that list is acted on: a left-click on the label used
to start renaming, so the row could not be clicked at all without falling into an
edit, and the × for removal sat a few pixels from that same target. Removal asks
first, and says the notes are not affected. They move by dragging like a note does — a
divider that cannot move with what it divides is worse than none — and their
position is recorded as "how many notes come before me", so they stay put as the
notes around them are renamed, edited or opened. They are shown only in the
manual order: in "newest first" the notes would slide past the line on every
edit.
Reordering also has its own route, `POST /api/notes/reorder`. Going through
`PUT /api/notes/:id` would bump `updated_at` and snapshot a version — so
dragging a note would move it to the top of "last changed" and spend a slot of
its history on a change that is not to the note at all.
---
## Notes pinned to a passage
The panel beside a PDF is for notes about the document. A note that belongs to
*one paragraph* is a different thing — and a note that has to begin "on page 14,
third paragraph" has already lost its place. So a note can be pinned to a spot:
click **Randnotiz**, click the passage, write.
**The note is written beside the page, not on it.** Only a small numbered marker
sits on the document, with a hairline out to a card in the margin, level with the
passage. Writing onto the page would cover the sentence the note is about, which
is the one thing it must not do. Cards keep one height, so two notes close
together are laid out by pushing the second down until they no longer overlap;
the one being written in grows and rises above the rest.
**On a phone there is no margin.** A page narrow enough to read leaves no white
space beside it, and taking 200px for a column would make the document itself
unreadable. There, tapping a marker brings the note up from the bottom — over the
page, but only while it is open, within reach of a thumb, and with the passage
still visible above it.
The column is kept as soon as the document has one pinned note, or while one is
being placed, so the page does not jump sideways at the moment of writing the
first. Reserving it also exposed a loop worth remembering: a flex item is as wide
as its content unless told otherwise, so the column made the viewer wider, which
made there be room for the column — the page ended up at 254px inside a 420px
phone while the layout insisted it had 611px.
Positions are stored in **PDF points**, the same space as the drawings, so a
marker stays on its sentence at every zoom and on every screen. Being in the
overlay rather than in the note body has three consequences, each of which needed
handling rather than assuming:
- **Search.** Nothing else would index them, so the viewer feeds them to the
local index itself, with their page numbers — finding the words is only half of
finding the note.
- **Clearing the ink.** "Seite leeren" empties the drawing. It must not take the
notes pinned to that page: same page, different thing.
- **"Is this overlay empty?"** The client and the server each held their own
answer, and both counted strokes only. An overlay carrying nothing but pinned
notes was therefore stored as `NULL` — every note on it lost on the next save.
Both now ask about strokes *and* pins.
**The margin notes can be linked from the notes.** A remark in the margin is
short by design; when there is more to say about that passage it belongs in the
notes beside the document — and it has to be possible to say *which* passage. So
the pins are offered to the editor's `@` menu exactly like notes and notebooks
are, labelled with their page, and following one scrolls the document to the spot
and opens the remark. They are unhidden first: a link that led to something
invisible would read as broken.
A pin link is the same node with `kind: "pin"`, and is deliberately left out of
the links reported to the server. Its target is a marker inside this note's own
overlay, so no note has that id, and reporting it would fill the backlink table
with rows that match nothing.
**They can be hidden**, and so can the drawings — by a pair of switches at the
right-hand end of the toolbar. They are two halves of one question, *how much of
my own work do I want to see right now*, and they used to sit at opposite ends of
the bar, each beside the tool that made them. Each appears only once there is
something of its kind to hide. Hiding the margin notes takes the markers and the
column together, and the page takes the width back while they are away.
The drawing tools sit directly after the button that turns them on. They had
drifted to the far side of two other controls, so switching to Zeichnen made them
appear somewhere other than where the eye already was.
They come along when the document is exported with annotations: each becomes a
numbered marker plus a small box with its text, placed on whichever side of the
pin has room. Only Latin-1 survives — the standard PDF fonts cannot encode more,
and one em dash must not cost the whole export, so anything else is replaced.
---
## Import
The other half of the promise that notes are not hostage to this program. A
`.md` file becomes a note; a `.zip` this app exported comes back nearly whole —
text, tables, task boxes, and the images, re-uploaded as attachments so they are
encrypted at rest again like anything else.
The parser is a deliberate subset: what the exporter emits, plus what a
hand-written note is likely to use. It is not CommonMark and does not try to be.
A full parser is thousands of lines and a permanent maintenance surface, and the
failure mode here is mild — an unrecognised construct arrives as the literal
text that was written, which is still the user's words in the user's order.
Two things do not come back, and the import says so afterwards rather than
leaving them to be discovered:
- **A drawing returns as a picture, not as strokes.** It was exported as a PNG,
and vectors cannot be recovered from pixels. It stays visible, printable and
exportable, but the pen can no longer edit it.
- **An image the archive does not contain cannot be restored.** The place it sat
in the note is kept, so what used to be there is at least recorded.
Writing the exporter also turned up a real bug in it: blocks were emitted
without a blank line between them, so a list written straight after another list
became more items of the first, and a quote or code fence on the next line was
read as part of the item above. That is not one reader's quirk — it is what the
format says — so the file rendered wrongly everywhere, not only here.
---
## Export
Notes should not be hostage to this program, so everything can leave it — from a
single note up to every note in the account, reachable from the note's toolbar,
a notebook's context menu, and Account → *Alle Notizen exportieren*.
A single note is asked two questions rather than one, because the answers are
not independent — Markdown holds text only, a PNG holds the drawing only, and
PDF is the one format that holds both:
| Was soll mit? | In welcher Form? |
| --- | --- |
| Notiz mit Zeichnung | PDF *(Markdown puts the drawing in a PNG beside it)* |
| Nur den Notiztext | PDF · Markdown |
| Nur die Zeichnung | PNG · PDF |
A note without a drawing skips the first question instead of showing it with two
thirds greyed out.
**Markdown** keeps the notes editable elsewhere. Headings, lists, quotes, code,
tables and task state all have equivalents and are carried over; a link to
another exported note becomes a relative link to its file, and a link to a note
outside the export degrades to its plain title rather than a dead reference.
A note with images or a drawing arrives as a ZIP so the files travel with it,
and a whole notebook gets a `README.md` index grouped by notebook.
**PDF** goes through the browser's own print dialog rather than a bundled PDF
library — every browser already has a good one behind *Save as PDF*, and it
handles fonts and page breaks better than anything that would fit in here. This
is the format that keeps drawings inside the document.
Two things Markdown cannot represent, and what happens instead:
| | |
| --- | --- |
| drawings | rendered to PNG beside the note, with a line in the text saying so |
| underline | written as `<u>`, which every Markdown renderer passes through |
The ZIP writer is [`export/zip.ts`](frontend/src/export/zip.ts) — about eighty
lines for stored (uncompressed) entries, rather than a dependency. Note text is
small and images are already compressed, so there was nothing to gain by
compressing, and this is one fewer thing to keep updated.
Everything runs in the browser. Nothing is uploaded anywhere to produce a file.
---
## Drawings
Drawings are **vector strokes**, not bitmaps. Every stroke is recorded in one canonical
page space of 1200 × 1600 units:
```json
{ "v": 1, "w": 1200, "h": 1600, "strokes": [
{ "t": "pen", "c": "#e94560", "s": 5.5, "p": [100,100, 200,180, 300,140] }
]}
```
`t` is the tool (`pen` | `highlighter` | `eraser`), `c` the colour, `s` the width in
canonical units, and `p` a flat `x,y` point list. Each viewer scales that page to the
room it has and rasterizes at its own device pixel ratio, which is what makes a note
look the same on a phone and on a desktop — and stay crisp on both. Stroke data is
encrypted at rest like note content.
Notes drawn before this format still carry a PNG in `drawing_data_path`; it renders
underneath any newer strokes so nothing is lost.
---
## Conflict Resolution
Two devices can edit the same note without knowing about each other — a phone
that was offline all day, a laptop that was not. When the phone reconnects, its
save arrives last, and without care it would simply bury the newer work.
Each write carries `baseUpdatedAt`: the revision the editing device had when it
started. The server compares it with the note's current revision:
1. **Same** — an ordinary save, nothing to think about.
2. **Server is newer** — someone else saved in between. The incoming edit still
wins, because rejecting it would throw away whatever was just typed, and on a
phone that has been offline for a day that is the worse loss. But the state
it replaces is kept as a version marked `conflict`, so both exist, and the
history dialog is where you pick which one to keep.
This replaced vector clocks, which sat in the server unused: no client ever sent
one. They can tell a genuine conflict from a device merely catching up, which is
strictly more accurate — but that accuracy only pays for itself with a merge
interface to spend it on, and comparing revisions needs one field and reuses the
version history that already exists.
**What this does not do:** it will not tell you at the moment it happens. The
conflicting version appears in the history, labelled, and that is where you find
it. A notification is worth adding if it ever actually bites.
---
## Extending to Native Platforms
### React Native
Use `note-app.jsx` components with React Native adaptations (replace HTML canvas with `react-native-canvas` or `@shopify/react-native-skia` for drawing). The sync engine works as-is.
### Flutter
Use the sync engine via a JavaScript bridge, or port the API calls to Dart. For drawing, use Flutter's `CustomPainter`.
### Electron
Wrap the React app directly in Electron for desktop. Add native file system access for export.
---
## Security Notes
- All note content is encrypted with AES-256-CBC before database storage
- Passwords are hashed with bcrypt (12 rounds)
- JWT tokens expire after 30 days
- HTTPS should be enforced in production (use a reverse proxy like Nginx or Pangolin)
- The encryption key should be stored securely (e.g., environment variable, secrets manager)
---
## Tech Stack
| Layer | Technology |
|-----------|-------------------------------------|
| Frontend | React 18 + Vite |
| Editor | TipTap (ProseMirror) |
| Backend | Node.js, Express |
| Database | SQLite (dev), PostgreSQL (prod) |
| Real-time | WebSocket (ws library) |
| Auth | JWT + bcrypt |
| Encryption| AES-256-CBC |
| Container | Docker + docker-compose |
Reference in new issue View Git Blame Copy Permalink