InkNote/readme.md

# ✏️ InkNote

A cross-platform note-taking application with rich text editing, freehand drawing, PDF annotation, and encrypted sync — inspired by Samsung Notes.

---

## Features

### Core
- **Rich Text Notes** — Bold, italic, underline, headings, bullet/numbered lists, task lists, quotes, code blocks
- **Tables** — Insert, resize columns, add/remove rows and columns, merge cells, toggle header row
- **Note Links** — Type `@` to link to another note or notebook; every note shows what links back to it
- **Version History** — Past states of every note, restorable; restoring is itself reversible
- **Full-Text Search** — Searches note bodies, not just titles, and works offline
- **Image Attachments** — Pasted images are stored as deduplicated blobs, not inlined into the note
- **Freehand Drawing** — Pen, highlighter, eraser with adjustable thickness and color palette (20 colors, 7 pen sizes)
- **Undo/Redo** — Full history stack for drawing operations
- **PDF Import & Annotation** — Import PDFs, overlay drawing/highlighting, persistent annotations
- **Notebooks & Folders** — Create notebooks with custom gradient covers, organize notes by folder
- **Star/Favorite** — Quick access to important notes
- **Dark & Light Mode** — Full theme support with refined color schemes

### Sync & Storage
- **Offline-First** — Works fully offline, syncs when connection returns
- **Conflict-Aware Sync** — Vector clock–based conflict detection; notifies user of merge conflicts
- **WebSocket Real-Time Push** — Changes appear on other devices instantly
- **Encrypted at Rest** — AES-256-CBC encryption for all note content on the server
- **JWT Authentication** — Username + password accounts with bcrypt password hashing
- **Vector Drawings** — Strokes are stored as vectors in one canonical page space, so a drawing looks identical on every screen and stays sharp at any pixel density

### Architecture
- **Self-Hosted Backend** — Node.js + Express + SQLite (or PostgreSQL)
- **Docker Ready** — docker-compose up and you're running
- **Client Sync Engine** — Portable JS library for any frontend (React Native, Flutter JS bridge, Electron, web)

---

## Project Structure

```
inknote/
├── docker-compose.yml        # Deploy from the Gitea registry
├── docker-compose.dev.yml    # Build from source for local work
├── build.sh                  # Build & push images to Gitea
├── backend/
│   ├── server.js             # Express API + WebSocket server
│   ├── sync-client.js        # Client-side sync engine library
│   ├── package.json          # Node dependencies
│   ├── Dockerfile            # Server container
│   └── .env.example          # Environment variable template
├── frontend/
│   ├── src/
│   │   ├── App.jsx           # Root: where the user is, which dialog is open
│   │   ├── types.ts          # Shapes shared by server, IndexedDB and UI
│   │   ├── syncEngine.ts     # Offline-first sync + API client
│   │   ├── offlineStore.ts   # IndexedDB wrapper
│   │   ├── search.ts         # Local full-text index
│   │   ├── hooks/            # Note data, search, layout, history, drag
│   │   ├── ui/               # Sidebar, dialogs, icons, palette
│   │   ├── features/notes/   # List, editor pane, version history
│   │   ├── features/notebooks/ # Dashboard
│   │   ├── features/drawing/ # Canonical stroke model + render layer
│   │   └── editor/           # TipTap schema, toolbar, note links, attachments
│   ├── public/sw.js          # PWA service worker
│   ├── nginx.conf            # Static serving + API/WS proxy
│   └── dockerfile            # Web container
└── readme.md                 # This file
```

---

## Quick Start

### Where the data lives

Both compose files use **bind mounts**, not named volumes, so everything sits in
a plain directory on the Docker host and is picked up by whatever backs that
host up:

```
$INKNOTE_DATA_DIR/
├── db/        # inknote.db — notes, drawings, version history
└── uploads/   # legacy raster drawings, PDFs
```

Default is `./data` for development and `/srv/inknote` in production; override
with `INKNOTE_DATA_DIR`. The container runs as uid/gid `1000` so the files stay
owned by a normal user — set `INKNOTE_UID` / `INKNOTE_GID` if yours differ, and
create the directory with matching ownership before the first start.

> Back up the `.env` holding `ENCRYPTION_KEY` too. Note content is encrypted at
> rest, so a copy of the database without that key cannot be read.

### 1. Local development (Docker)

```bash
# Generate secrets once
printf 'JWT_SECRET=%s\nENCRYPTION_KEY=%s\n' "$(openssl rand -base64 32)" "$(openssl rand -hex 32)" > .env

# Build and run backend + nginx-served frontend
docker compose -f docker-compose.dev.yml up -d --build      # → http://localhost:3000

# Same, plus a Vite dev server with hot reload
docker compose -f docker-compose.dev.yml --profile dev up -d --build   # → http://localhost:5173
```

`docker-compose.dev.yml` builds from source and uses its own volumes, so it never
touches production data.

### 2. Docker Deployment

```bash
# Generate a secure encryption key
export ENCRYPTION_KEY=$(openssl rand -hex 32)
export JWT_SECRET=$(openssl rand -base64 32)

docker-compose up -d
```

The server runs on port `4000`, the web UI on port `3000`.

---

## API Reference

### Authentication

| Method | Endpoint              | Body                          | Response            |
|--------|-----------------------|-------------------------------|---------------------|
| POST   | `/api/auth/register`  | `{username, password, displayName}` | `{token, user}` |
| POST   | `/api/auth/login`     | `{username, password}`        | `{token, user}`     |
| GET    | `/api/auth/me`        | —                             | `{id, username, displayName}` |
| PUT    | `/api/auth/username`  | `{newUsername, password}`     | updated user        |
| PUT    | `/api/auth/password`  | `{currentPassword, newPassword}` | `{success}`      |

Usernames are 3–32 characters of letters, digits, `.`, `_` or `-`, and match case-insensitively.
Servers created before this carried e-mail logins; on first start they migrate automatically,
deriving each username from the local part of the address (logged to the console).

### Notebooks

| Method | Endpoint              | Description              |
|--------|-----------------------|--------------------------|
| GET    | `/api/notebooks`      | List all notebooks       |
| POST   | `/api/notebooks`      | Create notebook          |
| PUT    | `/api/notebooks/:id`  | Update notebook          |
| DELETE | `/api/notebooks/:id`  | Soft-delete notebook     |

### Notes

| Method | Endpoint                  | Description                   |
|--------|---------------------------|-------------------------------|
| GET    | `/api/notes`              | List all notes (metadata)     |
| GET    | `/api/notes/:id`          | Get note with decrypted content |
| POST   | `/api/notes`              | Create note                   |
| PUT    | `/api/notes/:id`          | Update note (`{content, links}`) |
| GET    | `/api/notes/:id/backlinks`| Notes linking to this one     |
| GET    | `/api/links`              | Every link, for backlink counts |
| DELETE | `/api/notes/:id`          | Soft-delete note              |
| PUT    | `/api/notes/:id/drawing`  | Save vector strokes           |
| POST   | `/api/notes/pdf`          | Upload PDF (multipart/form)   |

### Sync

| Method | Endpoint     | Description                              |
|--------|------------- |------------------------------------------|
| POST   | `/api/sync`  | Bidirectional sync with conflict reports  |

### WebSocket

Connect to `ws://localhost:4000/ws`, then send:
```json
{"type": "auth", "token": "JWT_TOKEN", "deviceId": "device-123"}
```

Receive real-time events: `note_created`, `note_updated`, `note_deleted`, `notebook_created`, etc.

---

## Sync Engine (Client)

```javascript
import { SyncEngine } from './sync-client';

const sync = new SyncEngine('https://your-server.com', 'device-abc');

// Authenticate
await sync.authenticate('username', 'password');

// Listen for events
sync.on('status', (status) => console.log('Sync:', status));
sync.on('conflict', (conflict) => {
  // Present both versions to user, let them choose
  console.log('Conflict:', conflict);
});
sync.on('remote_change', (change) => {
  // Update local state with remote changes
  console.log('Remote change:', change);
});

// Start syncing (every 30s + WebSocket push)
sync.start();

// CRUD operations auto-queue for sync
await sync.createNote({ type: 'text', title: 'Hello', content: '<p>World</p>' });
await sync.updateNote('note-id', { title: 'Updated' });
await sync.deleteNote('note-id');
await sync.uploadPDF(fileBlob);
await sync.saveDrawing('note-id', strokeDoc);
```

---

## Working on the frontend

```bash
npm run test        # Vitest — store, search and content migration
npm run typecheck   # tsc --noEmit
npm run format      # Prettier
npm run build       # production bundle
```

TypeScript is enabled with `allowJs`, so `.js` and `.ts` coexist and modules can
be converted one at a time. The whole data layer is typed — `types`,
`offlineStore`, `search`, `editor/content` and `syncEngine` — because that is
where the bugs have been: every serious one so far has been a shape bug, or a
failure distinguished by comparing an error's `.message` to a magic string.
Failures now carry a typed `SyncError.code`; `.message` still holds the same
string, so the remaining JavaScript callers are unaffected.

The 81 unit tests concentrate on the same places, and several exist because the
bug they cover actually happened:

- a queue entry that fails must stay queued, with an attempt counter
- `fullSync` must not drop notes created offline
- `hasPendingWrites` must cover the debounce window, not just requests in flight
- v1 note bodies must migrate without losing task state or smuggling in markup

### How the frontend is arranged

`App.jsx` holds only *where the user is* — the open notebook, the open note, the
search box, whichever dialog is up — and hands each of the three panes what it
needs. Everything else lives behind a hook:

| Hook | Owns |
| --- | --- |
| `useNoteData` | notes, notebooks, bodies, every mutation, background sync |
| `useSearchIndex` | the local index and the background backfill that fills it |
| `useResponsive` | phone vs desktop, sidebar and which pane a phone shows |
| `useNavigationHistory` | mapping the current location onto browser history |
| `useNotebookDnd` | drag-to-reorder, shared by sidebar and dashboard |
| `usePwaInstall` | the browser's install offer |

`useNoteData` deliberately has no confirmation prompts in it: whether to ask
before deleting something is a question for the interface, not the store. The
panes (`Sidebar`, `Dashboard`, `NoteList`, `EditorPane`) take props and hold no
state of their own, so they can be read on their own.

---

## Note Content

The body of a note is a **TipTap/ProseMirror document** stored as JSON:

```json
{ "v": 2, "doc": { "type": "doc", "content": [ … ] } }
```

Tables, task lists and note links are real nodes rather than HTML that happened
to survive `document.execCommand`. Notes written before this were
`{ html, tasks }`; they are converted to a document the first time they are
opened and rewritten in the new format on the next save, so nothing needs
migrating up front.

### Note links and backlinks

Typing `@` opens a search over notes and notebooks and inserts a `noteLink`
node holding the target's id — so a link survives the target being renamed, and
the label is refreshed from the current title whenever the note is opened.

The server cannot read encrypted note bodies, so it cannot discover links
itself. The client sends the ids it references alongside the content, and the
server maintains a `note_links` index from them. That index is what powers
`GET /api/notes/:id/backlinks` and the "Linked from N notes" bar under the
editor.

---

## Full-Text Search

Search runs entirely on the client. The server stores note bodies encrypted and
deliberately keeps no plaintext index, so it cannot search them — and a
server-side index would undo the point of encrypting at rest.

Bodies are normally fetched only when a note is opened, which would leave a
fresh device with nothing to search. After login a background pass backfills the
missing ones into IndexedDB, slowly enough to stay out of the way, indexing each
as it lands. Searching therefore also works offline.

Matching is AND across terms, case- and diacritic-insensitive (`kuhlschrank`
finds *Kühlschrank*), with title matches ranked above body matches. Results show
a snippet around the first match.

> Bodies of notes edited on another device are refreshed when that note is
> opened, so search results for untouched notes can lag behind until then.

---

## Attachments

Images used to be inlined into the note body as base64 data URIs — a few
screenshots turned one note into megabytes that were re-encrypted and re-uploaded
on every keystroke. They are now separate blobs; the body only keeps an id.

- Stored on disk under the SHA-256 of their bytes, so the same image pasted into
  ten notes is kept once
- Uploads are type-checked against the actual bytes, not the declared MIME type
- Downloads require the owner's token, so `<img>` cannot fetch them directly —
  the client fetches with its token, caches the blob in IndexedDB and renders a
  blob URL, which also makes images work offline
- Files with no database row are swept up daily, one hour after last write

| Method | Endpoint                | Description                        |
|--------|-------------------------|------------------------------------|
| POST   | `/api/attachments`      | Upload a base64 data URL           |
| GET    | `/api/attachments/:id`  | Fetch the bytes (owner only)       |

> Removing an image from a note does not yet delete its blob: the body is
> encrypted, so the server cannot tell what is still referenced. Unused blobs
> cost disk space, nothing else.

---

## Version History

Every write snapshots the state it is about to replace into `note_versions`,
encrypted the same way the note itself is.

Saves arrive about once a second while typing, so snapshots **coalesce**: within
a 5-minute window the existing newest version already represents "before this
editing session". Two cases bypass the window, because they are exactly the ones
worth undoing:

- a write that shrinks a note to under half its size (`large-deletion`)
- clearing a drawing (`drawing-cleared`)

Restoring snapshots the current state first (`before-restore`), so picking the
wrong version is not a one-way door. A restored body also rebuilds its entries
in the backlink index.

**Retention** — the newest 50 versions per note, trimmed on every write, plus a
daily sweep of anything older than 90 days. Both are configurable:

| Variable                   | Default | Meaning                                  |
|----------------------------|---------|------------------------------------------|
| `MAX_VERSIONS_PER_NOTE`    | `50`    | Versions kept per note                   |
| `VERSION_RETENTION_DAYS`   | `90`    | Age after which a version is dropped     |
| `VERSION_COALESCE_MINUTES` | `5`     | Window in which saves share one snapshot |

**Deleting a note** — moving one to the trash keeps its history, because the
note can still be restored. Deleting it permanently, or emptying the trash,
removes its versions along with it (`ON DELETE CASCADE`), leaving no orphaned
rows behind.

| Method | Endpoint                                  | Description                |
|--------|-------------------------------------------|----------------------------|
| GET    | `/api/notes/:id/versions`                 | List (metadata only)       |
| GET    | `/api/notes/:id/versions/:versionId`      | One version, decrypted     |
| POST   | `/api/notes/:id/versions/:versionId/restore` | Restore it              |

> Version history protects against **overwritten** data. It does not protect
> against a lost disk, because it lives in the same database — that is what
> backing up the data directory is for.

---

## Drawings

Drawings are **vector strokes**, not bitmaps. Every stroke is recorded in one canonical
page space of 1200 × 1600 units:

```json
{ "v": 1, "w": 1200, "h": 1600, "strokes": [
  { "t": "pen", "c": "#e94560", "s": 5.5, "p": [100,100, 200,180, 300,140] }
]}
```

`t` is the tool (`pen` | `highlighter` | `eraser`), `c` the colour, `s` the width in
canonical units, and `p` a flat `x,y` point list. Each viewer scales that page to the
room it has and rasterizes at its own device pixel ratio, which is what makes a note
look the same on a phone and on a desktop — and stay crisp on both. Stroke data is
encrypted at rest like note content.

Notes drawn before this format still carry a PNG in `drawing_data_path`; it renders
underneath any newer strokes so nothing is lost.

---

## Conflict Resolution

InkNote uses **vector clocks** to detect conflicts:

1. Each device maintains a logical clock counter
2. On save, the device increments its counter and includes the full vector clock
3. The server compares clocks — if the client clock is strictly newer, the update applies
4. If clocks are concurrent (both have changes the other doesn't), a **409 Conflict** is returned
5. The client receives both versions and can present a merge UI to the user

---

## Extending to Native Platforms

### React Native
Use `note-app.jsx` components with React Native adaptations (replace HTML canvas with `react-native-canvas` or `@shopify/react-native-skia` for drawing). The sync engine works as-is.

### Flutter
Use the sync engine via a JavaScript bridge, or port the API calls to Dart. For drawing, use Flutter's `CustomPainter`. For PDF, use `flutter_pdfview`.

### Electron
Wrap the React app directly in Electron for desktop. Add native file system access for PDF import/export.

---

## Security Notes

- All note content is encrypted with AES-256-CBC before database storage
- Passwords are hashed with bcrypt (12 rounds)
- JWT tokens expire after 30 days
- HTTPS should be enforced in production (use a reverse proxy like Nginx or Pangolin)
- The encryption key should be stored securely (e.g., environment variable, secrets manager)

---

## Tech Stack

| Layer     | Technology                         |
|-----------|-------------------------------------|
| Frontend  | React 18 + Vite                     |
| Editor    | TipTap (ProseMirror)                |
| Backend   | Node.js, Express                    |
| Database  | SQLite (dev), PostgreSQL (prod)     |
| Real-time | WebSocket (ws library)              |
| Auth      | JWT + bcrypt                        |
| Encryption| AES-256-CBC                         |
| Container | Docker + docker-compose             |