Read a PDF where it is imported, and keep the text with the file
A PDF was searchable on a device once that device had opened it, because the extracted text lived in that browser's IndexedDB and nowhere else. So a document imported on the desktop was invisible to a search on the phone — which is exactly the case that matters for filing invoices, since those are imported and then never opened again. The reading still never happens on the server. It has no PDF library and is not getting one: the bytes arrive encrypted and are stored that way. The device that imports the file reads it there and then, and the result is stored beside the file in attachments.text_index, encrypted at rest like everything else and opaque to the server. Every other device asks for it after login and can then find a document it has never downloaded. Not storing it was the earlier decision, on the reasoning that text derivable from bytes the server already holds should not be kept twice. True, and useless. A few hundred kilobytes against a file measured in megabytes is a trade worth making. A `null` answer is not an error — it means nobody has read this file yet. Documents extracted before this existed are sent up the next time they are opened. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
228552d453
commit
8e38404a69
8 files changed
+268
-23
No files matched your search
@@ -4,6 +4,7 @@ import * as sync from "../syncEngine";
|
||||
import * as searchIndex from "../search";
|
||||
import { sameStrokeDoc } from "../features/drawing/strokes.js";
|
||||
import { reindexEntries } from "../features/notes/noteSort";
|
||||
import { extractOnImport } from "../features/pdf/pdfText";
|
||||
import type { Backlink, NoteContent, NoteMeta, Notebook, StrokeDoc, User } from "../types";
|
||||
|
||||
/** How often the whole account is reconciled with the server. */
|
||||
@@ -203,6 +204,12 @@ export function useNoteData(user: User | null, activeNote: string | null): NoteD
|
||||
const title = file.name.replace(/\.pdf$/i, "") || "Dokument";
|
||||
const note = await sync.createPdfNote(folderId, title, attachment.id);
|
||||
setNotes((p) => [note, ...p]);
|
||||
/* Read the document here, now, on the machine that already has it open.
|
||||
*
|
||||
* Waiting until someone opens it means a filed invoice may never be read at
|
||||
* all, and until it is, no device can find it. Not awaited: the note exists
|
||||
* and should appear at once — the text catches up. */
|
||||
void extractOnImport(file, attachment.id);
|
||||
return note;
|
||||
}, []);
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@ import { useEffect, useMemo, useRef, useState } from "react";
|
||||
|
||||
import * as sync from "../syncEngine";
|
||||
import * as searchIndex from "../search";
|
||||
import { backfillPdfText } from "../features/pdf/pdfText";
|
||||
import type { NoteMeta, SearchHit, User } from "../types";
|
||||
|
||||
const DEBOUNCE_MS = 120;
|
||||
@@ -52,6 +53,12 @@ export function useSearchIndex(user: User | null, notes: NoteMeta[], query: stri
|
||||
},
|
||||
});
|
||||
if (!cancelled) setVersion((v) => v + 1);
|
||||
/* PDFs last: their text is stored with the file rather than derived from a
|
||||
* note body, so it has to be fetched separately — and a document nobody
|
||||
* has read yet is simply skipped. Searching a note is more urgent than
|
||||
* searching a scan, so this waits until the notes are in. */
|
||||
const indexed = await backfillPdfText(notesRef.current, signal);
|
||||
if (indexed && !cancelled) setVersion((v) => v + 1);
|
||||
running.current = false;
|
||||
})();
|
||||
|
||||
|
||||
Reference in new issue
Block a user