One scanned page should not cost you the document

You hand over a forty page report. Thirty eight pages are ordinary typeset text that any PDF reader will hand back as a string. Two pages in the middle are scanned exhibits, photographed and dropped in.

The tool reads the thirty eight. Then it reaches the two, cannot read them, and gives you nothing at all. Not thirty eight pages and a warning. Nothing.

That is a different failure from the one usually written about, which is the quiet one: the file reports success and the scanned pages are simply missing from the output, so nobody learns that anything was lost. We covered that one in a scanned PDF extracts as an empty page, and the mechanism is that "does this file have text" is a per file test of a per page property. This post is about the loud failure, which looks better and is sometimes worse, and about how far the damage travels.

The work is per page and the failure is not

Here is the shape, from a desktop chat application that attaches PDFs to a conversation (OpenMindAI issue 420, open on 2026-10-31). The function is extractPdfText in src/lib/pdf.ts, read at commit b884a42. The first loop is the honest part:

for (let pageNumber = 1; pageNumber <= pagesRead; pageNumber += 1) {
  const page = await pdf.getPage(pageNumber);
  const pageText = await selectablePageText(page);
  pageTexts[pageNumber - 1] = pageText;
  if (pageText.replace(/\s+/g, "").length < MIN_SELECTABLE_TEXT_CHARS) {
    scannedPages.push(pageNumber);
  }
}

Every page is read. Every page's text is stored in pageTexts. Pages that came back with fewer than MIN_SELECTABLE_TEXT_CHARS characters (the constant is 24) are collected into scannedPages as candidates for OCR. That test does not need a scan to trip it: a blank page or a page with only a short heading qualifies too, which is the case the issue describes. At the end of that loop the useful work is already done and already in memory.

The second loop sends the candidate pages off for OCR in batches of four, and this is where it goes:

let ocr: PdfOcrPageResult[];
try {
  ocr = await invoke<PdfOcrPageResult[]>("ocr_pdf_pages", { pages: rendered });
} catch (error) {
  const detail = error instanceof Error ? error.message : String(error);
  throw new Error(
    `OpenMindAI Lens could not read scanned PDF pages ${batchNumbers.join(", ")}: ${detail}`,
  );
}

There is a try around the OCR call, so somebody thought about failure. But the catch rethrows, and the only enclosing handler in extractPdfText is a try/finally that destroys the pdf.js task. There is no second catch anywhere in the function. So the throw leaves the function, and pageTexts leaves with it. The assembly step that turns pageTexts into the returned string sits below the OCR loop and is never reached.

Thirty eight pages of text that were read correctly, that needed no OCR, that nothing was wrong with, are discarded because pages nineteen and twenty could not be photographed into words.

The error message is precise about which pages failed. The caller never gets to use that precision, because there is no partial result to attach it to.

An optional dependency that fails a whole request

Why would the OCR call fail at all? The interesting case is not a corrupt page. It is that the OCR engine is optional and not installed.

On the Rust side, ocr_pdf_pages in src-tauri/src/pdf_ocr.rs starts like this:

let lens_status = installed_catalog_entry_by_id(&state, "qwen25-vl-3b-q4km")?
    .ok_or_else(|| {
        AppError::ModelNotFound(
            "OpenMindAI Lens is required to read scanned/image-only PDF pages. Download Lens from Settings > Models first."
                .to_string(),
        )
    })?;

The vision model is a separate download. If it is absent, the command returns an error before looking at a single page. That error is correct and its message is helpful. The problem is purely one of scope, and it compounds twice: a missing optional model fails the whole batch, and a failed batch fails the whole document.

So the user experience is that installing an optional add on becomes a precondition for reading PDFs that do not need it. A file whose text was sitting in a variable is refused because of a capability it never required.

The issue asks for exactly the change you would expect: keep the text that was verified, mark the image only pages as incomplete, and report which pages were skipped. On 2026-10-31 the issue was still open, and those two files were unchanged at the tip of the default branch.

Then the blast radius gets larger

That first case stops at one user and one file. The second one does not, and it is worth following because it shows what happens when the same mismatch sits inside a shared pipeline with a retry loop.

A Zotero indexing backend (zotero-rag issue 69, fixed by PR 70) sends attachments to a shared text extraction sidecar. Large PDFs are split into parts first, so that no single request can exhaust the sidecar's memory. The guard looked like this:

pdf_split_threshold: int = Field(
    default=50 * 1024 ** 2,
    description="PDFs larger than this are split into parts before sending to kreuzberg. "
                "Accepts '50MB', '1GB', or a raw byte count.",
)

A threshold in bytes, in the code since May. On 2026-10-07, commit 2a43323 added a second byte guard, a hard cap of 200 MiB per request, with a comment recording that production had seen an 8GB out of memory kill from a single 329MB scanned PDF. A byte shaped incident, answered with a byte shaped guard. Reasonable.

Two days later, an issue recorded the same sidecar being killed by a file less than a sixth of that size.

According to the issue that records it, the file was 50,878,921 bytes, which is roughly 48.5 MiB, holding 1,643 page objects with essentially no text operators in them: an image only scan where every page needs OCR. The default threshold is 50 * 1024 ** 2, that is 52,428,800 bytes, so the file was around 1.5 MiB under the line and went to the sidecar whole. The issue reports the sidecar in an out of memory crash loop against an 8GB limit.

Bytes were never the quantity that mattered. A scanned page of text compresses extremely well, so a byte budget buys you an enormous number of pages, and OCR is paid per page. The maintainer's own fix (commit 8b35032) says so, in the description of the setting it added alongside the byte test (the byte threshold stays, the page cap joins it):

pdf_scan_max_pages_per_part: int = Field(
    default=50,
    description="PDFs without a text layer (image-only scans, every page needs OCR) are split "
                "into parts of at most this many pages before being sent to kreuzberg. "
                "Byte size is a poor proxy for OCR memory cost, so this applies regardless "
                "of pdf_split_threshold.",
)

And again, in a one line comment on the new page cap argument to the splitter: max_pages_per_part is documented as "Optional page cap per part, applied on top of the byte target (OCR memory cost scales with pages, not bytes)."

A retry with no cap is a queue outage

The memory crash is only half of that incident. The other half is what a shared queue does with a file that fails every time.

Deferred uploads were drained oldest first:

def list_entries(data_path: Path, library_id: str) -> list[dict]:
    """Metadata for every pending entry in a library, oldest-enqueued first.

and a failure did this:

def record_failure(data_path: Path, library_id: str, attachment_key: str, error: str) -> None:
    """Bump `attempts` and set `last_error` on an existing entry. No-op if it's gone."""

Increment a counter, store the message, change nothing about the entry's position and nothing about whether it will be tried again. Those two behaviours together are a head of line block: the entry that cannot succeed is the oldest entry, so it is always first, so every run for that library begins by crashing the sidecar. Work queued behind it in that library never ran, and the sidecar is shared, so every other library lost it too. The issue reports the entry at eighteen attempts after two days in the queue.

The fix changes the ordering key and gives failure a limit. The docstring is the whole argument in one sentence:

def list_entries(data_path: Path, library_id: str, include_quarantined: bool = False) -> list[dict]:
    """Metadata for pending entries in a library, least-attempted first, then oldest-enqueued.

    Ordering by attempts keeps a permanently failing entry from blocking the
    never-tried ones queued behind it.

Along with it: a page count profile taken before extraction (cheap, no OCR), scans split into parts of at most fifty pages whatever their size, scans over two thousand pages refused outright rather than attempted, and an entry quarantined after five failures and surfaced to the user as a tag they can remove to retry. The commit message for the fix names the symptom directly: the sidecar "was OOM-crash-looping on them and blocking the shared extraction queue".

One caveat on that profile, since this post is otherwise approving of it: it decides whether a document has a text layer by sampling about twenty evenly spread pages. A mostly scanned document with text on one sampled page reads as having a text layer and skips the page cap. That is a cheap test and a sensible default, and it is still a sample based answer to a per page question.

Three units, three different right answers. The page is the unit of extraction. The document is the unit the user asked about. The queue entry is the unit of scheduling. In both projects the code had picked a unit that was convenient rather than the one the work was actually measured in.

Measured: equal bytes, very unequal work

The maintainer's comment says bytes are a poor proxy for OCR memory cost. That is worth testing rather than repeating, so I built two image only PDFs of nearly the same size, with no text layer in either.

file	pages	bytes	image pixels
A, large random noise JPEGs at 300 dpi	2	19,226,944	16,368,000
B, synthetic bilevel pages at 200 dpi, Group 4	350	19,776,473	1,309,000,000

File A is random noise because noise is what compresses worst, which makes it the extreme case of few pages and many bytes. File B has text like strokes, not real text. The sizes differ by under 3%, the pages by a factor of 175 and the pixels by a factor of 80. Run the Zotero fix's text layer pattern over every page, which looks for Tj or TJ operators, and neither file has a match. pypdf extracts no characters from either.

I rasterised every page at 200 dpi in three independent engines, three runs each in a fresh process. Page sizes are close, so each engine draws nearly the same number of pixels per page for both files.

Time:

file	poppler	MuPDF	PDFium
A, 2 pages	0.5 s	1.3 s	0.4 s
B, 350 pages	37.5 to 38.6 s	6.7 to 7.0 s	9.5 to 9.7 s

Peak RSS:

file	poppler	MuPDF	PDFium
A, 2 pages	40 MiB	210 to 212 MiB	96 MiB
B, 350 pages	40 to 41 MiB	392 MiB	55 to 56 MiB

At the same byte size, the many page file takes about 75 times as long in poppler, about 5 times in MuPDF and about 24 times in PDFium. The factors are far apart, partly because poppler writes each page to disk here and the other two do not, so read down each column rather than across. All three agree on the direction, and that is the claim about work.

Memory is the result worth slowing down for, because rendering does not grow with page count at all. Poppler peaks at 40 MiB rendering the first page alone, and at 40 to 41 MiB for all 350. PDFium's peak is lower on the 350 page file than on the two page one. MuPDF does appear to grow, but that is its cache: emptying the cache after every page takes file B from 392 MiB to 139 MiB, which is exactly what its first page costs alone.

So rendering one page at a time holds memory flat, whatever the page count and whatever the bytes. That fits what the Zotero issue says about its own crash: the sidecar handled the whole document in one non streaming request and had to assemble the entire result before responding. I did not run that sidecar, so this does not prove its cause. What it does show is that nothing about a PDF page forces the memory to pile up. Taking 1,643 pages as one unit of work is a choice in the pipeline, and it is the same lesson from the other direction.

Where AegisPDF fits, including where it does not

AegisPDF is our browser based PDF tool, and OCR is one of its paid features. The relevant question for this post is what its unit of failure is, and the answer is mixed.

It chooses targets per page, which is the part the earlier post was about. "OCR all" builds its target list from the pages that actually need OCR and skips pages that already have real text, since layering OCR over existing text duplicates every match.

But the failure unit is the whole document. The driver loop is:

const snapshot = doc.toBytes();
...
try {
  for (const p of targets) {
    await ocrPage(p);
  }
  ...
} catch (err) {
  // Roll back any partial application so doc matches the pre-OCR snapshot.
  doc.loadBytes(snapshot);

ocrPage throws on failure, there is no per page tolerance inside the loop, and the catch restores the pre OCR snapshot. If page 300 of a 400 page scan fails, the text layers written to the first 299 are discarded.

That is the same shape as the first case in this post, and unlike that case it is deliberate, which is why it is worth arguing about rather than just fixing. There is a real distinction here that the OpenMindAI code misses:

Extraction is read only. OCR that writes a text layer mutates the document. When you are reading text out of a file, a partial result is strictly better than no result, because nothing downstream is made inconsistent by the pages you did not get, as long as you say which ones they were. When you are writing an invisible text layer into a document, a partial result is a file in which some pages can be searched and some cannot, and nothing in the file records which. A search that finds nothing then looks like an answer.

All or nothing rollback is a defensible answer to that, and it is the one AegisPDF picked. The better answer is probably to keep the completed pages and record, in the document, which pages have a verified layer and which do not, so a partial result is legible rather than silent. We have not built that. Two things that follow from reading the code, stated plainly rather than softened:

• There is no page count or file size cap on the OCR path. Nothing refuses a 1,643 page scan the way the Zotero fix now does. It will try, in your browser tab.
• The blast radius is one tab. There is no shared sidecar to crash, no queue for your file to block, and nobody else's document is affected by yours, because the work happens on your device and never leaves it. That is a property of the architecture rather than of any care taken in the OCR path, and it is the reason the second half of this post cannot happen to an AegisPDF user.

What was verified here, and what was not

• The code was read at named commits. OpenMindAI at b884a42, and I confirmed its two files are unchanged at the default branch tip 7e3f11b. zotero-rag on both sides of 8b35032. AegisPDF at its production commit 019e0de. Every quoted line and comment comes from those checkouts.
• The incident figures are the issue's, not mine. The file size, page and operator counts, eighteen attempts, two days and the crash loop against the 8GB limit are what issue 69 reports, read in full. I did not have the file. The 329MB figure is from a code comment.
• The OpenMindAI failure was read, not run. The issue says its own smoke test is pending, and I did not run the app either, so this is a reading of its control flow.
• The measurement covers rendering, not recognition. Poppler 24.02, MuPDF 1.28.5 and PDFium, on synthetic files on one two core machine, so the times do not travel. Nothing went through Tesseract, PP-OCRv5, Kreuzberg or the Lens model. Rendering is the step before OCR, and the cost of recognition itself was not measured.
• AegisPDF was read, not run. The product section is a reading of the source. No large scan was put through the shipped tool, so I make no claim about how far it gets before the tab runs out of memory.
• No count of reports appears here. The two incidents are examples, not a survey.