Stop an empty env var from shadowing .env, and refuse a total OCR blackout
Two faults found by running the pipeline inside the Docker image for the first time, both of which let a 25-page run finish "successfully" having stored nothing. The image declares ENV DASHSCOPE_API_KEY= so the variable is visible to docker inspect and overridable at docker run. That leaves it set-but-empty in every container, and load_dotenv used os.environ.setdefault, so the empty string won and the mounted .env was ignored. An empty value now counts as unset; a real value still wins, because overriding on the command line is the point. The consequence was worse than a missing key. Every page failed, the per-page fallback kept the font-mangled embedded text -- exactly the text OCR is there to replace -- and the run went on to spend a full detection pass on it. Tolerating one unreadable page is right; tolerating every page is a configuration error wearing a success message, so Document now raises OcrUnavailable when OCR was attempted and not one page came back, quoting the first real failure. A page range that needed no OCR at all is still not an error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ci5T3gWDhhmqDec3e1XtYZ
This commit is contained in:
@@ -46,6 +46,10 @@ def normalise_page_text(text: str) -> str:
|
||||
return text
|
||||
|
||||
|
||||
class OcrUnavailable(RuntimeError):
|
||||
"""Raised when OCR was asked for and could not read a single page."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Page:
|
||||
number: int # 1-based, physical page order in the file
|
||||
@@ -178,6 +182,19 @@ class Document:
|
||||
replaced.append(page.number)
|
||||
say(f" page {page.number}: OCR (embedded text scored {quality})")
|
||||
|
||||
# One page the model chokes on is a bad page; every page failing is a broken
|
||||
# configuration -- a missing key, an unavailable model, no network. Falling
|
||||
# back page by page is right in the first case and disastrous in the second,
|
||||
# because the fallback text is exactly the mangled text OCR exists to replace:
|
||||
# the run completes, costs a full detection pass, and yields nothing. It is
|
||||
# cheaper to stop here than to explain the empty database afterwards.
|
||||
if skipped and not replaced:
|
||||
reason = next(iter(ocr.failures.values()), "no reason recorded")
|
||||
raise OcrUnavailable(
|
||||
f"OCR produced nothing for all {len(skipped)} page(s) via "
|
||||
f"{ocr.backend.name}; first failure: {reason}"
|
||||
)
|
||||
|
||||
if replaced:
|
||||
self.extractor = f"{self.extractor}+{ocr.backend.name}@{ocr.dpi}dpi"
|
||||
self.meta["ocr"] = {
|
||||
|
||||
Reference in New Issue
Block a user