screen-pinyin

A desktop app (Electron, Windows/Mac) that screenshots any region of the screen, OCRs the Chinese text, and shows pinyin and English definitions per character or word, with a spaced-repetition flashcard mode for review.

The problem #

Reading Chinese on screen (a webpage, a game, a PDF) means stopping to retype each unfamiliar character into a dictionary app. I built ScreenPinyin to skip that: a hotkey, a drag-select over the text, and the pinyin and definitions show up next to it.

What I built #

Electron 39 + React 19 + TypeScript, with Tailwind for styling and Electron Forge for packaging. Everything on the lookup path runs locally: OCR via Tesseract.js with a bundled chi_sim.traineddata, word segmentation via nodejieba, and dictionary lookups against a local SQLite database with FTS5 full-text search, built from CC-CEDICT by a custom build script (scripts/build-sqlite-dictionary.mjs). Only full-sentence translation leaves the machine, as a call to Azure Translator. HSK level tracking and an FSRS-based spaced-repetition review mode round it out.

The hard part #

The decision was to keep everything on the lookup path local. Only full-sentence translation goes over the network, because it's the one step that needs a full sentence of context a local model can't give cheaply; everything else stays local so a hotkey lookup doesn't wait on a round trip, still works offline, and costs nothing per lookup.

Second: after the feature set stabilized, I ran a cleanup pass. I extracted a monolithic results-view into a directory of small components, replaced ad hoc keyboard-event chains with a single useKeyboardShortcut hook, swapped a hand-rolled debounce for the use-debounce library, then added ESLint (import-x, jsx-a11y, unicorn, react-hooks) and dependency-cruiser.

What it shows #

I build tools to solve my own problem first, and go back to clean up my own code once the feature set is done.