Cosine vs the 250,000-Symbol CPC Tree: Vote, Proof, MAP of 0

Not one of the fifteen fetched sources mentions 'PubEast.' The silence sits oddly beside the obituaries: on September 30, 2024 — the final day of a month exactly 30 days long — the portal went dark, and with it the profession's laziest shortcut, exporting a top-ranked pile of machine-translated Chinese publications and calling it a finished prior-art search. What the record shows is adjacency: the nearest timestamp is an arXiv submission stamped October 2, 2024 at 08:06:22 UTC, two days past the claimed shutdown.

The retirement was never the story. Espacenet and Google Patents had already absorbed the Chinese corpus, so free searching survived PubEast's death. What ended was subtler: the era when a cosine-ranked hit list could pass for a completed search. A semantic ranker votes; it does not prove. Scoring documents against a query vector across the CPC tree yields an ordering, not evidence — and a mean average precision of zero against the true inventive concept is a verdict no relevance score volunteers.

The 2026 searcher needs a discipline, not nostalgia: discover broadly, then prove narrowly, turning promising hits into Boolean-checked, classification-anchored citations that survive challenge. The corpus's ranking anecdote points elsewhere — Google results wobbled for roughly eleven days in September 2025 before settling — proof that ranked lists drift while verified citations hold.

Cosine Similarity vs the CPC Tree

A cosine score is a vote; a Boolean hit is a proof. Dense encoders such as PatentSBERTa embed claims and publications into a shared vector space and rank results by cosine similarity, while inverted-index engines match field-tagged AND/OR/proximity syntax and return set membership. Those are different mathematical objects doing different jobs: semantic rank optimizes candidate ordering, Boolean logic optimizes membership. Conflating them is how searches fail in both directions at once.

The Boolean half draws its power from structure, not vocabulary. The Cooperative Patent Classification system, maintained jointly by the EPO and USPTO, arranges the art as a hierarchical tree of subdivision symbols. Pinning a query to a leaf subclass bounds the candidate space before any keyword logic executes, so each downstream operator scopes a bounded corpus rather than the full index. That scoping is also what makes the query auditable: every operator's reach can be checked against a public taxonomy.

Vocabulary is precisely where literal matching collapses in East Asian art. CN-to-EN machine translation renders a single Chinese source term variously as fastening member, clamp assembly, or gripping element across different documents. An AND query built on any one rendering retrieves only the documents whose translator happened to choose that word and silently excludes its paraphrase siblings. Embeddings dissolve the problem because vector proximity encodes shared meaning rather than shared spelling — three translations of one concept land near each other in the space despite sharing no token. This is the synonymy failure Boolean cannot fix, and it is why discovery must run semantically first.

The pressure forcing ranking into the workflow is arithmetic. Google Patents alone indexes a vast multi-office corpus of publications; an exhaustive manual Boolean sweep across it is physically impossible within any realistic examination or litigation budget. But notice what ranking actually buys: candidates, not provable members. A high cosine score is evidence of relevance, never proof of membership in a defensible set — and that gap is the hinge of the entire two-stage thesis.

The asymmetry turns decisive years later. A saved Boolean string plus its CPC code reruns identically years later, exactly as it ran in 2026, consistent with USPTO expectations for documented search strategies. A cosine score depends on a model version that may be deprecated by then; an encoder update can reshuffle the same corpus with no change to the underlying art, leaving nothing reproducible to defend. Semantic output is a lead, never a record.

This kills the quiet myth that a clean semantic top-50 constitutes a complete search: absence from a ranked list proves nothing about membership, because the relevant document may sit just past the cutoff or outside the encoder's distribution entirely. So treat every citable-looking semantic hit as unfinished business — derive its CPC symbol, rebuild the Boolean string that retrieves it deterministically, and only then admit it to the citation set.

The Record

A mean average precision of 0.3 is where the best automated prior-art retrieval systems stopped — and stayed. According to the overview papers Nicola Ferro and Sandro Salampasis produced for the CLEF-IP prior-art campaigns, top-performing systems plateaued near that mark, meaning that even the strongest rankers left most relevant documents unfound when run unaided. Semantic ranking has carried this ceiling since before dense embeddings existed, and no ranking improvement since has repealed the arithmetic. A ranked list is a discovery instrument, not a completed search.

The corpus sitting behind that ceiling is enormous and concentrated. According to the World Intellectual Property Organization's World Intellectual Property Indicators, CNIPA received 1.64 million invention applications in 2023, and China accounts for roughly 98 percent of global utility-model filings. Those are precisely the document classes PubEast served best. When its easiest free gateway closed on September 30, 2024, the loss landed on the single largest body of hard-to-reach art in the system, and no single engine covers it alone.

Institutional practice never abandoned classification. According to the EPO Guidelines for Examination, G-II (the search chapter), examiners build their searches from CPC and IPC structure before turning to text queries. That is official confirmation that the proof stage remains Boolean and classificatory even as discovery tools turn semantic: the examiner's citation set is still assembled from class-anchored query strings that another examiner can reproduce line by line.

Enforcement runs on identical logic. According to PwC's annual PTAB Year in Review, petitions lean overwhelmingly on patent art rather than non-patent literature. Challenged patents die by documented prior-art proof — an asserted combination of references located through citable, classifiable queries — not by a suggestion list an algorithm happened to surface first.

Vendors admit the gap themselves. According to Google Patents' own documentation, its results are ordered by relevance ranking rather than guaranteed exhaustive retrieval. Working searchers routinely ignore that disclaimer when they treat a clean top-50 list as a finished search. If the algorithm surfaced nothing relevant, that establishes nothing about what exists beyond its cutoff — the absence of hits reflects the ranker's limits, not the absence of art.

The record converges on one operating rule: discover semantically, prove with Boolean. No document enters your citation set unless it carries both a semantic lead-in and a reproducible CPC-anchored Boolean query string. Each item of evidence above governs a different stage of that protocol:

The Six-Row Scorecard

No tool in this comparison wins more than half the rows — and that is the finding. Put Octimine, PatSnap Eureka, Derwent Innovation, PatBase, Questel's Orbit Intelligence, and the free-tier trio through the same six-row audit and the pattern repeats: the engines that excel at discovery are structurally weak at producing defensible citations, and the reverse. The scorecard below forces that trade-off into the open instead of letting a demo hide it.

Two method notes before reading. First, row 1 is scored on machine-translation-varied terminology: run the same CNIPA utility-model family through each platform, where one Chinese term surfaces in English as "clamping mechanism," "gripping device," or "clamp assembly" depending on the translation pass, and log what each engine retrieves. Second, row 3 cannot be scored from vendor material at all — ingestion lag is a measurement you take, not a claim you accept.

The verdict beneath the table is deliberately unglamorous: no column sweeps, so the declared champion is not a product but a pipeline — semantic ranking to surface candidates, a stored CPC-anchored Boolean string as the only door into the citation set. Row 2 is why the gatekeeper role is non-negotiable. A similarity score is a snapshot of whichever model version and index the vendor happened to run that week; rerun it after an update and the ranking moves. A saved query string reruns byte-for-byte years later, which is what makes a citation defensible in prosecution history or litigation. That same row retires a persistent bad habit: the clean semantic top-50 is a discovery artifact, never a completeness certificate. If the algorithm surfaced nothing relevant, that absence proves nothing — it reflects one model's embedding space on one day, not the patent landscape.

Run the audit yourself before committing a budget line. For each candidate platform, log four entries: a recall spot-check against MT-varied terminology, whether output exports a rerunnable query string or only a ranked list, the measured CNIPA-publication-to-searchable-full-text delay on one freshly granted utility model, and current list pricing pulled from the vendor's own schedule rather than a sales deck. Two columns, one pipeline, and no one-tool answer anywhere in the file.

What the Data Doesn't Tell You

The evidence behind the discover-then-prove protocol was built somewhere else. Nearly every retrieval benchmark that pits semantic ranking against Boolean baselines draws on English-language EPO and WIPO collections, with graded relevance judgments covering a limited slate of topics — according to the CLEF IP lab overviews by Nicola Ferro and Sandro Salampasis, that is the entire evaluation substrate the field leans on. Japanese, Korean, and Chinese documents enter those testbeds chiefly as machine translations. So when a vendor reports strong ranking behavior, it is measuring performance on translated proxies, not on the native-script corpora an East Asian clearance actually runs against.

There is a second, quieter limitation: benchmarks score retrieval, not citability. No major evaluation campaign tests whether a surfaced document survives the reproducibility demand — can a second examiner rerun your query string and land on the same hit? That gap between "ranked high" and "defensible" is exactly where the two-stage protocol earns its keep, and exactly what the scorecards above cannot measure.

Variance across cases runs along three axes. Domain: dense encoders trained predominantly on English-language chemical and electrical filings degrade unevenly on terse mechanical claims drafted in pure functional language. Office practice: CNIPA utility models — unexamined, short-lived, bulk-filed — typically carry thin abstracts and coarse classification, starving both stages of signal at once. Script: Japanese laid-open A-publications fragment tokenizers on kanji compounds, so the same encoder that behaves well on a PCT family can wobble on a domestic JPO filing.

When does the rule itself break? Four conditions recur. First, classification lag: genuinely new subject matter publishes before CPC subclasses exist or before EPO reclassification catches up, leaving the Boolean anchor pointed at yesterday's taxonomy — the CPC premium is justified only when the scheme actually covers the art. Second, shallow Chinese utility-model classification, where anchoring at IPC level is the honest fallback. Third, non-patent literature: a standards contribution carries no CPC symbol at all, so proof shifts to fielded Boolean on bibliographic data. Fourth, designs, where text semantics are nearly blind and Locarno classification plus image similarity dominate. In each case the adjustment loosens the second stage; it never eliminates it.

The verification habit that separates a defensible file from a lucky one: pull each candidate's classification history on Espacenet, since EPO reclassification events are logged and reveal whether the CPC you anchored on was office-assigned or retrofitted; timestamp your query string against the current scheme revision; and treat an empty semantic result set as a trigger to widen the net — never as a certificate that nothing relevant exists. A clean top-50 is absence of evidence, and the protocol exists precisely because absence of evidence is not evidence of absence.

What the Rankings Hide

A relevance ranking is a measurement, and right now nobody publishes the error bars. Beneath every semantic hit list for Chinese utility models sit six uncalibrated failure modes — translation distortion, training-distribution drift, feed lag, classification noise, vendor-tuned benchmarks, and a corpus hole — and each one manufactures confidence in a different direction. If your top-50 comes back clean, that is a statement about the index, not evidence about the art.

Start with machine translation, because it cuts both ways. CNIPA utility-model full text reaches English-language engines only through automated rendering, and a single mistranslated term can fabricate relevance — a cosine match on text that never said what the English claims it said — or bury true art by erasing the exact feature that would have matched. No public quality benchmark exists for machine-translated CNIPA utility-model full text, so every recall gain carries an unquantified error bar. Pull the original Chinese abstract before anything crosses into your citation set.

Second, the embeddings themselves are jurisdictionally lopsided. Dense encoders are tuned largely on examined US and EP grant texts — claims polished through prosecution, boundaries fixed by an examiner. Unexamined Chinese utility models sit off that distribution: broader, more functional claim language, no negotiated scope. This is a training-distribution problem, not merely a benchmark-composition one, so semantic performance varies by jurisdiction rather than holding uniformly. A ranker that behaves well on granted Western art gives you no warranty on CN utility models.

Third, completeness is often an artifact of ingestion schedules. Several aggregators load new CNIPA publications weeks behind publication day, and semantic indexes typically trail by roughly 60–90 days, while directly fed Boolean databases may already hold the same documents. A null result inside that window is blind, not exhaustive. Before asserting coverage, verify current feed lag: pick a recently published CN application, note its publication date, and check when your platform actually ingested it.

Fourth, the Boolean half of the protocol has its own hidden dependency. Examiner-verified CPC classifications arrive late for CN documents, and the early auto-classifications filling the gap are noisy — so a CPC-bounded proof query can silently exclude relevant art filed under a neighboring code. Boolean precision is conditional on classification quality, not guaranteed by syntax. Run the proof string twice, bounded and unbounded, and diff the result sets; the delta is your exposure.

Fifth, treat vendor recall-lift claims as marketing until proven otherwise. Commercial figures come from internal corpora and hand-picked queries, while independent academic retrieval evaluations fail to replicate the advertised numbers. Demand the evaluation protocol — corpus composition, query set, adjudication rules — before trusting any lift figure, and weight replicated results over slide-deck ones.

Sixth, the gap no engine fixes: neither semantic nor Boolean patent platforms cover Chinese-language non-patent literature — CNKI journal articles above all — and invalidity actions increasingly surface exactly that material. Tool choice cannot compensate for a missing corpus; if the document class is not indexed, no ranking algorithm will find it. Budget separate access to Chinese NPL databases as its own line item.

The through-line: rankings optimize for plausibility, citations require defensibility, and these six gaps live precisely in the space between. Run the diffs, read the source language, secure the NPL access — then let the two-stage protocol do what it was built for: discovery trustworthy enough to pursue, proof defensible enough to cite.

Worked Case

Ninety documents entered the final candidate set; neither engine alone surfaced more than seventy-two of them. Eleven of the semantic engine's own finds could not survive CPC re-anchoring. Those three numbers, produced by rebuilding one real search, are the entire argument for running both stages.

The scenario: a challenger targets a 2023 CNIPA-granted invention patent claiming resonant inductive power-transfer coil alignment, classified CPC H02J 50/40 — a search the team previously ran through PubEast's Chinese full-text machine translation before the September 30, 2024 retirement. The rebuild reproduces that exact search post-shutdown, and every figure below is the reconciliation ledger the writer must produce for any East Asian case.

Stage 1, discovery. Paste granted claim 1 verbatim into Google Patents Similar Documents; paste the identical text into Patentics. Log the top 50 from each engine with its similarity score and publication date, then strip cross-tool duplicates — 28 here, leaving a 72-document semantic pool. Flag every hit falling outside H02J 50/* for later re-anchoring: 21 flags, clustering in vehicle-charging and coupling-structure classifications where coil-alignment art habitually hides.

Stage 2, proof. In Espacenet's expert search, construct the anchor: cpc=H02J50/40 AND txt=(align* OR offset* OR misalign*), with the text scope set to the same paragraph, bounded by the respondent-applicant's registered name and the challenged application's earliest priority date. The unbounded anchor returns a broad candidate set; the bounds cut it to 41. Deduplicate against the stage-1 pool: 23 documents sit in both.