S-OBI asks whether AI can read the whole divination

Inscribed ox bone at the Chinese Writing Museum in Anyang, photographed by Gary Todd on December 13, 2011. This contextual photograph is not identified as an S-OBI sample. Source: Wikimedia Commons, CC0.[5]

An inscribed ox bone presents two puzzles at once: which marks are writing, and what those marks say together. Cracks interrupt strokes; a familiar-looking sign can have several forms. Even after the characters have been identified, a reader still has to reconstruct a record of people, actions and circumstances.[4]

S-OBI makes that second problem an AI test. Its question is whether a model can connect ancient glyphs into the meaning and structure of a complete divination. The public release asks for interpretations, reading order, semantic roles, punctuation and missing characters. That is a substantial step beyond attaching a label to a cropped sign.[2]

The project comes from researchers affiliated with Nanjing University of Science and Technology, Shanghai Jiao Tong University and Shanghai Artificial Intelligence Laboratory. Its June 30, 2026 preprint reports a striking gap: Qwen3.6-Plus-thinking scores 89.2% on order selection, but 32.6% on meaning matching. Selecting a plausible sequence is evidently easier than recovering what the inscription says.[1]

A cleaner inscription, a harder question

The earlier OBI-Bench, accepted at ICLR 2025, assembled 5,523 images across recognition, fragment rejoining, classification, retrieval and deciphering. Its scope included original bones, rubbings and individual characters. That broad design asked where multimodal models might help along a scholar's workflow.[3]

S-OBI narrows the lens to a complete inscriptional group. The authors replace damaged glyphs with clearer examples while preserving the underlying inscription's content and organization. This controls some visual noise; it also moves the experiment away from the surface of an excavated object.[1]

The release documentation, available in both English and Chinese, records 95 base inscriptions and 695 questions. Those are different denominators: questions and image variants repeatedly draw on the same small collection. The archive includes original rubbing images alongside constructed benchmark images, but the scored tasks use the prepared inputs.[2]

Its three task families expose different kinds of understanding. T1 asks about meaning and order. T2 requests a structured record: subject, action, target, time, outcome, the introductory divination information and the proposition being tested. T3 asks for punctuation in an ordered grid or a missing character inferred from context.[2]

For a reader, the distinction is intuitive. Putting pieces of a sentence into a familiar sequence does not guarantee that an action has been assigned to the right person. A polished interpretation can conceal precisely that mistake. By asking for the components separately, the benchmark creates places where a fluent answer can be checked.

What the reported scores establish

The preprint evaluates ten models, using APIs for some systems and up to eight 24 GB RTX 4090 GPUs for locally run models, with each model's default generation settings. Its table gives Qwen3.6-Plus-thinking 33.5% overall. The authors also report zero completely correct slot frames across all evaluated models. These are results under the paper's setup, not a November 2026 leaderboard or a measured success rate on museum collections.[1]

The useful finding is the unevenness. A system might help arrange a candidate reading while remaining unreliable at filling in the whole record. For a research assistant, those are different jobs. The former can supply something to inspect; the latter requires enough accuracy to support a claim about an ancient event.

The scorer changes how to read the number

The public toolkit makes the distinction between progress and completion unusually concrete. Its T2 primary score averages several component measures. Some demand an exact match; others reward overlap in a list or similarity between character sequences. A separate measure records whether all slots are exact.[2]

Imagine a response that gets the subject and date right but assigns the wrong action. Component scoring can recognize the useful pieces without declaring the record correct. That is sensible for diagnosing improvement. It becomes misleading if a reader treats the resulting percentage as the fraction of inscriptions fully understood.

Aggregation matters too. T2 contributes 350 questions, compared with 190 for T1 and 155 for T3. Averaging every question therefore gives slot extraction more influence. The toolkit's recommended headline instead gives each of the three task families equal weight. Neither convention is automatically wrong; they answer different questions about performance.[2]

The maintainers explicitly caution that running the released scorer does not reproduce the paper's tables without matching model outputs and evaluation conditions. As checked on November 6, 2026, the release also defines no official training, validation or test split. Labels inherited from a source dataset must not be mistaken for such a split.[2]

That leaves a practical comparison rule: keep the archive version, prompts, model revision and scoring protocol together. Show the component results alongside the aggregate. Because variants share base inscriptions, any future experiment that trains on part of this material should also explain how related inscriptions are kept apart. Otherwise, success on a familiar family of examples could be mistaken for transfer to unseen writing.

Back to the bone

A 2026 survey of oracle-bone computing describes the material obstacles that standardized inputs reduce: fragmentation, variable glyph forms, confusingly similar signs and damage in rubbings. It also notes that collections dominated by rubbings leave a gap when systems must work from original bones.[4]

S-OBI is useful precisely because it isolates a question within that larger difficulty. If a model struggles with a cleaner, structured inscription, improving the camera alone will not supply the missing understanding. Conversely, progress on these prepared images would still need testing against the surfaces scholars actually encounter.

The next convincing result would join those two tests: better recovery of complete semantic records, followed by evidence that the improvement survives original photographs and unfamiliar inscriptions. Until then, S-OBI offers a productive way to ask where a reading goes wrong—and to keep partial understanding visible.

cronfeed.work

S-OBI asks whether AI can read the whole divination

A cleaner inscription, a harder question

What the reported scores establish

The scorer changes how to read the number

Back to the bone

Sources

Recommended In ai china