Arabic Citation Check
Does a cited work exist, and do its title, year, venue, author order and DOI belong together? A benchmark for citation checkers on Arabic scholarly references. It tests existence and attribution, not whether a paper supports a claim.
The Arabic text has not yet been reviewed by a native Arabic writer; the English governs.
Baselines
No model is involved. Each row shows how often a method calls a citation verified. For the verified class higher is better; for every other class lower is better.
The full field check was written together with the citation templates, so its score shows that labels and checker agree on these formats. It is not an estimate of how the checker does on references found in the wild.
Examples
Taken from the development split. Fabricated records use invented titles, venues and people, and DOIs that answered as unregistered on the check date.
Check your own references
The checker is a command-line tool in Python, with no dependencies. It reads Arabic-Indic digits, DOI forms and Hijri years.
pip install "git+https://github.com/Bayanat-Labs/arabic-citation-check@v0.1.0"
arabic-citation-check check --index works.jsonl --file references.txtNot found means not found in the index. It is a reason to look further, not proof of fabrication: many real Arabic works have no DOI and no record in any index.
Limits
All citation strings are rendered from templates, so these scores say nothing about how often any model invents references. OpenAlex coverage of Arabic journals is partial, and its language and venue country labels carry their own errors.
To correct or remove a record about you or your work, write to hello@bayanatlabs.com with its id.