Documentation

BUDOVA (Building Ukrainian Domain-Specific, Open Voice & Text Archives) is building an open Ukrainian construction-domain language dataset: a text corpus with named-entity (NER) annotations, a speech corpus and a bilingual lexicon.

Status: pre-release. Nothing has been published on Hugging Face or Zenodo yet. The first public release, v1.0 — a sample of the text corpus with a datasheet and the annotation guidelines — is planned for Q4 2026 on Hugging Face and Zenodo. Speech data, the lexicon and a baseline NER model will follow in later releases.

Available now on this site (as of October 2026):

• Corpus browser — 2,927 annotated segments from nine Ukrainian building codes (ДБН);
• Annotation guidelines — v0.22, 118 rules, each citable by anchor;
• Inter-annotator agreement for the merged entries;
• Source catalog — the document every task comes from;
• Dataset page — live samples, the record format and the citation;
• Changelog — what changed, month by month.

A living datasheet for the text corpus, organized around the questions of Datasheets for Datasets. It describes the corpus as of October 2026 and will be finalized for the v1.0 release.

Motivation. To provide an open, documented NER corpus of Ukrainian construction-domain text, starting with building codes. The project is run by the KNUCA AI Lab (Kyiv National University of Construction and Architecture) with support from the Microsoft AI for Good Lab through the LINGUA Open Call.

Composition. 2,927 segments from nine building codes (ДБН), with about 16,700 gold entity spans in nine labels. Each segment is a self-contained fragment of normative text, usually a complete clause or paragraph. The source editions date from 2008 to 2026.

• ДБН В.2.6-98:2009 — concrete and reinforced-concrete structures: 904 segments, 5,014 gold spans;
• ДБН Б.2.2-6:2013 — sanitary cleaning of settlements: 199 segments, 974 spans;
• ДБН В.1.2-8:2021 — hygiene, health and environmental protection: 89 segments, 231 spans;
• ДБН В.1.1-45:2017 — buildings and structures in difficult ground conditions: 367 segments, 2,593 spans;
• ДБН В.1.2-12:2026 — infill development: 133 segments, 344 spans;
• ДБН Б.2.2-1:2008 — cemeteries, crematoria and columbaria: 241 segments, 1,143 spans;
• ДБН В.2.6-220:2017 — roofs of buildings and structures: 559 segments, 4,392 spans;
• ДБН В.1.1-25:2009 — engineering protection of territories and structures from flooding and inundation: 328 segments, 1,446 spans;
• ДБН В.1.2-7:2021 — basic requirements for buildings and structures, fire safety: 107 segments, 581 spans.

Span counts come from each document’s gold record. Two fragments, one in the cemeteries code and one in the roofs code, repeated text already in the corpus and were merged into the existing record: the corpus keeps one task per identical text.

Collection and preprocessing. The sources are published editions of the codes, with the text extracted from PDF. Each document is cut into self-contained fragments — a clause or paragraph, normally up to 1,200 characters. Longer paragraphs are split on sentence boundaries, and notation entries (such as «k – коефіцієнт») are kept so that symbols resolve. An automated linter checks every fragment against the fragment standard before it is loaded, and identical fragments from different documents are stored once.

Annotation. Nine entity labels — material, tool, process, measurement, structure, safety, regulation, property, load_effect — are defined in the public annotation guidelines. Every segment receives three independent annotation passes (no pass sees another’s output), followed by adjudication against the guidelines and an audit of unanimous records, where all three passes agree. The guidelines grow from document to document (now v0.22, 118 rules), and a document counts as done only when every fragment has been ruled on and merged. Agreement metrics for the merged entries are on the IAA page; expert validation on a sample is still pending.

Record format. JSON lines, one merged record per segment — see Text corpus below.

Known limitations.

• One register: all text is formal normative prose from building codes. Speech, field notes and other genres are not represented yet.
• Eight documents: the domain mix follows those codes, and the labels tool and safety are rare.
• Segments with no entity are kept: “no label fits” is recorded as an empty annotation, not skipped.
• There is no official train/validation/test split yet. Segments from one document share wording, so evaluate with document-level splits.

Uses. Intended for research on named-entity recognition and terminology extraction in Ukrainian technical text. The segments are excerpts and do not replace the official text of a building code.

Distribution. Planned from v1.0 on Hugging Face (the BUDOVA organization) and Zenodo (with a DOI). The annotations will be released under CC BY 4.0; the basis for redistributing each source text will be documented before the release.

Maintenance. Changelog versions (v0.1–v0.10) are platform milestones, not dataset releases; the first dataset release will be v1.0. The annotation guidelines are versioned separately (currently v0.22). Report errors through the contact form on the Join page.

Target: 100M+ tokens of raw construction text. Now: the working collection has passed 49 million raw tokens — building codes, construction legislation, estimating norms, open-access journals and other open sources — not yet annotated or released.

Roadmap: further document types from the launch plan (technical specifications, safety protocols, project documentation, construction contracts).

NER-annotated subset (live): 2,927 segments from nine building codes (ДБН). Search them in the corpus browser; the source catalog lists the documents, and the datasheet above gives the composition.

Record format. JSON lines, one merged record per segment:

• final_text — the annotated segment;
• final_labels — entity spans {start, end, label, text, agreement}: start and end are 0-based character offsets into final_text (end exclusive), text equals final_text.slice(start, end), label is one of the nine labels, and agreement is the share of passes that support the span;
• domain — one of 31 construction-domain tags;
• validation_score — a composite agreement score from 0 to 1;
• agreement_meta — agreement details (number of passes, κ, span F1, per-label figures);
• merged_at — when the record was merged (ISO 8601).

A real record from the corpus (agreement_meta abridged):

{
  "final_text": "Арматура повинна бути нерозривною у межах з'єднання і анкерування у прилеглих елементах.",
  "final_labels": [
    {"start": 0, "end": 8, "label": "material", "text": "Арматура", "agreement": 1},
    {"start": 42, "end": 51, "label": "structure", "text": "з'єднання", "agreement": 1},
    {"start": 54, "end": 65, "label": "process", "text": "анкерування", "agreement": 1}
  ],
  "domain": "concrete",
  "validation_score": 1,
  "agreement_meta": {"version": 2, "numRaters": 3, "entityKappa": 1, "cohenKappa": 1, "fleissKappa": 1, "avgSpanF1": 1},
  "merged_at": "2026-07-27T11:03:33.390Z"
}

The release datasheet will confirm the final record shape for v1.0.

Target: 100+ hours of transcribed construction speech from all three Ukrainian dialect groups. Speech data is planned for a later release, after the text corpus.

Now: since October 2026, contributors read short construction texts aloud in the browser and may edit a word to say it the way people do where they come from. Recordings join the corpus only after moderation and with the speaker’s consent. Selected recordings are on the Voices page, and collection progress is on the Coverage page.

Roadmap: recording spontaneous speech on site (briefings, technical consultations, planning meetings), as foreseen in the launch plan.

Dialect groups (standard classification): Northern — East Polissian, Central Polissian, West Polissian; Southwestern — Volhynian, Podolian, Upper Dniester, Upper Sian, Pokuttia–Bukovyna, Hutsul, Boyko, Transcarpathian, Lemko; Southeastern — Middle Dnieper, Slobozhanian, Steppe.

Formats as recorded. The browser recorder captures 48 kHz mono WebM/Opus (MP4/AAC on Safari), with automatic gain control, noise suppression and echo cancellation switched off. Uploads are accepted as WAV, MP3, OGG, FLAC, WebM or M4A (up to 50 MB per file).

Recording specification for external recordings: WAV (16-bit PCM; 24-bit also accepted), mono, 48 kHz (16, 32 or 44.1 kHz also accepted); peaks between −12 and −3 dBFS; a noise floor at or below −50 dBFS; no gain control, noise suppression, echo cancellation, compression or EQ. The target SNR is ≥ 20 dB: it is measured on the server for WAV and in the browser for other formats, and a lower value triggers a warning, not a rejection.

Annotations: the transcript (for read-aloud takes, the text as actually read); a speaker pseudonym (SPK-NNN); and the region — the speaker’s dialect origin as the speaker states it, not the recording location.

Privacy note: uploaded files are stored as uploaded, so metadata embedded in a file (for example tags in WAV, MP3 or M4A files) is not removed yet. Removing it is planned before any speech release; until then, strip tags before uploading.

Planned for a later release: a bilingual (Ukrainian–English) dictionary of construction terminology. Target: 5–10K domain terms.

Planned coverage:

• Building materials and composites
• Construction techniques and methods
• Safety standards and equipment
• Regulatory and legal terminology
• Tools, machinery, and instrumentation

Planned format: structured JSON with the Ukrainian term, its English equivalent, a definition, usage examples from the corpus and a domain category. Terms will appear in the glossary as they are added.

Planned API — available from v1.0 (Q4 2026). The repositories below are not public yet, so these snippets will work only after the release.

pip install datasets

Text corpus (v1.0):

from datasets import load_dataset

corpus = load_dataset("BUDOVA/text-uk-construction")
print(corpus)
print(corpus["train"][0])

No official train/validation/test split is defined yet (see the datasheet); train above is the default split name.

Later releases: speech (BUDOVA/speech-uk-construction) and the lexicon (BUDOVA/construction-lexicon).

Until then, browse the annotated segments in the corpus browser.

Planned workflows once v1.0 is out. A baseline NER model is on the roadmap for after v1.0; its name will be announced with that release.

Count entity labels in the text corpus:

from collections import Counter
from datasets import load_dataset

corpus = load_dataset("BUDOVA/text-uk-construction", split="train")
labels = Counter(
    span["label"] for record in corpus for span in record["final_labels"]
)
print(labels.most_common())

Field names follow the record format in Text corpus; the release datasheet will confirm them.

BUDOVA resources will be released under these licenses:

• Datasets: CC BY 4.0;
• Baseline models (roadmap): Apache 2.0;
• Code: MIT License.

To cite the project, use the BibTeX entry on the Dataset page.