Download

All resources associated with NL2SHACL-Bench are publicly available.

---

Dataset

Platform	Link	Description
GitHub	DE-TUM/NL2SHACL-Dataset	Dataset repository
Zenodo	10.5281/zenodo.20082565	Archived dataset with DOI (CC-BY-SA 4.0)

---

Framework

Resource	Link	Description
GitHub	DE-TUM/NL2SHACL-Framework	Source code for the full benchmarking pipeline (MIT)
GitHub	DE-TUM/NL2SHACL-LLM-Eval	Prompts, LLM outputs, and evaluation results

---

Data Format

Each subset in the dataset is stored as three files that are linked by a shared id field.

*-descriptions.jsonl — one record per line:

{
  "id": "snik-1",
  "label": "meta:EntityTypeShape",
  "description": "Every entity type must have at least one human-readable label..."
}

*-ontology_snippets.jsonl — one record per line:

{
  "id": "snik-1",
  "label": "meta:EntityTypeShape",
  "ontology_snippet": {
    "http://www.snik.eu/ontology/meta/EntityType": {
      "types": ["http://www.w3.org/2002/07/owl#Class"],
      "label": "entity type",
      "description": "An entity type is any kind of information that is consumed, produced or modified by a task."
    }
  }
}

shacl/<id>.ttl — one Turtle file per record:

meta:EntityTypeShape a sh:NodeShape ;
    sh:closed true ;
    sh:targetClass meta:EntityType ;
    sh:property [ sh:path rdfs:label ; sh:minCount 1 ] ;
    ...

---

License

The dataset is released under CC-BY-SA 4.0. The framework source code is released under the MIT License.