---
tags:
  - gliner2
  - ner
  - data-mention-extraction
  - lora
  - two-pass-hybrid
base_model: fastino/gliner2-large-v1
library_name: gliner2
license: apache-2.0
---

# GLiNER2 Data Mention Extractor — datause-extraction

Fine-tuned GLiNER2 LoRA adapter for extracting structured data mentions from
development economics and humanitarian research documents.


## Architecture: Two-Pass Hybrid

- **Pass 1** (`extract_entities`): Finds ALL data mention spans using 3 entity types
  (`named_mention`, `descriptive_mention`, `vague_mention`). Bypasses count_pred entirely.
- **Pass 2** (`extract_json`): Classifies each span individually (count=1).

## Entity Types

- `named_mention`: Proper names and acronyms (DHS, LSMS, FAOSTAT)
- `descriptive_mention`: Described data with identifying detail but no formal name
- `vague_mention`: Generic data references with minimal identifying detail

## Classification Fields

- `typology_tag`: survey / census / administrative / database / indicator / geospatial / microdata / report / other
- `is_used`: True / False
- `usage_context`: primary / supporting / background

## Installation

```bash
pip install git+https://github.com/rafmacalaba/GLiNER2.git@feat/main-mirror
```

## Usage
```python
from gliner2 import GLiNER2
import re

extractor = GLiNER2.from_pretrained("fastino/gliner2-large-v1")
extractor.load_adapter("ai4data/datause-extraction")

ENTITY_SCHEMA = {
    "entities": ["named_mention", "descriptive_mention", "vague_mention"],
    "entity_descriptions": {
        "named_mention": "A proper name or well-known acronym for a data source (DHS, LSMS, FAOSTAT).",
        "descriptive_mention": "A described data reference with identifying detail but no formal name.",
        "vague_mention": "A generic or loosely specified reference to data.",
    },
}

def extract_sentence_context(text, char_start, char_end, margin=1):
    boundaries = [0] + [m.end() for m in re.finditer(r"[.!?]\s+", text)] + [len(text)]
    for i in range(len(boundaries) - 1):
        if boundaries[i] <= char_start < boundaries[i + 1]:
            s = max(0, i - margin)
            e = min(len(boundaries) - 1, i + margin + 1)
            return text[boundaries[s]:boundaries[e]].strip()
    return text

json_schema = (
    extractor.create_schema()
    .structure("data_mention")
    .field("mention_name", dtype="str")
    .field("typology_tag", dtype="str", choices=["survey","census","administrative","database","indicator","geospatial","microdata","report","other"])
    .field("is_used", dtype="str", choices=["True", "False"])
    .field("usage_context", dtype="str", choices=["primary", "supporting", "background"])
)

text = "The analysis draws on the DHS 2018 and administrative records from the National Statistics Office."

# Pass 1 — span detection
pass1 = extractor.extract(text, ENTITY_SCHEMA, threshold=0.3, include_confidence=True, include_spans=True)
entities = pass1.get("entities", {})

# Pass 2 — classification per span
results = []
for etype in ["named_mention", "descriptive_mention", "vague_mention"]:
    for span in entities.get(etype, []):
        mention_text = span.get("text", span) if isinstance(span, dict) else span
        char_start = span.get("start", text.find(mention_text)) if isinstance(span, dict) else text.find(mention_text)
        char_end = span.get("end", char_start + len(mention_text)) if isinstance(span, dict) else char_start + len(mention_text)
        context = extract_sentence_context(text, char_start, char_end)
        tags = extractor.extract(context, json_schema)
        tag = (tags.get("data_mention") or [{}])[0]
        results.append({
            "mention_name": mention_text,
            "specificity": etype.replace("_mention", ""),
            "typology": tag.get("typology_tag"),
            "is_used": tag.get("is_used"),
            "usage_context": tag.get("usage_context"),
        })

for r in results:
    print(r)
```