Understanding GLiFormer’s Benchmarks and Schema-Driven Extraction
Overview
gliformer-large-v1 is a 575.6-million-parameter, Apache-2.0 model from knowledgator for schema-driven information extraction and text representation. It uses a shared DeBERTa encoder with task heads for named entity recognition, classification, joint relation extraction, structured records, and text embeddings. Its layout-aware architecture supports text and document-layout inputs, including PDF-oriented processing, but the published examples and evaluation results focus on English text rather than visual document understanding. You run it through the gliformer Python library or the GLiFormer source framework in Python 3.10 or newer. The most important consideration is its breadth: one checkpoint accepts labels and schemas at inference time, but its quality varies by task and dataset. Reported mean strict entity F1 is 50.91 across 26 NER datasets, mean macro-F1 is 75.03 across 13 classification datasets, joint relation extraction is weak on several benchmarks, and the strong 91.10 structuring score uses order-free, boundary-tolerant JSON evaluation rather than exact JSON matching. No context window, maximum sequence length, training-step count, VRAM requirement, inference latency, or checkpoint-specific training dataset is provided.
Best use cases
Schema-driven entity extraction from English text. Use the model when entity types vary by project and you want to provide labels at inference time instead of training a separate fixed-label NER model. For example, extract person, organization, and location from "Alice works at Acme in London." and receive each entity’s text, label, character offsets, and score. The shared DeBERTa encoder and task-conditioned interface support custom entity labels, but benchmark results show substantial variation: strict F1 reaches 80.16 on WikiNeural and 71.48 on CrossNER politics, while it falls to 13.81 on HarveyNER and 21.99 on FabNER.
Text classification with custom candidate classes. Use classify for jobs such as assigning support tickets to product or support, detecting sentiment, or routing documents among business categories. The model accepts candidate labels at inference time and supports named groups such as separate sentiment and topic label sets. It performs well on some conventional datasets, including 97.95 macro-F1 on Enron spam, 96.77 on Financial Phrasebank, 93.93 on IMDB, and 92.97 on SST-2, but performance is much lower on SST-5 at 40.33, Emotion at 48.07, and the SOTU dataset at 49.00.
Structured record extraction from semi-structured prose. Use structure when the output must contain fields and parent-child relationships, such as converting company descriptions into nested companies, departments, and employees. The decoder can produce Python dictionaries from simple field lists or nested Pydantic schemas. The reported multilevel structuring score is 91.10 on 500 examples, with scores from 89.94 at JSON depth 3 to 95.11 at depth 4. Treat this as source-grounded extraction, not factual verification: Pydantic validation checks shape and types, not whether extracted values are correct.
Joint relation extraction with a closed schema. Use inference with joint_relations when you need entities and relations in one configured task, such as extracting (Alice, works_at, Acme). This checkpoint has a joint relation head, so the schema must provide entity and relation labels together. It is not a strong choice for high-recall relation mining: reported micro-F1 is 12.78 on DocRED, 12.61 on CrossRE, 24.08 on FewRel, and 35.83 on zero-shot CoNLL04.
Embedding short text for similarity workflows. embed_text returns 1,024-dimensional vectors that you can compare with cosine similarity. This fits semantic search prototypes, duplicate detection, clustering, and retrieval features where a single local encoder is useful alongside extraction tasks. The provided material does not include embedding benchmark scores, so validate retrieval quality on your own corpus before selecting it for production search.
Limitations
The model does not provide a guaranteed extraction of every fact. NER can omit mentions, select incorrect spans, or assign incorrect labels. Its strict NER score requires both span and type to match, and the dataset results range from 13.81 to 80.16 F1. Labels, schema wording, domain, input length, and thresholds affect predictions.
Relation extraction is a major weakness in the published results. The reported runs use predicted entities, and the scores are not controlled comparisons between the base and large releases because they use different-sized subsets. The model also lacks an open relation head: the convenience method predict_relations requires one, so this checkpoint should use joint_relations instead.
Structured extraction can attach records to the wrong parent, omit fields, or produce incorrect values. Its 91.10 score is not exact JSON match. Evaluation flattens JSON value paths, aligns records without requiring original order, and permits limited boundary repairs. Fixed record anchors and configured span width constrain extraction capacity.
The layout-aware design supports text and document-layout inputs, but the supplied quality evidence does not establish performance on PDFs, other document layouts, or visual inputs. The checkpoint has no dedicated vision or audio capability. Results cover English tasks; quality for other languages is not established. The NER transfer groups also do not prove that every evaluated domain was absent from training.
The model is large at 575.6M parameters. The documentation provides no VRAM minimum, latency, throughput, maximum context length, or practical batch-size guidance beyond the NER example using batch_size=8. CUDA attention kernels are optional; CPU inference uses eager attention. You should benchmark memory and latency on your target hardware.
The Apache-2.0 license permits commercial use subject to the license terms. The provided material does not document additional dataset-specific restrictions, warranty terms, or safety evaluations. It also does not state whether the checkpoint is actively maintained.
How it compares
gliner-x-large
Choose gliformer-large-v1 over gliner-x-large when you need one interface for NER, classification, joint relations, structured records, and embeddings, or when nested record extraction matters. Choose GLiNER X-Large when the primary requirement is flexible zero-shot NER and you want a model positioned as a practical alternative to fixed-label NER systems and larger language models. The supplied information does not provide matched parameter counts, latency, cost, or benchmark scores, so it does not support a numeric quality or speed ranking between them.
modern-gliner-bi-large-v1.0
Choose gliformer-large-v1 when multi-task extraction, classification, relation extraction, structured records, and embeddings must share one checkpoint. Choose modern-gliner-bi-large-v1.0 for NER workl