Veyra-20M: Compact Many-Label Intent Classification

Veyra-20M is a small, encoder-only text classification model built by Dl26. It is designed for fast English banking-intent and topic-style classification using a BERT-style bidirectional Transformer encoder.

The released checkpoint, Dl26/Veyra-20M, is a 20M-parameter-class classifier trained from random initialization. It uses the standard Transformers bert model type, so it loads with AutoModelForSequenceClassification and does not require trust_remote_code=True.

Why this model

• Compact encoder-only classifier
• Many-label classification setup with 77 labels
• Standard Hugging Face Transformers compatibility
• No custom architecture Python files
• Fast batch inference on CPU or GPU
• Built for intent routing, query classification, triage, and lightweight encoder research

Model details

Property	Value
Model name	Veyra-20M
Developer	Dl26
Model type	Encoder-only sequence classifier
Transformers model type	bert
Architecture	BertForSequenceClassification
Parameters	21,342,285
Hidden size	512
Layers	6
Attention heads	8
Intermediate size	2,048
Max positions	256
Vocabulary size	3,892
Number of labels	77
Training objective	Supervised single-label classification
License	Apache 2.0

Supported labels

ID	Label
0	activate_my_card
1	age_limit
2	apple_pay_or_google_pay
3	atm_support
4	automatic_top_up
5	balance_not_updated_after_bank_transfer
6	balance_not_updated_after_cheque_or_cash_deposit
7	beneficiary_not_allowed
8	cancel_transfer
9	card_about_to_expire
10	card_acceptance
11	card_arrival
12	card_delivery_estimate
13	card_linking
14	card_not_working
15	card_payment_fee_charged
16	card_payment_not_recognised
17	card_payment_wrong_exchange_rate
18	card_swallowed
19	cash_withdrawal_charge
20	cash_withdrawal_not_recognised
21	change_pin
22	compromised_card
23	contactless_not_working
24	country_support
25	declined_card_payment
26	declined_cash_withdrawal
27	declined_transfer
28	direct_debit_payment_not_recognised
29	disposable_card_limits
30	edit_personal_details
31	exchange_charge
32	exchange_rate
33	exchange_via_app
34	extra_charge_on_statement
35	failed_transfer
36	fiat_currency_support
37	get_disposable_virtual_card
38	get_physical_card
39	getting_spare_card
40	getting_virtual_card
41	lost_or_stolen_card
42	lost_or_stolen_phone
43	order_physical_card
44	passcode_forgotten
45	pending_card_payment
46	pending_cash_withdrawal
47	pending_top_up
48	pending_transfer
49	pin_blocked
50	receiving_money
51	Refund_not_showing_up
52	request_refund
53	reverted_card_payment?
54	supported_cards_and_currencies
55	terminate_account
56	top_up_by_bank_transfer_charge
57	top_up_by_card_charge
58	top_up_by_cash_or_cheque
59	top_up_failed
60	top_up_limits
61	top_up_reverted
62	topping_up_by_card
63	transaction_charged_twice
64	transfer_fee_charged
65	transfer_into_account
66	transfer_not_received_by_recipient
67	transfer_timing
68	unable_to_verify_identity
69	verify_my_identity
70	verify_source_of_funds
71	verify_top_up
72	virtual_card_not_working
73	visa_or_mastercard
74	why_verify_identity
75	wrong_amount_of_cash_received
76	wrong_exchange_rate_for_cash_withdrawal

Installation

pip install -U transformers torch accelerate

Quick start

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-20M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Can you help me reset my password?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)

with torch.no_grad():
    logits = model(**inputs).logits

label_id = int(logits.argmax(dim=-1))
print(model.config.id2label[label_id])

Batch inference

texts = [
    "Book me a flight to Zurich tomorrow morning.",
    "What is the weather like today?",
    "Please cancel my card.",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    truncation=True,
    padding=True,
    max_length=64,
)

with torch.no_grad():
    logits = model(**inputs).logits

for text, label_id in zip(texts, logits.argmax(dim=-1).tolist()):
    print(model.config.id2label[label_id], "-", text)

Evaluation highlights

Evaluation	Result
Validation accuracy	0.8890
Test accuracy	0.8976
Remote code required	No

Intended use

Veyra-20M is intended for:

• intent classification
• query routing
• customer-support triage
• lightweight text classification
• many-class encoder experiments
• CPU-friendly classifier deployments

Limitations

• The model is specialized for the supported intent labels.
• It is trained for English short utterances.
• It may fail on long documents or out-of-domain language.
• Confidence scores should be calibrated for production systems.
• It is not a generative model or a semantic embedding model.

Citation

@misc{dl26_2026_veyra_20m,
  title        = {Veyra-20M: Compact Many-Label Intent Classification},
  author       = {Dl26},
  year         = {2026},
  url          = {https://huggingface.co/Dl26/Veyra-20M}
}