KoichiYasuoka
/

deberta-base-thai

Model card Files Files and versions

deberta-base-thai / tokenizer_config.json

KoichiYasuoka's picture

initial release

e8a1658 about 4 years ago

history blame contribute delete

297 Bytes

	{"do_lower_case": false, "bos_token": "[CLS]", "eos_token": "[SEP]", "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "split_by_punct": true, "keep_accents": true, "model_max_length": 512, "tokenizer_class": "DebertaV2TokenizerFast"}