dlouapre/SmolChessLM_780k_30Mg

A 🤗 transformers model playing chess by predicting chess moves, where each move is encoded in a custom explicit format , for instance "WPe2e4".

The model is based on the Llama architecture (128 width, 4 heads, 4 layers) and trained over 30 million games from the lichess database.

Tokenizer : dlouapre/chess_tokenizer_pairs, which tokenizes using pairs of characters like "WP" or "e2".

On Lichess games, the model plays with 96% of valid moves, but for a given position, only 38% of the moves are identical to the actual game.