You won't be able to train anything that is anything like what most people think language models are, unfortunately.
alright, but training aside, surely they could at least fine-tune an existing model to some extent anyway