hmm doesnt pytorch lightning handle all that D:
If you are doing everything according to their tutorials / examples, it should be, but odds are you still have to define something for it to know which GPU(s) to use