prediction_step - Performs an evaluation/test step.training_step - Performs a training step.compute_loss - Computes the loss on a batch of training inputs.create_scheduler - Sets up the learning rate scheduler if it wasn’t passed at init.create_optimizer - Sets up the optimizer if it wasn’t passed at init.
Note, that you can also subclass or override the create_optimizer and create_scheduler methods To inject custom behavior you can subclass them and override the following methods: The Trainer contains the basic training loop which supports the above features. The API supports distributed training on multiple GPUs/TPUs, mixed precision through NVIDIA Apex and Native AMP for PyTorch. It’s used in most of the example scripts.īefore instantiating your Trainer, create a TrainingArguments to access all the points of customization during training. The Trainer class provides an API for feature-complete training in PyTorch for most standard use cases.
