fundaml.trainers
Module Contents
Classes
Functions
Return the list of available devices (CPUs, CUDA GPUs, or Metal Performance Shaders) on the current platform. |
- fundaml.trainers.get_available_devices()
Return the list of available devices (CPUs, CUDA GPUs, or Metal Performance Shaders) on the current platform.
This function checks if CUDA or Metal Performance Shaders (MPS) is available and returns the list of available devices. It first checks if CUDA is available, and if it is, adds all the CUDA devices to the list. Next, it checks if the MPS backend is available and, if it is, adds the MPS device to the list. As of the latest knowledge cutoff in September 2021, PyTorch does not support device indexing for MPS, so we cannot distinguish between different MPS devices. If no GPU devices are found (either CUDA or MPS), the function adds the CPU device to the list of devices.
- Returns:
List of available devices. Each device is represented as a torch.device object.
- Return type:
List[torch.device]
Example
>>> devices = get_available_devices() >>> for device in devices: >>> print(device)
- class fundaml.trainers.NNTrainer(device='cpu', model=None, loss_function=None, optimizer=None, scoring_functions={}, verbose_level=1, log_dir='_tb_log_dir')
- SUPPORTED_DEVICES
- with_device(device)
Sets the device for the current instance.
This function sets the device attribute of the current instance to the requested device. If the requested device is ‘cuda’ and CUDA is available, it will set the device to ‘cuda’. If the requested device is ‘mps’ and the MPS backend is available, it will set the device to ‘mps’. In all other cases, or if the requested device is not available, the device will be set to ‘cpu’.
- Parameters:
device (str) – The requested device. Valid values are ‘cpu’, ‘cuda’, and ‘mps’.
Example
>>> trainer = NNTrainer() >>> trainer.with_device('cuda')
- with_loss_function(loss_function)
Sets the loss function for the current instance and returns the instance.
- Parameters:
loss_function (torch.nn.Module) – The loss function to be used.
- Returns:
The instance of the class.
- Return type:
self
Example
>>> trainer = NNTrainer() >>> trainer.with_loss_function(torch.nn.CrossEntropyLoss())
- with_optimizer(optimizer)
Sets the optimizer for the current instance and returns the instance.
- Parameters:
optimizer (torch.optim.Optimizer) – The optimizer to be used.
- Returns:
The instance of the class.
- Return type:
self
Example
>>> trainer = NNTrainer() >>> trainer.with_optimizer(torch.optim.SGD(model.parameters(), lr=0.01)) >>> trainer.with_optimizer(torch.optim.AdamW(model.parameters(), lr=0.01, weight_decay=0.001, betas=(0.9, 0.99), eps=1e-8))
- with_model(model)
Sets the model for the current instance and returns the instance.
- Parameters:
model (torch.nn.Module) – The model to be used
- Returns:
The instance of the class.
- Return type:
self
Example
>>> trainer = NNTrainer() >>> trainer.with_model(MyModel())
- with_scoring_functions(scoring_functions)
Sets the scoring functions for the current instance and returns the instance.
- Parameters:
scoring_functions (dict) – The scoring functions to be used.
- Returns:
The instance of the class.
- Return type:
self
Example
>>> trainer = NNTrainer() >>> trainer.with_scoring_functions({'accuracy':score_accuracy})
- with_verbose_level(verbose_level)
Sets the verbosity level for the current trainer and returns the instance.
- Parameters:
console_verbose_level (int) – The level of verbosity for console output. This could be implemented as follows: - 0: No output - 1: Basic output - 2: Detailed output
- Returns:
The instance of the class.
- Return type:
self
Example
>>> trainer = NNTrainer() >>> trainer.with_console_verbose(2)
- _report_scores(y_true, y_pred, prefix, loss, epoch_num, current_in_batch, total_in_batch, step)
Internal function, subject to change This function reports the scores of a neural network training.
- _compute_loss(X, y)
Internal function, subject to change
- train_loop(train_dataloader, update_every_n_batches=10, epochs=1)
Performs the training loop for the model, using only training data
The function loops over the given number of epochs, and for each epoch, it loops over all batches in the train dataloader. For each batch, it computes the loss, performs backpropagation, and updates the model parameters. Reporting (e.g., printing training loss, accuracy) is done every update_every_n_batches batches or at the end of each epoch.
- Parameters:
train_dataloader (torch.utils.data.DataLoader) – The dataloader that provides the training data in batches.
update_every_n_batches (int, optional) – How often to report training progress, in number of batches. Default is 10.
epochs (int, optional) – The number of epochs to go through the training data. Default is 1.
- Returns:
- The scores of the model on the training data after the last epoch. The contents of the dictionary
depend on the implementation of the _report_scores method.
- Return type:
dict
Example
>>> trainer = NNTrainer() # assuming this class contains the train_loop method >>> scores = trainer.train_loop(train_dataloader, update_every_n_batches=20, epochs=5) >>> print(scores)
- test_loop(test_dataloader, update_every_n_batches=10)
Performs a testing (evaluation) loop on the model.
The function loops over all batches in the test dataloader, computes the loss, and reports the scores (e.g., testing loss, accuracy) every update_every_n_batches batches or at the end of the testing.
- Parameters:
test_dataloader (torch.utils.data.DataLoader) – The dataloader that provides the testing data in batches.
update_every_n_batches (int, optional) – How often to report testing progress, in number of batches. Default is 10.
- Returns:
None. The scores are printed but not returned. To make this function return the scores, you need to modify the function to store the scores in an instance variable or return them directly.
Example
>>> tester = NNTrainer() # assuming this class contains the test_loop method >>> tester.test_loop(test_dataloader, update_every_n_batches=20)
- train_test_loop(train_dataloader, test_dataloader, update_every_n_batches=10, epochs=1, patience=2, min_delta=0.1)
Performs a training and testing loop on the model with early stopping.
The function loops over the given number of epochs. For each epoch, it loops over all batches in the train_dataloader and performs a training step. Then, it loops over all batches in the test_dataloader and performs a validation step. The validation loss is compared to the best validation loss seen so far, and early stopping is implemented based on the patience and min_delta parameters.
- Parameters:
train_dataloader (torch.utils.data.DataLoader) – The dataloader that provides the training data in batches.
test_dataloader (torch.utils.data.DataLoader) – The dataloader that provides the testing data in batches.
update_every_n_batches (int, optional) – How often to report progress, in number of batches. Default is 10.
epochs (int, optional) – The number of epochs to go through the training data. Default is 1.
patience (int, optional) – The number of epochs to wait before stopping if the validation loss does not decrease by at least min_delta. Default is 2.
min_delta (float, optional) – The minimum decrease in validation loss to qualify as an improvement. Default is 0.1.
- Returns:
- The scores of the model on the training data after the last epoch. The contents of the dictionary
depend on the implementation of the report_scores method.
- Return type:
dict
Example
>>> trainer = NNTrainer() # assuming this class contains the train_test_loop method >>> scores = trainer.train_test_loop(train_dataloader, test_dataloader, update_every_n_batches=20, epochs=5, patience=3, min_delta=0.05) >>> print(scores)