Why logging for rank=[0,1] only in function engine.py -> _save_checkpoint()

[dunalduck0]

Link to the source code

Line 2997 only produces logging for rank 0 and 1. In multi-node training, the local main process ranks can be other values. For example, 2 nodes, each with 8 GPUs. The rank of the main process of the 2nd node is normally 8, and there would be no logging for this rank. I think the logging is useful for all ranks other than just [0,1]

log_dist(message=f'Saving model checkpoint: {save_path}', ranks=[0, 1])