FAQ
torch.distributed.barrier()
hangs in DDP
Use model.module
instead of model
during validation.
Check https://discuss.pytorch.org/t/torch-distributed-barrier-hangs-in-ddp/114522/6 for more details.