Pruning and Distilling LLMs Using NVIDIA TensorRT Model Optimizer | NVIDIA Technical Blog
… In practice, we recommend multinode training for faster training. torchrun --nproc per node 8 /opt/NeMo/scripts/llm/gpt train.py \ --name Qwen3-8B-nemo-depth-pruned-distill \ --devices 8 \ --num nodes 1 \ --tp size 8 \ --model path Qwen3-8B-nemo-depth-pruned \ --teacher path Qwen3-8B-nemo \ --legac… …