To validate the practical impact of low-precision training for real-world large-model pretraining, the team evaluated both the training convergence and downstream task performance across two widely used dense transformer architectures: Llama 3 8B and an NVIDIA internal research 8B model (Research-8B with dense grouped query attention (GQA) architecture that is similar to Llama 3 8B). The models were trained on 1 trillion tokens.
Using NVFP4 Low-Precision Model Training for Higher Throughput Without Losing Accuracy | NVIDIA Technical Blog…both training and inference. Her current focus is shaping the direction of NVIDIA's AI software to better serve the community. She holds an M.S. in computer science from National Tsing…
…모델 품질을 유지하면서 연산·메모리 요구량을 낮추므로, AI 모델이 리소스가 제한된 환경에서도 더 효율적으로 동작하도록 돕습니다. 이 글에서는 NVIDIA Model Optimizer 를 활용해 CLIP 모델을 학습 후 양자화(post-training quantization, PTQ…
…She joined from Gretel, where she was the VP of marketing, and led a team promoting synthetic data generation for AI model training. Prior to this role, she served as the head…
…Based in San Jose, Adi brings a unique perspective to speech AI development, having successfully navigated the transition from legacy systems to modern AI-driven platforms while addressing complex requirements of real…
…Smith is the senior developer relations manager for AI in Chemistry and Materials Science at NVIDIA. He is a computational chemist who earned his PhD from the University of Florida in 2018…
…Enterprise-Ready Multimodal AI May 28, 2026 By Anu Srivastava Discuss (0) Discuss (0) L T F R E AI-Generated Summary Like Dislike Step 3.7 Flash from StepFun is a…
…is an AI Developer Technology Engineer (Devtech) at NVIDIA. He obtained a master's degree in computer science from TU Munich in 2002 and a PhD in discrete mathematics from ETH Zurich…
…This post shows how users can benefit from NVIDIA DSX Air through accelerated deployment timelines and simplified, full-stack cluster management. How DSX Air enables AI factory simulation To make AI factory…
…And while the majority of tokens have been generated from humans interacting with AI, the new era is one in which most tokens will be generated from AI interacting with AI. Modern…
…Build secure agentic AI workflows with DGX Spark AI developers and enthusiasts benefit from the GB10 Grace Blackwell Superchip paired with 128 GB of unified memory in DGX Spark, providing the resources…