Tracked topic
Qwen3 is an AI model family developed by Alibaba, released as a set of large language models for natural-language tasks.
With agents running 24 hours a day, seven days a week on increasingly complex tasks, efficient local compute matters even more. NVIDIA has collaborated with the open source community to enhance the top inference backends for agents, llama.cpp and vLLM. llama.cpp now delivers 2x performance on Qwen 3.5 and 3.6 27B dense models, and 1.6x performance on Qwen 3.5 and 3.6 35B mixture-of-expert (MoE) models. The following two techniques make this possible: Multi-Token Prediction (MTP): An advanced speculative decoding technique, where a smaller draft model proposes several tokens ahead that the targ
Build Personal AI Agents on Windows PCs with New Tools from Microsoft and NVIDIA | NVIDIA Technical BlogOne popular way to run AI locally has been to use multiple GPUs to access more memory and compute. While cloud frameworks like vLLM are well optimized for multiple GPUs thanks to their use in data centers, PC frameworks like llama.cpp and the ComfyUI implementation in PyTorch are not optimized for it. To solve this challenge, NVIDIA has collaborated with both llama.cpp and ComfyUI to enhance performance for RTX PCs with two equivalent GPUs. This enables you to run larger models and use the compute of both GPUs for better performance. llama.cpp now supports tensor parallelism (TP), fully utiliz
Build Personal AI Agents on Windows PCs with New Tools from Microsoft and NVIDIA | NVIDIA Technical Blog… NVIDIA In-Game Inferencing Plugins & Model Support The NVIDIA In-Game Inferencing SDK provides plugin support for a number of different language and speech models including: NVIGI Plugin Supported Inference Hardware Supported Models Speech - ASR Local TRT CUDA-Enabled GPU Riva ASR Speech - TTS Loca… …
… There are also model performance improvements with Qwen3.6 and a guided multi-node cluster setup for teams that need to scale beyond a single device. …
… Native multimodal interaction is achieved through optimized Qwen3-TTS and Qwen3-ASR models, allowing end-to-end, low-latency voice dialogue with a Thinker-Talker framework, and Cosmos Reason 2 enables advanced spatio-temporal reasoning, 3D localization, and long-context processing for humanoid robo… …
… NVIDIA has collaborated with the open source community to enhance the top inference backends for agents, llama.cpp and vLLM. llama.cpp now delivers 2x performance on Qwen 3.5 and 3.6 27B dense models, and 1.6x performance on Qwen 3.5 and 3.6 35B mixture-of-expert MoE models. …
… Using synthetic training data generated by large teacher models, developers can distill character behaviors into lightweight, deployable LoRA-powered performances that run locally on RTX hardware. …
… Discuss 0 Discuss 0 Tags Agentic AI / Generative AI | General | Blackwell | GB200 | Hopper | TensorRT-LLM | Intermediate Technical | Tutorial | AI Inference | featured | Inference Performance | LLM Techniques About the Authors About Laikh Tewari Laikh Tewari is part of the AI Platform Software grou… …
… This places extreme demands on local compute. NVIDIA DGX Spark provides the performance necessary for autonomous agents to execute these complex workflows efficiently and locally. …
… With 34 billion parameters, it leads Qwen2.5-VL 72B by 17.0 points, Qwen3-VL 32B by 7.0 points, Gemini 2.5 Pro by 15.1 points, and GPT-4o by 23.2 points. Lower minADE 6 values indicate better trajectory predictions; higher reasoning scores indicate better performance. …
… Muon training performance on NVIDIA GB300 NVL72 Table 1 summarizes training throughput of the Kimi K2 and Qwen3 30B models with Muon and the AdamW optimizer on the NVIDIA GB300 NVL72 system. …
… Install Dynamo Operator + Grove helm fetch https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform-${RELEASE VERSION}.tgz helm install dynamo-platform dynamo-platform-${RELEASE VERSION}.tgz --namespace ${NAMESPACE} --create-namespace --set "grove.enabled=true" Step 3: Verify Grove insta… …