Tracked topic
Large language models are machine learning models trained to predict and generate text and other language-based outputs.
…Workers reference logical services such as llm , agent_llm , vlm , stt , and tts through configuration, letting developers swap endpoints, use cloud-hosted models, or bring OpenAI-compatible APIs without changing agent logic…
…Learn more In LLM training, Expert Parallel (EP) communication for hyperscale mixture-of-experts (MoE) models is challenging. EP communication is essentially all-to-all, but due to its dynamics and sparseness…
…Inference engine that executes the model and owns the KV cache manager (SGLang, vLLM, TRT-LLM) Layer 1: The frontend Multi-protocol support Agent harnesses are increasingly adopting v1/responses and v1…
…It provides GPU-optimized building blocks for training and post-training workflows, so teams can build custom systems with the performance, flexibility, and scale required for modern LLM, MoE, and multimodal development…
…Such malicious instructions to the LLM can result in it taking attacker-influenced actions with adverse consequences. Manual approval of actions performed by the agent is the most common way to manage…
…Learn more LLM context lengths are exploding, and architectures are moving toward complex attention schemes like Multi-Head Latent Attention (MLA) and Grouped Query Attention (GQA). As a result, AI ”speed of…
…Since joining NVIDIA in 2022, Annamalai has been instrumental in shaping the LLM landscape, previously working on the NeMo suite of products. Drawing on a diverse background spanning deep learning, firmware, and…
…the model to specific tasks or domains. NeMo AutoModel enables users to fine-tune models (LLMs, VLMs and DiffusionLMs) directly on top of HuggingFace checkpoints without conversion, so users can start rapid…
…SkillSpector scanning coverage is grounded in recognized AI security governance, including OWASP guidance for LLM , agentic AI risks , and MITRE ATLAS . The NVIDIA Skills scanning documentation lists the current coverage and is…
…Tokenize transactions on the GPU General-purpose LLM tokenizers waste capacity on tabular financial data. For example, a byte pair encoding (BPE) tokenizer splits a single transaction into roughly 39 subword tokens…