Tracked topic
Large language models are machine learning models trained to predict and generate text and other language-based outputs.
…Broader Large Language Model (LLM) support and more model compression techniques. OpenVINO™ runtime optimized for Intel® Xe Matrix Extensions (Intel® XMX) systolic arrays on built-in GPUs for efficient matrix multiplication resulting…
…Learn More The AutoRound Quantization Algorithm Achieve near-lossless weight-only quantization (WOQ) compression for popular large language models (LLMs). Learn More Quantize LLMs with SmoothQuant LLMs tend to have large-magnitude…
…Broader LLM model support and more model compression techniques. LLM compilation time reduced through additional optimizations with compressed embedding. Improved 1st token performance of LLMs on 4th and 5th generations of Intel…
…We have improved the performance of various LLM models, including LLaMA 2 7B/13B/70B FP8 for pre-training, and LLaMA 2 7B/70B FP8 and Mistral 7B for inference. For more…
…Explore AI Model Benchmarks Download this comprehensive white paper on LLM optimization that uses compression techniques. Learn to use the OpenVINO toolkit to compress LLMs, integrate them into AI applications, and deploy…
…These are then sent together to the large language model (LLM). Even if the LLM has no prior knowledge pertaining to the user’s original query, it can comprehend the supplied related…
…AI development and integration of deep learning in domains like computer vision, large language models (LLM), and generative AI. What's New in 2026.3 AI Programming Workshops for OpenVINO Toolkit Learn…
…Neural Network Compression Framework (NNCF) Added data-free Int4 weights compression support for LLMs in OpenVINO IR with nncf.compress_weights(). Improved quantization time of LLMs with NNCF PTQ API for nncf…
…We added support for vLLM, a fast and easy-to-use library for LLM inference and serving. For more information, see the Intel Gaudi vLLM. We have added software backward/forward compatibility…
…Broader Large Language Model (LLM) support and more model compression techniques. Preview: Addition of Prompt Lookup to GenAI API improves 2nd token latency for LLMs by effectively utilizing predefined prompts that match…