… Analysis of the roofline model suggests that the kernel scales effectively relative to the respective roofline, demonstrating that Tile IR is a viable option to scale workloads. The kernel considered is the attention decode kernel and the kernel is optimized using Tile IR. …
… Most of the kernel code is involved with setting up views for the data and loading/storing the data. The compute portion of the kernel is simple. cView.store masked accTile, xBlock, yBlock ; Launch the kernel. …
… In this setting, constant per-kernel overhead and little work per kernel lead to an unfavorable ratio between overhead and actual work. Changing this requires altering the paradigm from many small kernels to a few larger kernels. …
… Get started using agent skills to translate Python kernels to Julia Use the following code to try the Julia subproject and the conversion skill: cd TileGym Explore the Julia kernels ls julia/kernels/ add.jl, matmul.jl, softmax.jl Explore the conversion skill ls .claude/skills/converting-cutile-to-j… …
… Using cuda.compute we can solve this using pure Python by calling a device-wide primitive. import cuda.compute from cuda.compute import OpKind Build-time tensors used to specialize the callable build A = torch.empty 2, 2, dtype=torch.float16, device="cuda" build B = torch.empty 2, 2, dtype=torch.fl… …
… Some Julia language features notably iterator-based ‘for’ loops aren’t supported in kernels or generate inefficient code The integration with CUDA.jl needs to improve to facilitate coexistence with SIMT kernels. …
… Cutting transformer layer time by 34% with kernel-level optimizations The NVIDIA and Sarvam AI teams systematically targeted the most expensive kernels identified in the trace, replacing standard PyTorch implementations with fused, architecture-specific kernels. …
… As a result, three distinct kernels are implemented, all sharing the same memory layout for weights and all following similar structural and timing conventions. …
… Warp targets that kernel-heavy layer: you control execution, fuse kernels to cut memory traffic and launches, and use CUDA Graphs to reduce repeated dispatch. …
… Get setup First, install cuTile BASIC with PIP: pip install git+https://github.com/nvidia/cuda-tile.git@basic-experimental The full hardware and software requirements for running cuTile BASIC are listed at the end of this post 64k of RAM or more recommended . cuTile BASIC example If you’ve learned … …