Mitigating a token-length side-channel attack in our AI products
… An example request vulnerable to the side-channel attack looks like this: curl -X POST \ https://api.cloudflare.com/client/v4/accounts/ /ai/run/@cf/meta/llama-2-7b-chat-int8 \ -H "Authorization: Bearer " \ -d '{"stream":true,"prompt":"tell me something about portugal"}' Let’s use Wireshark to inspe… …