Rendered at 23:02:19 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
recroad 2 hours ago [-]
That is a LOT of code for a pretty basic feature.
daybox 3 hours ago [-]
> Never spend more than you budgeted
I assume that this is "$ spent on search + $ spent on LLM" < budget, but how do you handle the LLM spending more than you would expect on a request? Or is this handled by max_tokens and some form of pricing table? (and if so, how does caching play a role?)
I assume that this is "$ spent on search + $ spent on LLM" < budget, but how do you handle the LLM spending more than you would expect on a request? Or is this handled by max_tokens and some form of pricing table? (and if so, how does caching play a role?)