Managing LLM spend across providers

TL;DR

Jackie Grubbs, Senior IT Manager at Doximity, built a set of Serval automations that track and manage AI spend for hundreds of developers across three different LLM providers. Serval pulls usage and cost data directly from each provider's API, DMs every developer their month-to-date spend each day, and lets them move budget between providers on request while keeping each person under their monthly limit.

The problem

Doximity's developers work across several LLM providers, and each one bills separately with its own admin console. A small IT team was logging into each service to adjust per-user limits by hand, blocking developers waiting for changes, and leadership had no single view of the AI bill.

The build

Jackie connected each provider to Serval as a custom integration and built the tracking and budget controls on top. Pulling directly from the provider APIs gives the team one reliable source of truth. One caveat for builders: most usage APIs are only available on certain plans for LLM providers.

1. 1.Connected the providers as custom integrations in Serval. Three different LLM providers connected via API with their own usage and budget limits.
2. 2.Defined each user's monthly budget so Serval knows the allotment per developer and per provider and can check any request against it.
3. 3.Drafted a daily spend DM. A daily scheduled workflow pulls month-to-date spend per user, service, and model and sends developers over a defined level of spend activity a Slack DM breakdown so they can watch their own trend.
4. 4.Built self-service reallocation. If a user is getting close to their limit, they can ask Serval’s Help Desk agent to move their budget between providers via Slack DM or in the tech-support channel. Serval will check that the new split stays under their monthly limit; if it meets the bar, then it will directly write the updated limits back to each provider through its API.
5. 5.Added an escalation path and leadership reporting. Requests that would exceed the monthly allotment are reviewed by Serval and require business justification before proceeding. Exceptions and budget overages are reported back to leadership monthly for review, with all budgets returning to baseline at the start of each month.
6. 6.Created custom AI spend dashboard. All users’ historical AI spend is aggregated and stored in a custom Serval dashboard for ongoing finance and leadership review.

The results

• Resolves over 450 tickets per month automatically related to AI cost management and allocation, replacing manual log-ins from IT users
• Keeps costs under control by enforcing each user's monthly budget across providers.
• Gives developers flexibility to dynamically move budget between providers to match the work they're doing without waiting on IT.