← Back to Command Center
Choosing an LLM Serving Path at Datavant
TLDR: Datavant has three ways to serve LLMs: Bedrock, Databricks, and Direct APIs. For production, Databricks is recommended for its built-in governance, cost attribution, and platform alignment — at only ~4% more cost than Bedrock. Databricks endpoints are early access with SLA-backed production targeted for Q2 2026.
The Three Paths
Datavant teams currently have three viable ways to serve foundation models:
- Amazon Bedrock — AWS-native, IAM-based auth, mature SLAs
- Databricks LLM Endpoints — Built-in cost attribution, usage tracking, endpoint isolation, open-source flexibility
- Direct Model APIs (e.g., Anthropic Claude directly) — Best for quick prototyping
All three provide access to production-grade LLMs. The decision is not about model quality alone. It is about governance, cost visibility, operational overhead, compliance surface area, platform maturity, and long-term flexibility.
Note: Databricks endpoints are currently early access and not yet SLA-backed, with a productionized release targeted for Q2 2026.
Estimated Timeline per Use Case
| Dimension |
Bedrock |
Databricks LLM Endpoints |
| Time to First Call (0→1) |
1–3 days (AWS-native) |
2–5 days (PR + provisioning) |
| Authentication Setup |
Same day (IAM role) |
1–2 days (OAuth setup) |
| Secret Handling Setup |
<1 day (IAM policy) |
1 day (credential setup) |
| Endpoint Isolation |
1–3 days if separate |
Built in at provisioning |
| Rate Limiting |
1–2 days (AWS/app config) |
Built in |
| Usage Tracking Setup |
2–5 days (observability setup) |
Built in |
| Cost Attribution Setup |
1–3 days (AWS tagging) |
Built in (cost tag required) |
| Experimentation Setup |
3–7 days (app-managed) |
1–2 days (config change) |
| Evaluation / ML Workflow |
1–3 weeks (separate tooling) |
2–5 days (platform-aligned) |
| Open Source Model Setup |
Limited support |
2–5 days (provisioned endpoint) |
| Production Hardening |
Mature; minimal extra |
Not SLA-ready yet |
| Internal Ramp-Up |
Low (AWS familiar) |
1–2 weeks familiarity |
What You Get "Free" with Databricks
Non-obvious benefits that come built in:
- Dedicated endpoint per service — isolation + clean blast radius
- Built-in rate limiting — per-endpoint QPM caps
- Observability built in — automatic request logs + usage metrics (tokens, latency, volume) per endpoint
- Central dashboards — usage and cost visibility with no custom logging required
- Easier alerting — centralized metrics vs app-by-app
- Automatic cost attribution — required dv-cost-owner tag
- Usage tracking at the platform layer — no extra Datadog tracing setup
- Easy experimentation — swap models / adjust limits via config changes
- Open-source model support — provisioned throughput options like Llama
- Native evaluation + ML workflow path — aligned with Databricks tooling
Model Availability
Model availability varies across platforms and does not imply production approval.
- Databricks supports a broad set of commercial (Claude, GPT, Gemini) and open-source models within a unified serving framework.
- Amazon Bedrock provides strong support for Anthropic (Claude) and some Google and OpenAI offerings, but does not include all frontier models (e.g., GPT-5).
- Direct APIs (OpenAI, Google, Anthropic) provide the most complete and up-to-date model access.
At Datavant:
- Anthropic (Claude) is the current default and approved commercial provider.
- Other commercial providers (e.g., OpenAI, Google) may require additional compliance and security review before production use.
- Open-weight / open-source models (e.g., Llama, Qwen) do not require MSAs or BAAs, but remain subject to infrastructure, data handling, and usage compliance requirements.
While Databricks enables access to both commercial and open-source models through a single platform, teams should confirm provider-level approval before onboarding a commercial model outside the current standard.
If a team requires a model beyond the approved commercial providers, that should trigger a broader discussion with DataOps and Security.
Pricing
Pricing is roughly the same across all three platforms. We've currently negotiated a cheaper rate to talk to Claude via Bedrock at a 4% discount. We also agreed to hit a minimum threshold — otherwise we will incur penalties.
Watch item: Overall usage of Claude in Bedrock has decreased, likely due to the rollout of Claude Code. Unknown if this will impact our contract.
View all Models Available + Pricing →