Cost of Service
I'm being extremely transparent. It's a core value of mine. Hopefully you can learn from our decisions.
Last updated
Was this helpful?
I'm being extremely transparent. It's a core value of mine. Hopefully you can learn from our decisions.
Last updated
Was this helpful?
We're becoming an production Illinois service. Let's take a look at our costs before we launch to campus (anticipated full campus advertising campeign in September 2025).
LLM inference is the most expensive part of the app, but we pass that onto the user with a "BYO API Keys" model. $0.
Hosted on Vercel, had to upgrade to pro tier for greater usage. We're doing 600k function invokations, that's dominated by our "polling" during document uploads. I'm working to reduce tons of unnecessary polling.
We host our Python backend and a few supporting services on Railway.
We upgraded from the "Small" to "Medium" instance in Febuaruy 2025. Still, it seems a little under-sized for our needs and occasionally locks up under heavy load.
I’m falling out of love with Supabase. (1) It “locks up” under heavy load, e.g. a user exporting their files while another user adds tons of new file uploads. (2) Using their (optional) SDK creates vendor lock-in. (3) The pricing is good, better than most, but only on-par with AWS Aurora RDS. I’d use managed AWS RDS in the future, or self hosted vanilla Postgres + PGBouncer.
We're going to move this somewhere else more cost effective.
This ranges from $10-$30/mo, depending on egress costs.
Mailgun + Ghost (self hosted) powers news.uiuc.chat. Mailgun is the only supported provider for Ghost, we pay Mailgun a base of $15/mo + usage, averaging $16/mo.
A fantastic startup creating visualizations of embedding spaces. We use this to (1) visualize all the documents a user has uploaded and (2) visualize all the conversations in each chatbot. Both have great filtering, search, clutering, hierarchical topic labeling. It's pretty great. They give us $100/mo education pricing.
Frontend
$30
Backend
$82
Databases
$439
$329 is Qdrant, which we're moving somewhere cheaper.
Supporting services
$116
Mailgun + Nomic Atlas.
Total
$667
Soon to be $367 w/ cheaper Qdrant. Largest costs are covered by AWS credits.
Beam.cloud runs our document ingest queue, and a few supporting functions for .
Hosted on AWS EC2 i3en.xlarge
with all data stored in-memory - this is not the most cost effective. Using AWS credits supplied to the .
Purchased via Redis Cloud on AWS Marketplace, just a flat rate $5/mo. Using AWS credits supplied to the .
We run highly scalable jobs, primarily document ingest, on Beam.cloud. It's . Highly recommend. Steady-state average of $5/mo so far.
for error and latency monitoring. Free tier.
for usage monitoring and custom logs. Free tier... mostly.
for maps of embedding spaces. Educational free tier.
for public documentation. Free tier.