Cloud architecture · AI

LLM Platform, multi-tenant AI assistants on AWS.

The one with many brandsThe need“Each of our brands wants its own AI assistant, without mixing data or blowing up costs.”
Hector · Monit4devThe answerLLM Platform: multi-tenant RAG on AWS, with WhatsApp, quotas and Kubernetes.
See cloud infrastructure Let's talk
LLM platform architecture diagram on AWS: CI/CD, VPC with EKS, data and infrastructure as code
AWS architecture: keyless CI/CD, EKS with 12 services, encrypted data and Terraform.
What I built

LLM Platform

A multi-tenant LLM platform: each brand gets an assistant with RAG over its own documents, serving WhatsApp and web, running on AWS EKS with infrastructure as code, WAF, managed secrets and per-tenant quotas.

What's inside

  • One assistant per brand with RAG over its own documents
  • Serves WhatsApp and web
  • Microservices on Amazon EKS with node autoscaling
  • Infrastructure as code with Terraform and remote state
  • Layered security and a daily quota per tenant
  • CI/CD with GitHub Actions and OIDC, no static keys
Technical highlights

Technical highlights

  1. 01

    12 microservices on Kubernetes with node autoscaling

  2. 02

    5 layers guarding AI spend: WAF, hardening, per-IP and per-user limits, and a daily per-tenant quota

  3. 03

    Zero static credentials: OIDC, IRSA and Secrets Manager

Stack

  • AWS EKS
  • Terraform
  • PostgreSQL + pgvector
  • Redis
  • AWS WAF
  • GitHub Actions + OIDC
Architecture
The five security layers that cap AI spend
Five security layers: AI spend has a ceiling by design.

The architecture, in depth

This platform has no public demo: see how each layer is solved on AWS or Azure in the cloud section.

See cloud infrastructure

Got a similar need?

Tell me about it and I'll tell you how I'd solve it.

Let's talk

Use ↑ ↓ to move and Enter to open