AI Infrastructure Template
Self-hosted MLOps platform for rapid AI product development
Tech Stack
A production-ready, self-hosted MLOps platform designed to build any type of AI product with built-in best practices. Implements the three-phase AI Engineering maturity model: Development (prompts + evals), Production (traces + scorecards), and Evolution (backtests + leaderboards). Features a compositional prompt architecture with auto-versioning, systematic evaluation framework, response caching to optimize costs, and complete observability with Prometheus, Grafana, and Loki. Deployed on VPS with Coolify, supporting multiple LLM providers (AWS Bedrock, OpenAI, Anthropic). Demonstrates production-grade infrastructure patterns and DevOps maturity.
Screenshots
Key Highlights
Prompt management system with auto-versioning
Systematic evaluation framework with multiple matcher types
Multi-provider LLM support (Bedrock, OpenAI, Anthropic)
Complete observability stack (Prometheus, Grafana, Loki)
Response caching and cost optimization
Production-grade infrastructure patterns
Self-hosted on VPS with Coolify
Metrics
Microservices
Status