AI Operating System is a self-hosted, full-stack AI system software for building, deploying, and orchestrating autonomous AI agents. It is designed for technical founders, engineering teams, and developers who need a complete, production-architecture AI system they can own outright, extend, and operate — without rebuilding core AI infrastructure from the ground up.
The system is built on a Python 3.12 / FastAPI async backend and a Next.js 15 / TypeScript / Tailwind CSS frontend. Infrastructure is covered by Docker Compose for development and production, Kubernetes manifests for container orchestration, and Terraform scripts targeting AWS EKS for cloud deployment. A GitHub Actions CI/CD pipeline is included. The REST API covers agents, tasks, memory, knowledge, tools, models, and system health, with a WebSocket layer for real-time task streaming.
The core kernel delivers: a Graph-DAG multi-agent engine with parallel execution and full audit trails; four-layer memory comprising Episodic and Semantic memory via Qdrant, Working memory via Redis, and Procedural memory via PostgreSQL with background consolidation; a hybrid RAG pipeline combining BM25 sparse retrieval and dense embeddings fused via Reciprocal Rank Fusion and re-ranked by a cross-encoder; ReAct, Reflexion, Chain-of-Thought with majority vote, and Self-Refinement reasoning engines; Monte Carlo Tree Search and BFS-based task planning with DAG dependency resolution; and a LiteLLM-based LLM gateway with three-tier Mixture-of-Experts routing, fallback chains, and semantic caching.
A fine-tuning pipeline supports LoRA and QLoRA (4-bit), Supervised Fine-Tuning and Direct Preference Optimization, and evaluation via ROUGE-L and perplexity scoring. Synthetic training data generation is built in — the system produces instruction, preference, and tool-use samples in JSONL format across configurable domains, reducing dependency on real user data and helping protect user privacy during model adaptation workflows. Sample datasets in all three formats are included. Constitutional safety guardrails apply pre- and post-call LLM filters, prompt injection detection, and refusal enforcement. Observability is provided by Prometheus metrics, Grafana dashboards, OpenTelemetry traces, and structured logging. Security includes JWT and API key authentication, bcrypt password hashing, rate limiting, and a full audit log.
The system connects to OpenAI, Anthropic, Google Gemini, Mistral, and local Ollama models via LiteLLM. Local embedding runs on Sentence Transformers; API embedding uses OpenAI. Vector storage uses Qdrant. Built-in tools include a sandboxed code executor, web search, file I/O, HTTP client, calculator, and vector search. Unit and integration test suites are included using pytest and pytest-asyncio. System requirements: Python 3.12+, Node.js 20+, Docker and Docker Compose 24+, and Make.
