How PostHog built open source product analytics that scales
PostHog is an open source suite for product analytics, session replay, feature flags and experiments. Its stack is a Django and React app on top of ClickHouse, with Rust services and Kafka taking care of the firehose of incoming events.
The first version was built by co-founder Tim Glaser in four weeks during Y Combinator. It was a plain Django web app that queried events through the Django ORM, ran on Heroku Postgres and could be deployed with one click.
That simplicity got PostHog its first users. As more teams adopted it and sent more events, Heroku Postgres was pushed to its limits and queries got slow.
In 2021 the team benchmarked Pinot, Presto, Druid, TimescaleDB, Citus and ClickHouse on speed, ease of self-hosting and query interface. ClickHouse came out roughly an order of magnitude faster, so the test cluster was promoted to production. PostHog said Postgres struggled beyond about 10k monthly active users.
They ran both databases side by side, ingesting into each, and moved features over one at a time behind their own feature flags. Once everything had switched, they stopped writing events to Postgres. Not everything was smooth: firing hundreds of mutations per minute, far above what ClickHouse recommends, caused an outage.
Today the hot path is split from the Django app. High-throughput Rust services capture events, evaluate feature flags and ingest session replays, and Kafka is the central bus that connects ingestion to storage.
A Node.js CDP worker processes events as they arrive and exports them to destinations. Django still serves the web app and the API, and the handbook reserves Rust for high-scale services.
PostHog uses Celery only for small, low-latency tasks, since the handbook says it can be unreliable for larger or long-running workflows. New medium and large jobs go elsewhere.
Temporal handles user-facing and mission-critical workflows thanks to exactly-once guarantees, and it is much cheaper than Dagster for recurring work. Dagster stays with internal data jobs, where data lineage and testing matter most.
On PostHog Cloud, the application services run in an EKS Kubernetes cluster on AWS, while the data stores are managed separately. ClickHouse is a self-managed cluster on EC2 with shards and replicas, Postgres runs on Aurora behind PgBouncer, and session recordings and exports live in S3.