Case study · Stack

How PostHog built open source product analytics that scales

PostHog is an open source suite for product analytics, session replay, feature flags and experiments. Its stack is a Django and React app on top of ClickHouse, with Rust services and Kafka taking care of the firehose of incoming events.
Copy this stackposthog.com Product analytics and developer tools · United States · founded 2020

UI library

Hosting

  • AWSposthog.com

    Hosts PostHog Cloud, including the ClickHouse cluster on EC2.

  • Kubernetesposthog.com

    Runs the application services in an EKS cluster.

Database

  • ClickHouseposthog.com

    Stores and queries all analytics events.

    Why: In their benchmarks it was consistently about an order of magnitude faster than Pinot, Presto, Druid, TimescaleDB and Citus.

  • PostgreSQLposthog.com

    Holds app data on Amazon Aurora, accessed through PgBouncer.

Build and runtime

Data fetching

  • Keaposthog.com

    Manages front-end state on top of Redux.

  • Apache Kafkaposthog.com

    Acts as the central message bus between ingestion and storage.

  • Redisposthog.com

    Backs the task queue for background jobs.

  • Dagsterposthog.com

    Runs internal data jobs.

    Why: It offers rich data lineage and testing.

Backend framework

  • Djangoposthog.com

    Powers the web app and the API.

  • Rustposthog.com

    Runs the high-throughput services for event capture, feature flag evaluation and session replay ingestion.

  • Node.jsposthog.com

    Runs the CDP worker that processes events on ingestion and exports them to destinations.

  • Temporalposthog.com

    Orchestrates complex, user-facing and mission-critical workflows.

    Why: It gives exactly-once guarantees for complex failure scenarios and is much cheaper than Dagster for recurring workloads.

Testing

File storage

How they built it

An MVP on Django and Heroku Postgres

The first version was built by co-founder Tim Glaser in four weeks during Y Combinator. It was a plain Django web app that queried events through the Django ORM, ran on Heroku Postgres and could be deployed with one click.

That simplicity got PostHog its first users. As more teams adopted it and sent more events, Heroku Postgres was pushed to its limits and queries got slow.

Sources:posthog.com

Moving events from Postgres to ClickHouse

In 2021 the team benchmarked Pinot, Presto, Druid, TimescaleDB, Citus and ClickHouse on speed, ease of self-hosting and query interface. ClickHouse came out roughly an order of magnitude faster, so the test cluster was promoted to production. PostHog said Postgres struggled beyond about 10k monthly active users.

They ran both databases side by side, ingesting into each, and moved features over one at a time behind their own feature flags. Once everything had switched, they stopped writing events to Postgres. Not everything was smooth: firing hundreds of mutations per minute, far above what ClickHouse recommends, caused an outage.

Sources:posthog.com posthog.com

Rust and Kafka on the ingestion path

Today the hot path is split from the Django app. High-throughput Rust services capture events, evaluate feature flags and ingest session replays, and Kafka is the central bus that connects ingestion to storage.

A Node.js CDP worker processes events as they arrive and exports them to destinations. Django still serves the web app and the API, and the handbook reserves Rust for high-scale services.

Sources:posthog.com posthog.com

Three tools for background work

PostHog uses Celery only for small, low-latency tasks, since the handbook says it can be unreliable for larger or long-running workflows. New medium and large jobs go elsewhere.

Temporal handles user-facing and mission-critical workflows thanks to exactly-once guarantees, and it is much cheaper than Dagster for recurring work. Dagster stays with internal data jobs, where data lineage and testing matter most.

Sources:posthog.com

PostHog Cloud on AWS

On PostHog Cloud, the application services run in an EKS Kubernetes cluster on AWS, while the data stores are managed separately. ClickHouse is a self-managed cluster on EC2 with shards and replicas, Postgres runs on Aurora behind PgBouncer, and session recordings and exports live in S3.

Sources:posthog.com

Also detected on their website

Found on the homepage with the open Wappalyzer rules (HTML, scripts and headers). It shows what runs in the browser, not the whole backend.

  • Gatsby
  • HSTS
  • Naver Analytics
  • Vercel
  • Webpack