Apache kafka

Apache Kafka

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

Core Capabilities: - Deliver messages at network limited throughput using a cluster of machines with latencies as low as 2ms. - Scale production clusters up to a thousand brokers, trillions of messages per day, petabytes of data, hundreds of thousands of partitions. Elastically expand and contract storage and processing. - Store streams of data safely in a distributed, durable, fault-tolerant cluster. - Stretch clusters efficiently over availability zones or connect separate clusters across geographic regions.

Ecosystem: - Process streams of events with joins, aggregations, filters, transformations, and more, using event-time and exactly-once processing. - Kafka’s out-of-the-box Connect interface integrates with hundreds of event sources and event sinks including Postgres, JMS, Elasticsearch, AWS S3, and more. - Read, write, and process streams of events in a vast array of programming languages. - Large ecosystem of open source tools: Leverage a vast array of community-driven tooling.

For more info: https://kafka.apache.org/