Apache Kafka: Event Monitor
Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. A Kafka Event Monitor is a Universal Extension responsible for monitoring events (messages) from topics in Kafka, and consuming an event based on filtering criteria via consumer group subscription.
Key Features
This Universal Extension supports the following main features:
Monitor messages and consume a Kafka message by consumer group topic subscription.
Authenticate against Kafka using PLAINTEXT, SASL_SSL or SSL security protocol
Capability to filter Kafka messages based on the value of the message. Number, String and JSON filter patterns are supported.
Capability to fetch dynamically Kafka topics, during task creation.
Capability to control the partition assignment strategy, as well as session-related timeout values
What's New in Version 1.2.0
Enhancements
- Upgraded kafka-python to version 2.1.6, bringing major enhancements in stability, performance, and compatibility, along with numerous bug fixes.
- Added support for PLAIN mechanism under SASL authentication.
- Added environment variables for configuring Kafka connection and polling parameters.
- Added validation step after initializing a consumer, for improved stability and robustness.
- Improved consumer shutdown handling to ensure tasks terminate gracefully without leaving open connections.
- Improved log messages.
- Optimized the message retrieval logic for better efficiency.
- Optimized the pipeline stages flow for better efficiency.
Fixes
- Fixed bug where the last message was not commited when consuming multiple messages.
| Extension Name: | ue-kafka-monitor |
|---|---|
| Universal Template Name: | Apache Kafka: Event Monitor |
| Vendor Product: | Apache Kafka |
| Version: | 1.2.0 |
| Vendor: | Apache Software Foundation |
| Type: | Free |
| Compatibility : | UC/UA 7.6 and above |
| Support: | Stonebranch Certified |