Automate MSK Topics to S3 Data Lake: New AWS Delivery Service
Automate AWS MSK Kafka topics from MSK into your S3 Data Lake. AWS announced a new managed delivery service from MSK to your S3 data lake on Oct. 27 2023. This new managed service can pull topics from Kafka and dump them into S3 buckets via Kinesis Firehouse. There are no additional pipes, functions or code required. (source) Are you tired of building pipes and serverless functions to move data around? AWS's new fully managed delivery stream will handle the pipes from your topics into your S3 data lake via Kinesis. You don't have to write and manage additional code. There are also no additional costs for this managed pipeline as it is built into the output of your MSK cluster. The new feature is designed to ease data flow from MSK to S3 Data Lake, leveraging Amazon Kinesis Data Firehose for the Extract, Transform, Load (ETL) tasks. This noteworthy…
