AWS Managed Streaming for Kafka
AWS Managed Streaming for Kafka (Govern how your clients interact with Apache Kafka using API Gateway) Watch my You tube video on this: Follow the steps below to create the data pipeline: STEP1 Networking Part VPC , Public and Private Subnets, Internet Gateway, NAT Gateway , S3 end point , Route Tables , Security Groups STEP2 Launch MSK Cluster in the same VPC as created in STEP1 , unauthorized access allowed , plaintext encryption Make sure the security group remain as it is. STEP3 Launch Linux EC2 Choose the Same VPC , create in Public Subnet In the list Auto-assign Public IP, choose Enable. STEP4 Once the client for Amazon MSK has been created, the security group rules must be configured to allow the connection between the cluster and the client ec2 machine that we have just created. For that , Add the security group id of ec2 to MSK cluster security group all traffic Repeat these steps to add an inbound rule in the security group that corresponds to your client computer