SNS/SQS seems to do the job so far for communicating between microservices, I am looking at streaming events from different microservices now and would like to know at what scale SNS/SQS combination could start to become too expensive and when should I consider kinesis based on the scale?

I don't see any reasons other than scale to just communicate events from different microservices to events microservice.

CodePudding user response：

To be honest, I don't think kinesis is cheaper at any scale. It's more related to if you need the features kinesis has to offer, like lower latency than the sqs/sns setup, data retention (e.g. for stream analytics) or message ordering or increased message sizes.

Since you are not using those features now, I'd think you don't need them and can stick to the current setup. It's also simpler to manage.

CodePudding user response：

Kinesis and SQS serve different use-cases.

SQS is for when you want to process each message once, by a single consumer (recognizing that your actual guarantee is "at least once").

Kinesis is for when you have multiple consumers, all of them need to process the same message, and they may not all be running at the same time.

Prior to Kinesis, you could achieve the same thing by sending the message to SNS, and then subscribing an SQS queue for each consumer.