I am currently creating a Step Functions workflow which contains some Lambdas and other services that share states between one task and other.

Mainly I am doing some external requests on my Lambda and sharing between tasks to process this data.

When I am trying to test my Step Functions workflow I am receiving the following error: States.DataLimitExceeded, but when I test on the Lambda console everything works properly.

Error:

Can anyone help me to understand what I am doing wrong in Step Functions workflow?

CodePudding user response：

The Step Functions service has a limit for the inputs/payloads of 256KB (more necessarily 262,144 bytes) of data as a UTF-8 encoded string. This quota affects tasks (activity, Lambda function, or integrated service), state or execution output, and input data when scheduling a task, entering a state, or starting an execution.

In your case looks like you are sending more than this limit between states, this is not a problem for the Lambda functions because the Lambda limit is 6MB for events.

To work with this, you need will to share these huge payloads in another service like S3 and then send this reference between your functions/states.

For example, if you decide to use S3 for this case we can do something like this:

-> Lambda do the request 
-> Save on S3 the data 
-> Send the response with the S3 object ARN 
-> Get this file on the Lambda function and process the data

This is preventing you to send the payload directly in your Step functions and you will not need to worry with this 256Kb input limit.

For understand more about the Step Functions quotas/limits take a look here: Step Functions - Quotas