Posted On: May 10, 2023
Amazon Step Functions expands support for iterating and processing large sets of data such as images, logs and financial data in Amazon Simple Storage Service (Amazon S3), a cloud object storage service.
Amazon Step Functions is a visual workflow service capable of orchestrating over 10,000 API actions from over 220 Amazon Web Services services to automate business processes and data processing workloads.
Now, Amazon Step Functions can iterate over objects such as images or logs stored in Amazon S3, then launch and coordinate thousands of parallel workflows to process the data, and save the results of executions to Amazon S3. You can scale thousands of parallel workflow executions, running concurrently, with the new distributed mode of the Map state. You can use the distributed Map mode to analyze millions of log files for security risks, iterate terabytes of data for business insights or scan images, and video files for specific objects. To process your data, use compute services such as Amazon Lambda and write code in any language supported, or choose from over 220 purpose-built Amazon Web Services services to accelerate your development. Amazon Step Functions manages the concurrency and error thresholds, provides visibility into the progress of each individual workflow execution in a visual operator console, and gives you control over how faults and errors are handled.
Start using the new distributed Map mode today in the console with Workflow Studio, Amazon Command Line Interface (CLI) or Amazon SDK. To learn more, please see the Amazon Step Functions developer guide.
Amazon Step Functions’ distributed Map mode is available in Amazon Web Services China (Beijing) Region, operated by Sinnet and Amazon Web Services China (Ningxia) Region, operated by NWCD.