Posted On: Feb 10, 2025
Amazon Step Functions now supports additional data sources and output options for Distributed Map, enabling more flexible large-scale parallel processing workflows. Distributed map can now process data from JSON Lines (JSONL) and a broader range of delimited file formats stored in Amazon S3. Additionally, distributed map offers new output transformations for greater control over result formatting.
Amazon Step Functions is a visual workflow service capable of virtually any Amazon Web Services service to build distributed applications and data processing workloads. With this update, you can more easily iterate over and process diverse datasets using Step Functions distributed map. In addition to existing JSON and comma separated value (CSV) file support, distributed map now supports JSONL, and new delimited file formats, such as semicolon-delimited files and tab delimited files. This expands processing capabilities to a wider range of data formats without custom pre-processing. The new flexible output transformations give you more control over how results are formatted, enabling easier aggregation of results and simplifying integration with downstream systems.
This update is available in all the Amazon Web Services regions, including the Amazon Web Services China (Beijing) Region, operated by Sinnet and the Amazon Web Services China (Ningxia) Region, operated by NWCD where Amazon Step Functions is available. To get started, you can use the distributed map mode today in the Amazon Web Service Management console. To learn more, visit the Step Functions developer guide.