The best way to get started with Amazon Kinesis Data Analytics is to get hands-on experience by building a sample application. Simply go to the Amazon Kinesis Data Analytics console and create a new Amazon Kinesis Data Analytics application. Use the following steps, depending on whether you choose (i) an Apache Flink application using an IDE (Java, Scala, or Python) or an Apache Beam application (Java), (ii) Studio notebook application (Apache Flink SQL, Python, or Scala via an interactive development experience), or (iii) a Kinesis Data Analytics SQL-based application via a console editor.

To get started, create a Kinesis Data Analytics application that continuously reads and processes streaming data. Download the open source Apache Flink libraries using your favorite IDE, and then write your application code and test it with live streaming data. You can configure destinations where you want Kinesis Data Analytics to send the results.
 You can get instructions on how to download the libraries and create your first application in the Amazon Kinesis Data Analytics for Apache Flink Developer Guide. You can find equivalent code in other languages that Apache Flink supports in the official Apache Flink documentation for the Apache Flink version you are using on Amazon Kinesis Data Analytics.

Step 1: Download the open source libraries into your favorite IDE

You can start by downloading the open source libraries that include the Amazon SDK, Apache Flink, and connectors for Amazon Web Services services.

You write your Apache Flink application code using data streams and stream operators. Application data streams are the data structure you perform processing against using your application code. Data continuously flows from the sources into application data streams. One or more stream operators are used to define your processing on the application data streams.

Step 3: Upload your code to Kinesis Data Analytics

Once built, upload your code to Amazon Kinesis Data Analytics and the service takes care of everything required to run your real-time applications continuously including scaling automatically to match the volume and throughput of your incoming data.

The Getting Started with Amazon Kinesis Data Analytics for Apache Flink Applications section of the Developer Guide provides a simple walkthrough of building your first application.

Apache Flink provides several stream processing examples on the Apache Flink GitHub repository.

It’s easy to get started with Amazon Kinesis Data Analytics Studio

To get started, create a Kinesis Data Analytics Studio application. Start the application and open the Apache Zeppelin notebook and write your application code in SQL, Python, and Scala, with a stream processing engine powered by Apache Flink. Test your application with live streaming data and explore how your streaming data looks with SQL queries and built-in visualization. You can configure destinations where you want Kinesis Data Analytics to send the results. Optionally, you can promote your code in the note to a long running Apache Flink streaming application with durable state and autoscaling. You can get instructions on how to get started in the Amazon Kinesis Data Analytics Studio Developer Guide. You will also find samples to try out various SQL queries, and sample Python and Scala programs on your streaming data.

Step 1: Create an Amazon Kinesis Data Analytics Studio application

You can start from the Amazon Kinesis Data Analytics, Amazon MSK, or Amazon Kinesis Data Streams console. You can also use custom connectors to connect to any other data source.

You can run individual paragraphs in the note, view results in context, and use Apache Zeppelin’s built-in visualization to accelerate development. You can also use user-defined functions in your code.

Step 3: Build and deploy as a Kinesis Data Analytics streaming application

You can deploy your code as a continuously running stream processing application with a just few clicks. Your deployed application will be a Kinesis Data Analytics for Apache Flink application with durable state and autoscaling. You will also get the opportunity to change sources, destinations, logging, and monitoring levels before you productionize your code.

It's easy to get started with Kinesis Data Analytics for SQL

To get started, create a new Amazon Kinesis Data Analytics application. Select the demo stream we provide as input, pick a template, and edit the SQL query. You can then view the results right there in the console or load the output into Amazon Elasticsearch Service and visualize using Kibana. Within a few minutes, you will be able to deploy a complete streaming data application.

Step 1: Configure input stream

First, go to the Amazon Kinesis Data Analytics console and select a Kinesis data stream or Kinesis Data Firehose delivery stream as input. Amazon Kinesis Data Analytics ingests the data, automatically recognizes standard data formats, and suggests a schema. You can refine this schema, or if your input data is unstructured, you can define a new schema using our intuitive schema editor.

Step 2: Write your SQL queries

Next, write your SQL queries to process the streaming data using the Amazon Kinesis Data Analytics SQL editor and built-in templates, and test it with live streaming data.

Step 3: Configure output stream

Lastly, point to the destinations where you want the results loaded. Amazon Kinesis Data Analytics integrates out-of-box with Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose so it’s easy to send processed results to Amazon S3, Amazon Redshift, Amazon Elasticsearch Service, or your own custom destination.

Getting started examples

These resources provide example streaming data applications and step-by-step instructions so you can try them out and gain hands-on experience.

How it works

This SQL Developer Guide gives you an overview of the Amazon Kinesis Data Analytics architecture, creating applications, and configuring inputs and outputs.

Getting started

In the Getting Started guide, we step you through setting up an Amazon Web Services Account, the Command Line Interface (Amazon CLI), and creating your starter Amazon Kinesis Data Analytics application.

Example applications

This Example Applications guide provides code examples and step-by-step instructions to help you create Amazon Kinesis Data Analytics applications and test your results.