Amazon Kinesis Analytics was launched yesterday by Amazon Web Services Inc.
Spokesperson Jeff Barr stated that the new service was created because of one simple reason. “We want you to be able, regardless of whether you are a procedural or data scientist, to process voluminous clickstreams coming from Web applications, telemetry, sensor reports, server logs, and other connected devices using a standard query language in real-time, no matter what your skill level. Yesterday’s blog post by Barr included these words.
Barr said, “Today, I am happy to announce the availability Amazon Kinesis Analytics.” “You can now run continuous SQL statements against streaming data, filtering and transforming it as it arrives. Instead of spending your time on infrastructure, you can concentrate on processing the data and extracting business benefits from it. In 5 minutes, you can create a powerful, end to end stream processing pipeline without needing to write more than a SQL query.
This tool uses the Firehose Streams and Streams components to provide real time analysis via SQL queries. Firehose is used for loading streaming data into AWS services like Redshift (data warehouse), S3 (cloud storage), and Amazon Elasticsearch Service. Streams is used to create custom applications that work with streaming data to meet a variety of requirements.
Kinesis Analytics uses a three-step workflow to configure an input stream from a console, write SQL queries using a built-in SQL editor, and then configure an output stream. You can specify where you want the processed result to be loaded, such the Elasticsearch Service or S3, Redshift.
Analytics tools can be used to create alerts or respond to changing data. This is useful for IoT applications. Built-in machine learning algorithms provide stream processing functionality, such as anomaly detection, top K analysis, and approximate distinct items. These functions are exposed as SQL functions.
Kinesis Analytics infrastructure, like other AWS services can be scaled up or down as required and users only pay for what they use.
Kinesis Analytics, in conjunction with IoT scenarios can be used to serve up personalized content to Web surfers based upon clickstream data or real-time placement of appropriate ads. AWS stated that time-series analytics and real-time dashboards are the most popular usage patterns.
Barr gives an example of Kinesis Analytics “getting Started” in his blog post. Ryan Nienhuis provides more detailed guidance in a blog posting yesterday, the first part of a series titled “Writing SQL for Streaming Data with Amazon Kinesis Analytics Part 1.”
Nienhuis stated that “previously, real-time streaming data processing was only available to those who had the technical skills to manage complex applications.” Amazon Kinesis Analytics allows anyone who is familiar with the ANSI SQL standard to build and deploy a stream processing application in minutes.
“This application you have just built allows for a managed and elastic data processing process using Analytics. It calculates useful results over streaming. Results are calculated as they arrive and you can set a destination to deliver them into a persistent store such as Amazon S3.
Nienhuis stated that part two of his blog series would explore stream processing concepts.