nav search
Data Center Software Security Transformation DevOps Business Personal Tech Science Emergent Tech Bootnotes BOFH

Artificial intelligence, filesystems, containers ... Amazon showers cloud gold on devs

Google, who?

By Neil McAllister, 9 Apr 2015

AWS Summit Amazon piled on the new cloud features at its AWS Summit event in San Francisco, including new services for file storage and machine learning and updates to features that debuted last year.

Speaking at Thursday morning's keynote, AWS boss Andy Jassy described cloud computing as "the new normal" for businesses of all sizes and said Amazon offered more and broader functionality than any other cloud IT vendor.

The cloud mega-giant rolled out 516 major new features and services in 2014, Jassy said, and he had several new ones to announce at this week's conference that should please developers and data center admins.

Objects, blocks, and now files

The first was Amazon Elastic File System (EFS), a new storage service that allows customers to store their files directly in the AWS cloud.

Previously, Amazon has only offered object-based storage via its Simple Storage Service (S3), SAN-style block-based storage via the Elastic Block Store (EBS), or archival storage via the Amazon Glacier service. This new offering mimics a shared file system in the cloud, accessible via the NFSv4 protocol and scalable up to petabytes in size.

"Whether you want to use this for an application that's small, development test, or if you want to use it for something that's very large with high demand and scalability; because it grows to petabyte-scale or more, it handles all of those use cases," Jassy said.

File systems can be created and managed using GUI, command-line tools, and APIs. Each file system is backed by multiple Elastic Compute Cloud (EC2) instances, with SSD-based storage for maximum performance. To ensure high availability, all files, directories, and links are replicated across multiple Availability Zones.

Fees for the service are straightforward at $0.30 per gigabyte of storage used, billed monthly based on the average usage throughout the month. You'll have to wait to get your hands on it, though; Amazon says EFS will become available in preview "in the near future," but it's accepting applications to try it out now.

Machine learning for the masses

Big data is another topic that has been on Amazon's mind, and on Thursday it announced a new service designed to enable developers to add machine learning to their applications, even if they have no direct experience in the field.

Jassy explained that Amazon has been experimenting with machine learning since its very early days as an online bookseller, for things like recommendation engines and fraud detection. Over the years it has developed in-house tools to make creating new machine learning models easier, and those tools have now become the basis of its new public service, Amazon Machine Learning.

Amazon data scientist Matt Wood took the stage on Thursday to explain how the new service can automatically pull data from S3, Amazon Redshift, or MySQL databases hosted on the AWS Relational Database Service, run that data through its built-in machine learning algorithms, and use it to generate predictive models.

The service is designed to be usable by developers with no experience in statistics, data analysis, or any of the other mathematical drudgery that's ordinarily necessary to implement machine learning. All that's needed to create models is the AWS Management Console or the service's APIs. Once the models are built, predictions can be generated in real time or in batches, as needed.

"All of this is using the battle-hardened technology that's been tried and tested inside Amazon.com," Wood explained.

Mind you, Amazon isn't the only cloud vendor interested in machine learning. Microsoft introduced a similar service, Azure ML, last year, and Google offers its Prediction API.

Amazon Machine Learning is available now and there are no fixed up-front costs. Data analysis and model building are billed at a rate of $0.42 per hour. After that, predictions are billed at $0.10 per batch of 1,000, or $0.0001 per real-time prediction.

Containers are go

Amazon has also made generally available two major features it introduced at the AWS re:Invent conference in November. The first is the EC2 Container Service (ECS), a way of spinning up Docker containerized applications as first-class citizens on the AWS cloud.

In addition to declaring ECS production-ready, Amazon has added a new Service scheduler to make it easier to manage long-running containerized services. The Service scheduler can scale clusters up or down by starting or stopping containers, use Elastic Load Balancing (ELB) to distribute traffic across multiple containers, and restart containers that die or fail ELB health checks. It will also automatically relaunch containers when you change the images or container definition files used by your application.

There's also a new web-based console that makes it easier to get ECS clusters up and running, although ECS can also be managed via command-line tools or APIs.

Unlike most AWS services, Amazon said it doesn't plan to charge for ECS, so the production version is still free – although you do have to pay for the other AWS resources your applications use, including compute, storage, and so on.

AWS Lambda nods, winks at mobile devs

Another feature that Amazon has declared ready for production workloads is AWS Lambda, a way for developers to write software functions that run directly within the AWS cloud, without the need to manage any infrastructure.

Lambda functions generally work by being associated with triggers tripped by events in the AWS ecosystem. They run automatically whenever a certain S3 storage bucket is modified, for example, or when they receive a notification from Amazon Kinesis. As of the general-availability release, they can now also be triggered by Simple Notification Service (SNS) messages or Cognito Events.

But the production version adds a new twist, too. Lambda functions can now be called synchronously by invoking them directly from applications and waiting for a response. Jassy said this latest feature was added largely at the behest of mobile developers, who have been particularly smitten by Lambda because of its potential for creating "server-free" backends for their apps.

All of the above features are available today, but Jassy said another new feature is coming soon that's an even bigger deal. When Lambda launched in preview at re:Invent last year, it only supported functions written in JavaScript/Node.js. But "in the next few weeks," Jassy said, AWS will introduce support for Lambda functions written in Java – although we'll have to wait and see just how that pans out.

Pricing for Lambda hasn't changed since the preview version, but it's complicated – you can check out the gory details, with examples, here. ®

The Register - Independent news and views for the tech community. Part of Situation Publishing