As someone who’s had the privilege of attending all five AWS re:Invent conferences, I wanted to share my perspective on the 2016 conference which took place last week in Las Vegas, as well as some reflections on past re:Invents. The first re:Invent in 2012 was a modest affair with around 6,000 attendees, dominated by startups and emerging technology partners. Fast forward four years, and the conference has grown to 30,000+ attendees, with an emphasis on all-in enterprise adoption of AWS.
In this post, you’ll find the overall themes, selected announcements that stand out in retrospect, and my general takeaways from each year’s event, counting back to the very beginning.
• Lambda functionality extended into new services: Snowball Edge, Lambda@Edge, and Step Functions.
o The next generation of Snowball–Snowball Edge–has onboard Lambda compute capabilities that enable intriguing options for processing of data being collected offline. An example from one of the sessions was deduplication and ETL processing of data being collected at long-term survey sites which can be remote and detached from the Internet.
o Lambda@Edge (announced as a preview service) allows developers to code Lambda functions on the CloudFront CDN edge network. This feature will be of huge benefit to high-traffic website developers, allowing dynamic web page processing to run on the last-mile edge servers, significantly reducing the number of round-trip calls back to origin servers.
o AWS Step Functions introduces a visual orchestration tier allowing for easy stitching of microservices components. This will significantly reduce the time required to build microservices apps using Lambda. My AHEAD colleagues were already discussing the potential benefit of this service as an orchestration engine alternative for our AWS clients.
• Snowmobile is by far the coolest announcement at the conference, in my opinion. Snowmobile is a large white tractor-trailer semi-truck that AWS will send to your data center and literally have it drive away with your data (up to 100PB per run). They actually drove one of these on stage when the service was being announced. Military-grade security escort is an optional feature!
• In the DevOps category, new services include OpsWorks for Chef Automate, a new fully managed service for Chef which eliminates the need for a traditional Chef build server, and CodeBuild, a service that provides Jenkins-style build server functionality. Both of these announcements align with a number of existing DevOps managed services (including CodePipeline, CodeDeploy, and CodeCommit) that have been released over the past few years. Development teams benefit from these services to accelerate CI/CD release capabilities, and in turn, systems teams have a reduced number of servers (many of which sit around doing next to nothing) to support.
• PostgreSQL support for RDS Aurora, the initial version of Aurora announced in 2014, was a game-changing data service. It enabled MySQL as an enterprise-grade database and gave AWS customers a viable option to move away from expensive database platforms such as Oracle. AWS has continued to invest in the platform, announcing a PostgreSQL option at this year’s conference. AWS customers now have more reasons than ever to revisit their database needs with this compelling offering to significantly lower IT costs without sacrificing performance.
• Athena is a new service that provides SQL query capabilities for semi-structured data stored on S3. Traditionally, data files had to be converted into relational database format via ETL processes which added time, overhead, and complexity.
• EC2 is one of the oldest AWS services, and new instance type announcements have come to be expected at AWS events. Notable amongst this year’s announcements were higher-end instances in the T2 class: t2.xlarge and t2.2xlarge. These instance types add 4 CPU/16GB RAM and 8 CPU/32GB RAM capacity, respectively. In my experience, the T2 class is one of the underutilized EC2 classes. It is designed for burstable use cases where the server may require a high amount of CPU utilization, but only for a short time. This is the case with many more workloads than most customers suspect. They utilize a CPU credit model (think rollover minutes on your cell phone provider) which allows you to have the capacity you need when you need it in exchange for a much lower price, so long as you do not exceed your baseline credit amount. According to AWS, use of the T2 instances works out for customers well over 90% of the time. The t2.xlarge and t2.2xlarge types are significant as they increase the number of enterprise software applications and databases that can run on the T2 class.
• New Artificial Intelligence (AI) Services. Amazon Rekognition (facial recognition and categorization), Amazon Polly (a text to speech service), and Amazon Lex (“what’s inside aLEXa”, a natural language understanding and speech recognition service) are some of the new AI services announced this year. AI has been a strong AWS theme recently, and these services seem designed to fill gaps in the AWS platform as compared to competitors such as Google. What’s particularly compelling about them (as is the case with many AWS services) is they come “fully-baked” and at a significantly lower price point compared to third-party alternatives. AWS continues to provide its user community with all the tools they need to innovate on AI. Startups to large enterprises have access to the same feature set. One interesting trend to follow will be the development of an increased number of AI “skills” on platforms like Alexa. It seems that companies are moving away from developing mobile applications as a vehicle for their content and brands, and that AWS is banking on companies developing Alexa-like skills as the next standard.
AWS is acknowledging the realities of the hybrid states that many of its larger customers find themselves in. AWS is innovating to bring the cloud to customers in non-traditional (and ironic) ways. There are physical devices (like Snowball Edge and Snowmobile) with embedded compute capabilities and massive payload capacities that can be used for more than just one-time data transfers. This is in the same spirit as Amazon opening brick & mortar retail stores after being online-only for so many years. There were other services such as AWS Glue, which provides ETL capabilities for multiple dataset formats, that will connect to on-premises databases in addition to AWS services. These features blur the lines between cloud and on-premises data centers and allow for greater flexibility for customers.
Finally, AHEAD was a Platinum Sponsor of this year’s conference and was prominently featured in the expo area. In addition to the enjoying the free popcorn we handed out, visitors to our booth had the opportunity to talk with our team about our enterprise cloud solutions and see demos of a few of our solution offerings on Amazon WorkSpaces, ServiceNow integrations with AWS, and DevOps CI/CD release processes. We had some great conversations with attendees which echoed the realities of what we see with our larger enterprise customers, many of which want to move rapidly to AWS but are also dealing with the realities of a complex, existing data center infrastructure. They also want to ensure compatibility with their existing toolset and organizational ESM standards as they move toward AWS. The people I talked to seemed relieved to find out that they are not alone in facing these issues, and impressed that AHEAD has not only tackled these migration complexities with other customers, but also had demonstrable solutions built out in the AHEAD Lab. Amazon WorkSpaces in particular drew a lot of attention, helped in part by a session my colleague Nick Frank presented on a customer case study rollout of WorkSpaces. The value proposition of WorkSpaces resonated with attendees from small (<5 employees) to large: no more upfront infrastructure investments and complex architecture to support along with pay-as-you-go cost flexibility and global placement options.
Check out a recording of Nick’s session, Deploying Amazon WorkSpaces at Enterprise Scale.
Attendance was up to 18,000 creating massive crowds and human pileups on the escalators (time for a venue change that came in 2016!). Full-on enterprise adoption of AWS was a major theme, with featured talks from companies such as Capital One and GE who were shutting down data centers to move to AWS. Big data services continued to mature along with the growth of Internet-of-Things (IoT) services that allow for real-time analysis of massive amounts of streaming data (example: Major League Baseball Advanced Media’s (MLBAM) products which provide real-time analysis of in-game camera and telemetry data for every play). Amazon Echo/Alexa devices were prominently featured in both the keynotes and the expo area (along with the realization that background noise and Alexa do not play nice together!).
• IoT Platform: This service allows for more rapid creation of IoT services on AWS, fed by an increasing number of hardware and embedded collection devices. Side note: the IoT Button was the hottest commodity in the expo area!
• Database Migration Service and Snowball: These services are important because they enable easier AWS migration paths for large amounts of enterprise customer data. Even though it’s generally a one-time operation, the sheer amount of work (and time) involved with migrating data on-premises to the cloud has been a barrier to cloud adoption in the past. These services help make the process easier.
Along with the maturation of the enterprise AWS migration features, IoT was my main takeaway from this conference. While IoT remains an immature space with many players trying to figure out what to do with it, it’s an ideal fit for the AWS platform. IoT yields massive amounts of data that can only be streamed and stored in a cost-effective way in the cloud. Once stored in the cloud, you can take advantage of AWS data solutions to analyze and transform the data in near real time. This processing can happen intermittently, which is ideal for the AWS “pay as you go” cost model.
Attendance was up to 13,000 and there was a noticeable change in tone from past events. Rather than focusing on startups and differentiating AWS from its competitors, this conference marked a shift into the era of full-on enterprise adoption of AWS.
• Lambda – I’m hard-pressed to think of an AWS service announcement more revolutionary than Lambda. It started the cascade of microservices offerings on AWS and changed the way that cloud-first applications are designed. Rather than architecting applications with servers as the runtime vehicle, Lambda introduced the notion of the platform (AWS itself) as the vehicle for your code rather than a server. Given that most servers sit around doing very little, this is ideal from an optimization standpoint. It also opened up design patterns that interconnect AWS services via Lambda. The way it was first described by Werner Vogels, AWS’s CTO and Vice President, as the “connective tissue” for AWS services still stands out.
• EC2 Container Service (ECS) – This is a ready-to-go Docker container service on AWS. No more needing to go through a complicated server cluster setup and tuning exercise. Fast forward a few years to present day, and Docker is becoming a de-facto standard for how application teams architect their apps.
This was the first re:Invent conference that felt like a major technology conference. It also had the biggest list of groundbreaking service announcements. Gone was the startup and developer vibe, replaced with a larger more enterprise-heavy audience. Also noticeable was the number of big-name technology and consulting companies in attendance, many of which also had prominent sponsorship and speaker slots. The expo area was also much larger with an arms race of booth designs (think double-decker booths, couches, bartenders with baristas, and more) as well as stepped-up booth swag like RC helicopters and other cool items.
To further show the increased size and importance of this conference, AWS had Skrillex, a popular, animated DJ, with tons of lights and aggressive bass thumping, perform at the annual re:Play Party. Although I hadn’t heard of him before, my 11-year-old boys were immediately jealous when they found out he was performing.
The second annual re:Invent conference grew to 9,000 attendees. Redshift was growing as the fastest service in AWS history, opening up data warehouse capabilities and massive amounts of storage to companies who never would have been able to afford it in the past. For the first time, serious enterprise usage of AWS was mentioned for more than emergency DR and storage offload, with companies such as Boeing and Nike featured during the keynotes.
• Amazon WorkSpaces – WorkSpaces is the AWS platform for Virtual Desktop Infrastructure (VDI) that allows you to provision cloud-based Windows desktops for users and access them via any physical device. WorkSpaces has had a relatively long runway compared to other AWS services, but this service is starting to see major traction within global enterprises as a viable, flexible, and cost-effective alternative to expensive, traditional VDI offerings that are often hard to maintain.
• AWS CloudTrail API – This service filled a noticeable gap in the AWS platform, enabling support for full logging and auditing of all activity within your AWS account. Before CloudTrail, AWS was a non-starter for companies with strict compliance and audit requirements.
This was the second re:Invent conference and it helped solidify that this “cloud thing” has legs. More attendees, more references to bigger name customers, and the first serious discussion of migrating and running enterprise workloads on AWS were common themes.
On the heels of some notable outages earlier in the year, the inaugural conference focused on AWS as the go-to solution for startups and small companies, and a viable storage, backup, and disaster recovery solution for enterprises. There was little focus on enterprise usage, and more focus on customer success stories (Netflix, NASA, and the Obama 2012 campaign). Sessions focused on topics such as “How to Make the Case for Cloud Adoption at Your Company”. This was also the only AWS conference thus far that was attended by Amazon CEO, Jeff Bezos, who participated in a “fireside chat” with Werner Vogels. Bezos talked about lean startup principles, a culture of innovation, and Amazon’s business model of scalable offerings sold at thin margins, used as the basis for AWS. We also got to hear about his side interests, including his private spaceflight company called Blue Origin.
• Redshift – Redshift is the AWS data warehouse service that started the “big data” era on AWS. While AWS offered data services such as EMR and RDS, prior to Redshift, AWS consisted almost exclusively of Infrastructure-as-a-Service (IaaS) services which provided basic compute, relational database, and storage services. Redshift led the way for a plethora of future big data services over the next few years.
I went to this conference knowing little about the cloud, only having heard it was becoming more important. I attended a pre-conference bootcamp (Architecting on AWS) and was completely blown away by the features and agility of the platform. In a matter of hours, I was able to spin up a production-ready, load-balanced, auto-scaling web application stack – all through the AWS console and CLI. Recalling a time not long ago when procuring new infrastructure could be a multi-month affair, this was fascinating stuff! I let convinced that AWS was the wave of the future for enterprise IT, which led me to a series of career moves that accelerated my professional growth, culminating in my current role as a Principal Consultant at AHEAD.
To learn more about AHEAD’s capabilities or offerings with AWS, contact us today to meet with our experts. You can experience (or relive!) the excitement of AHEAD’s re:Invent sponsorship with a personalized demonstration of ServiceNow, DevOps, or Amazon WorkSpaces that you can set up when you click the button below.