AWS re:Invent Recap
There were hundreds of announcements at AWS re:Invent. The AWS User Group shortlisted a few on the Re:Cap.
Application Development
A promising preview is Console to Code. You can execute a section of actions in the console, which are exported to CloudFormation or CDK. When launching an EC2 instance through the console, you select an AMI, set the KeyPair, set the security group and so on..In the Console to Code console, there will be a list of recorded actions. You can select the ones you want to generate as code and export to CloudFormation.
Second announcement in DevOps is CloudFormation management through Git. You create your Stack and reference the repository during the setup. When you make a commit on the code, CloudFormation will see the change and apply it. When using input parameters, you can define them in Git, and CloudFormation can also see them.
New SDKs for Rust and Kotlin More information here.
For those of you who are a visual builder, you can now use Application Composer and the Builder for Step Functions from within Visual Studio Code, which is the wildest thing I've seen today. Read more. I'm not really sure yet how this thing deploys. You drag a lot of components into the canvas, and in the end you save it and it exports the stack as CloudFormation.
CodeCatalyst the one stop shop for DevOps is now Enterprise Tier. I could write more, but I've never worked in an enterprise that really required this, so my opinion is not relevant.
Amazon Inspector now has Container Inspector to scan your container images during your build to shift security a little more to the left.
AWS Fault Injector has new scenarios, like Cross-Region failures. But let's be honest, it's hard enough to test single region failures. Another nice feature is a multi-account strategy, which can orchestrate failures across multiple accounts. It leverages IAM roles to act on other accounts.
SageMaker Studio got a little bit prettier and more features. AI is a hot topic everywhere, so we'll get back to this topic later. CodeWhisperer has been in preview for a while, but it can now train on your own codebase. CodeWhisperer works on 10 languages and scans for vulnerabilities, for most of them you get an automatic hint for remediation. And the thing works on Terraform wieeee
If you aren't tired yet of chatting to bots instead of a person, you'll have an amazing time using Amazon Q. It looks like you can add it to your IDE so you can interact with it in your trusted environment and ask it AWS related questions. It's still in preview, so it's still free. If you don't like using an IDE, you can use it straight from the command line. Amazon Q Code Transform Support can take actions like: upgrade from Java 8 to Java 17, which is pretty dope. Amazon Q integrated with CodeCatalyst can read a feature request, write some code and create a Pull Request, so it looks like it's finally time for AI to replace me lol.
Amazon Q transforms the developer experience
We'll see about that
Analytics & Databases
Welcome back to the second part of our journey, where the AI fun is over! Just kidding—still plenty of AI excitement ahead.
Amazon Aurora: Limitless Scaling
Let's kick things off with Amazon Aurora, the powerhouse of database management. Imagine automatic scaling to handle millions of transactions per second! With serverless fast scaling and limitless horizontal scaling, Aurora is here to make your database dreams come true.
Next up, Elasticache Serverless—a game-changer in distributed, vertically scaling cache nodes. Check out the details here and witness the magic of effortlessly scalable cache solutions.
Skipping over RDS for DB2; let's face it, sometimes legacy is just legacy.
Wanting for that sweet AI content?
Enter the Vector Database—a specialized marvel designed for processing vector data. In a world where data is growing exponentially, having a dedicated database for vector search becomes crucial. Perfect for powering generative AI applications, Vector DB integrates seamlessly with Amazon Bedrock.
And that's not all! Vector Search now extends its capabilities to Amazon DocumentDB and MemoryDB for Redis, providing the same vector search prowess without the need to switch databases.
Amazon Redshift Serverless AI-Scaling: Where AI Meets Optimization
Prepare to be amazed by Amazon Redshift's serverless AI-driven scaling optimizations. AI learns from past queries' resource requirements, and Redshift scales automatically to deliver optimal performance—claiming to be 10x better than regular Redshift. Explore new features like CTAS Scaling, Multi-data Warehouse writes, Multi-dimensional Layouts, and Incremental refresh for Data Lake Materialized Views.
Real-time insights are crucial in today's landscape, especially for IoT, anomaly monitoring, and fraud detection. Amazon Redshift offers zero-ETL integration, providing near real-time data propagation when used as your data warehouse.
Looking for more analytics power? Check out Amazon Neptune Analytics.
AWS Glue: Data Monkeys Rejoice!
For the data monkeys out there, AWS Glue introduces Data Quality anomaly detection and Dynamic Rules—now that's something to cheer about! Plus, Glue now supports multi-engine views for added flexibility.
EFS Archive Storage Class: Your Data, Your Way
Experience a new dimension in storage with EFS Archive Storage Class. Choose between Standard, Infrequent Access, and now Archive—offering 50% less cost than IA, with sub-millisecond access and GB/s throughput.
AWS S3 Express One Zone: High-Performance Storage
Last but not least, meet the new high-performance storage class—AWS S3 Express One Zone.
A new S3 Storage class boasting a 10x better performance than the S3 Standard storage class while handling hundreds of thousands of requests per second with consistent single-digit millisecond latency, making it a great fit for your most frequently accessed data and your most demanding applications.
Networking & Security
IAM Access Analyzer simplifies inspecting unused access. You can configure an analyzer which will do an actions when unused roles and policies and whatever are found. You could also configure external access analysis, which will also scan for external identities accessing your resources. With Custom Policy Checks, you can get a report when someone tries to broaden IAM policies. This can be executed straight from the CI pipeline.
IAM Identity Center Trusted Token Issuers simplifies access through external identity providers. Based on attributes in the tokens, the AWS Analytics services can delegate authorization to the source.
GuardDuty Runtime Monitoring detects runtime events on both EC2 and ECS (EC2 and Fargate). This integrates with Amazon Detective which includes all findings from GuardDuty.
Amazon Detective Investigations for IAM enables you to identify attack patterns and let's you investigate findings from GuardDuty. As expected, there's some Generative AI integration that can summarize a lot of findings to make your job a bit easier.
Route53 Application Recovery Controller Zonal Autoshift, that name though, will start to move traffic from one zone to another when failures are being detected. During the failures, there's no interaction required. When AWS notices that one zone is about to get problems, they already start shifting traffic to minimize downtime and improve availability. They recommend to over-provision a little bit so AWS can shift without interruptions. AWS will do monthly checks to test failover on target applications. You don't have to enable cross-zone load balancing, because AWS will do this for you.
Application Load Balancer finally supports mTLS between client and server.
ALB Automatic Target Weights helps distribute traffic between target groups based on their health checks.
MyApplications on AWS gives you an overview of all your applications on AWS. It shows the cost, health, SLOs etc. You have a dashboard to easily monitor costs and get some insights, specific to your applications.
Application Signals monitors application health of EKS workloads. It's going to cost a lot, but it has a lot of metrics out of the box. Look out, New Relic! You get a dashboard for each workload, so it's easier to troubleshoot where an error is coming from. You can configure SLO metrics and alert when you need to panic.
CloudWatch Infrequent Access helps you shed less tears when your AWS bill arrives! When searching through the logs, use Query Assist so you don't have to remember the syntax but instead use natural language and let an AI handle it for you.
The Cost Hub has been redefined a bit. It's easier to find your cost and Cost Optimization Hub has bene introduced. For example, it helps you to properly scale your instances and notifies you about over-provisioned instances. Using Amazon QuickSight, you can get detailed reporting. Read more, it's easy to setup, has a lot of visuals out of the box and is customizable if you like.
Compute
Okay let's rattle off some quick compute updates.
Amazon EC2 trn2 instance (trainium2) Optimized ec2 for training FM's with hundreds of billions of trillions of data points. AWS Trainium2 will power the highest performance compute on AWS for training foundation models faster and at a lower cost, while using less energy
Lambda Scaling Improvements Scale up at a 12x faster rate. With this improvement each function can scale up to a rate of 1,000 concurrent executions every 10 seconds, up to your account concurrency limit.
EKS Pod Identity Running EKS pods, you know having your pods having write acces. Attach IAM roles to K8s Service and watch the magic happen.
Machine Learning
Amazon Bedrock
Easily access foundational models without having all the hassle of maintaining them.
Both AWS FM's and 3rd party FM's.
The tweaked and used FM's are also completely private, they never leave your VPC and aren't shared back with AWS.
Some use cases:
Model Evaluation on Bedrock
Bedrock privately customize your FM by fine-tuning or pre-training your FM.
Knowledge bases for Bedrock - Build your private organizational knowledge base using Bedrock and integrate it with for example a company chatbot.
Agent for Bedrock—
Guardrails for Bedrock—now it's possible to add guardrails and filter or guard against toxicity of your models. You can also specifically redact data or types of data.
AWS step functions for Bedrock—wanna build serverless gen AI apps? Step functions for Bedrock will be your friend.
But wait, do you wanna have a managed service for all of these managed services?! Use Q offerings.
Q for Quicksight-Quickly generate insights and infographics on datasets.
SAGEMAKER UPDATES: - Sagemaker Pipelines Developer Experience - Sagemaker Clarify FM Evaluations—Bias and Toxicity detections. Also noteworthy an open-source library called FM-eval provided so everyone can collaborate on more responsible and sustainable AI.
- Sagemaker Smart Sifting—reduces training time and cost up to 35% by detecting useless reports or datasets for your specific training purposes.
- Sagemaker Inference Capabilities—Reduce cost by 50% and latency by 20%. All the other words didn't make any sense to me ;)
- Sagemaker HyperPod—helps maintaining pods. All pods are self-healing. Reduce training time by 20% once again.
- Sagemaker Canvas—a no-code GenAI tool for business analysts. The first step in completely throwing out your IT department of software engineers.
Responsible AI with AI Service Cards. An attempt of AWS to make their customers understand and be more responsible using their AI solutions. If you still have irresponsible thoughts, have a look here