About this role
Do you want to build the large-scale data systems that help millions of AWS customers cut their cloud costs and run more efficiently? Join a team that turns massive streams of infrastructure and usage data into the intelligent recommendations customers rely on every day. We are the Optimization Data team, the data backbone behind AWS Compute Optimizer and AWS's broader cost optimization products. Our job is to identify customer resources, collect the utilization metrics that describe how they run, and turn that raw telemetry into clean, reliable, well-labeled signals that our science and engine teams use to model cost-savings recommendations. Every day we ingest and process enormous volumes of metrics and billing data across EC2, Auto Scaling groups, EBS, Lambda, ECS, EKS, RDS/Aurora, DynamoDB, and networking resources — sourcing data from internal systems like CloudWatch, EBS Stats, Lambda Stats, and the Fargate control plane, as well as third-party products such as Datadog. If you enjoy working at the intersection of big data, distributed systems, and real customer impact, this is the place for you. As a Software Development Engineer on our team, you will: - Deeply understand the AWS services we optimize — how compute, storage, database, and networking services emit utilization and configuration data — and use that domain knowledge to design a high-throughput, highly available ingestion layer that reliably captures metrics for millions of resources with tight freshness SLAs - Design, build, and scale data pipelines and services that collect, clean, aggregate, and label huge volumes of metrics with high reliability and low latency - Own systems end to end — from architecture and implementation to deployment, monitoring, and operations across many regions, including GovCloud and isolated partitions - Partner with data scientists and engineering teams to define the data they need and deliver it in clean, well-structured, intelligently partitioned data lakes that fuel modeling and recommendations - Solve hard problems around data security, scalability, and cost efficiency, favoring serverless and decoupled distributed architectures so components scale independently and operational burden stays low - Establish and meet ingestion and publishing SLAs, build in backfill and data-correction paths, and continuously raise the bar on data quality, operational excellence, and the customer experience You'll join a collaborative group of engineers who care deeply about doing things the right way — secure by default, reliable at scale, and always focused on the value we deliver to customers. We invest in your growth, give you real ownership from day one, and celebrate the impact you make.