Akhil.

Akhil is a Senior Software Engineer at AWS’s Annapurna Labs, where he builds high-performance machine learning systems for AWS’s custom AI accelerators. He works across distributed inference, ML frameworks, compilers, kernels, and hardware optimization. His current focus is maximizing Trainium inference performance for frontier AI models.

Akhil architected and built Trainium’s first distributed PyTorch inference framework, which powered LLM inference at Amazon scale for Amazon Bedrock and Rufus, Amazon’s AI shopping assistant. In 2024, he led the inference framework team for the launch of Trainium2, delivering performance that surpassed competing cloud accelerator offerings at the time and helped establish Trainium as a major platform for LLM inference.

He also made Trainium easier for developers to use with mainstream ML frameworks and open-source serving tools. He designed and implemented Trainium’s torch.compile backend, allowing standard PyTorch models to run on Trainium through the AWS Neuron compiler. He then architected and built vLLM Neuron, adapting vLLM’s dynamic, GPU-first design to Trainium’s statically compiled execution model.

His role combines hands-on architecture with technical leadership: he sets direction, turns ambitious performance goals into clear engineering plans, and leads execution across geographies. As one of the first engineers on the Trainium inference team, Akhil helped scale the organization by establishing its technical foundations and mentoring engineers who now lead key initiatives. Earlier at AWS, he helped build the inference platforms behind Amazon SageMaker Autopilot and Amazon Forecast.