Senior Software Engineer - AI Frameworks

About this role

Architect and implement efficient tensor computation primitives and software abstractions for custom AI accelerators. Develop and extend PyTorch features for model onboarding, optimization, and execution on custom AI accelerators. Contribute to and improve AI inference stacks such as vLLM and SGLang, including scheduling, KV cache management, and serving pipelines

Design, develop, profile, and optimize high-performance kernels for NPUs (MAIA) and GPUs to accelerate LLM inference and training workloads. Collaborate across disciplines to define requirements and Deliver practical solutions to new technical challenges. Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR equivalent experience

Master's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR Bachelor's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, or Python OR equivalent experience. Experience with PyTorch internals, custom operators, hardware backend, or torch.compile/Dynamo-based optimization flows. Experience with AI inference stacks such as vLLM, SGLang, or similar large-scale model serving systems

Experience with NPU or GPU kernel development and optimization (e.g., CUDA, Triton, or accelerator-specific toolchains). Familiarity with common LLM concepts such as attention mechanisms, KV caching, quantization (PTQ/QAT), and distributed parallelism strategies (TP, PP, DP).

Apply for this role on Microsoft’s official careers site.