AI Software Engineer Intern

About this role

Job Details: Job Description: We are building a next-generation LLM inference system, spanning model optimization, inference runtime, and system-level design. This is a research + engineering role where you will: Study cutting-edge work (LLM inference, MoE, system optimization) Implement and optimize core techniques Work across the stack: model → kernels → runtime → distributed system A key focus is GPU kernel and runtime optimization, including exploring Triton-like programming models and compiler approaches, as part of building an end-to-end AI rack software system for LLM inference. Key Responsibilities 1

Research & Prototyping Read and reproduce state-of-the-art work (LLM inference, MoE, systems) Translate ideas into working, optimized implementations Identify bottlenecks and iterate beyond baseline performance 2. LLM Inference Optimization Implement and evaluate techniques such as: Continuous / dynamic batching KV cache optimization and memory management Speculative decoding Flash / paged attention Quantization (INT8 / FP8 / low-bit) Optimize for latency, throughput, and GPU utilization 3. MoE (Mixture-of-Experts) Systems Explore efficient inference for sparse models: Routing strategies and load balancing Expert parallelism and sharding Communication vs computation trade-offs Improve scalability and efficiency of MoE inference 4

Kernel & Runtime Optimization Develop and optimize GPU kernels using modern approaches: Triton-like programming models CUDA or equivalent low-level frameworks Investigate: Memory access patterns and layout optimization Operator fusion and kernel efficiency Compiler-style optimization for tensor workloads Compare different kernel/runtime strategies and integrate into the system 5. End-to-End Inference System Development Build and optimize a full inference stack: Model execution layer (vLLM, TensorRT-LLM, or similar) Runtime scheduling and batching Distributed inference across GPUs/nodes Work on: Multi-GPU / multi-node scaling NCCL / co

Apply for this role on Intel’s official careers site.