[Remote] Software Co-Design AI HPC Systems

Note: The job is a remote job and is open to candidates in USA. Microsoft is a leading technology company focused on architecting next-generation AI systems at datacenter scale. The role involves leading the co-design of AI systems, optimizing performance and efficiency, and collaborating with various teams to drive high-impact ideas into production.

Responsibilities

• Lead the co-design of AI systems across hardware and software boundaries, spanning accelerators, interconnects, memory systems, storage, runtimes, and distributed training/inference frameworks
• Drive architectural decisions by analyzing real workloads, identifying bottlenecks across compute, communication, and data movement, and translating findings into actionable system and hardware requirements
• Co-design and optimize parallelism strategies, execution models, and distributed algorithms to improve scalability, utilization, reliability, and cost efficiency of large-scale AI systems
• Develop and evaluate what-if performance models to project system behavior under future workloads, model architectures, and hardware generations, providing early guidance to hardware and platform roadmaps
• Partner with compiler, kernel, and runtime teams to unlock the full performance of current and next-generation accelerators, including custom kernels, scheduling strategies, and memory optimizations
• Influence and guide AI hardware design at system and silicon levels, including accelerator microarchitecture, interconnect topology, memory hierarchy, and system integration trade-offs
• Lead cross-functional efforts to prototype, validate, and productionize high-impact co-design ideas, working across infrastructure, hardware, and product teams
• Mentor senior engineers and researchers, set technical direction, and raise the overall bar for systems rigor, performance engineering, and co-design thinking across the organization

Skills

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python + OR equivalent experience
• Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python + OR equivalent experience
• Strong background in one or more of the following areas: AI accelerator or GPU architectures, Distributed systems and large-scale AI training/inference, High-performance computing (HPC) and collective communications, ML systems, runtimes, or compilers, Performance modeling, benchmarking, and systems analysis, Hardware–software co-design for AI workloads
• Proficiency in systems-level programming (e.g., C/C++, CUDA, Python) and performance-critical software development
• Proven ability to work across organizational boundaries and influence technical decisions involving multiple stakeholders
• Experience designing or operating large-scale AI clusters for training or inference
• Deep familiarity with LLMs, multimodal models, or recommendation systems, and their systems-level implications
• Experience with accelerator interconnects and communication stacks (e.g., NCCL, MPI, RDMA, high-speed Ethernet or InfiniBand)
• Background in performance modeling and capacity planning for future hardware generations
• Prior experience contributing to or leading hardware roadmaps, silicon bring-up, or platform architecture reviews
• Publications, patents, or open-source contributions in systems, architecture, or ML systems are a plus

Benefits

• Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

Company Overview

Company H1B Sponsorship