Senior Machine Learning Systems Engineer

As a Senior ML System Engineer on the AI & ML Platform’s Inference team, you will design and optimize large-scale model serving systems end-to-end. You will own everything from distributed infrastructure (global KV cache, continuous batching, load balancing, auto-scaling) to deep low-level optimizations (GPU kernels, quantization, speculative decoding).

* Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache). * Optimize latency and throughput of model inference under real production workloads. * Build reliable, high-concurrency serving systems that serve billions of requests reliably * Benchmark, fine-tune, and accelerate inference engines. * Create robust CI/CD infrastructure for seamless model deployment and inference engine updates. * Partner with senior ML engineers to fine‑tune open-source LLMs and deploy

* 5+ years of software engineering experience, 2+ years of system performance optimization experience * Deep low-level systems programming (C/C++ or Rust) * Experience with large-scale, high-concurrent production serving. * Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM, etc.). * Strong background in system optimizations: batching, caching, load balancing, parallelism.

* Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. Our offerings include health and wellbeing resources, paid volunteer days, and so much more.