Inference Software Engineer

Posted: 2025-06-17

About Etched

Etched is building hardware for frontier intelligence. We co-design chips, racks, software, and manufacturing to deliver best-in-class throughput and latency across both prefill and decode workloads. Our first products are heavily focused on inference. Backed by hundreds of millions from top-tier investors and staffed by leading engineers, Etched is redefining the infrastructure layer for the fastest growing industry in history.

- Support porting state-of-the-art models to our architecture. Help build programming abstractions and testing capabilities to rapidly iterate on model porting.

- Build, enhance, and scale our runtime, including multi-node inference, intra-node execution, state management, and robust error handling.

- Optimize routing and communication layers using our collectives.

- Utilize performance profiling and debugging tools to identify bottlenecks and correctness issues.

You may be a good fit if you have

- Proficiency in C++ or Rust.

- Understanding of performance-sensitive or complex distributed software systems like Linux internals, accelerator architectures (e.g. GPUs, TPUs), Compilers, or high-speed interconnects (e.g. NVLink, InfiniBand).

- Familiarity with PyTorch or JAX.

- Ported applications to non-standard accelerator hardware or hardware platforms.

Strong candidates may also have experience with (Nice-to-have qualifications)

- Developed low-latency, high-performance applications using both kernel-level and user-space networking stacks.

- Deep understanding of distributed systems concepts, algorithms, and challenges, including consensus protocols, consistency models, and communication patterns.

- Solid grasp of Transformer architectures, particularly Mixture-of-Experts (MoE).

- Built applications with extensive SIMD (Single Instruction, Multiple Data) optimizations for performance-critical paths.