Senior Data Engineer

DRW is a diversified trading firm with sophisticated technology operations across multiple asset classes and global markets. As a Senior Data Engineer in the Unified Platform organization, you will design, build, and operate the data platform and managed services used by traders, researchers, and back-office personnel, combining technical design ownership, software development, production support, and mentorship.

* Design, build, and operate managed platform services for data storage, distributed query execution, batch and streaming processing, and workload orchestration * Own technical designs and implementation decisions for services and components within the broader platform architecture, leading delivery from requirements through deployment and support * Automate provisioning, configuration, upgrades, and monitoring for managed services to reduce manual operations and improve consistency across deployments * Build and integrate metadata management tools, including data catalogs and lineage tracking, to help users discover datasets and understand their structure, ownership, and dependencies * Diagnose integration and performance issues with platform users, clarify technical requirements, and improve the services their workloads depend on * Improve reliability and performance of managed services through observability, capacity management, and testing; participate in production support and incident response * Mentor less experienced engineers through hands-on collaboration and feedback, promoting maintainable code, automated testing, and sound operational practices

* 7+ years of experience working with modern data technologies and/or building data-intensive distributed systems * Expertise in Java and/or Python with proven ability to deliver high-quality, maintainable production software; strong familiarity with SQL, Bash, and Linux environments * Experience building and operating managed data or infrastructure services offered to other teams, including operational automation and production support * Demonstrated ownership of engineering projects or service components from technical design through deployment and ongoing operation * Strong understanding of distributed systems trade-offs including fault tolerance, concurrency, partitioning, and recovery from partial failures * Production experience with batch and streaming systems, including practical knowledge of state management, retries, replay, and data consistency * Experience with data processing or query technologies such as Flink, Spark, or ClickHouse, and ability to evaluate their performance and operational trade-offs * Experience with multiple storage technologies including object storage, relational or NoSQL databases, and event streaming systems such as Kafka * Practical experience with data modeling, schema governance and evolution, and data quality controls * Experience with data formats and serialization systems such as Arrow, Parquet, Protobuf, or Avro * Experience with cloud-native data infrastructure including Kubernetes, workflow orchestration, automated deployment, and production monitoring * Strong communication and problem-solving skills with ability to troubleshoot issues with platform users and collaborate across teams