Lead Data Engineer
at Smart Working Solutions
London; Reading; Guildford; Milton Keynes; Chelmsford; St Albans; Woking, hybrid, senior, contract, 7+ years
posted 20 Aug 2026
- Azure
- pgvector
- PostgreSQL
- Qdrant
- Polars
- Kafka
- Apache Spark
- OpenSearch
- Apache Airflow
- NoSQL
- Elasticsearch
- GCP
- MySQL
- Pandas
- MongoDB
- Milvus
- AWS
- Python
About the role
Posted: 2026-08-20
About Smart Working At Smart Working, we believe your job should not only look right on paper but also feel right every day. This isn’t just another remote opportunity - it’s about finding where you truly belong, no matter where you are. From day one, you’re welcomed into a genuine community that values your growth and well-being. Our mission is simple: to break down geographic barriers and connect skilled professionals with outstanding global teams and products for full-time, long-term roles. We help you discover meaningful work with teams that invest in your success, where you’re empowered to grow personally and professionally. Join one of the highest-rated workplaces on Glassdoor and experience what it means to thrive in a truly remote-first world.
We are looking for a Lead Data Engineer to build and lead the data infrastructure powering an intelligent AI assistant platform. This role will architect and scale the systems that power our AI products, from real-time data pipelines and analytics infrastructure to vector databases and machine learning data workflows. You will work closely with AI engineers, backend engineers, and product teams to ensure our platform can process large volumes of operational data reliably and intelligently. You will define our data architecture, tooling, and engineering standards, and play a key role in building the foundations of the future data team. This role requires you to be in the office 4 days / week.
What you would do
- Architect and build scalable data pipelines and infrastructure to support AI and product systems.
- Design and maintain data ingestion, transformation and storage architectures for operational and AI workloads.
- Develop and manage batch and real-time data pipelines.
- Build and optimise systems for vector search, retrieval and machine learning data pipelines.
- Ensure data reliability, security and governance across the platform.
- Collaborate with AI and backend engineering teams to support training, inference and product features.
- Implement monitoring, observability and data quality frameworks.
- Optimise the performance of large-scale datasets and query systems.
- Contribute to technical architecture decisions and long-term data strategy.
Read from jobs.lever.co, last seen 5d ago.