Senior Data and Analytics Engineer
at Dataroles
North Chicago, hybrid, senior, full time, USD 1,09,500–2,08,500 / year, 5+ years
posted 02 Nov 2026
- Spark Structured Streaming
- Apache Iceberg
- AWS Step Functions
- Infrastructure as Code
- PySpark
- Amazon S3
- Alation
- AWS CloudWatch
- AWS Glue
- AWS Lake Formation
- Apache Airflow
- Redshift Spectrum
- Python
- Scala
- Amazon Kinesis
- AWS CloudTrail
- SQL
- Apache Spark
- Git
- Amazon EMR
- Amazon Redshift
- dbt
- CI/CD
- Apache Kafka
- AWS IAM
- AWS Secrets Manager
About the role
Data Analysis & BI work from LinkedIn, X, Reddit and Upwork, in one list. Find me work ×
dataroles
What you would do
* Build and support production data pipelines and products for structured, semi-structured and unstructured information * Use AWS lakehouse and warehouse components to model data, apply medallion layers and improve query performance * Turn business domains into governed data products for dashboards, self-service analysis, APIs, data science and AI * Set up ingestion and orchestration with monitoring, recovery, retries and backfills across batch and streaming workloads * Guide designs and delivery, manage technical risks, and own solutions through deployment, support and ongoing improvement
* Hands-on engineering and solution delivery: Design, develop, test, deploy, and support production-grade data pipelines and data products for structured, semi-structured, and unstructured data. Build for both batch and near-real-time/streaming patterns, with reusable components, parameterized frameworks, and automated recovery. * Lakehouse and modern data warehouse architecture: Implement solutions using Amazon S3, Apache Iceberg/open table formats, AWS Glue Data Catalog, Amazon EMR/Spark, Amazon Redshift and Redshift Spectrum. Apply medallion patterns (Bronze, Silver, Gold), dimensional and domain-oriented modeling, workload isolation, partitioning, clustering/sort strategies, and performance optimization. * Data product and data mesh enablement: Translate business domains into discoverable, interoperable, and governed data products. Define product contracts, ownership, metadata, quality expectations, service-level objectives, lineage, and approved consumption patterns for dashboards, self-service analytics, APIs, data science, and AI. * Ingestion and orchestration: Engineer reliable ingestion from enterprise applications, databases, files, APIs, and event streams. Develop orchestration using tools such as Apache Airflow, dbt, AWS Step Functions or equivalent approved platforms, with dependency management, idempotency, observability, alerting, and backfill capability. * Data quality, governance, and security: Embed automated data-quality controls, reconciliation, schema validation, lineage, cataloging, retention, encryption, least-privilege access, and auditability. Partner with data governance, security, privacy, architecture, and platform teams to meet AbbVie standards and regulated-industry expectations. * DevSecOps and engineering excellence: Establish source-control, code-review, automated testing, CI/CD, infrastructure-as-code, environment promotion, release management, and operational monitoring practices. Improve reliability, cost, performance, and developer productivity through measurable engineering standards. * Business partnership and technical leadership: Understands organization's vision, goals, and strategies and partner with DATA, BTS, Operations stakeholders to translate complex requirements into scalable designs. Provide technical direction, resolve design and delivery issues, communicate risks and trade-offs, and contribute to portfolio prioritization and roadmap decisions. * Operational ownership: Own solutions through the full lifecycle, including production readiness, deployment, support transition, incident resolution, root-cause analysis, technical debt reduction, and continuous improvement. Work with minimal oversight while escalating material risks early.
Read from dataroles.co, last seen 3d ago.