Data Engineer
at Paypay
Hybrid Workstyle, hybrid, full time, 4+ years
seen 54d ago
- Airflow
- GitOps
- PySpark
- GitHub
- Kafka
- Prefect
- Prometheus
- AWS Lake Formation
- Datadog
- Grafana
- Terraform
- Confluence
- AWS
- Python
- AWS DMS
- Jira
- Unity Catalog
- SQL
- Slack
- CloudWatch
- Dagster
- Delta Lake
- CI/CD
- Databricks
- Spark
About the role
The assigned organization is primarily advancing projects related to a next-generation digital financial platform. Our team takes the lead in designing, building, and operating modern data engineering infrastructure from scratch to enable data-driven innovation across the group.
PayPay Bank’s strengths lie in its ability to develop products in-house from concept through implementation, its marketing capabilities that deliver value to tens of millions of users, and its robust financial and corporate foundations that maintain the trust expected of critical social infrastructure. As a Data Engineer, you will play a key role in supporting these strengths across the organization and contribute to the challenge of redefining what a bank can be in the digital era.
What you would do
* Design, develop, and operate robust and scalable data ingestion pipelines using technologies such as Databricks, AWS DMS, and Terraform * Develop large-scale data processing workflows using Spark, Python, and SQL * Design and build data integration platforms that connect banking services, payment systems, internal business applications, and external data sources * Optimize data pipelines by combining Change Data Capture (CDC), streaming, and batch processing architectures * Build and maintain data lakehouse and data catalog platforms leveraging Databricks Lakehouse and Unity Catalog * Design and operate data governance capabilities, including data quality management, metadata management, data lineage, access control, and audit logging * Improve performance, optimize costs, and enhance reliability for large-scale data processing environments * Implement SRE and observability practices for data platforms, including monitoring, alerting, dashboards, and incident response * Build, automate, and standardize data infrastructure using Infrastructure as Code (IaC) technologies such as Terraform * Develop and maintain shared frameworks, tools, templates, and CI/CD pipelines to improve the efficiency and consistency of data engineering operations * Partner with product, analytics, risk management, security, compliance teams, and other PayPay Group companies to drive data utilization and business value * Operate and enhance data platforms in compliance with the security, privacy, regulatory, and internal control requirements expected of a financial institution
What they are looking for
Read from job-boards.greenhouse.io, last seen 3d ago.