SOCAR Data Engineer
Batch platform, operational DB → warehouse pipelines, real-time pricing server, CDC lakehouse
Data Engineer · Seunghyeon Yang
I build and operate data pipelines
A data engineer who keeps looking for ways to automate repetitive pain points and build resilient data infrastructure.
- 4+ yrsData Engineering
- Batch · CDCPipelines
- GCP · K8sInfrastructure
- End-to-endIngest → Mart → Use
About
- 01
Pipeline optimization & automation
Hardened batch pipelines on Airflow and automated cloud infrastructure to keep data platforms reliably running in production.
- 02
Building the data environment
Owned the full path from raw data processing to a BigQuery-based warehouse, so the organization can make decisions grounded in data.
- 03
System design & operations
Handled high-traffic workloads with gRPC/FastAPI and shipped MSA services reliably through GitOps (ArgoCD, Helm) — that combination is my strong suit.
Experience
- 2022.06 — Present
- 2021.06 — 2022.06
Haezoom Software Engineer
Solar power data ingestion platform, weather data pipeline automation
Orchestration
- Airflow
- dbt
Warehouse & Lakehouse
- BigQuery
- Apache Iceberg
Streaming & CDC
- Kafka
Infrastructure
- Docker
- Kubernetes
- GKE
- EKS
- Helm
- Datadog
Language
- Python
- SQL
Projects
Iceberg-based CDC Pipeline
2025.12 — 2026.03A PoC that swapped a once-a-day batch for CDC to fix stale reads and missing deletes.
- Apache Iceberg
- Kafka
- Debezium
- Strimzi
CDC Medallion Layer
2026.04 — 2026.06A dbt layer that turns raw CDC change logs into tables analysts can query directly.
- dbt
- BigQuery
- BigLake
- Airflow
Airflow Build-out and Operation
2022.06 — 2023.05Moved Docker Compose Airflow onto Kubernetes with high availability and proper secrets handling.
- Airflow
- Kubernetes
- Helm
- Workload Identity
Operational DB → Warehouse Batch Pipeline
2023.06 — 2024.01A multi-stage ingestion path with a dbt transformation layer, designed to be gentle on the operational DB and safe to reprocess.
- Airflow
- BigQuery
- dbt
- Airbyte
Real-time Pricing & Waiver Server
2024.06 — 2024.12A dynamic pricing server that sets prices per region and vehicle type in real time, based on utilization and idle-rate demand.
- gRPC
- FastAPI
- NumPy
- EKS
Self-service BI Deployment Platform
2026.07An internal dashboard platform where analysts push SQL and HTML, and CI ships each as an independent service.
- Go
- BigQuery
- Cloud Run
- Firebase
Let's talk
Always up for a conversation about data engineering, data infrastructure,
or interesting technical problems and opportunities.