About this opportunity
Elios AI lists this Senior Platform Data Engineer opportunity in madison, Wisconsin. Review the employer’s description below for duties, qualifications and application requirements.
Job description
Senior Platform Data Engineer
We are seeking a Senior Platform Data Engineer. This is a production engineering role focused on building and operating a centralized, enterprise-grade data platform that institutionalizes proprietary knowledge and delivers data as a product — not a side effect.
You will design, build, and operate models, pipelines, integrations, and infrastructure across the full medallion architecture (Bronze → Silver → Gold). This role requires true end-to-end ownership: design, build, test, deploy, monitor, and continuously improve.
What You’ll Do
Build and maintain ingestion pipelines from raw Bronze landing through curated Gold domain models
Develop transformation models using dbt with testing, Slim CI, and exposure tracking
Author and manage production-grade Airflow DAGs with SLA monitoring and backfill strategies
Design and implement streaming and event-driven pipelines (Kafka, Glue Streaming, Delta Live Tables)
Collaborate with data scientists, platform engineers, and product teams to ship reliable data products
Implement data quality frameworks and enforce governance standards
Containerize workloads and deploy to Kubernetes (EKS)
Contribute to infrastructure as code (Terraform provisioning for S3, IAM, Glue, etc.)
Document lineage, metadata, ownership, and sensitivity classification
This is not a “pipeline-only” role — it is platform ownership at scale.
Core Technical Skills
Python
Production-level standards
SQL
Advanced analytical queries, window functions, optimization
Experience with Redshift, Snowflake, or Databricks SQL
dbt
Gold-layer modeling and testing
Slim CI, exposures, meta configurations
Orchestration
Apache Airflow (DAG authoring, task dependencies, SLAs, backfills)
Prefect or Dagster is a plus
Streaming & Event Pipelines
Kafka topic consumption
Bronze landing jobs
Cloud Data Platforms (hands-on with at least one)
Databricks (Delta Lake, Unity Catalog, Auto Loader, DLT)
AWS (S3, Glue, Lake Formation, Redshift)
Data Quality
Great Expectations, dbt tests, or equivalent
Docker & Kubernetes
Containerized jobs
Kubernetes Jobs and CronJobs
EKS deployments
Infrastructure as Code
Terraform familiarity
Data Modeling
Dimensional modeling
Slowly Changing Dimensions
Medallion architecture design
Data Platform-Specific Experience
SaaS ingestion via Fivetran, Airbyte, or custom connectors (REST, SFTP, JDBC)
Change Data Capture (Debezium or connector-based CDC modes)
Handling semi-structured and unstructured data (PDF, Excel, JSON)
Designing Bronze schemas for fidelity and downstream flexibility
Vector embedding pipelines (chunking strategies, embeddings, pgvector/OpenSearch)
Data lineage tooling (OpenLineage, DataHub, Unity Catalog)
Governance standards: PII handling, sensitivity classification, column masking, partition-level access
Ways of Working
Agile sprint participation (planning, standups, retros)
Treat pipelines as production software (tests, runbooks, SLAs, alerts)
Migration-safe schema changes (expand/contract pattern)
Proactive data quality ownership
Rigorous peer code reviews
Close collaboration with data science and platform engineering teams
Nice to Have
Migration of legacy data assets (Excel, Access, shared drives)
ML feature engineering and point-in-time correct pipelines
Open-source data tooling contributions
Qualifications
5–8+ years of experience developing data products and platforms
Strong background in Agile software development
Demonstrated success in prior engineering roles
Strong analytical and problem-solving capabilities
What We’re Looking For
Clear communicator who can document complex data flows
Results-driven and impact-focused
Proactive and self-directed
Comfortable interfacing across technical and business stakeholders
Entrepreneurial mindset with a willingness to innovate
If you’re a production-minded data engineer who thrives on building scalable platforms and owning pipelines end-to-end, we’d love to connect.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.