About this opportunity
EXL lists this Senior Azure Databricks Data Engineer opportunity in new york, New York. Review the employer’s description below for duties, qualifications and application requirements.
Job description
Job Summary
We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
Job Description: Azure / Databricks Data Engineer (9-15 Years Experience)
Senior Azure Databricks Data Engineer
Experience
9-15 Years of IT Experience
Location
Hybrid
Job Summary
We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
Key Responsibilities
Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.
Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.
Develop data models and implement data quality, validation, and governance frameworks.
Integrate data from multiple sources into a unified Lakehouse architecture.
Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
Implement security controls, access management, and data governance using Unity Catalog.
Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.
Monitor, troubleshoot, and resolve data pipeline issues.
Support CI/CD, DevOps, and infrastructure automation practices.
Maintain technical documentation and best practices.
Required Technical Skills Core Technologies
Databricks Lakehouse Platform
Apache Spark / PySpark
Delta Lake
SQL
Python
Data Engineering
ETL / ELT Development
Data Modeling
Data Warehousing
Data Quality & Validation
Streaming & Real-Time Processing
Governance & Security
Unity Catalog
Data Lineage
Row-Level Security
Access Control & Compliance
Data Governance Frameworks
Cloud & DevOps
Azure / AWS / GCP
Terraform
GitHub Actions / Azure DevOps
CI/CD Pipelines
Analytics & AI
Semantic Layers
Data Products
BI Platforms
Machine Learning Support
Generative AI & RAG Architectures
Required Qualifications
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
9-15 years of experience in Data Engineering and Data Warehousing.
Minimum 5+ years of hands-on experience with Azure Data Engineering technologies.
Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem.
Strong understanding of data lake, lakehouse, and cloud-native architecture patterns.
Experience in handling large-scale structured and unstructured datasets.
Strong analytical, problem-solving, and troubleshooting skills.
Preferred Qualifications
Microsoft Certified: Azure Data Engineer Associate (DP-203).
Databricks Certified Data Engineer Associate/Professional.
Experience with Snowflake, Power BI, or Microsoft Fabric.
Experience in real-time streaming solutions using Kafka/Event Hubs.
Exposure to Data Governance and Master Data Management initiatives.
Soft Skills
Strong stakeholder management and communication skills.
Ability to lead technical initiatives and drive architecture discussions.
Experience working in Agile/Scrum environments.
Excellent documentation and presentation skills.
Strong mentoring and team leadership abilities.
Nice to Have
Microsoft Fabric
Power BI
Azure Event Hubs
Kafka
Machine Learning data pipelines
Data Governance tools such as Purview
#J-18808-Ljbffr
Worksite address
new york, NY, 10261, US
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.