About this opportunity
Torque Technologies LLC lists this Data Engineer opportunity in new york city (manhattan), New York. Review the employer’s description below for duties, qualifications and application requirements.
Job description
Position: Data Engineer
Location:Manhattan NY (Hybrid 3 days a week to office)
Long Term
Key Skills
SQL | Python | Spark | Airflow / ADF | Cloud data services | CI/CD
Required Qualifications
8-10+ years in data engineering
Strong SQL and Python (or Scala) skills
Experience with orchestration tools such as Airflow or Azure Data Factory
Hands-on experience with cloud data services and big data frameworks such as Spark
Understanding of data warehousing and lakehouse concepts
Familiarity with Git, CI/CD and infrastructure-as-code
Experience supporting ML/AI data needs is a plus
Key Responsibilities
Design, build and optimize batch and streaming data pipelines
Implement data models and transformations defined by the Data Architect
Ingest and integrate data from multiple source systems and APIs
Implement data quality checks, monitoring and alerting
Optimize performance and cost of data workloads
Collaborate with AI Developers and analysts to provision datasets
Maintain documentation, version control and CI/CD for data code
Work with offshore data engineers on knowledge transfer and delivery
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.