About this opportunity
Techvilla Solutions lists this Big Data Developer opportunity in Pelham Bay, New York. Review the employer’s description below for duties, qualifications and application requirements.
Job description
We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala, along with performance tuning and troubleshooting in the Cloudera Big Data ecosystem.
Key Responsibilities
Develop, maintain, and optimize big data applications and data processing pipelines.
Work with Kafka and Apache Spark for real-time and batch data processing.
Perform data analysis, data wrangling, and processing of large volumes of structured and unstructured data.
Write and optimize complex SQL queries using Hive and Impala.
Monitor, troubleshoot, and resolve performance issues in the Cloudera Big Data ecosystem.
Use Splunk for log analysis, monitoring, and troubleshooting.
Manage and monitor batch workflows and job scheduling using AutoSys.
Work in Linux-based environments for development, deployment, and operational support.
Collaborate with business users and cross-functional teams to deliver high-quality data solutions.
Ensure data quality, reliability, scalability, and performance of big data applications.
Required Technical Skills
Strong hands-on experience with Kafka, Spark, Scala, and SQL.
Experience with Hive, Impala, Hadoop, and the Cloudera ecosystem.
Proficiency in data analysis and data wrangling for large datasets.
Experience in performance analysis, tuning, troubleshooting, and issue resolution.
Hands-on experience with Splunk and AutoSys.
Strong knowledge of Linux-based infrastructures.
Experience in distributed data processing and big data technologies.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.