About this opportunity
2T Consulting lists this Kafka Data Engineer opportunity in Lumberville, Pennsylvania. Review the employer’s description below for duties, qualifications and application requirements.
Job description
We are seeking an experienced Kafka/Spark Data Engineer with strong expertise in real-time data processing, streaming technologies, and distributed systems. The role will focus on Kafka, Spark, Redis, Python, and cloud-based Big Data solutions.
Roles and Responsibilities
Design, develop, and support real-time data streaming and Big Data solutions.
Develop streaming applications using Apache Kafka, KStreams, and KTables.
Work with Apache Spark for large-scale data processing.
Develop applications using Python, Scala, Java, and Bash/Shell scripting.
Work with NoSQL databases such as Redis, MongoDB, and HBase.
Design scalable, distributed, and fault-tolerant data architectures.
Implement data integration, security, and authentication solutions, including Kerberos.
Work with streaming technologies such as Flink or Storm.
Support cloud-based data solutions across AWS, Azure, or GCP.
Apply data analysis, statistical methods, and machine learning techniques where required.
Create and support data visualizations using Tableau.
Troubleshoot business-critical applications and provide effective technical solutions.
Required Qualifications
9+ years of overall IT/industry experience, with strong Big Data and real-time processing experience.
Bachelor's or Master's degree in Computer Science, Engineering, Science, or related field.
Strong hands-on experience with Kafka and real-time streaming.
Experience with Spark, Redis, and Python.
Strong understanding of distributed systems, data partitioning, and fault-tolerant architectures.
Experience with NoSQL databases and cloud platforms.
Strong problem-solving, analytical, and communication skills.
Preferred Qualifications
Experience with Flink, Storm, Scala, or Java.
Knowledge of machine learning and predictive analytics.
Experience with Tableau and real-time data visualization.
Who can apply
Review the original listing for work authorization, qualifications and employer requirements.