Job Title: AWS Data Engineer
Notice Period:: Immediate to 45days
Experience: 5-8 years in Data Engineering
Job Description:
We are seeking skilled and dynamic Cloud Data Engineers specializing in AWS. The ideal candidate will have a strong background in data engineering, with a focus on data ingestion, transformation, and warehousing. They should also possess excellent knowledge of PySpark or Spark, and a proven ability to optimize performance in Spark job executions.
Key Responsibilities:
- Design, build, and maintain scalable data pipelines for cloud platform AWS.
- Implement data ingestion and transformation processes to facilitate efficient data warehousing.
- Utilize cloud services to enhance data processing capabilities:
- AWS: Glue, Athena, Lambda, Redshift, Step Functions, DynamoDB, SNS.
- Optimize Spark job performance to ensure high efficiency and reliability.
- Stay proactive in learning and implementing new technologies to improve data processing frameworks.
- Collaborate with cross-functional teams to deliver robust data solutions.
- Work on Spark Streaming for real-time data processing as necessary.
Qualifications:
- 5-8 years of experience in data engineering with a strong focus on cloud environments.
- Proficiency in PySpark or Spark is mandatory.
- Proven experience with data ingestion, transformation, and data warehousing.
- In-depth knowledge and hands-on experience with cloud services(AWS)
- Demonstrated ability in performance optimization of Spark jobs.
- Strong problem-solving skills and the ability to work independently as well as in a team.
- Cloud Certification in AWS is a plus.
- Familiarity with Spark Streaming is a bonus.
