Overview
TekWissen is a global workforce management provider throughout India and many other countries in the world. The below job opportunity is to one of our clients who is a part of a trusted global innovator of IT and business services headquartered in Tokyo. We help clients transform through consulting, industry solutions, business process services, IT modernization and managed services. This client enables us to move confidently into the digital future. This client committed to Long Term success and combine global reach with local client attention to serve them in over 50 Countries.
Position: Cloud Engineer
Location: Hyderabad/Bangalore/Noida/chennai/Pune/Gurgaon
Work Type: Hybrid
Job Type: Full Time
Job Description
Required Qualifications
- Data Orchestration: Design and maintain Directed Acyclic Graphs (DAGs) using Apache Airflow for complex ETL pipelines and dependencies.
- Event Streaming: Build, configure, and monitor real-time data pipelines using Apache Kafka for data ingestion, decoupling, and processing.
- AWS Cloud Infrastructure: Provision, scale, and manage Kafka and Airflow using AWS services (e.g., Amazon MSK for Kafka, MWAA for Airflow, or self-hosted on EKS). System Integration: Connect Airflow with Kafka to create event-driven workflows that trigger DAGs upon message arrival.
- Monitoring & Optimization: Set up alerting and logging using AWS CloudWatch and Kafka's Control Center; optimize broker configurations for high throughput and fault tolerance.
- AWS: Deep understanding of core AWS services (S3, EC2, VPC, IAM) and managed streaming/orchestration tools (Amazon MSK, MWAA).
- Apache Kafka: Hands-on experience with Kafka architecture, topics, partitions, consumer groups, and KSQL/Kafka Connect.
- Apache Airflow: Proficiency in writing Python-based DAGs and operators, including the Apache Kafka Provider for Airflow.
- Programming: Advanced proficiency in Python and SQL; experience with scripting and building data integration layers.
- Data Warehousing & Processing: Experience integrating with modern data warehouses (e.g., Snowflake, Redshift) and stream-processing engines (e.g., Apache Spark).
TekWissen® Group is an equal opportunity employer supporting workforce diversity
