Tittle: Lead Data Engineer (Tech Lead)
Location: Alpharetta GA (Day 1 onsite)
The pay range for this role is $120k - $130k per annum including any bonuses or variable pay. Tech Mahindra also offers benefits like medical, vision, dental, life, disability insurance and paid time off (including holidays, parental leave, and sick leave, as required by law). Ask our recruiters for more details on our Benefits package. The exact offer terms will depend on the skill level, educational qualifications, experience, and location of the candidate.
Job Description:
Responsible for designing, developing, and managing data engineering solutions to enable the efficient collection, transformation, and storage of data. The Lead Data Engineer will work closely with cross-functional teams, including data architects, data scientists, and business analysts, to build and optimize data pipelines, ensuring data is accessible, reliable, and of high quality for analytical and operational purposes.
Key Responsibilities:
- Architect, design, and develop scalable and high-performance data pipelines for data ingestion, transformation, and storage using modern cloud technologies like Azure Data Factory, Databricks, and SQL-based solutions.
- Integrate and manage structured, semi-structured, and unstructured data sources from multiple platforms, ensuring data is appropriately cleansed, transformed, and loaded (ETL/ELT processes).
- Design and develop data ingestion pipelines utilizing Delta Live Tables (DLT) and Structured Streaming to handle both batch and real-time data ingestion requirements.
- Collaborate with data engineers, analysts, and business teams to understand data needs and translate those into technical solutions.
- Optimize data pipelines and workflows to ensure high performance, scalability, and cost-efficiency across large-scale data environments.
- Lead the design and development of cloud-based data solutions on platforms such as Azure, ensuring data systems are reliable, secure, and compliant with industry standards.
- Implement data quality checks, validation rules, and data governance processes to maintain data integrity and consistency.
- Provide technical leadership and mentorship to junior and mid-level data engineers, fostering a culture of continuous learning and improvement.
- Set up monitoring systems for data pipelines and troubleshoot data issues to ensure smooth operations and prompt issue resolution.
- Document data engineering processes, architectures, and workflows to ensure transparency, traceability, and maintainability.
- Stay up to date with new technologies, tools, and best practices in data engineering, and continuously improve the performance, scalability, and efficiency of data systems.
Skills/Qualifications:
- Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
- 8+ years of experience in data engineering or data-related roles.
- 5+ years of experience in designing and implementing data pipelines using Azure Data Factory, Databricks, SQL, and other relevant technologies.
- Strong experience with building, managing, and optimizing ETL/ELT workflows.
- Strong experience in designing and developing data ingestion pipelines using Delta Live Tables (DLT) and Structured Streaming.
- Proficiency in programming languages such as Python, Scala, and SQL for data engineering tasks.
- Experience in handling large datasets and working with distributed computing frameworks (e.g., Spark, Databricks).
- Familiarity with data warehousing concepts, star/snowflake schemas, and cloud-based storage solutions.
- Azure Data Engineer, Azure Solutions Architect, or related cloud certifications are highly desirable.
- Demonstrated ability to lead data engineering teams, guide technical projects, and mentor junior engineers.
- Excellent analytical and troubleshooting skills, with a focus on optimizing performance and resolving data pipeline issues.
- Strong interpersonal and communication skills, with the ability to effectively communicate technical concepts to both technical and non-technical stakeholders.
Tech Mahindra is an Equal Employment Opportunity employer. We promote and support a diverse workforce at all levels of the company. All qualified applicants will receive consideration for employment without regard to race, religion, color, sex, age, national origin or disability. All applicants will be evaluated solely on the basis of their ability, competence, and performance of the essential functions of their positions with or without reasonable accommodations. Reasonable accommodations also are available in the hiring process for applicants with disabilities. Candidates can request a reasonable accommodation by contacting the company ADA Coordinator at ADA_Accomodations@TechMahindra.com.”
