Position Overview: As a Technology Lead, you will be responsible for overseeing and guiding the development and maintenance of data pipelines and applications. You will leverage your expertise in Azure Data Factory, Azure Databricks, Python, and Azure DevOps to ensure the successful implementation of data solutions. The role requires at least 5 years of experience and involves managing technical aspects of data engineering projects while working closely with cross-functional teams.

Key Responsibilities:

Leadership and Project Management:

Lead and mentor a team of data engineers and developers, providing guidance and support on technical challenges.
Oversee the design, development, and deployment of data pipelines and applications using Azure Data Factory and Azure Databricks.
Collaborate with stakeholders to understand project requirements and translate them into technical solutions.

Data Pipeline Development:

Design, develop, and maintain scalable and efficient data pipelines using Azure Data Factory (ADF) and Azure Databricks.
Implement data integration solutions, including ETL/ELT processes, data transformation, and data movement.

Python Development:

Develop and maintain applications and scripts in Python to extract, manipulate, and analyze data.
Utilize Python libraries for data processing, including Pandas, NumPy, and others, to support data engineering tasks.

Azure DevOps Practices:

Apply Azure DevOps practices for continuous integration and continuous deployment (CI/CD) of data solutions.
Manage source control, automated testing, and deployment pipelines to ensure high-quality and reliable releases.

Technical Expertise:

Utilize expertise in Azure Data Factory and Azure Databricks to optimize data workflows and performance.
Develop Python scripts for various data extraction, transformation, and loading (ETL) tasks.

Collaboration and Communication:

Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to ensure alignment on project goals and deliverables.
Communicate technical concepts effectively to both technical and non-technical stakeholders.

Problem-Solving and Innovation:

Identify and address technical challenges and inefficiencies in data engineering processes.
Propose and implement innovative solutions to enhance data management and processing capabilities.

Mandate Skills and Qualifications:

Experience: Minimum of 5 years of work experience in data engineering or a related field.
Azure Data Factory: Proficient in designing and managing data pipelines using Azure Data Factory.
Azure Databricks: Experience in developing and maintaining data solutions with Azure Databricks.
Python Development: Strong experience in Python programming for data extraction, transformation, and application development.
Azure DevOps: Familiarity with Azure DevOps practices, including CI/CD pipelines, source control, and automated testing.
Technical Expertise: Proven track record of implementing data solutions and optimizing data workflows using Azure technologies.

eJAmerica

New Jobs

Infra and Devops

Technology Lead