We are looking for a highly skilled
Senior DevOps Engineer with deep expertise in Azure cloud and strong hands-on experience with Databricks. The role involves designing, implementing, and optimizing DevOps processes, ensuring seamless CI/CD, automation, and infrastructure scalability to support data and AI-driven projects.
Key Responsibilities
- Design, implement, and maintain CI/CD pipelines for applications and data platforms.
- Deploy, manage, and optimize Azure infrastructure using IaC (Terraform/ARM/Bicep).
- Automate provisioning, monitoring, and scaling of Databricks clusters and jobs.
- Collaborate with data engineering and ML teams to streamline MLOps and DataOps workflows.
- Manage Azure DevOps/GitHub Actions/Jenkins pipelines for code deployments.
- Implement logging, monitoring, and alerting solutions (App Insights, Log Analytics, Prometheus, Grafana, etc.).
- Drive cost optimization and performance tuning across cloud and Databricks environments.
- Ensure best practices in security, compliance, and governance within Azure and Databricks.
- Mentor junior engineers and actively contribute to DevOps strategy and roadmap.
Key Skills
- 8+ years of experience in DevOps, Cloud, and Infrastructure automation.
- Strong hands-on expertise in Microsoft Azure (AKS, ADF, Storage, Key Vault, Networking, Security, etc.).
- Proven experience in Databricks administration and automation (cluster management, job orchestration, integration with Azure services).
- Proficiency with Infrastructure as Code (Terraform, ARM, Bicep).
- Strong scripting skills (Python, PowerShell, Bash).
- Solid experience with CI/CD tools (Azure DevOps, GitHub Actions, Jenkins).
- Expertise in Docker, Kubernetes (AKS preferred) for containerized workloads.
- Familiarity with monitoring & logging tools (Azure Monitor, ELK, Prometheus, Grafana).
- Strong understanding of networking, security, and compliance practices in Azure.
