Devops (Data Platform)
Purpose of the job
The DataOps Engineer will play a critical role in ensuring the reliability, security and operational excellence of Collinson's modern data platform. As a key member of the Data & Analytics team, you will be responsible for automating platform operations, maintaining cloud infrastructure, supporting CI/CD pipelines and ensuring the platform remains secure, scalable and highly available.
In this hands-on role, you will work closely with Data Engineers, Security, Infrastructure and Product teams to support enterprise-wide data initiatives through automation, observability and platform standardisation.
Key Responsibilities
- Manage and maintain Snowflake, Kafka, AWS, MongoDB and supporting data platform infrastructure.
- Build and maintain CI/CD pipelines and deployment automation.
- Develop and maintain Infrastructure as Code using Terraform.
- Manage AWS networking including VPCs, VPC Peering, PrivateLink and secure private networking.
- Work with the Central Network team to implement and support enterprise networking solutions.
- Support Confluent Kafka platform operations and event-driven data integration patterns.
- Perform platform patching, upgrades, and lifecycle management.
- Ensure timely remediation of security vulnerabilities and compliance with security standards.
- Implement monitoring, alerting and observability using Datadog and platform monitoring tools.
- Support production incidents, root cause analysis, and operational improvements.
- Maintain platform documentation, operational runbooks, and engineering standards.
- Optimize platform performance, reliability, and cloud costs.
- Support platform governance, access management, and disaster recovery activities.
- Evaluate emerging technologies and recommend improvements to platform engineering practices.
- Own operational excellence of the data platform.
- Define and improve platform reliability standards.
- Drive automation and reduce manual operational effort.
- Improve developer experience for data engineering teams.
- Develop and maintain operational metrics, SLAs and SLOs to improve platform reliability and availability.
Knowledge, skills and experience required
Mandatory Technical Skills
DataOps & Platform Engineering
- AWS fundamentals
- Snowflake Administration
- MongoDB
- Confluent Kafka Platform
- Python
- SQL
- Git
- CI/CD pipelines
- Infrastructure as Code (Terraform)
- Data Observability and Monitoring (Datadog)
- Incident Management
- Production Support
- Security Patching
- Vulnerability Management
- Performance optimisation
- Cost optimisation
AWS Networking
- VPC
- VPC Peering
- AWS PrivateLink
- Route 53
- Security Groups
- IAM
- Private networking
- Central Network integration
Preferred Skills
- AWS Transit Gateway
- AWS Direct Connect / VPN concepts
- Datadog dashboards and alerting
- Airflow
- Platform Engineering / SRE practices
Nice to Have
- Generative AI fundamentals
- LLM integration patterns
- AI Agent frameworks
- Machine Learning platform awareness
- Real-time analytics architectures
