Cloud Data Engineer ROCGJP00027944
A leading biotechnology company is seeking a highly skilled and motivated Cloud Data Engineer to join our growing data team. The ideal candidate will have a strong background in developing data pipelines, implementing data models, and executing best practices for developing data products. This role requires expertise in AWS, GitLab CI/CD, dbt, Snowflake, SQL, Python, Git, and DevOps, as well as hands-on experience with orchestrators such as Airflow and AutoMateNow, and data governance technologies like Monte Carlo and Collibra.
Cloud Data Engineer Pay And Benefits
- Hourly pay: $70-$80/hr (pay varies based on candidate's experience)
- Worksite: Leading biotechnology company (South San Francisco, CA 94080)
- W2 Employment, Group Medical, Dental, Vision, Life, Retirement Savings Program, PSL
- 40 hours/week, 7 Month Assignment
Cloud Data Engineer Responsibilities
- Develop Data Pipelines: Design, develop, and maintain robust, scalable, and efficient ETL/ELT pipelines to support diverse data sources and large-scale data processing.
- Data Modeling: Create and maintain data models and data architecture to ensure data integrity and optimum performance.
- Best Practices Implementation: Apply industry best practices in data warehousing, data governance, and data lifecycle management.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to gather requirements and deliver high-quality data products.
- Automation and CI/CD: Implement CI/CD pipelines using GitLab for automated testing, deployment, and integration of data workflows.
- Database Management: Manage and optimize Snowflake databases for performance, scalability, and cost-effectiveness.
- Coding and Scripting: Write efficient SQL queries and Python scripts to extract, load, and transform data.
- Version Control: Utilize Git for version control, ensuring traceable and manageable code changes.
- Orchestration: Utilize orchestration tools like Airflow and AutoMateNow to manage and automate data workflows.
- Data Governance: Implement and manage data governance and data observability solutions using technologies like Monte Carlo and Collibra.
- Monitoring and Optimization: Monitor data pipeline performance and troubleshoot issues to ensure reliability and efficiency.
- Documentation: Maintain comprehensive documentation for data pipelines, models, and processes.
Cloud Data Engineer Qualifications
- 5+ years of experience in data engineering or a similar role.
- Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
- AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect.
- Proven experience with AWS cloud services related to data processing, such as S3, Redshift, Lambda, Glue, and Data Pipeline.
- Proficiency in designing and implementing CI/CD pipelines with GitLab.
- Experience with dbt (data build tool) for transforming data within the warehouse.
- Experience with orchestrators like Airflow and AutoMateNow.
- Experience with real-time data processing and streaming technologies (e.g., Kafka, Kinesis).
- Experience in a regulated industry
- Experience in a manufacturing environment
- Strong knowledge of Snowflake with hands-on experience in data warehousing solutions.
- Expertise in SQL for data querying, manipulation, and optimization.
- Proficiency in Python for scripting and automation tasks.
- Familiarity with DevOps practices and tools.
- Knowledge of data governance and observability tools such as Monte Carlo and Collibra.
- Strong working knowledge of Git for version control.
- Strong problem-solving and analytical skills.
- Excellent communication skills, both written and verbal.
- Ability to work collaboratively in a fast-paced, dynamic environment.
- High attention to detail and commitment to producing high-quality work.
Compensation: From $70.00 to $80.00 per hour
