Our client is seeking a Cloud Data Engineer with experience in architecting, designing, and developing enterprise-scale data warehouse and analytics systems.
The ideal candidate will have deep expertise in building cloud-based data platforms using Azure technologies, SQL Server products, and DevOps processes. You will be responsible for developing data pipelines, implementing security models, and optimizing cloud infrastructure solutions for large-scale data environments.
Duration: 12 months
Location: Hybrid 2d/week NYC
Responsibilities:
- Architect, design, and develop enterprise-scale data warehouse systems using both cloud (Azure) and on-premise solutions.
- Build end-to-end data warehouse and analytical systems using SQL Server, Analysis Services, Power BI, and related tools.
- Develop and implement data platforms using Azure SQL Database, Azure Data Factory, Azure Storage, Azure Pipelines, and Azure IaaS/PaaS services.
- Lead the design and development of data pipelines using API ingestion and streaming ingestion methods.
- Work with Azure services such as Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service to deliver scalable data solutions.
- Apply best practices in data modeling using Kimball and Inmon methodologies.
- Implement and manage DevOps processes, including CI/CD pipelines, and Infrastructure as Code for efficient data platform delivery.
- Develop NoSQL solutions using Azure Cosmos DB and integrate with the data platform.
- Work with cloud infrastructure offerings from both Azure and AWS, ensuring optimal deployment and management of resources.
- Utilize Azure Databricks, Azure IoT, Azure HDInsight + Spark, Azure Stream Analytics, and Power BI to support advanced analytics and real-time data processing.
- Develop and maintain security models to protect sensitive data and ensure compliance with relevant regulations.
- Leverage scripting languages such as C#, PowerShell, and Python to automate data processes and optimize performance.
- Collaborate with cross-functional teams to develop business intelligence solutions, including working knowledge of Microsoft BI Stack (SSRS/SSAS/SSIS).
Requirements:
- 12+ years of experience in architecting, designing, and developing data warehouse systems (SQL Server-based) in both cloud and on-premise environments.
- Expertise in Azure cloud services, including Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service.
- Experience developing data pipelines and integrating data using API and streaming methods.
- Strong understanding of DevOps processes (CI/CD) and Infrastructure as Code.
- Expertise in NoSQL solutions using Azure Cosmos DB.
- Thorough knowledge of both Azure and AWS cloud infrastructure.
- Experience with Azure Databricks, Azure IoT, Azure HDInsight + Spark, and Azure Stream Analytics.
- Knowledge of Microsoft BI Stack, including SSRS, SSAS (Tabular with DAX and OLAP with MDX), and SSIS.
- Proficiency in C#, PowerShell scripting, and Python for automation and data management.
- Experience with security model development for cloud data solutions.
- Knowledge of data warehouse modeling principles (Kimball, Inmon).
- Hands-on experience with Python, C#, and PowerShell scripting.
