Total Required Experience in Years: 10+ Years

Job Description:The client is seeking a senior-level AWS Data Engineer with strong Databricks expertise to design, build, and maintain scalable cloud-based data pipelines and health insurance applications. The role focuses on architecting data engineering solutions from scratch, implementing ETL/ELT processes, supporting enterprise integrations, and ensuring data quality and reliability. The ideal candidate should have strong AWS cloud expertise, Databricks experience, and the ability to collaborate across distributed Agile teams.

Key Responsibilities

Design, build, and maintain scalable data pipelines using AWS and Databricks
Implement ETL/ELT processes for loading and transforming data into data lakes and downstream systems
Ensure data quality, reliability, observability, and performance
Build and automate unit, integration, and performance testing
Integrate internal systems, vendors, mobile applications, and data marts
Support retail health insurance applications
Monitor pipelines and proactively resolve production issues
Participate in SAFe/Agile ceremonies and collaborate with cross-functional teams

Additional Responsibilities

Support migration of on-premise data pipelines to Databricks cloud environments
Work with Delta Lake, Apache Iceberg, and Apache Hudi technologies
Implement partitioning strategies and metadata management
Handle JSON/XML, document databases, and real-time ingestion processes
Apply Lakehouse architecture, dimensional modeling, and Data Vault concepts
Provide technical leadership and architecture recommendations

Required Skills

10+ years designing and architecting solutions from scratch
Strong AWS expertise: Glue, ECS/EKS, Fargate, Aurora, DocumentDB, S3, Lambda, SNS, IAM, ECR
Strong Databricks expertise: Delta Live Tables (DLT), Spark, SQL, Python/PySpark, Java, Databricks workflows or Apache Airflow
Strong knowledge of ETL/ELT, data integration, and data warehousing principles
Experience in DevOps practices, performance tuning, and design patterns
Familiarity with Tableau, Alation, and Collibra
Strong analytical, communication, and problem-solving skills

Qualifications & Certifications

Bachelors degree in Information Systems or related field
Advanced degree (MBA or equivalent) preferred
Equivalent experience accepted
AWS/Databricks certifications preferred but not mandatory

SIRITECH SOLUTIONS CORP

New Jobs

AWS Data Engineer with Databricks Experience