Sr. DevOps Engineer

– Develop and manage CI/CD pipelines for Databricks notebooks,
libraries, and Azure Data Factory (ADF) workflows.
– Implement Infrastructure-as-Code (IaC) using Terraform or Bicep to
provision and manage Azure resources.
– Automate Databricks job and notebook deployments using CLI,
REST APIs, and integration with Git-based workflows.
– Establish branching and merging strategies (e.g., GitFlow, trunk-
based development) and enforce code review processes.
– Integrate automated testing, data validation, and quality gates into
CI/CD pipelines.
– Manage release processes for Databricks jobs, ADF pipelines, and
tabular models in Azure Analysis Services.
– Monitor pipeline runs, investigate failures, and resolve issues in
collaboration with engineering teams.
– Secure CI/CD processes by leveraging Key Vault for secrets
management, service principals, and role-based access controls.
– Document pipeline processes and provide training to development
and analytics teams.

Skills

Proficiency in Azure DevOps or GitHub Actions, with hands-
on experience automating CI/CD pipelines.
– Strong knowledge of Databricks (clusters, jobs, CLI, APIs)
and Azure Data Factory.
Experience with Terraform, ARM templates, or Bicep for
cloud automation.