Principal Data Engineer

Job Title: Principal Data Engineer

Location: Toronto, ON (Onsite)
    
Estimated Duration: Fulltime

Principal Data Engineer
Lead the design, development, and implementation of scalable data pipelines on Azure Databricks and ADLS. This role requires expertise in Azure, PySpark, Azure SQL, CDC, and ADF to enable efficient and near real-time data movement from SQL Data Warehouses to an Enterprise Data Lake. The ideal candidate has experience architecting incremental data ingestion solutions, optimizing Databricks pipelines, and enforcing data governance best practices.

Responsibilities

• Architect and implement an end-to-end data platform on Azure Data Lake and Databricks
• Design incremental ingestion pipelines using ADF, CDC, and Databricks Auto Loader
• Build and optimize PySpark workflows for batch and streaming data processing
• Manage Delta Lake architecture (Bronze, Silver, Gold layers)
• Lead performance tuning and cost optimization of Databricks and Azure SQL workloads
• Enforce security, access controls, and data governance practices
• Collaborate across business, data science, and BI teams
• Mentor engineers and set best practices across the data team

Requirements

• 10+ years in data engineering with a strong Azure cloud focus
• Deep expertise in Databricks, PySpark, and Delta Lake
• Proven experience with ADF, Azure SQL, and CDC methods
• Skilled in both batch and streaming architectures
• Strong SQL capabilities and performance tuning
• Knowledge of cloud security, governance, and compliance
• Experience leading teams in Agile environments

Preferred Qualifications

• Terraform experience (nice to have)
• Certifications in Azure Data Engineering, Databricks, or Terraform
• Background in high-scale enterprise data environments
• Familiarity with Power BI integration with Databricks/ADLS