Technical Architect

Responsibilities:

• Enabling end-to-end data ingestion and transformation pipeline for project application data

• Delivering business-ready datasets to meet reporting and operational requirements post-application shutdown

• Supporting or establishing medallion data architecture (bronze, silver, gold layers)

• Migrating risk associated with data usability and reporting readiness, which is identified as the highest risk area in the program

• Designing and delivery end-to-end data products in Microsoft Fabric using repeatable patterns that support business flows

• Builds and optimizes Fabric Data Factory pipelines and notebook-based, Spark Job based transformations to populate OneLake/Lakehouse and/or Warehouse layers and downstream data marts.

• Defines technical requirements and delivers solutions that meet service level agreements through efficient modeling, partitioning, caching, and query optimization across Fabric compute engines.

• Implements monitoring and observability for pipelines and models (refreshes, failures, data drift) and proactively resolves performance, reliability, and cost and/or capacity issues.

• Designs, integrates, and documents technical components (data contracts, lineage, definitions) for seamless data extraction, governance, and analysis.

• Presents insights on data and its quality via presentations, and visualizations.

• Defines and develops automation within Microsoft Fabric.

• Supports the corporation’s broader technical and operational requirements by adopting best practices in our data systems.

• Works in a team environment, interacting with multiple groups daily, within the department, as well as throughout the corporation.

• Designing and implementing transformations across, bronze, silver, gold layers

• Performing:

• Data cleansing and normalization

• Schema interpretation and mapping (including XML-based data structures)

• Transformation to business-aligned data models

• Enabling creation of analytics-ready and reporting-ready datasets aligned with business requirements

• Ingesting extracted application data into the enterprise data platform bronze layer as the authoritative raw dataset

• Supporting ingestion of Policy data, Billing data, Insights and related datasets

• Ensuring data is properly structured for downstream processing

•  Ensuring transformed data supports day-one operational reporting needs post-application shutdown and validation of expected vs. actual outputs and metrics

• Collaborating with business stakeholders to align on required attributes and outputs and support reconciliation and validation processes

• Contributing to or lead implementation of medallion architecture (Fabric / enterprise data platform)

• Supporting decision path depending on program direction, enhance existing EDP pipelines, or build required transformation architecture within project scope

• Ensuring scalability and maintainability of data pipelines

• Developing and maintaining pipelines for Data ingestion, Data transformation and Data movement across layers

• Ensuring performance, scalability, and reliability of data workflows

• Assisting in validating completeness and integrity of ingested data and supporting gap identification and remediation where required

• Collaborating on resolving discrepancies between source and target data

Qualifications

• Deep hands-on expertise with Microsoft Fabric workloads: Data Factory (pipelines), Lakehouse, Warehouse, Real-Time Analytics, Semantic Models, and (where applicable) Data Activator.

• Strong understanding of OneLake concepts, shortcuts, and Lakehouse architecture; ability to design scalable medallion-style (bronze/silver/gold) data models and performant analytics solutions.

• Strong knowledge of Python and Spark; ability to build and optimize notebook jobs in Fabric or Databricks for ELT/ETL, data quality, and advanced transformations.

• Working knowledge of governance and administration in Fabric: workspaces, domains, sensitivity labels, RLS/OLS, and capacity management understood.

• Knowledge of analytics engineering practices: metric definitions, semantic layer design, data quality expectations, and documentation for trusted self-service BI.

• Strong Power BI expertise, including Semantic Models, DAX fundamentals, and performance tuning.

• Experience ingesting batch and streaming/event data (e.g. Event Hub, Kafka, logs) and using it for analytics using Fabric pipelines, notebooks, and Real-Time Analytics.

• Experience in performing root cause analysis on data pipelines, models, and reports to answer business questions, improve reliability, and reduce time-to-insight.

• Working knowledge of DevOps/DataOps and Git; experience with Azure DevOps deployment pipelines, CI/CD automation, and environment promotion.

• Working knowledge of Azure fundamentals (identity, networking, security) and Entra ID; familiarity with private endpoints and secure data access patterns.

• Experience integrating enterprise data sources (SQL Server, APIs, files, SaaS) and designing robust ingestion patterns (incremental loads, CDC, scheduling, monitoring).

• Minimum of 4+ years of advanced Python development (performance tuning) and strong understanding of data warehousing and data Lakehouse concepts.

• Fabric DP-700 (Microsoft Certified: Fabric Data Engineer Associate) completion.