Abhishek K.

About Me

Abhishek is a Senior Data Engineer with 6+ years of work experience in advanced analytics and data engineering with a focus on building scalable data pipelines, data lakes/warehouses, data model & visualization, and cloud-based analytics infrastructure. He is proficient in cloud-based data engineering, specializing in Azure, AWS, and Databricks, having worked on end-to-end data platforms, ETL pipelines, real-time streaming, and data lake architecture. Abhishek is currently leading the design and implementation of end-to-end data pipelines and architecting data ingestion techniques for Canada's PHSA.

AI, ML & LLM

Database

SQL Data Build Tool (dbt)

DevOps

Workflow

Other

Work history

Provincial Health Services Authority
Provincial Health Services Authority
Senior Specialist (Data Engineering & Business Intelligence)
2023 - Present (2 years)
Vancouver, Canada
  • Leading the design and implementation of end-to-end data pipelines, automating the extraction, transformation, and delivery of critical healthcare data using Azure Data Factory and PySpark for efficient data handling and processing.

  • Architecting data ingestion techniques (batch) to consolidate data from multiple sources into Azure Synapse Data Warehouse, improving data accessibility and reducing manual intervention.

  • Guiding and mentoring 6 junior analysts on best practices in data modeling, ETL/ELT processes, and cloud infrastructure, fostering a culture of continuous learning and improvement.

  • Drove the migration from legacy reporting systems (SSRS) to Power BI dashboards, optimizing resource utilization and improving decision-making processes by 10%.

  • Implementing an error logging system to ensure data integrity and enable troubleshooting of data pipelines, improving reliability.

  • Led the preparation of high-impact reports for the BC Ministry of Health, aggregating data across multiple systems to deliver accurate and timely insights.

  • Optimized SQL queries, reducing execution time by 25% and enhancing the performance of complex queries used for reporting.

Data EngineeringBusiness IntelligenceData pipelinesData ModelingData VisualizationAzure Data Studio Azure Data FactoryAzure Synapse Analytics Azure DatabricksPower BI Microsoft Fabric AWS S3AWS GlueAWS Lambda AWS RedshiftTerraformSQL Server Management Studio (SSMS) PysparkData Preprocessing Data WarehouseData Integration (ELT/ETL)Cloud InfrastructureData Aggregation SQLHealthcare
GyanSys
GyanSys
Data Engineering Consultant
2022 - 2023 (1 year)
Vancouver, Canada
  • Built data pipelines to ingest real-time data from SAP DW using Azure Synapse using SAP OData API, 100% of data migrated.

  • Converted SAP ABAP code to PySpark code to perform data transformation in Azure Databricks, completing 80% of conversion.

  • Led requirements gathering and process map creation efforts for client’s microservices architecture for cart management.

  • Designed a Delta data model for data batch processing in AWS S3 and AWS Glue for a multimillion-dollar project.

  • Built Power BI dashboards for resource optimization and client billable hours and optimized the reporting efficiency for key insight.

  • Led data reconciliation and migration of ETL to new org, migrating 95% of data.

  • Routinely updated data dictionaries and ETL for client’s historical data to maintain data quality.

  • Coordinated a team of 7+ developers for Fortune 500 client’s product development, adding 2 feature enhancements to the production code environment.

  • Managed Agile project delivery for an enterprise-level account, with 100% deployment on Azure Cloud within stipulated time and budget.

Data EngineeringOdataSAPData WarehouseData pipelinesAzure Synapse PysparkABAP Data Transformation Azure DatabricksMicroservices Architecture Data ModelingAzure Delta Lake DevelopmentBatch ProcessingAWS S3AWS GluePower BI Dashboards ETLData MigrationMicrosoft Azure Cloud Server
Canada Drives
Canada Drives
Analytics Contractor
2021 - 2021
Vancouver, Canada
  • Built a sales forecasting model using the ARIMA model in R to predict future sales with a model accuracy of 83%.

  • Built dbt models to transform raw data into structured datasets, optimizing data pipelines for analytics and forecasting.

  • Analyzed data quality issues on merging data from external sources and built an automated model with 95% accuracy to report errors.

  • Built SQL queries in Snowflake to assess, clean, validate, and analyze large datasets to support the forecasting model and various in-house analysis.

AnalyticsForecastingARIMA Models Data Build Tool (dbt) Sales Forecasting Datasets Data pipelinesData Analytics Data Quality Analysis SQLData Queries SnowflakeData Visualization
Atom Motors
Atom Motors
Market Analyst
2018 - 2020 (2 years)
Delhi, India
  • Spearheaded the home automation segment in the company portfolio based on middle-income group real estate spending pattern.

  • Improved reporting efficiency by 15% through dashboard redesign and modeling using Tableau for the managerial team.

  • Analyzed large datasets for customer segmentation and product recommendation for the electric vehicle market.

Education

Academy Accreditation - Generative AI Fundamentals (Expires in Aug 2025)
Academy Accreditation - Generative AI Fundamentals (Expires in Aug 2025)
Databricks
2023 - 2023
AWS Partner: Technical Accredited
AWS Partner: Technical Accredited
AWS
2022 - 2022
Master's Degree, Business Analytics
Master's Degree, Business Analytics
UBC Sauder School of Business - Canada
2021 - 2022 (1 year)
Data Science for Business | Big Data Fundamentals with PySpark | Understanding Cloud Computing
Data Science for Business | Big Data Fundamentals with PySpark | Understanding Cloud Computing
DataCamp
2020 - 2020