Create your free profile

About

Azure Data Engineer with 4+ years of experience building scalable ETL/ELT pipelines for batch and streaming workloads using Azure Data Factory, Azure Databricks, PySpark, Python, SQL, ADLS Gen2, Delta Lake, Kafka, and Azure Synapse Analytics. Skilled in Medallion Architecture, CDC and incremental loading, dimensional data modeling, data quality, Spark performance tuning, and production support across banking, healthcare, and asset management domains.

Experience4

  1. Software Engineer II

    Jan 2026 — Present

    Venus Universal · Bangalore

    Developed PySpark transformation jobs on Azure Databricks, converting 20+ complex SQL transformation scripts and processing approximately 5M records/day across Silver and Gold layers. Designed and orchestrated Azure Data Factory and Databricks pipelines for raw ingestion, Parquet storage, and Silver/Gold processing. Managed multi-stage ETL pipelines using Databricks Workflows and ADF triggers, including scheduling, dependencies, retries, and error handling. Implemented data validation, data-quality checks, exception handling, monitoring, and production issue resolution.

    • Converted 20+ complex SQL transformation scripts
    • Processed approximately 5M records/day

  2. System Engineer I

    Apr 2025 — Dec 2025

    Infosys Limited · Bengaluru

    Delivered end-to-end ETL/ELT pipelines using Azure Data Factory and Azure Databricks, owning 40+ tables/pipelines to ingest structured and semi-structured clinical and patient data into ADLS Gen2. Established a Delta Lake Medallion Architecture (Bronze/Silver/Gold) delivering reliable, auditable, analytics-ready datasets for clinical reporting and data science teams. Built incremental load and Change Data Capture (CDC) frameworks in PySpark to process high-volume clinical and transactional data efficiently. Applied data quality checks, schema validation, and reconciliation logic to ensure patient data accuracy and completeness across pipelines.

    • Owned 40+ tables/pipelines
    • Delivered reliable, auditable datasets for clinical reporting

  3. Software Engineer I, Data Engineering

    Aug 2023 — Sep 2024

    Jana Small Finance Bank

    Built Kafka-integrated Spark Structured Streaming pipelines for payment and loan transaction data, replacing nightly batch processing with approximately 10-minute processing latency. Developed and maintained 40+ SQL stored procedures, complex queries, CTEs, and window functions for data modeling, loan analytics, payment reconciliation, customer analysis, and MIS reporting. Set up pipeline monitoring and alerting for processing failures and invalid records and supported production issue resolution. Handled data validation, reconciliation, deduplication, and quality checks across core banking datasets.

    • Replaced nightly batch processing with approximately 10-minute processing latency
    • Developed and maintained 40+ SQL stored procedures

  4. Software Engineer, Data Engineering

    Feb 2022 — Apr 2023

    Zscaler Softech India Pvt. Ltd. (Contract) · Contract

    Built metadata-driven Azure Data Factory pipelines and developed PySpark transformations, aggregations, and business rules for healthcare data. Designed Delta Lake fact and dimension tables using data modeling principles for Power BI reporting through Azure Synapse Analytics and implemented data-quality tests and incremental loads.

Education1

  • Bachelor of Computer Applications (BCA)

    2019

    Bangalore University (VCDC College)

Certifications2

  • Microsoft Certified: Azure Data Engineer Associate (DP-203)

    2026

    Microsoft

  • Databricks Lakehouse Fundamentals

    2024

    Databricks