Create your free profile

About

Staff AI/ML Systems and Distributed Foundation Infrastructure Engineer with over eight years of experience directing large-scale GPU training clusters, low-latency deep learning inference engines, and multimodal foundation model deployment serving over 65 million daily inference queries. Specializes in closing the gap between mathematical loss landscapes and bare-metal compute efficiency.

Experience3

  1. Staff AI/ML Infrastructure & Systems Engineer

    Jan 2023 — Present

    Apex Foundation AI Labs

    Orchestrated distributed pre-training and continuous parameter-efficient adaptation of 8B–70B parameter multimodal foundation models across a 1,024 NVIDIA H100 cluster. Re-architected the production serving tier handling 65M daily inference queries, migrating legacy single-stream containers to an asynchronous vLLM cluster. Designed an enterprise agentic execution runtime enabling LLMs to issue deterministic API calls and synthesize SQL.

    • Raised Model FLOPs Utilization from 36% to 58.4%
    • Compressed training runs from 48 to 28 days
    • Cut $14.2M in cloud compute burn
    • Reduced Time-to-First-Token by 64% (420ms to 14ms)
    • Sustained 128 tokens/sec per stream under concurrent multi-tenant load
    • Drove API execution reliability to 99.98% across 12,000 requests/minute
    • Cut agentic round-trip latency by 72%

  2. Senior AI/ML Systems Engineer — Model Alignment & Acceleration

    Feb 2021 — Dec 2022

    Hyperion Neural Platforms

    Directed the post-training alignment and domain adaptation platform for enterprise customer intelligence, establishing distributed DPO and QLoRA fine-tuning workflows across 256 A100 GPUs. Built automated synthetic dataset filtering and debiasing pipelines over multi-billion token corpora.

    • 41% improvement on MT-Bench domain tasks
    • 52% shorter fine-tuning runs
    • $6.8M in hardware savings
    • Eliminated redundant HBM-to-SRAM roundtrips
    • Shrunk production weight footprints from 140GB to 36GB
    • Allowed 70B models to serve on single 80GB SXM4 nodes
    • Improved context recall by 29% while holding end-to-end retrieval latency under 18ms

  3. Deep Learning Systems Engineer — Computer Vision & Inference Platforms

    Aug 2018 — Jan 2021

    Cerebro AI Systems

    Engineered scalable deep learning architectures for real-time video understanding and multimodal feature extraction across 14,000 concurrent video feeds. Built automated MLOps infrastructure on Kubeflow, Ray Train, and MLflow across hybrid on-premise and GCP GPU clusters.

    • Raised average GPU allocation efficiency from 28% to 84%
    • $3.4M in annualized infrastructure savings
    • Shrunk deployment iteration intervals from six weeks to 36 hours

Projects2

  • Triton-FlashDecode

    Architected and released an open-source OpenAI Triton kernel optimizing the autoregressive decoding phase for sequence lengths beyond 32k tokens.

  • DistriServe-Orchestrator

    Designed a distributed inference orchestration engine separating compute-bound prefill from memory-bandwidth-bound autoregressive decoding across dedicated GPU sub-clusters.

Education1

  • B.Sc. in Computer Science & Applied Mathematics

    2022

    Tribhuvan University