● Open to Work

SHUBHAM SINGH

Senior LLM Engineer
Varanasi, UP, India

About Me

Senior LLM Engineer with 5+ years of hands-on experience building, fine-tuning, and shipping production-grade LLMs and agentic AI systems. Core expertise spans the full LLM lifecycle — pre-training data pipelines, SFT/RLHF post-training, RAG architecture, multi-agent orchestration, and systematic prompt engineering.

Skills & Technologies

LLM EngineeringHugging Face TransformersLoRA/PEFTLangChainLangGraphRAGFlowPineconeFAISS

Work Experience

LLM Pre/Post-Training Specialist & AI Evaluator (Freelance)

Upwork — Productive Playhouse / Large Programs
Jan 2020NOW

Curated and structured large-scale pre-training corpora for language models, maintaining 98%+ annotation accuracy across 30k+ examples. Delivered SFT datasets and preference pairs for LLM fine-tuning projects. Evaluated Hindi-language AI responses, achieving a 100% Job Success Score.

LLM Pre/Post-Training Specialist & AI Evaluator (Freelance)

Upwork — Productive Playhouse / Large Programs
Jan 2020NOW

Curated and structured large-scale pre-training corpora for language models, maintaining 98%+ annotation accuracy across 30k+ examples. Delivered SFT datasets and preference pairs for LLM fine-tuning projects. Evaluated Hindi-language AI responses, achieving a 100% Job Success Score.

Senior AI Data Trainer & Annotator

RWS Community / Outlier.ai / Welocalize / Sigma AI
Jun 2018Dec 2020

Annotated over 50,000 samples with 98%+ accuracy and led a sub-team of 6 annotators. Developed RLHF workflows for conversational AI and performed various NLP tasks including sentiment analysis and multi-turn prompt evaluation.

Projects

Agentic RAG System with LangGraph + RAGFlow

Built a stateful multi-agent RAG system using LangGraph for agent orchestration and RAGFlow for document ingestion and retrieval, achieving sub-400ms E2E latency and <4% hallucination rate.

PythonLangGraphRAGFlowLangChain+4

LLM Pre-Training Data Pipeline & SFT Fine-Tuning

Designed an end-to-end LLM pre-training data pipeline for a 7B-param domain LLM, followed by SFT using LoRA on SageMaker, reducing perplexity by 18% and hallucination rate by 42%.

PythonHugging FaceLoRA/PEFTAWS SageMaker+3

Multi-Agent LLM Orchestration Platform

Engineered a multi-agent framework using LangGraph stateful graphs and persistent Redis memory, achieving a 55% reduction in task completion time.

PythonLangGraphLangChain AgentsOpenAI Function Calling+4

Computer Vision Defect Detection Pipeline (Deep Learning)

Trained and deployed a YOLOv8-based defect detection model for industrial quality control, achieving 94.3% [email protected] and deployed as a real-time FastAPI microservice.

PythonPyTorchYOLOv8OpenCV+3

Automated LLM Evaluation & CI/CD Quality Gate

Created a real-time evaluation dashboard tracking hallucination rate and answer relevancy, integrated with GitHub Actions CI/CD to cut regression detection time from days to minutes.

PythonDeepEvalRagasStreamlit+3

Decibench — Voice Agent Benchmarking Framework

Developed an open-source benchmarking framework for voice AI agents, supporting automated multi-turn conversation simulation and latency profiling.

PythonFastAPIWebSocketsWhisper+3

NearestDoctor — AI-Powered Healthcare Search

Built a full-stack healthcare discovery platform with LLM-powered symptom-to-specialist matching and real-time doctor availability, achieving 99.5% uptime.

Next.jsReactSupabaseLangChain+3

Education

B.Tech in Computer Science & Engineering

Computer Science

Punjab Engineering College, Chandigarh2016 - 2020
SHUBHAM SINGH - Senior LLM Engineer | HiringAnt