As a research assistant in the Machine Learning, Perception, and Cognition Lab at UCSD, I worked under the supervision of PhD students and Professor Zhouwen Tu. My role was that of a research engineer, focused on making experiments run reliably rather than driving research direction. I implemented and adapted PyTorch-based training pipelines from published papers, validated results within ±3% of reported benchmarks, and built experiment tracking and pipeline automation to reduce manual overhead across training, evaluation, and validation.

Reproducing State-of-the-Art Results

I reproduced results from papers including Re-Confusion, PixelSplat, and NeRF, achieving results within ±3% of reported benchmarks. This required careful control of hyperparameters, preprocessing steps, and randomness. ML training has randomness at multiple levels, including weight initialization, data shuffling, dropout, and data augmentation, so fixing random seeds and running multiple seeds to understand variance was essential. Papers often report the best of several runs rather than a typical result, and undocumented details like learning rate schedule specifics, warmup steps, and dataset preprocessing differences can all compound into larger deviations.

Profiling Memory and Improving Training Efficiency

I SSHed into UCSD’s GPU servers and used nvidia-smi to monitor real-time GPU utilization. I noticed the hardware was being underutilized, with GPU utilization sitting around 60 to 70% between batches, meaning memory was not the bottleneck but throughput was. I wrote a Python sweep script that systematically tried increasing batch sizes, measured throughput in samples per second and GPU utilization at each setting, and automatically stopped when it hit an OOM error. The output identified exactly where utilization was high and throughput had plateaued.

In Gaussian Splatting, batching works differently from standard models. Rather than batching samples in the traditional sense, you iterate over training images. The efficiency gains were more about GPU utilization per iteration and pipeline throughput than tuning a single batch size number. The tradeoffs are well understood: larger batches improve utilization and throughput but gradients become less noisy, which can hurt generalization. Smaller batches produce noisier gradients that act as a form of regularization. Memory is the hard constraint, as batch size is ultimately bounded by GPU VRAM.

Stabilizing Gradient Behavior

During batch size sweeps away from the paper’s exact configuration, loss would spike erratically mid-run and explode, forcing full restarts from scratch. This was the exploding gradient problem, made worse by running on different hardware. I added gradient clipping, which caps the gradient norm before the optimizer step, preserving the update direction but preventing dangerously large updates. When increasing batch size, I also scaled the learning rate accordingly using the linear scaling rule and warmed up the learning rate at the start to avoid early instability. Together, gradient clipping and batch size tuning produced a ~25% training efficiency improvement.

The broader toolkit for gradient stability includes learning rate warmup schedules and normalization layers. BatchNorm normalizes across the batch dimension and works well for vision models but breaks down with small batches. LayerNorm normalizes per sample, which is why it is standard in transformers where batch sizes can be small and sequences are variable length.

Novel Research Development

I collaborated with the lab team on developing novel deep learning methods, with the goal of publication in top conferences.