Distributed content delivery is critical to scalable systems like YouTube, especially when reliability and fault tolerance are required. TritonTube simulates this at a systems level, allowing videos to be uploaded, segmented using MPEG-DASH, and distributed across a network of storage nodes using consistent hashing. As part of a networking course, I built this application from the ground up in Go and achieving full functionality.

Approach

The system consists of three components: a web server, a set of storage servers, and an admin CLI. I implemented a custom consistent hashing algorithm to assign each video segment to a specific node in a distributed storage ring. Each file is deterministically mapped using SHA-256 to maintain a balanced load across nodes. The system supports dynamic resizing—when storage servers are added or removed, video files are automatically migrated to the appropriate new node to preserve the consistency of the ring.

Implementation

  • Implemented NetworkVideoContentService, which uses consistent hashing to determine the correct storage node for reading and writing files.
  • Developed gRPC-based communication between the web server and storage servers using a protocol I designed in .proto files.
  • Extended the web server to support dynamic commands via an admin gRPC interface, enabling file migration and live updates to the hash ring.
  • Used SQLite to manage video metadata and ffmpeg to convert uploaded videos into streamable MPEG-DASH formats.
  • Wrote file migration logic that ensures each video exists on exactly one node, even after topology changes.

Key Takeaways

  • I learned how to implement consistent hashing from scratch and apply it to real-world data partitioning problems.
  • Writing gRPC services in Go helped me understand how to design scalable, typed APIs for distributed services.
  • Debugging issues related to node state, file synchronization, and fault tolerance improved my systems programming skills.
  • I understand and know how to manage cluster membership, dynamic scaling, and maintain system availability during server failures.

This project deepened my understanding of distributed systems and taught me to design reliable and scalable backend architectures using Go. TritonTube remains functional even when servers crash or rejoin, demonstrating resilience through careful design of hashing, migration, and service coordination.

Please contact me to view the code written for this project.