Snap’s Secret to Processing 10 Petabytes a Day: GPU-Accelerated Spark | NVIDIA AI Podcast Ep. 298

In short

In this episode of the NVIDIA AI Podcast, Noah Kravitz interviews Prudhvi Vatnala from Snap about how they accelerated their data processing pipeline using NVIDIA technology on Google Cloud. They discuss the challenges of processing massive amounts of data, the importance of AB testing, and the benefits of leveraging idle GPU capacity. Prudhvi highlights the significant cost savings and performance improvements achieved through their partnership with NVIDIA and Google Cloud, and the impact on Snap's future roadmaps for data and AI.

Key takeaways

• NVIDIA's Spark Rapids enables significant performance improvements in data processing with minimal code changes.
• Leveraging idle GPU capacity can lead to substantial cost savings and resource optimization.
• AB testing is crucial for ensuring new features add value and don't negatively impact user experience.
• Partnerships between technology companies can drive innovation and accelerate the development of new solutions.
• Building a bottom-up data platform can empower teams to leverage GPU capacity and NVIDIA libraries effectively.

Chapters

1. 00:00

   Introduction to the NVIDIA AI Podcast and Prudhvi Vatnala

   Noah Kravitz introduces Prudhvi Vatnala, Head of Engineering Platforms at Snap, to discuss data processing and how Snap accelerates its data pipeline.

2. 01:10

   Snap's Current Role and Camera-Centric Philosophy

   Prudhvi explains that Snapchat is a household name and Snap believes the camera is central to improving communication and lives in the digital world, focusing on augmented reality, AI, and visual communication, serving close to a billion monthly active users.

3. 02:28

   Accelerating Data Processing at Snap

   Prudhvi discusses the scale at which Snap operates, processing over 10 petabytes of data daily for experimentation, and the importance of meeting strict SLAs for developers and data scientists to act on results quickly. Accelerating data processing means flattening the scale curve rather than just adding more CPUs.

4. 03:48

   Experimentation at Snap: Product Philosophy and AB Testing

   Prudhvi explains that experimentation, safety, and privacy are core pillars of Snap's product development. AB testing is crucial for bringing statistical rigor to decision-making at scale, ensuring new features add value and don't regress user experience.

5. 06:15

   Integrating NVIDIA Tech: Spark Rapids and Google Cloud

   Prudhvi discusses how Snap integrated NVIDIA's Spark Rapids to speed up PySpark workloads, achieving significant performance improvements. Snap's stack is entirely on Google Cloud, and they benchmarked various job types to optimize performance.

6. 08:43

   Migrating to Kubernetes and Leveraging Idle GPU Capacity

   Prudhvi explains the process of migrating workloads to a Kubernetes-based Spark runtime on GKE to leverage idle online inference GPU capacity. This involved building a data platform from the ground up to ensure any team at Snap could leverage the capacity.

7. 10:55

   Accelerated Apache Spark Pipeline and Fallback Mechanisms

   Prudhvi discusses Snap's accelerated Apache Spark pipeline, split into daily and hourly cadences. They implemented fallback mechanisms to gracefully switch from GPUs to CPUs or DataProc clusters if resources were constrained, ensuring operational reliability.

8. 12:19

   Key Learnings and NVIDIA's Direction

   Prudhvi highlights the positive direction NVIDIA is heading with Spark Rapids, enabling zero-code changes. NVIDIA Ether also helped with Spark tuning, ensuring consistent tuning across different versions. He emphasizes the importance of image building and environment differences.

9. 13:48

   Impact of Partnerships and Future Roadmaps

   Prudhvi emphasizes the phenomenal three-way partnership between Snap, NVIDIA, and Google Cloud, leading to a 76% reduction in job costs and significant improvements in core requirements and memory footprint. This partnership has shaped Snap's roadmaps for data and AI.

10. 15:44

    Snap's Evolution and Impact on Visual Communication

    Prudhvi reflects on Snap's evolution from camera messaging to AI-driven stories and AR lenses, highlighting its innovative culture and impact on visual communication. He encourages listeners to try Snapchat and visit Snap's engineering blog for more information.

Get the next The AI Podcast recap

A short recap of every new episode, by email. Free for up to 5 shows.