19 - Google BigQuery / Dremel (CMU Advanced Databases / Spring 2023)
CMU Database Groupγ»2 minutes read
The class shifts focus to real-world systems from individual techniques, emphasizing how industry papers apply discussed methods and understanding fundamentals for decoding marketing claims. Topics like BigQuery, Spark SQL, and Snowflake are covered, highlighting the separation of compute from storage in database systems and challenges like lack of data statistics and adaptive query optimization.
Insights
- Industry papers often lag behind real system development, but understanding fundamental concepts and reading these papers can aid in deciphering marketing claims about new techniques, providing a diverse catalog of problem-solving approaches.
- Google's significant influence in the database realm, showcased through products like Dremel evolving into BigQuery, emphasizes the importance of open-source software for large systems, with unique features like in-memory Shuffle operations and in-situ data processing setting these systems apart in query processing efficiency and fault tolerance.
Get key ideas from YouTube videos. Itβs free
Recent questions
What is the main focus of the class?
Real systems over individual techniques from papers.