Hellow guys, Welcome to my website, and you are watching Everyday Probabilistic Data Structures for Humans. and this vIdeo is uploaded by Databricks at 2020-07-28T08:13:59-07:00. We are pramote this video only for entertainment and educational perpose only. So, I hop you like our website.
| Name | Everyday Probabilistic Data Structures for Humans |
| Video Uploader | Video From Databricks |
| Upload Date | This Video Uploaded At 28-07-2020 15:13:59 |
| Video Discription | Processing large amounts of data for analytical or business cases is a daily occurrence for Apache Spark users. Cost, Latency and Accuracy are 3 sides of a triangle a product owner has to trade off. When dealing with TBs of data a day and PBs of data overall, even small efficiencies have a major impact on the bottom line. This talk is going to talk about practical application of the following 4 data-structures that will help design an efficient large scale data pipeline while keeping costs at check. 1. Bloom Filters 2. Hyper Log Log 3. Count-Min Sketches 4. T-digests (Bonus) We will take the fictional example of an eCommerce company Rainforest Inc and try to answer the business questions with our PDT and Apache Spark and and not do any SQL for this. 1. Has User John seen an Ad for this product yet? 2. How many unique users bought Items A , B and C 3. Who are the top Sellers today? 4. Whats the 90th percentile of the cart Prices? (Bonus) We will dive into how each of these data structures are calculated for Rainforest Inc and see what operations and libraries will help us achieve our results. The session will simulate a TB of data in a notebook (streaming) and will have code samples showing effective utilizations of the techniques described to answer the business questions listed above. For the implementation part we will implement the functions as Structured Streaming Scala components and Serialize the results to be queried separately to answer our questions. We would also present the cost and latency efficiencies achieved at the Adobe Experience Platform running at PB Scale by utilizing these techniques to showcase that it goes beyond theory. About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business. Read more here: https://databricks.com/product/unifie... Connect with us: Website: https://databricks.com Facebook: https://www.facebook.com/databricksinc Twitter: https://twitter.com/databricks LinkedIn: https://www.linkedin.com/company/databricks Instagram: https://www.instagram.com/databricksinc/ Databricks is proud to announce that Gartner has named us a Leader in both the 2021 Magic Quadrant for Cloud Database Management Systems and the 2021 Magic Quadrant for Data Science and Machine Learning Platforms. Download the reports here. https://databricks.com/databricks-named-leader-by-gartner |
| Category | Science & Technology |
| Tags | Databricks |
