Developing Feast, the Leading Open Source Feature Store, with Willem Pienaar (Gojek, Tecton)

Willem Pienaar is the co-creator of Feast, the leading open source feature store, which he leads the development of as a tech lead at Tecton. Previously, he led the ML platform team at Gojek, a super-app in Southeast Asia. Learn more: https://twitter.com/willpienaar https://feast.dev/ Every Thursday I send out the most useful things I’ve learned, curated specifically for the busy machine learning engineer. Sign up here: https://www.cyou.ai/newsletter Follow Charlie on Twitter: https://twitter.com/CharlieYouAI Subscribe to ML Engineered: https://mlengineered.com/listen Comments? Questions? Submit them here: http://bit.ly/mle-survey Take the Giving What We Can Pledge: https://www.givingwhatwecan.org/ Timestamps: 02:15 How Willem got started in computer science 03:40 Paying for college by starting an ISP 05:25 Willem's experience creating Gojek's ML platform 21:45 Issues faced that led to the creation of Feast 26:45 Lessons learned building Feast 33:45 Integrating Feast with data quality monitoring tools 40:10 What it looks like for a team to adopt Feast 44:20 Feast's current integrations and future roadmap 46:05 How a data scientist would use Feast when creating a model 49:40 How the feature store pattern handles DAGs of models 52:00 Priorities for a startup's data infrastructure 55:00 Integrating with Amundsen, Lyft's data catalog 57:15 The evolution of data and MLOps tool standards for interoperability 01:01:35 Other tools in the modern data stack 01:04:30 The interplay between open and closed source offerings Links: Feast's Github Gojek Data Science Blog Data Build Tool (DBT) Tensorflow Data Validation (TFDV) A State of Feast Google BigQuery Lyft Amundsen Cortex Kubeflow MLFlow