Practical workflow orchestration
15 Oct 2024
• 58 min
• EN
Workflow orchestration has always been a pain for data scientists, but this is exacerbated in these AI hype days by agentic workflows executing arbitrary (not pre-defined) workflows with a variety of failure modes. Adam from Prefect joins us to talk through their open source Python library for orchestration and visibility into python-based pipelines. Along the way, he introduces us to things like Marvin, their AI engineering framework, and ControlFlow, their agent workflow system.
From "Practical AI: Machine Learning, Data Science, LLM"
Comments
Add comment Feedback