Jason & Duncan discuss the results of Sama's ML Pulse Report, focusing on how to measure model effectiveness, how to increase confidence when moving models into production, and whether they believe generative AI is worth the hype.
Joining us today are our panelists, Duncan Curtis, SVP of AI products and technology at Sama, and Jason Corso, a professor of robotics, electrical engineering, and computer science at the University of Michigan. Jason is also the chief science officer at Voxel51, an AI software company specializing in developer tools for machine learning. We use today’s conversation to discuss the findings of the latest Machine Learning (ML) Pulse report, published each year by our friends at Sama. This year’s report focused on the role of generative AI by surveying thousands of practitioners in this space.Its findings include feedback on how respondents are measuring their model’s effectiveness, how confident they feel that their models will survive production, and whether they believe generative AI is worth the hype. Tuning in you’ll hear our panelists’ thoughts on key questions in the report and its findings, along with their suggested solutions for some of the biggest challenges faced by professionals in the AI space today. We also get into a bunch of fascinating topics like the opportunities presented by synthetic data, the latent space in language processing approaches, the iterative nature of model development, and much more. Be sure to tune in for all the latest insights on the ML Pulse Report!Key Points From This Episode:
Quotes:“It's really hard to know how well your model is going to do.” — Jason Corso “With debugging and detecting errors in your data, I would definitely say look at some of the tooling that can enable you to move more quickly and understand your data better.” — Duncan Curtis “Work with experts – there's no replacement for good experience when it comes to actually boxing in a problem, especially in AI.” — Jason Corso “It's not just about how your model performs. It's how your model performs when it's interacting with the end user.” — Duncan Curtis “Remember, what we do in this field, and in all fields really, is by humans, for humans, and with humans. And I think if you miss that idea you will not achieve – either your own potential, the group you're working with, or the tool.” — Jason Corso