Machine Learning Operations

Recommendation systems are ubiquitous in consumer and enterprise applications alike for retail, media, advertising, and general search use cases, among many others. These systems incorporate multiple models and rule-based components; they also process enormous amounts of data and can have tremendous economic impact.

Because recommendation systems are often deployed in highly dynamic environments, retrieval and scoring models may need to be retrained multiple times per day, and data scientists will often need to figure out how to tailor their performance to maximize business metrics. This is even more complicated because the overall system will depend on the interaction between trained models (for finding relevant suggestions and scoring a filtered list) and business rules (for filtering irrelevant suggestions and ordering the final results).  

Data scientists thus need a flexible environment to design and track experiments, test hypotheses, and define metrics to monitor in production. Machine learning engineers need tooling to define, execute, and monitor training pipelines, as well as to monitor the performance of the overall system.