Seattle-based Algorithmia has announced Insights, a solution for monitoring the performance of machine learning models.
Algorithmia specialises in artificial intelligence operations and management. The company is backed by Google LLC and focuses on simplifying AI projects for enterprises just getting started.
Diego Oppenheimer, CEO of Algorithmia, says:
“Organisations have specific needs when it comes to ML model monitoring and reporting.
For example, they are concerned with compliance as it pertains to external and internal regulations, model performance for improvement of business outcomes, and reducing the risk of model failure.
Algorithmia Insights helps users overcome these issues while making it easier to monitor model performance in the context of other operational metrics and variables.”
Insights aims to help enterprises to monitor the performance of their machine learning models. Many organisations currently don’t have that ability, or use a complex variety of tools and/or manual processes.
Operational metrics like execution time and request identification are combined with user-defined metrics such as confidence and accuracy to identify data skews, negative feedback loops, and model drift.
Model drift, in layman’s terms, is the degradation of a model’s prediction power due to changes in the environment—which subsequently impacts the relationship between variables. A far more detailed explanation can be found here for those interested.
Algorithmia teamed up with monitoring service Datadog to allow customers to stream operational – as well as user-defined inference metrics – from Algorithmia, to Kafka, and then into Datadog.
Ilan Rabinovitch, Vice President of Product and Community at Datadog, comments:
“ML models are at the heart of today’s business. Understanding how they perform both statistically and operationally is key to success.
By combining the findings of Algorithmia Insights and Datadog’s deep visibility into code and integration, our mutual customers can drive more accurate and performant outcomes from their ML models.”
Through integration with Datadog and its Metrics API, customers can measure and monitor their ML models to immediately detect data drift, model drift, and model bias.
(Photo by Chris Liverani on Unsplash)
