{"id":134,"date":"2021-07-16T16:10:27","date_gmt":"2021-07-16T16:10:27","guid":{"rendered":"https:\/\/felixaugenstein.com\/blog\/?p=134"},"modified":"2021-07-16T16:10:27","modified_gmt":"2021-07-16T16:10:27","slug":"monitor-your-deployed-machine-learning-models-with-watson-openscale","status":"publish","type":"post","link":"https:\/\/felixaugenstein.com\/blog\/monitor-your-deployed-machine-learning-models-with-watson-openscale\/","title":{"rendered":"Monitor your deployed machine learning models with Watson OpenScale"},"content":{"rendered":"\n<p>In this hands-on tutorial you will learn how Watson OpenScale can be used to monitor your deployed machine learning models<\/p>\n\n\n\n<p>If you don\u2019t have one already, please Sign up for an <a href=\"https:\/\/cloud.ibm.com\/registration\" target=\"_blank\" rel=\"noreferrer noopener\">IBM Cloud account<\/a>.<\/p>\n\n\n\n<p>This tutorial consists of 6 parts, you can start with part I or any other part, however, the necessary environment is set up in part I.<br><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial\" target=\"_blank\">Part I \u2013 data visualization, preparation, and transformation<\/a><br><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-ii\" target=\"_blank\">Part II \u2013 build and evaluate machine learning models by using AutoAI<\/a><br><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-iii\" target=\"_blank\">Part III \u2013 graphically build and evaluate machine learning models by using SPSS Modeler flow<\/a><br><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-iv\" target=\"_blank\">Part IV \u2013 set up and run Jupyter Notebooks to develop a machine learning model<\/a><br><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-v\" target=\"_blank\">Part V \u2013 deploy a local Python app to test your model<\/a><br><a rel=\"noreferrer noopener\" href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-vi\" target=\"_blank\">Part VI \u2013 monitor your model with OpenScale<\/a><\/p>\n\n\n\n<p>Deployed models can be <strong>biased<\/strong> or become <strong>less accurate over time<\/strong>, making precise predictions difficult. In order to <strong>trust<\/strong> machine learning models and artificial intelligence, deployed models need to be monitored. This is where Watson OpenScale comes into play, because it helps us with 3 kind of monitors:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"507\" src=\"https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-1024x507.png\" alt=\"\" class=\"wp-image-136\" srcset=\"https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-1024x507.png 1024w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-300x148.png 300w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-768x380.png 768w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-1536x760.png 1536w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-830x411.png 830w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-230x114.png 230w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-350x173.png 350w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score-480x237.png 480w, https:\/\/felixaugenstein.com\/blog\/wp-content\/uploads\/2021\/07\/fairness_score.png 1995w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\"><li>Fairness monitor &#8211; looks for biased outcomes from your model. If there is a fairness issue, a warning icon appears.<\/li><li>Quality monitor &#8211; determines how well your model predicts outcomes. When quality monitoring is enabled, it generates a set of metrics every hour by default.<\/li><li>Drift monitor &#8211; determines if the data the model is processing is causing a drop in accuracy over time.<\/li><\/ul>\n\n\n\n<p>To access the complete tutorial go to this <a href=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-vi\" data-type=\"URL\" data-id=\"https:\/\/github.com\/FelixAugenstein\/cloud-pak-for-data-tutorial-part-vi\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub Repository<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this hands-on tutorial you will learn how Watson OpenScale can be used to monitor your deployed machine learning models If you don\u2019t have one already, please Sign up for an IBM Cloud account. This tutorial consists of 6 parts, you can start with part I or any other part, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":135,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,29],"tags":[],"class_list":["post-134","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ibm-watson","category-openscale"],"_links":{"self":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts\/134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/comments?post=134"}],"version-history":[{"count":2,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts\/134\/revisions"}],"predecessor-version":[{"id":138,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/posts\/134\/revisions\/138"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/media\/135"}],"wp:attachment":[{"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/media?parent=134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/categories?post=134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/felixaugenstein.com\/blog\/wp-json\/wp\/v2\/tags?post=134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}