{"id":572,"date":"2018-01-19T14:55:22","date_gmt":"2018-01-19T14:55:22","guid":{"rendered":"http:\/\/www.nullplug.org\/ML-Blog\/?p=572"},"modified":"2018-01-20T12:24:15","modified_gmt":"2018-01-20T12:24:15","slug":"572","status":"publish","type":"post","link":"http:\/\/www.nullplug.org\/ML-Blog\/2018\/01\/19\/572\/","title":{"rendered":"Problem Set  12"},"content":{"rendered":"<h2>Problem Set 12<\/h2>\n<p>This is to be completed by January 25th, 2018.<\/p>\n<h3>Exercises<\/h3>\n<ol>\n<li><a href=\"https:\/\/www.datacamp.com\/home\">Datacamp<\/a>\n<ul>\n<li>Complete the lesson:<br \/>\na. Python Data Science Toolbox (Part I)<\/li>\n<\/ul>\n<\/li>\n<li>Let $S\\subset \\Bbb R^n$ with $|S|&lt;\\infty$. Let $\\mu=\\frac{1}{|S|}\\sum_{x_i\\in S} x_i$. Show that $$ \\frac{1}{|S|}\\sum_{(x_i,x_j)\\in S\\times S} ||x_i-x_j||^2 = 2\\sum_{x_i\\in S} ||x_i-\\mu||^2.$$<\/li>\n<li>Prove that the $K$-means clustering algorithm converges.<br \/>\n<!--- 4. Do problems 2.3 and 2.4 from [Elements of Statistical Learning](https:\/\/web.stanford.edu\/~hastie\/Papers\/ESLII.pdf).) ---><\/li>\n<\/ol>\n<h3>Python Lab<\/h3>\n<ol>\n<li>Implement a $K$-Nearest Neighbors classifier and apply it to the MNIST dataset (you will probably need to apply PCA, you can use a <a href=\"http:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.decomposition.PCA.html\">library<\/a> for this at this point).<\/li>\n<li>Implement a $K$-Means clustering algorithm and apply it to the MNIST dataset (after removing the labels and applying a PCA transformation) with $K=10$. Compare the cluster labelings with the actual labelings. <\/li>\n<li>Complete the implementation of the decision tree algorithm from last week.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Problem Set 12 This is to be completed by January 25th, 2018. Exercises Datacamp Complete the lesson: a. Python Data Science Toolbox (Part I) Let $S\\subset \\Bbb R^n$ with $|S|&lt;\\infty$. Let $\\mu=\\frac{1}{|S|}\\sum_{x_i\\in S} x_i$. Show that $$ \\frac{1}{|S|}\\sum_{(x_i,x_j)\\in S\\times S} ||x_i-x_j||^2 = 2\\sum_{x_i\\in S} ||x_i-\\mu||^2.$$ Prove that the $K$-means clustering algorithm converges. Python Lab Implement &hellip; <a href=\"http:\/\/www.nullplug.org\/ML-Blog\/2018\/01\/19\/572\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Problem Set  12&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-572","post","type-post","status-publish","format-standard","hentry","category-general"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/s9dIpN-572","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":531,"url":"http:\/\/www.nullplug.org\/ML-Blog\/2017\/11\/17\/problem-set-5\/","url_meta":{"origin":572,"position":0},"title":"Problem Set 5","author":"Justin Noel","date":"November 17, 2017","format":false,"excerpt":"Problem Set 5 This is to be completed by November 23rd, 2017. Exercises Datacamp Complete the lesson: a. Machine Learning Toolbox R Lab: Write a function in R that will take in a vector of discrete variables and will produce the corresponding one hot encodings. Write a function in R\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"http:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":63,"url":"http:\/\/www.nullplug.org\/ML-Blog\/2017\/09\/26\/computer-science-background\/","url_meta":{"origin":572,"position":1},"title":"Computer Science Background","author":"Justin Noel","date":"September 26, 2017","format":false,"excerpt":"If you find that you're spending almost all your time on theory, start turning some attention to practical things; it will improve your theories. If you find that you're spending almost all your time on practice, start turning some attention to theoretical things; it will improve your practice. - Donald\u2026","rel":"","context":"In &quot;Supplementary material&quot;","block_context":{"text":"Supplementary material","link":"http:\/\/www.nullplug.org\/ML-Blog\/category\/supplementary-material\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":214,"url":"http:\/\/www.nullplug.org\/ML-Blog\/2017\/10\/04\/linear-regression\/","url_meta":{"origin":572,"position":2},"title":"Linear Regression","author":"Justin Noel","date":"October 4, 2017","format":false,"excerpt":"Prediction is very difficult, especially about the future. - Niels Bohr The problem Suppose we have a list of vectors (which we can think of as samples) $x_1, \\cdots, x_m\\in \\Bbb R^n$ and a corresponding list of output scalars $y_1, \\cdots, y_m \\in \\Bbb R$ (which we can regard as\u2026","rel":"","context":"In &quot;Regression&quot;","block_context":{"text":"Regression","link":"http:\/\/www.nullplug.org\/ML-Blog\/category\/regression\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/trace.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/trace.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/trace.png?resize=525%2C300 1.5x"},"classes":[]},{"id":579,"url":"http:\/\/www.nullplug.org\/ML-Blog\/2018\/01\/26\/problem-set-13\/","url_meta":{"origin":572,"position":3},"title":"Problem Set 13","author":"Justin Noel","date":"January 26, 2018","format":false,"excerpt":"Problem Set 13 This is to be completed by February 1st, 2018. Exercises Datacamp * Complete the lesson: a. Python Data Science Toolbox (Part II) For a logistic regressor (multiclass ending in softmax) write down the update rules for gradient descent. For a two layer perceptron ending in softmax with\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"http:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":344,"url":"http:\/\/www.nullplug.org\/ML-Blog\/2017\/10\/10\/parameter-estimation\/","url_meta":{"origin":572,"position":4},"title":"Parameter Estimation","author":"Justin Noel","date":"October 10, 2017","format":false,"excerpt":"\u2026the statistician knows\u2026that in nature there never was a normal distribution, there never was a straight line, yet with normal and linear assumptions, known to be false, he can often derive results which match, to a useful approximation, those found in the real world. - George Box (JASA, 1976, Vol.\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"http:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_polyreg_normal.gif?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_polyreg_normal.gif?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_polyreg_normal.gif?resize=525%2C300 1.5x"},"classes":[]},{"id":33,"url":"http:\/\/www.nullplug.org\/ML-Blog\/2017\/09\/26\/machine-learning-overview\/","url_meta":{"origin":572,"position":5},"title":"Machine Learning Overview","author":"Justin Noel","date":"September 26, 2017","format":false,"excerpt":"Science is knowledge which we understand so well that we can teach it to a computer; and if we don't fully understand something, it is an art to deal with it. Donald Knuth Introduction First Attempt at a Definition One says that an algorithm learns if its performance improves with\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"http:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/web.stanford.edu\/class\/cs234\/images\/header2.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/web.stanford.edu\/class\/cs234\/images\/header2.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/web.stanford.edu\/class\/cs234\/images\/header2.png?resize=525%2C300 1.5x, https:\/\/i0.wp.com\/web.stanford.edu\/class\/cs234\/images\/header2.png?resize=700%2C400 2x"},"classes":[]}],"_links":{"self":[{"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts\/572","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/comments?post=572"}],"version-history":[{"count":5,"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts\/572\/revisions"}],"predecessor-version":[{"id":578,"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts\/572\/revisions\/578"}],"wp:attachment":[{"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/media?parent=572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/categories?post=572"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/tags?post=572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}