{"id":486,"date":"2017-11-03T08:29:34","date_gmt":"2017-11-03T08:29:34","guid":{"rendered":"http:\/\/www.nullplug.org\/ML-Blog\/?p=486"},"modified":"2017-11-10T08:49:24","modified_gmt":"2017-11-10T08:49:24","slug":"problem-set-3","status":"publish","type":"post","link":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/11\/03\/problem-set-3\/","title":{"rendered":"Problem Set 3"},"content":{"rendered":"<h2>Problem Set 3<\/h2>\n<p>This is to be completed by November 9th, 2017.<\/p>\n<h3>Exercises<\/h3>\n<ol>\n<li>[Datacamp](https:\/\/www.datacamp.com\/home\n<ul>\n<li>Complete the lesson &#8220;Introduction to Machine Learning&#8221;.<\/li>\n<li>This should have also included  &#8220;Exploratory Data Analysis&#8221;. This has been added to the next week&#8217;s assignment. <\/li>\n<\/ul>\n<\/li>\n<li>MLE for the uniform distribution.\n<ul>\n<li>(Source: Kaelbling\/Murphy) Consider a uniform distribution centered on 0 with width 2a. The density function is given by: $$ p(x) = \\frac{\\chi_{[-a,a]}}{2a}.$${<br \/>\na. Given a data set $x_1,\\cdots, x_n,$ what is the maximum likelihood estimate $a_{MLE}$ of $a$?<br \/>\nb. What probability would the model assign to a new data point $x_{n+1}$ using $a_{MLE}$?<br \/>\nc. Do you see any problem with the above approach? Briefly suggest (in words) a better approach.<\/li>\n<\/ul>\n<\/li>\n<li>Calculate the expected value and mode of $\\theta$ when $\\theta \\sim \\textrm{Beta}(\\alpha, \\beta)$,<\/li>\n<li>Change of variables:\n<ul>\n<li>Let $X\\colon S\\to T_1$ be a discrete valued random variable with pmf $p_X\\colon T_1\\to [0,1]$ and let $Y\\colon T_1\\to T_2$ be a function. Derive the pmf $p_{Y\\circ X}\\colon T_2\\to [0,1]$ in terms of $p_X$ and $Y$. <\/li>\n<li>Let $X^n\\colon S^{\\times n}\\to &#92;{0,1&#92;}^{\\times n}$ be the random variable whose values give $n$-independent samples of a Bernoulli random variable $X$ with parameter $\\theta$ (i.e., $p_X(1)=\\theta$). Show that $$p_{X^n}(v_1,\\cdots, v_n)=\\theta^{\\sum{v_i}}(1-\\theta)^{n-\\sum v_i}.$$ Now let $\\sigma \\colon &#92;{0,1&#92;}^{\\times n}\\to &#92;{0,&#8230;n&#92;}$ be defined by taking the sum of the entries. The composite $\\sigma\\circ X^{n}$ is called a <em>Binomial random variable with parameters $n$ and $\\theta.$<\/em> Determine $p_{\\sigma\\circ X^n}(k)$. <\/li>\n<li>Let $X\\colon S\\to \\Bbb R$ be a random variable with piecewise continuous pdf $p_X$ and let $Y\\colon \\Bbb R\\to \\Bbb R$ be a differentiable monotonic function.  Show that $Y\\circ X$ is a random variable and determine $p_{Y\\circ X}$.<\/li>\n<\/ul>\n<\/li>\n<li>Uninformative prior for log-odds ratio:\n<ul>\n<li>(Source: Murphy) Let $$ \\phi = \\textrm{logit}(\\theta) = \\log \\frac{\\theta}{1-\\theta}.$$ Show that if $p(\\phi)\\propto 1,$ then $p(\\theta)\\propto \\textrm{Beta}(0,0)$. <\/li>\n<\/ul>\n<\/li>\n<li>R Lab:\n<ul>\n<li>Construct and apply a Naive Bayes classifier for a specific text classification problem (e.g., <a href=\"https:\/\/www.kaggle.com\/c\/spooky-author-identification\">spooky author identification<\/a>) from scratch. In other words, do not use any modeling libraries. Feel free to use any libraries you like to get the data into an acceptable format.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Problem Set 3 This is to be completed by November 9th, 2017. Exercises [Datacamp](https:\/\/www.datacamp.com\/home Complete the lesson &#8220;Introduction to Machine Learning&#8221;. This should have also included &#8220;Exploratory Data Analysis&#8221;. This has been added to the next week&#8217;s assignment. MLE for the uniform distribution. (Source: Kaelbling\/Murphy) Consider a uniform distribution centered on 0 with width 2a. &hellip; <a href=\"https:\/\/www.nullplug.org\/ML-Blog\/2017\/11\/03\/problem-set-3\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Problem Set 3&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-486","post","type-post","status-publish","format-standard","hentry","category-general"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9dIpN-7Q","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":344,"url":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/10\/10\/parameter-estimation\/","url_meta":{"origin":486,"position":0},"title":"Parameter Estimation","author":"Justin Noel","date":"October 10, 2017","format":false,"excerpt":"\u2026the statistician knows\u2026that in nature there never was a normal distribution, there never was a straight line, yet with normal and linear assumptions, known to be false, he can often derive results which match, to a useful approximation, those found in the real world. - George Box (JASA, 1976, Vol.\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_polyreg_normal.gif?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_polyreg_normal.gif?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_polyreg_normal.gif?resize=525%2C300 1.5x"},"classes":[]},{"id":214,"url":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/10\/04\/linear-regression\/","url_meta":{"origin":486,"position":1},"title":"Linear Regression","author":"Justin Noel","date":"October 4, 2017","format":false,"excerpt":"Prediction is very difficult, especially about the future. - Niels Bohr The problem Suppose we have a list of vectors (which we can think of as samples) $x_1, \\cdots, x_m\\in \\Bbb R^n$ and a corresponding list of output scalars $y_1, \\cdots, y_m \\in \\Bbb R$ (which we can regard as\u2026","rel":"","context":"In &quot;Regression&quot;","block_context":{"text":"Regression","link":"https:\/\/www.nullplug.org\/ML-Blog\/category\/regression\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_linreg_normal.gif?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_linreg_normal.gif?resize=350%2C200 1x, https:\/\/i0.wp.com\/www.nullplug.org\/ML-Blog\/wp-content\/uploads\/2017\/10\/compressed_linreg_normal.gif?resize=525%2C300 1.5x"},"classes":[]},{"id":508,"url":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/11\/09\/problem-set-4\/","url_meta":{"origin":486,"position":2},"title":"Problem Set 4","author":"Justin Noel","date":"November 9, 2017","format":false,"excerpt":"Problem Set 4 This is to be completed by November 16th, 2017. Exercises Datacamp Complete the lessons: a. Supervised Learning in R: Regression b. Supervised Learning in R: Classification c. Exploratory Data Analysis (If you did not already do so) Let $\\lambda\\geq 0$, $X\\in \\Bbb R^n\\otimes \\Bbb R^m$, $Y\\in \\Bbb\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":286,"url":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/10\/05\/statistical-inference-2\/","url_meta":{"origin":486,"position":3},"title":"Statistical Inference","author":"Justin Noel","date":"October 5, 2017","format":false,"excerpt":"All models are wrong, but some are useful. - George Box Introduction The general setup for statistical inference is that we are given some data $D$ which we assume arise as the values of a random variable that we assume is distributed according to some parametric model $m(\\theta)$. The goal\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":468,"url":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/10\/27\/problem-set-2\/","url_meta":{"origin":486,"position":4},"title":"Problem Set 2","author":"Justin Noel","date":"October 27, 2017","format":false,"excerpt":"Problem Set 2 This is to be completed by November 2nd, 2017. Exercises Datacamp Complete the lesson \"Data Visualization in R\". Probabilities are sensitive to the form of the question that was used to generate the answer: (Source: Minka, Murphy.) My neighbor has two children. Assuming that the gender of\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":555,"url":"https:\/\/www.nullplug.org\/ML-Blog\/2017\/12\/18\/problem-set-9\/","url_meta":{"origin":486,"position":5},"title":"Problem Set 9","author":"Justin Noel","date":"December 18, 2017","format":false,"excerpt":"Problem Set 9 This is to be completed by December 21st, 2017. Exercises Datacamp Complete the lesson: a. Intermediate R: Practice R Lab: Consider a two class classification problem with one class denoted positive. Given a list of probability predictions for the positive class, a list of the correct probabilities\u2026","rel":"","context":"In &quot;General&quot;","block_context":{"text":"General","link":"https:\/\/www.nullplug.org\/ML-Blog\/category\/general\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts\/486","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/comments?post=486"}],"version-history":[{"count":10,"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts\/486\/revisions"}],"predecessor-version":[{"id":526,"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/posts\/486\/revisions\/526"}],"wp:attachment":[{"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/media?parent=486"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/categories?post=486"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nullplug.org\/ML-Blog\/wp-json\/wp\/v2\/tags?post=486"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}