Problem Set 4
This is to be completed by November 16th, 2017.
Exercises
 Datacamp
 Complete the lessons:
a. Supervised Learning in R: Regression
b. Supervised Learning in R: Classification
c. Exploratory Data Analysis (If you did not already do so)
 Complete the lessons:
 Let $\lambda\geq 0$, $X\in \Bbb R^n\otimes \Bbb R^m$, $Y\in \Bbb R^n$, and $\beta \in \Bbb R^m$ suitably regarded as matrices.
 Identify when $$\textrm{argmin}_\beta (X\betaY)^t(X\betaY)+\lambda \beta^t\beta$$ exists, and determine it in these cases.
 How does the size of $\lambda$ affect the solution? When might it be desirable to set $\lambda$ to be positive?
 Bayesian approach to linear regression. Suppose that $\beta\sim N(0,\tau^2)$, and the distribution of $Y$ conditional on $X$ is $N(X\beta,\sigma^2I)$, i.e., $\beta$, $X$, and $Y$ are vector valued random variables. Show that, after seeing some data $D$, the MAP and mean estimates of the posterior distribution for $\beta$ correspond to solutions of the previous problem.

R Lab:
 Write a linear regression function that takes in a matrix of $x$values and a corresponding vector of $y$values and returns a function derived from the linear regression fit.
 Write a function that takes in a nonnegative number (the degree), a vector of $x$values and a corresponding vector of $y$values and returns a function derived from the polynomial regression fit.
 Write a function that takes in a number $n$, a vector of $x$values, and a corresponding vector of $y$values and returns a function of the form: $$f(x)=\sum_{i=0}^n a_i \sin(ix)+b_i\cos(ix).$$
 Generate suitable testing data for the three functions constructed above and plot the fitted functions.