Homework 6

Deadline

Due Sunday November 3 at midnight on Crowdmark.

This homework covers three classes.

• Mon 10/28, we covered 7.1 polynomial and step functions

• Weds 10/30, we covered 7.2-7.3 finished step functions, basis functions and also started splines (7.4)

• Fri 11/1, we finished splines (7.4)

• Question 1 For this question, you will use the polynomial toy data set from the course webpage. This is a very simple data set, you will just predict y using X. We’ve learned three models in class this week.

  • (A) Polynomial Regression

  • (B) Step Functions

  • (C) Cubic splines

  For each of these, do the following.

  • (i) Identify the hyperparameter that is relevant to be chosen in that model (degree, number of cuts, etc.).

  • (ii) Use k-fold CV to find the best choice of that hyperparameter.

  • (iii) Train the model on all of the data using that chosen hyperparameter.

  Finally, make a plot of the data, along with all three of the learned models plotted on top. What do you notice? Is one a better (or worse choice) than the others? Which would you choose and why?

• Question 2

  • Part A. I am learning a step function of some data, and I’m using knots \(c_1 = 3\) and \(c_2 = 7\).

    • (i) Write equations for each of the basis functions \(C_0(X)\), \(C_1(X)\), and \(C_2(X)\). Sketch the three functions.

    • (ii) If the model learned was

      \[ f(X) = \beta_0 + \beta_1C_1(X) + \beta_2C_2(X) \]

      with \(\beta_0 = 2\), \(\beta_1 = 3\), and \(\beta_2 = -1\), sketch the graph learned.

  • Part B.

    • I am learning a cubic spline of some data with a single knot at \(c_1 = 4\). As noted in class and in Sec 7.4.3, we have a basis for learning cubic spline data. (As a side note, my lectures have knots as \(c_1,\cdots,c_K\) and the book uses \(\xi_1, \cdots, \xi_K\) but they’re the same thing.) I’m going to build a cubic spline with basis functions

      • \(b_1(X) = X\)

      • \(b_2(X) = X^2\)

      • \(b_3(X) = X^3\)

      • \(b_4(X) =h(x,4)= \begin{cases} (x-4)^3 & x >4 \\ 0 & \text{else} \end{cases}\)

    • Assume the learned model was

      \[ f(X) = 3 + b_1(X) - 2 b_2(X) + 3 b_3(X) - 4b_4(X)\]

    • (i) Write the equation for the piecewise polynomial that this function represents. Draw a graph of the function.

    • (ii) What are the requirements for a piecewise polynomial function to be a cubic spline?

    • (iii) Check that your piecewise polynomial from (i) fits these requirements.

Important

Standard instructions for submissions and deadlines can be found on the Homework Info Page.