{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Pre-Class Assignment: Further exploration of SVMs\n", "# Day 21\n", "# CMSE 202\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##

✅ Put your name here

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Goals for today's pre-class assignment\n", "\n", "\n", "1. Observing non-linear separability (`sklearn.datasets.make_circles`)\n", "1. Graphing data in 3D\n", "1. Explaining how a RBF kernel works like a nearest neighbors model\n", "\n", "\n", "\n", "\n", "## Assignment instructions\n", "\n", "**This assignment is due by 11:59 p.m. the day before class** and should be uploaded into the appropriate \"Pre-class assignments\" submission folder in the Desire2Learn website. \n", "\n", "### Imports:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# imports for the day\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn import svm\n", "from sklearn.datasets import make_circles\n", "from mpl_toolkits.mplot3d import Axes3D" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 1. Nonlinear data\n", "\n", "Working with linearly separable data certainly makes life easy, but that is simply not the case many times when we're working with data. Review data in the figure at the beginning of the assignment. Clearly the data in that plot are not linearly separable. It turns out, we can find a way to make it linearly separable and we will explore this during class.\n", "\n", "### 1.1 `sklearn.datasets.make_circles` data generators\n", "\n", "Sticking with the scikit-learn datasets page and in particuar Section 7.3, we see `make_circles` which allows us to make data similar to that seen in the figure above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Using the [documentation](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_circles.html), create an example data set with two \"circles\", a 2-class data set. It should:\n", "- have 100 samples\n", "- two classes\n", "- set the `random_state` value so that you can generate the same \"circles\" every time." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# put your answer here\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 2. Graph the circles\n", "\n", "Did it work? As always, it's hard to tell if we ended up with what we wanted without visualizing it.\n", "\n", "✅ **Do This:** Graph the result of the `make_circles` operation. Color all the elements of one class one color and the elements of the other class the other color. \n", "\n", "As a reminder:\n", "- you can create a list/array with color assignments for each data element based on their class. Your colors can be strings like `'red'`, integer numbers (that will map to whatever matplotlib `cmap` you use), or hex code values -- it's up to you.\n", "- try playing with the `random_state` value and see if you can get some \"blobs\" that are relatively \"close\" to each other. It should be the case that if you use the same `random_state` value each time that you get the same distribution of data. Remember, \"random\" means \"relatively random\" here. That is, predicting the next value in a sequence should be (statistically speaking) random. However, start with the same seed value (which is what `random_state` is for this process) you get the exact same sequence." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# put your answer here\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 3. Adding a dimension to linearly separate the data\n", "\n", "It's hard to see how we could separate these two classes given their overlap/shape, and that is likely true in the 2D format we are looking at currently. But what if we added a 3rd dimension based on the features of the data that we have? That is, can we use the existing X and Y values to make a new dimension that would that help to linearly separate the data?\n", "\n", "*As you saw in [this video](https://www.youtube.com/watch?v=efR1C6CvhmE) from the previous pre-class assignment, creating these new dimensions from the data is part of what an SVM does. For this assignment, we will try to do this manually.*\n", "\n", "First, let remind ourselves of 3D plots." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.1 Making a 3D scatter plot\n", "\n", "To setup a 3D scatter plot, you need the following steps (found in this tutorial; you can see the code if you click on the plot). We can generate some random values and try it out." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "x = np.random.randint(-5, 5, 100)\n", "y = np.random.randint(-5, 5, 100)\n", "z = np.random.randint(-5, 5, 100)\n", "from mpl_toolkits.mplot3d import Axes3D\n", "fig = plt.figure(figsize = (10, 7))\n", "ax = plt.axes(projection =\"3d\")\n", " \n", "# Creating plot\n", "ax.scatter3D(x, y, z, color = \"blue\")\n", "plt.title(\"simple 3D scatter plot\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "✅ **Do This:** Plot your circle data in 3D. Because we don't have a third/$z$ dimension you can just pass a fixed value for $z$. In fact, you can just use a single number. As before, mark the two classes in two different colors." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# put your answer here\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3.2 Trying to make the data linearly separable\n", "\n", "Let's try to separate these circles of data using the existing values to compute $z$-values that would make it possible to separate the.\n", "\n", "✅ **Do This:** Try to add a third dimension to the data, by calculated $z$-values based on the existing values. This is called a \"transformation\" of the data.\n", "\n", "What might be a procedure to do that? You might have to think about this for a moment or perhaps poke around a bit on the internet. Do your best to come up with something that you think should work. \n", "\n", "Make a 3D plot of your transformed data." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# put your code here\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "✅ **Question:** What transformation did you use to separate the data in the third, $z$ dimension? Do you feel you were successful in separating the data?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "✎ Do this - Erase this and put your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## 4. Alternative Kernels for SVM\n", "\n", "The linear kernel is one of a number of possible kernels that an SVM can use. One common choice for dealing with nonlinear data is the Radial Basis Function (RBF) Kernel. The video from the previous class assignment has a bit of information about this kernel.\n", "\n", "✅ **Do This:** Review this section of the video for a conceptual description how the RBF works like a nearest neighbors model. Deeper, more technial details on the RBF are readily available online, but extend a bit beyond the scope of this course at the present time. Feel free to dig into them if you wish!" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import YouTubeVideo\n", "from datetime import timedelta\n", "start=int(timedelta(hours=0, minutes=17, seconds=29).total_seconds())\n", "YouTubeVideo(\"efR1C6CvhmE\",width=800,height=450,start=start)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "✅ **Question:** How does a the RBF work like a nearest-neighbors model?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "✎ Do this - Erase this and put your answer here." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## Follow-up Questions\n", "\n", "Copy and paste the following questions into the appropriate box in the assignment survey include below and answer them there. (Note: You'll have to fill out the assignment number and go to the \"NEXT\" section of the survey to paste in these questions.)\n", "\n", "1. When we can't easily linearly separate classes in our data in its default form, what can we do with the data in order to find a linear decision boundary that *can* separate the data?\n", "\n", "2. How does the RBF act similarly to the nearest-neighbor algorithm?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "# Assignment wrap-up\n", "\n", "Please fill out the form that appears when you run the code below. **You must completely fill this out in order to receive credit for the assignment!**" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import HTML\n", "HTML(\n", "\"\"\"\n", "\n", "\"\"\"\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---------\n", "### Congratulations, you're done with your pre-class assignment!\n", "\n", "Now, you just need to submit this assignment by uploading it to the course Desire2Learn web page for today's submission folder (Don't forget to add your name in the first cell)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "© Copyright 2023 Department of Computational Mathematics, Science and Engineering at Michigan State University" ] } ], "metadata": { "anaconda-cloud": {}, "interpreter": { "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" } }, "nbformat": 4, "nbformat_minor": 4 }