{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Function approximation with linear models and neural network\n",
"* Are Linear models sufficient for approximating transcedental functions? What about polynomial functions?\n",
"* Do neural networks perform better in those cases?\n",
"* Does the depth of the neural network matter?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Import basic libraries"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Global variables for the program"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"N_points = 75 # Number of points for constructing function\n",
"x_min = 1 # Min of the range of x (feature)\n",
"x_max = 15 # Max of the range of x (feature)\n",
"noise_mean = 0 # Mean of the Gaussian noise adder\n",
"noise_sd = 10 # Std.Dev of the Gaussian noise adder"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Generate feature and output vector following a non-linear function\n",
"The ground truth or originating function is as follows:\n",
"\n",
"$$ y=f(x)= (20x+3x^2+0.1x^3).sin(x).e^{-0.1x}+\\psi(x) $$\n",
"\n",
"$$ {OR} $$\n",
"\n",
"$$ y=f(x)= (20x+3x^2+0.1x^3)+\\psi(x) $$\n",
"\n",
"$${where,}\\ \\psi(x) : {\\displaystyle f(x\\;|\\;\\mu ,\\sigma ^{2})={\\frac {1}{\\sqrt {2\\pi \\sigma ^{2}}}}\\;e^{-{\\frac {(x-\\mu )^{2}}{2\\sigma ^{2}}}}} $$"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Definition of the function with exponential and sinusoidal terms\n",
"def func_trans(x):\n",
" result = (20*x+3*x**2+0.1*x**3)*np.sin(x)*np.exp(-0.1*x)\n",
" return (result)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Definition of the function without exponential and sinusoidal terms i.e. just the polynomial\n",
"def func_poly(x):\n",
" result = 20*x+3*x**2+0.1*x**3\n",
" return (result)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Densely spaced points for generating the ideal functional curve\n",
"x_smooth = np.array(np.linspace(x_min,x_max,501))\n",
"\n",
"# Use one of the following\n",
"y_smooth = func_trans(x_smooth)\n",
"#y_smooth = func_poly(x_smooth)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Linearly spaced sample points\n",
"X=np.array(np.linspace(x_min,x_max,N_points))"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Added observational/measurement noise\n",
"noise_x = np.random.normal(loc=noise_mean,scale=noise_sd,size=N_points)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Observed output after adding the noise\n",
"y = func_trans(X)+noise_x"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" X | \n",
" Ideal y | \n",
" y | \n",
"
\n",
" \n",
" \n",
" \n",
" | 0 | \n",
" 1.000000 | \n",
" 17.588211 | \n",
" 26.157284 | \n",
"
\n",
" \n",
" | 1 | \n",
" 1.189189 | \n",
" 23.232586 | \n",
" 13.896885 | \n",
"
\n",
" \n",
" | 2 | \n",
" 1.378378 | \n",
" 28.672889 | \n",
" 44.913735 | \n",
"
\n",
" \n",
" | 3 | \n",
" 1.567568 | \n",
" 33.434022 | \n",
" 31.334087 | \n",
"
\n",
" \n",
" | 4 | \n",
" 1.756757 | \n",
" 37.046351 | \n",
" 25.576298 | \n",
"
\n",
" \n",
"
\n",
"