{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "import pylab as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "x = np.array([[1, 2, 3, 4], \n",
    "              [1, 3, 3, 5]])\n",
    "w = np.array([0.1, 0.2, 0.3, 0.4])\n",
    "\n",
    "y = np.array([0, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как посчитать квадратичный функционал если X — матрица объектов, y — ответы, а w — некоторый вектор весов:\n",
    " -  Честно посчитать по формуле"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15.76\n"
     ]
    }
   ],
   "source": [
    "q = 0\n",
    "for i in range(x.shape[0]):\n",
    "    q += (np.sum(w * x[i]) - y[i]) ** 2\n",
    "print q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " -  Воспользоваться Numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15.76\n"
     ]
    }
   ],
   "source": [
    "q = np.sum((np.sum(w * x, axis=1) - y) ** 2)\n",
    "print q"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Градиентный спуск\n",
    "\n",
    "Градиентный спуск позволяет оптимизировать (обычно — минимизировать) некоторый функционал.\n",
    "\n",
    "В начале просто попробуем найти минимум некоторой функции. \n",
    "\n",
    "Предположим что у нас есть зависимость $y = 2 * x_1^2 + 3 * x_2^2$. Давайте найдем минимум этой функции (он просто вычисляется аналитически, однако попробуем сделать это с помощью данного метода). То есть для заданной функции мы хотим найти точку $(x_1^*, x_2^*)$, в которой достигается минимум.\n",
    "\n",
    "Для начала опишем функции, которые нам понадобятся: \n",
    " - function — вычисляет значение функции в данной точке\n",
    " - grad — вычисляет градиент в заданной точке"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def function(x1, x2):\n",
    "    return 2 * x1 ** 2 + 3 * x2 ** 2 \n",
    "\n",
    "def grad(x1, x2):\n",
    "    return np.array([4 * x1, 6 * x2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Тогда градиентный спуск будет выглядеть как функция (уже из питона, которая принимает на вход):\n",
    " - начальное приближение (x1 и x2)\n",
    " - максимальное количество итераций max_iters\n",
    "\n",
    "Будем считать что шаг — это константа и не зависит от номера итерации.\n",
    "\n",
    "И которая будет возвращать:\n",
    " - points — точки, которые были получены во время градиентного спуска\n",
    " - values — значения функции в пройденных точках\n",
    " \n",
    "Так как мы решаем задачу минимизации функции, то хотим, чтобы с номером итерации значение функции уменьшалось."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def grad_descent(x1, x2, step, max_iters):\n",
    "    points = [(x1, x2)]\n",
    "    values = [function(x1, x2)]\n",
    "    for _ in range(max_iters):\n",
    "        x1_new, x2_new = np.array([x1, x2]) - step * grad(x1, x2)\n",
    "        x1, x2 = x1_new, x2_new\n",
    "        points.append((x1, x2))\n",
    "        values.append(function(x1, x2))\n",
    "    return points, values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Запустим наш градиентный спуск из случайной точки и построим график:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0.16870319358849653, 0.041097495410471981) 0.061988547441\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10c912b50>]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEACAYAAACznAEdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHNhJREFUeJzt3XuUFPWd9/H3h8sERJigCWAG5CKCgEFlI16isY0nXLIn\nymYTgrmpScye1RxdY9wFn7Mye/YWs2vUfZ6YHLPGNVkvS+IayK4icbGT4xoDBhHDIIwi18CoCYzx\nFkC/zx9VI+04wNDT09WXz+ucOl316+qqbxd95svvVqWIwMzMrE/WAZiZWWVwQjAzM8AJwczMUk4I\nZmYGOCGYmVnKCcHMzIBuJARJt0lqk7Smi/eulvSmpKMKyhZIapW0TtKMgvJpktZI2iDpptJ9BTMz\nK4Xu1BBuB2Z2LpQ0EvgIsLmgbBIwF5gEzAZukaT07W8DX4yICcAESe84ppmZZeeQCSEiHgF2dfHW\njcA1ncouAO6JiH0RsQloBaZLGgEMjoiV6X7fB+YUHbWZmZVcUX0Iks4HtkbEU53eagK2FmxvT8ua\ngG0F5dvSMjMzqxD9DvcDkgYC15I0F5mZWY047IQAHAeMAZ5M+wdGAqskTSepERxbsO/ItGw7MKqL\n8i5J8g2WzMyKEBE69F5d626TkdKFiPh1RIyIiHERMZak+eeUiHgeWAJ8SlKDpLHAeGBFROwE2iVN\nT5PI54HFBzthRHiJYOHChZnHUCmLr4Wvha/FwZee6s6w07uAR0lGBm2RdEnnv93sTxYtwCKgBbgf\nuCz2R3k5cBuwAWiNiKU9jt7MzErmkE1GEfHpQ7w/rtP2PwL/2MV+vwLef7gBmplZeXimcoXL5XJZ\nh1AxfC3287XYz9eidFSKdqdSkxR79gT9+2cdiZlZ9ZBElKFTuey2bMk6AjOz+lKxCWHjxqwjMDOr\nLxWbEJ59NusIzMzqixOCmZkBFZwQ3GRkZlZeFZsQXEMwMyuvik0IGzdCBY6INTOrWRWbEPr1gxdf\nzDoKM7P6UbEJ4bjj3GxkZlZOFZ0Q3LFsZlY+FZsQxo1zDcHMrJwqNiG4ycjMrLwqNiGMG+cmIzOz\ncqrYhOAagplZeVXs7a/37QsGDYJdu2DgwKwjMjOrfDV7++u+fWH0aHjuuawjMTOrDxWbEMDNRmZm\n5VTxCcEdy2Zm5VHxCeGZZ7KOwsysPlR0QpgwAdavzzoKM7P6UNEJYeJEJwQzs3I5ZEKQdJukNklr\nCsq+IWmdpNWS7pU0pOC9BZJa0/dnFJRPk7RG0gZJN3UnuDFj4Pnn4dVXD/NbmZnZYetODeF2YGan\nsmXAlIg4GWgFFgBImgzMBSYBs4FbJHWMif028MWImABMkNT5mO/Qt28yY7m1tVvfxczMeuCQCSEi\nHgF2dSp7KCLeTDcfA0am6+cD90TEvojYRJIspksaAQyOiJXpft8H5nQnQDcbmZmVRyn6EL4A3J+u\nNwFbC97bnpY1AdsKyrelZYfkhGBmVh79evJhSf8H2BsRd5conrc0NzcDsG4dvPJKDsiV+hRmZlUt\nn8+Tz+dLdrxu3ctI0mjgJxExtaDsYuBS4MMR8Ye0bD4QEXF9ur0UWAhsBh6OiElp+TzgnIj48wOc\nLzri+sUv4IorYOXKrvY0M7MO5bqXkdKl46SzgGuA8zuSQWoJME9Sg6SxwHhgRUTsBNolTU87mT8P\nLO7OiTuajCrwHnxmZjWlO8NO7wIeJRkZtEXSJcD/BY4EfipplaRbACKiBVgEtJD0K1wW+6sglwO3\nARuA1ohY2p0AjzoK3vUu2LnzML+ZmZkdloq9/XVhXGedBX/3d5DLZReTmVmlq9nbXxfySCMzs97n\nhGBmZoATgpmZpZwQzMwMqJJO5T17YMgQaG9PRhyZmdk71UWnckMDHHusH6dpZtabqiIhgJuNzMx6\nmxOCmZkBTghmZpZyQjAzM6CKEsIJJyS3wq7AQVFmZjWhahLCsGHQvz/s2JF1JGZmtalqEgLAlCmw\ndm3WUZiZ1aaqSggnngi//nXWUZiZ1SYnBDMzA5wQzMwsVRX3MuqwezeMHAkvvQR9qiqVmZn1vrq4\nl1GHd78bhg6FzZuzjsTMrPZUVUIANxuZmfUWJwQzMwOcEMzMLOWEYGZmQJWNMgJ49VV4z3uSkUb9\n+pU5MDOzCtbro4wk3SapTdKagrKhkpZJWi/pQUmNBe8tkNQqaZ2kGQXl0yStkbRB0k3FBnzEEdDU\nBM88U+wRzMysK91pMrodmNmpbD7wUERMBJYDCwAkTQbmApOA2cAtkjqy1beBL0bEBGCCpM7H7DY3\nG5mZld4hE0JEPALs6lR8AXBHun4HMCddPx+4JyL2RcQmoBWYLmkEMDgiVqb7fb/gM4fNCcHMrPSK\n7VQeFhFtABGxExiWljcBWwv2256WNQHbCsq3pWVFcUIwMyu9UnXLlrxnurm5+a31XC5HLpd7a/vE\nE2HhwlKf0cysuuTzefL5fMmO161RRpJGAz+JiKnp9jogFxFtaXPQwxExSdJ8ICLi+nS/pcBCYHPH\nPmn5POCciPjzA5zvgKOMAPbsgcZG2LULBgw4nK9rZla7ynUvI6VLhyXAxen6RcDigvJ5khokjQXG\nAyvSZqV2SdPTTubPF3zmsDU0wPjxySM1zcysNLoz7PQu4FGSkUFbJF0CfB34iKT1wHnpNhHRAiwC\nWoD7gcsK/qt/OXAbsAFojYilPQl86lR48smeHMHMzApV3cS0DjfcAFu2wM03lykoM7MKV1e3vy50\nyimwalXWUZiZ1Y6qrSHs2gWjRycPzfHDcszM6riGMHQoHH20b2FhZlYqVZsQAKZNgyeeyDoKM7Pa\nUNUJwf0IZmalU9UJYdo0JwQzs1Kp6oRwyilJk1EF9oubmVWdqk4IxxyTPCRn69ZD72tmZgdX1QkB\n3LFsZlYqVZ8Q3LFsZlYaVZ8QXEMwMyuNqk8IriGYmZVG1SeEsWPh5Zfh+eezjsTMrLpVfUKQ9g8/\nNTOz4lV9QgD3I5iZlUJNJIQ/+iNYuTLrKMzMqltNJITTToNf/jLrKMzMqltNJIRx4+APf4Bt27KO\nxMysetVEQpDg9NNdSzAz64maSAiQJITHHss6CjOz6lUzCeG005wQzMx6omqfqdxZezs0NSXPWu7f\nv5cCMzOrYHX7TOXOGhthzBh46qmsIzEzq049SgiSFkhaK2mNpDslNUgaKmmZpPWSHpTU2Gn/Vknr\nJM3oefhv534EM7PiFZ0QJI0GLgVOiYipQD/gQmA+8FBETASWAwvS/ScDc4FJwGzgFklFV2264vkI\nZmbF60kN4SVgDzBIUj9gILAduAC4I93nDmBOun4+cE9E7IuITUArML0H538H1xDMzIpXdEKIiF3A\nDcAWkkTQHhEPAcMjoi3dZycwLP1IE1D4sMvtaVnJTJ4MO3bAb39byqOamdWHfsV+UNI44CpgNNAO\n/FDSZ4DOw4OKGsbU3Nz81noulyOXyx3yM337wqmnwooVMHt2MWc1M6se+XyefD5fsuMVPexU0lzg\nIxFxabr9OeB04MNALiLaJI0AHo6ISZLmAxER16f7LwUWRsQ7Wv2LGXba4dproaEBCvKJmVldyHLY\n6XrgdEkD0s7h84AWYAlwcbrPRcDidH0JMC8diTQWGA+s6MH5u+R+BDOz4vRoYpqka0j++L8BPAF8\nCRgMLAJGAZuBuRGxO91/AfBFYC9wZUQsO8Bxi64hPP88TJyY9CP0qZlZFmZmh9bTGkLNzFQuNHEi\nLFoEJ51UwqDMzCqcZyp34Zxz4Gc/yzoKM7Pq4oRgZmZAjTYZbd2aPGf5+eeTZyWYmdUDNxl1YdQo\nGDwY1q3LOhIzs+pRkwkB3GxkZna4nBDMzAyo0T4EgOeegzPPhN/8xv0IZlYf3IdwAGPGQL9+0Nqa\ndSRmZtWhZhOC5GYjM7PDUbMJAZKE8POfZx2FmVl1qPmE8LOfQQV2k5iZVZyaTgjHHw979sCmTVlH\nYmZW+Wo6IUhw7rmwfHnWkZiZVb6aTggAM2bAgw9mHYWZWeWr2XkIHbZvh6lTk/sa9e1bkkOamVUk\nz0M4hKYmOOYYePzxrCMxM6tsNZ8QAGbOdLORmdmh1E1CWNblwzrNzKxDzfchALz2GgwbBtu2QWNj\nyQ5rZlZR3IfQDQMHJje6+5//yToSM7PKVRcJAdxsZGZ2KHWTEDrmI1RgC5mZWUWom4QwZQrs3evb\nYZuZHUiPEoKkRkk/lLRO0lpJp0kaKmmZpPWSHpTUWLD/Akmt6f4zeh7+4cSa1BLcbGRm1rWe1hBu\nBu6PiEnAScDTwHzgoYiYCCwHFgBImgzMBSYBs4FbpPI+y2zmTLj//nKe0cysehQ97FTSEOCJiDiu\nU/nTwDkR0SZpBJCPiBMkzQciIq5P93sAaI6IX3Zx7JIOO+3Q3g6jRiW3sxg8uOSHNzPLVJbDTscC\nL0q6XdIqSbdKOgIYHhFtABGxExiW7t8EbC34/Pa0rGwaG+GMMzxr2cysK/16+NlpwOUR8bikG0ma\nizr/176o/+o3Nze/tZ7L5cjlcsVF2cmcOfDjH8MnPlGSw5mZZSafz5PP50t2vJ40GQ0HfhER49Lt\ns0gSwnFArqDJ6OGImNRFk9FSYGE5m4wgaS56//uhrQ369++VU5iZZSKzJqO0WWirpAlp0XnAWmAJ\ncHFadhGwOF1fAsyT1CBpLDAeWFHs+YvV1ATjx/tZy2ZmnfWkyQjgCuBOSf2BjcAlQF9gkaQvAJtJ\nRhYRES2SFgEtwF7gsl6rBhzCBRfA4sVw3nlZnN3MrDLVxc3tOlu7FmbPhs2bk/kJZma1wDe3K8Lk\nydDQAKtXZx2JmVnlqMuEIO0fbWRmZom6TAiQ9CM4IZiZ7Ve3CeHMM5Ohpxs2ZB2JmVllqNuE0Lcv\nfOpTcPfdWUdiZlYZ6jYhAFx4YZIQKnCglZlZ2dV1QjjtNNizB554IutIzMyyV9cJQYJPf9rNRmZm\nUKcT0wqtXQuzZiWT1PrUdXo0s2rniWk9NGUKHHUUPPJI1pGYmWWr7hMCJJ3Ld92VdRRmZtmq+yYj\ngE2b4NRTk1tjNzSU7bRmZiXlJqMSGDMGJk70k9TMrL45IaQuvhhuuy3rKMzMsuMmo9TLL8OoUdDS\nAsccU9ZTm5mVhJuMSuTII5PnLP/bv2UdiZlZNlxDKLBiRTLiqLXVcxLMrPq4hlBCp56a1BTy+awj\nMTMrPyeEAhJ86Uvw3e9mHYmZWfm5yaiTXbtg7Fh45hl4z3syCcHMrChuMiqxoUPhYx+Df//3rCMx\nMysv1xC68Mgj8IUvwNNPu3PZzKqHawi94IMfhCFD4P77s47EzKx8epwQJPWRtErSknR7qKRlktZL\nelBSY8G+CyS1SlonaUZPz91bJLjqKvjmN7OOxMysfEpRQ7gSaCnYng88FBETgeXAAgBJk4G5wCRg\nNnCLpKKrNr3tk5+EDRtg9eqsIzEzK48eJQRJI4GPAv9aUHwBcEe6fgcwJ10/H7gnIvZFxCagFZje\nk/P3poYG+MpX4Kabso7EzKw8elpDuBG4BijsAR4eEW0AEbETGJaWNwFbC/bbnpZVrC9/GRYvhh07\nso7EzKz39Sv2g5L+GGiLiNWScgfZtajhQs3NzW+t53I5crmDnaJ3HHVUciuLW26Bv/3bsp/ezOyg\n8vk8+RLeWqHoYaeS/gH4LLAPGAgMBu4DPgDkIqJN0gjg4YiYJGk+EBFxffr5pcDCiPhlF8fOdNhp\noQ0b4Kyz4LnnYNCgrKMxMzuwzIadRsS1EXFsRIwD5gHLI+JzwE+Ai9PdLgIWp+tLgHmSGiSNBcYD\nK4o9f7lMmAAf+hB85ztZR2Jm1rt6Yx7C14GPSFoPnJduExEtwCKSEUn3A5dVTDXgEK67Dv75n+HV\nV7OOxMys93imcjf96Z8mE9a++tWsIzEz61pPm4ycELrpySdh1ix49lk44oisozEzeyffuqJMTjoJ\nzjgDbr0160jMzHqHawiHYfVq+OhHk1rCwIFZR2Nm9nauIZTRySfDaafBt76VdSRmZqXnGsJhevpp\nOPtsWLfOD9Axs8riTuUMfOUryXMS/uVfso7EzGw/J4QMvPACTJoEjz6aTFwzM6sE7kPIwHvfC3/5\nl8liZlYrXEMo0uuvJ7WE22+HDO67Z2b2Dq4hZGTAAPj61+HKK2Hv3qyjMTPrOSeEHpg7F4YP90N0\nzKw2uMmoh559NpmbsHIljB2bdTRmVs/cZJSx446Dr30NLrsMqiSHmZl1yQmhBK6+GrZvh//4j6wj\nMTMrnpuMSuSxx+DjH4c1azyD2cyy4YlpFeRrX4ONG+Hee0FF/5OYmRXHfQgV5O//Pulk/t73so7E\nzOzwuYZQYmvXJhPVHn0Ujj8+62jMrJ64hlBhpkyBv/5r+OxnPWHNzKqLawi9ICJ5kM7UqXD99VlH\nY2b1wjWECiTBD36QDEP90Y+yjsbMrHtcQ+hFq1bBzJmQzydNSWZmvck1hAo2bRrccAPMmQO7d2cd\njZnZwRWdECSNlLRc0lpJT0m6Ii0fKmmZpPWSHpTUWPCZBZJaJa2TNKMUX6DSff7zMGsWfPrTsG9f\n1tGYmR1Y0U1GkkYAIyJitaQjgV8BFwCXAL+NiG9I+itgaETMlzQZuBM4FRgJPAQc31XbUK00GXXY\nuxc+9jEYNQpuvdWT1sysd2TWZBQROyNidbr+MrCO5A/9BcAd6W53AHPS9fOBeyJiX0RsAlqB6cWe\nv5r07590Lj/xBPzN32QdjZlZ10rShyBpDHAy8BgwPCLaIEkawLB0tyZga8HHtqdldeHII+G//zsZ\nffTd72YdjZnZO/Xr6QHS5qIfAVdGxMuSOrf1FNX209zc/NZ6LpcjVwPPqRw+HJYuhQ99CIYMgU99\nKuuIzKya5fN58vl8yY7Xo2GnkvoB/wU8EBE3p2XrgFxEtKX9DA9HxCRJ84GIiOvT/ZYCCyPil10c\nt6b6EDpbsyYZjnrTTU4KZlY6WQ87/R7Q0pEMUkuAi9P1i4DFBeXzJDVIGguMB1b08PxVaepUWLYM\n/uIv/AwFM6scRTcZSfog8BngKUlPkDQNXQtcDyyS9AVgMzAXICJaJC0CWoC9wGU1XQ04hPe/P0kK\nM2Yko5A++9msIzKzeueZyhlraYHZs+Hyy+Gaazwk1cyK5wfk1IBt25Kb4Z1zTtKv0Ldv1hGZWTVy\nQqgR7e3wJ38CjY3w/e/D4MFZR2Rm1SbrTmUrkcZGeOCB5HnMp58OGzZkHZGZ1RsnhAryrnclk9au\nvBLOOguWLMk6IjOrJ24yqlCPPQaf/CTMnZs8q3nAgKwjMrNK5yajGnX66bB6NWzaBNOnw69/nXVE\nZlbrnBAq2NFHJzfFu+oqOPdc+Kd/8i20zaz3uMmoSmzcCH/2Z/Dii8kttE89NeuIzKzSuMmoTowb\nl8xsvvrq5NkKV1wBv/td1lGZWS1xQqgiUnKLi7VrYc8eOOEEuPnmZN3MrKecEKrQ0UfDd74DDz+c\n3E77xBPhnnvgjTeyjszMqpn7EGrAT38K112XzHa+7rpkuKpvf2FWf3zrCgMgIkkMCxfCb3+b3Fr7\nootg0KCsIzOzcnFCsLeJgEcegW9+M3m99FL48pdhzJisIzOz3uZRRvY2Epx9Ntx3H/ziF/Dqq/CB\nD8CsWfCf/+kOaDM7MNcQ6sBrr8G99ybzF9atg098IhmtdOaZfv6CWS1xk5Edlk2b4O674c474aWX\nYM4c+PjHk5vp9Sv6+XlmVgmcEKwoEUlt4b77kqakzZuTx3nOng0zZ8KwYVlHaGaHywnBSmLr1mRO\nwwMPwPLlSSf0uecmy9lnw9ChWUdoZofihGAlt3cv/OpXSWJ4+OHkVtzHHpv0OZxxRnL31RNOcBOT\nWaVxQrBet28frFkDjz6aLI8/Dr/5DZx8MpxyCkydCiedBFOmeN6DWZacECwT7e2walXyzIY1a+DJ\nJ5M+iREjYNKkZJkwAY4/PlmamqCPBzmb9aqqSwiSZgE3kcyBuC0iru9iHyeEKvTGG/Dcc9DSkiSH\nDRugtTVZdu+G0aOTu7aOHZs0QR17LIwalSSL970veYSomRWvqhKCpD7ABuA84DfASmBeRDzdaT8n\nhFQ+nyeXy2UdRo+98koy5HXjxuR161bYsiVZtm+HnTthyBA45piklnHMMTB8eDLa6b3vTV43bcoz\nY0aOo4+Gxsb6nkNRK7+LUvC12K+nCaHc3YLTgdaI2Awg6R7gAuDpg36qjtXKj33QoKSPYcqUrt9/\n80144QXYsSNJDh3Ljh1Jc9QLL8CaNXm+8Y0cL76YTLYbOjRZjjoqSRDvfnfy2tiYJJchQ2Dw4GQ5\n8sjkddCgty9HHAENDeW9FqVQK7+LUvC1KJ1yJ4QmYGvB9jaSJGF1rk+fpEYwfPiB92luThZIbsGx\nezfs2pU8KKi9PVl2704m3L30Ejz7LPz+9/Dyy8nr73+f1FReeSUpe/XVZF2CgQOT5YgjktcBA/a/\nDhiQNGd1vDY0vPO1oQH693/7ev/+yUiszuv9+u1f79t3/3bfvvu3O9Y7L336JK979iRJsWO7T5/k\ne9Rzrcl6zgMHrSo1NCTNSKWYQLd3b5IcXntt/+sf/pC8vv56st75tWPZuzf549zenqx3bO/Zk4zO\n6igrXH/jjf1l+/Yl24XrHdsd6x3Lm2/uX3/9dbjxxv3lb76ZTDaUkuTQ1dL5vY4EUrje1faByrpa\n4OBlHevFlnUo3N6yJRkifaj9DrTe1fbh7nc4x+jue8Xs11Pl7kM4HWiOiFnp9nwgOncsS3IHgplZ\nEaqpU7kvsJ6kU3kHsAK4MCLWlS0IMzPrUlmbjCLiDUlfAZaxf9ipk4GZWQWoyIlpZmZWfhU1d1TS\nLElPS9og6a+yjqecJI2UtFzSWklPSboiLR8qaZmk9ZIelNSYdazlIqmPpFWSlqTbdXktJDVK+qGk\ndenv47Q6vhYL0muwRtKdkhrq5VpIuk1Sm6Q1BWUH/O7ptWpNfzczunOOikkI6aS1/wfMBKYAF0o6\nIduoymof8NWImAKcAVyefv/5wEMRMRFYDizIMMZyuxJoKdiu12txM3B/REwCTiKZt1N310LSaOBS\n4JSImErS5H0h9XMtbif5+1ioy+8uaTIwF5gEzAZukQ49VqliEgIFk9YiYi/QMWmtLkTEzohYna6/\nDKwDRpJcgzvS3e4A5mQTYXlJGgl8FPjXguK6uxaShgBnR8TtABGxLyLaqcNrAbwE7AEGSeoHDAS2\nUyfXIiIeAXZ1Kj7Qdz8fuCf9vWwCWunGnK9KSghdTVpryiiWTEkaA5wMPAYMj4g2SJIGUC+PrrkR\nuAYo7OSqx2sxFnhR0u1p89mtko6gDq9FROwCbgC2kCSC9oh4iDq8FgWGHeC7d/57up1u/D2tpIRg\ngKQjgR8BV6Y1hc69/jU/CkDSHwNtaY3pYNXcmr8WJM0i04BvRcQ04BWSZoJ6/F2MA64CRgPvI6kp\nfIY6vBYH0aPvXkkJYTtwbMH2yLSsbqTV4B8BP4iIxWlxm6Th6fsjgOeziq+MPgicL2kjcDfwYUk/\nAHbW4bXYBmyNiMfT7XtJEkQ9/i4+APxvRPwuIt4A7gPOpD6vRYcDffftwKiC/br197SSEsJKYLyk\n0ZIagHnAkoxjKrfvAS0RcXNB2RLg4nT9ImBx5w/Vmoi4NiKOjYhxJL+D5RHxOeAn1N+1aAO2SpqQ\nFp0HrKUOfxckk1pPlzQg7SA9j2TQQT1dC/H2WvOBvvsSYF46CmssMJ5kIvDBD15J8xDSZyXczP5J\na1/POKSykfRB4OfAUyTVvgCuJflHXESS7TcDcyNid1Zxlpukc4CrI+J8SUdRh9dC0kkknev9gY3A\nJUBf6vNaXEPyB/AN4AngS8Bg6uBaSLoLyAFHA23AQuDHwA/p4rtLWgB8EdhL0gS97JDnqKSEYGZm\n2amkJiMzM8uQE4KZmQFOCGZmlnJCMDMzwAnBzMxSTghmZgY4IZiZWcoJwczMAPj/FF68EfC8ISIA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c8531d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(10, 20, 0.01, 100)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Заметим, что значение градиентный спуск не успевает найти минимум. Возможно стоит увелчить число итераций:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1.8673814466701882e-17, 2.6846247849867386e-26) 6.97422693474e-34\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10ca6cd10>]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEACAYAAABYq7oeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFfdJREFUeJzt3X+MXeV95/H3B4wh/Ijjthh37QZMEWCikIR03eymiKtE\nhdBKhj8q1mxIIET5o7BK1F1la2dV4b82YaUqpM0SCZUS0yQlTrqVnZQFlsIoSnYTEgIxa7vGq9TE\nOPHQCOqWtAk2fPePexzfzrU94/HMnDNz3y9p5HOe+5xznvMwuh+e5zl3bqoKSZIGndJ2AyRJ3WM4\nSJKGGA6SpCGGgyRpiOEgSRpiOEiShkwaDknuTTKeZNtRXvtPSV5L8gsDZRuS7E6yM8nVA+VXJNmW\n5Nkkd83cLUiSZtpURg73AddMLEyyEvhN4LmBstXADcBq4Frg7iRpXv4M8MGquhi4OMnQOSVJ3TBp\nOFTV14GXjvLSJ4GPTii7Dnigqg5V1R5gN7AmyXLgnKr6dlPvfuD6abdakjSrprXmkGQtsLeqnpnw\n0gpg78D+vqZsBfD8QPnzTZkkqYMWnegBSV4HfIz+lJIkaQE64XAAfhW4APhes56wEvhukjX0Rwpv\nHKi7sinbB/zKUcqPKol/8EmSpqGqMnmtyU11WinND1X1f6tqeVVdWFWr6E8Rva2qXgC2Av8uyeIk\nq4CLgCeqaj9wIMmaJlDeD2w53gWryp8q7rjjjtbb0JUf+8K+sC+O/zOTpvIo6xeA/03/CaMfJPnA\nxPdxjgTHDmAzsAN4ELitjrT4duBe4Flgd1U9NDO3IEmaaZNOK1XVv5/k9Qsn7H8c+PhR6j0JvPlE\nGyhJmnt+Qrrjer1e203oDPviCPviCPtidmSm56lmQpLqYrskqcuSUHO8IC1JGiGGgyRpiOEgSRpi\nOEiShhgOkqQhhoMkaYjhIEkaYjhIkoYYDpKkIYaDJGmI4SBJGmI4SJKGGA6SpCGGgyRpiOEgSRpi\nOEiShhgOkqQhhoMkaYjhIEkaYjhIkoYYDpKkIZOGQ5J7k4wn2TZQ9t+S7EzydJK/SPL6gdc2JNnd\nvH71QPkVSbYleTbJXTN/K5KkmTKVkcN9wDUTyh4B3lRVbwV2AxsAklwG3ACsBq4F7k6S5pjPAB+s\nqouBi5NMPKckqSMmDYeq+jrw0oSyR6vqtWb3m8DKZnst8EBVHaqqPfSDY02S5cA5VfXtpt79wPUz\n0H5J0iyYiTWHW4EHm+0VwN6B1/Y1ZSuA5wfKn2/KJEkdtOhkDk7yX4CDVfXnM9Sen9u4cePPt3u9\nHr1eb6YvIUnz2tjYGGNjY7Ny7lTV5JWS84GvVNXlA2W3AB8C3lVVP2vK1gNVVXc2+w8BdwDPAY9X\n1eqmfB1wVVX97jGuV1NplyTpiCRUVSavObmpTiul+TncgPcAHwXWHg6GxlZgXZLFSVYBFwFPVNV+\n4ECSNc0C9fuBLTNxA5KkmTfptFKSLwA94BeT/ID+SOBjwGLgfzUPI32zqm6rqh1JNgM7gIPAbQND\ngNuBzwJnAA9W1UMzfC+SpBkypWmluea0kiSduDamlSRJI8RwkCQNMRwkSUMMB0nSEMNBkjTEcJAk\nDTEcJElDDAdJ0hDDQZI0xHCQJA0xHCRJQwwHSdIQw0GSNMRwkCQNMRwkSUMMB0nSkM6Gg9/1I0nt\n6Ww4vPpq2y2QpNHV2XA4dKjtFkjS6OpsODhykKT2dDYcHDlIUnsMB0nSEMNBkjRk0nBIcm+S8STb\nBsqWJnkkya4kDydZMvDahiS7k+xMcvVA+RVJtiV5Nsldk13XcJCk9kxl5HAfcM2EsvXAo1V1CfAY\nsAEgyWXADcBq4Frg7iRpjvkM8MGquhi4OMnEc/4LhoMktWfScKiqrwMvTSi+DtjUbG8Crm+21wIP\nVNWhqtoD7AbWJFkOnFNV327q3T9wzFEZDpLUnumuOSyrqnGAqtoPLGvKVwB7B+rta8pWAM8PlD/f\nlB2T4SBJ7Vk0Q+eZ8T928Ud/tJFzz+1v93o9er3eTF9Ckua1sbExxsbGZuXcqSn8EaMk5wNfqarL\nm/2dQK+qxpspo8eranWS9UBV1Z1NvYeAO4DnDtdpytcBV1XV7x7jevW97xWXXz4DdyhJIyIJVZXJ\na05uqtNKaX4O2wrc0mzfDGwZKF+XZHGSVcBFwBPN1NOBJGuaBer3DxxzVAcPTrFlkqQZN+m0UpIv\nAD3gF5P8gP5I4BPAl5LcSn9UcANAVe1IshnYARwEbqsjQ5Pbgc8CZwAPVtVDx7uuaw6S1J4pTSvN\ntST1ta8VV17Zdkskaf5oY1ppzjmtJEnt6Ww4vPJK2y2QpNFlOEiShnQ2HJxWkqT2dDYcHDlIUns6\nGw6OHCSpPZ0NB0cOktSezoaDIwdJak9nw8GRgyS1p7Ph4MhBktrT2XBw5CBJ7elsODhykKT2dDYc\nHDlIUns6Gw6OHCSpPZ0NB0cOktQew0GSNKSz4eC0kiS1p7Ph4MhBktrT2XBw5CBJ7elsODhykKT2\ndDYcHDlIUns6Gw6OHCSpPZ0NB0cOktSekwqHJBuSbE+yLcnnkyxOsjTJI0l2JXk4yZIJ9Xcn2Znk\n6uOd25GDJLVn2uGQ5HzgQ8DbqupyYBFwI7AeeLSqLgEeAzY09S8DbgBWA9cCdyfJsc7vyEGS2nMy\nI4d/AF4BzkqyCHgdsA+4DtjU1NkEXN9srwUeqKpDVbUH2A2sOdbJHTlIUnumHQ5V9RLwh8AP6IfC\ngap6FDivqsabOvuBZc0hK4C9A6fY15QdleEgSe1ZNN0Dk1wI/B5wPnAA+FKS9wI1oerE/Sn50Y82\nsnFjf7vX69Hr9abbVElakMbGxhgbG5uVc6dqWu/dJLkB+M2q+lCz/z7gHcC7gF5VjSdZDjxeVauT\nrAeqqu5s6j8E3FFV3zrKuWvVquL735/eTUnSKEpCVR1zLfdEnMyawy7gHUnOaBaW3w3sALYCtzR1\nbga2NNtbgXXNE02rgIuAJ451chekJak9055WqqrvJbkfeBJ4FXgKuAc4B9ic5FbgOfpPKFFVO5Js\nph8gB4Hb6jjDFtccJKk9055Wmk1JaunS4sUX226JJM0fXZlWmlWOHCSpPZ0NB9ccJKk9nQ2HV16B\nDs54SdJI6Gw4nHoqvPpq262QpNHU2XA47TTXHSSpLZ0Nh8WLDQdJakunw8FFaUlqR2fDwWklSWpP\nZ8PBkYMktaez4eDIQZLa09lwcOQgSe3pbDg4cpCk9nQ2HE4/HX72s7ZbIUmjqbPhcMYZhoMktaXT\n4fDTn7bdCkkaTZ0Nh9NPNxwkqS2dDQdHDpLUnk6Hg2sOktSOToeDIwdJaofhIEkaYjhIkoYYDpKk\nIYaDJGnISYVDkiVJvpRkZ5LtSX49ydIkjyTZleThJEsG6m9Isrupf/Xxzm04SFJ7Tnbk8Cngwapa\nDbwF+BtgPfBoVV0CPAZsAEhyGXADsBq4Frg7SY51YsNBktoz7XBI8nrgyqq6D6CqDlXVAeA6YFNT\nbRNwfbO9FnigqbcH2A2sOdb5DQdJas/JjBxWAT9Ocl+S7ya5J8mZwHlVNQ5QVfuBZU39FcDegeP3\nNWVHZThIUnsWneSxVwC3V9V3knyS/pRSTag3cX9KtmzZyLZtsHEj9Ho9er3eSTRVkhaesbExxsbG\nZuXcqZrWezdJzgP+T1Vd2Oz/Bv1w+FWgV1XjSZYDj1fV6iTrgaqqO5v6DwF3VNW3jnLu+qu/Kj79\naXjwwendmCSNmiRU1THXck/EtKeVmqmjvUkuboreDWwHtgK3NGU3A1ua7a3AuiSLk6wCLgKeONb5\n/dtKktSek5lWAvgw8PkkpwHfBz4AnApsTnIr8Bz9J5Soqh1JNgM7gIPAbXWcYYtrDpLUnmlPK82m\nJPXkk8WHPgRPPtl2ayRpfujEtNJs88t+JKk9nQ0Hp5UkqT2GgyRpiOEgSRpiOEiShnQ2HA4vSHfw\nYSpJWvA6Gw6LFsEpp8ChQ223RJJGT2fDAZxakqS2dD4c/BMakjT3Oh8Ojhwkae4ZDpKkIYaDJGlI\n58Phn/+57VZI0ujpdDiceSb80z+13QpJGj2dDoezzoKf/KTtVkjS6Ol0OJx9Nrz8ctutkKTR0+lw\ncOQgSe0wHCRJQzofDk4rSdLc63Q4nH22IwdJakOnw8FpJUlqR+fDwWklSZp7nQ8HRw6SNPdOOhyS\nnJLku0m2NvtLkzySZFeSh5MsGai7IcnuJDuTXD3ZuV1zkKR2zMTI4SPAjoH99cCjVXUJ8BiwASDJ\nZcANwGrgWuDuJDneiZ1WkqR2nFQ4JFkJ/BbwJwPF1wGbmu1NwPXN9lrggao6VFV7gN3AmuOd32kl\nSWrHyY4cPgl8FKiBsvOqahygqvYDy5ryFcDegXr7mrJjMhwkqR2Lpntgkt8Gxqvq6SS941St47x2\nTBs3buTFF2HvXhgb69HrHe8SkjR6xsbGGBsbm5Vzp2pa790k+a/ATcAh4HXAOcBfAr8G9KpqPMly\n4PGqWp1kPVBVdWdz/EPAHVX1raOcu6qK/fvhLW+B8fFpNVGSRkoSquq4a7lTNe1ppar6WFW9saou\nBNYBj1XV+4CvALc01W4GtjTbW4F1SRYnWQVcBDxxvGs4rSRJ7Zj2tNJxfALYnORW4Dn6TyhRVTuS\nbKb/ZNNB4LaaZNhy+Mt+XnsNTun0JzIkaWGZ9rTSbDo8rQT9gPi7v+uPIiRJx9aJaaW54tSSJM29\neREOfhBOkuZW58PBP6EhSXNvXoSDIwdJmludD4clS+DAgbZbIUmjpfPh8IY3wN//fdutkKTRYjhI\nkoYYDpKkIYaDJGmI4SBJGmI4SJKGGA6SpCGGgyRpiOEgSRpiOEiShhgOkqQhnQ+HM86AKvjpT9tu\niSSNjs6HQ+LoQZLmWufDAQwHSZpr8yIcli6Fl15quxWSNDrmRTicey688ELbrZCk0TEvwuG88wwH\nSZpL8yYcxsfbboUkjY5ph0OSlUkeS7I9yTNJPtyUL03ySJJdSR5OsmTgmA1JdifZmeTqqV7LcJCk\nuXUyI4dDwH+sqjcB/wa4PcmlwHrg0aq6BHgM2ACQ5DLgBmA1cC1wd5JM5ULLlhkOkjSXph0OVbW/\nqp5utl8GdgIrgeuATU21TcD1zfZa4IGqOlRVe4DdwJqpXMs1B0maWzOy5pDkAuCtwDeB86pqHPoB\nAixrqq0A9g4ctq8pm5TTSpI0txad7AmSnA18GfhIVb2cpCZUmbg/JRs3bvz59tve1mN8vDfdJkrS\ngjQ2NsbY2NisnDtV03rv7h+cLAK+CvzPqvpUU7YT6FXVeJLlwONVtTrJeqCq6s6m3kPAHVX1raOc\ntwbb9dpr/b+x9PLLsHjxtJsrSQtaEqpqSmu5kznZaaU/BXYcDobGVuCWZvtmYMtA+boki5OsAi4C\nnphSI0+BX/ol1x0kaa5Me1opyTuB9wLPJHmK/vTRx4A7gc1JbgWeo/+EElW1I8lmYAdwELitTmDY\ncnjdYeXK6bZYkjRVJzWtNFsmTisBrF0Lt94K119/jIMkacR1aVppzlxwAezZ03YrJGk0GA6SpCGG\ngyRpyLwKh7/927ZbIUmjYV6Fw549/e+TliTNrnkTDkuX9oPBrwuVpNk3b8IhcWpJkubKvAkHgEsv\nhZ07226FJC188yocLr8ctm1ruxWStPDNq3B485vhmWfaboUkLXzzKhwuv9xwkKS5MK/C4fzz4cAB\neOmltlsiSQvbvAqHU07pTy099VTbLZGkhW1ehQPAlVfC177WdiskaWGbd+HQ68EsfSueJKkxb77P\n4bB//Ef45V+GH/+4/9WhkqS+kfw+h8POOaf/1JJTS5I0e+ZdOAD8zu/AF7/YdiskaeGad9NKAPv2\n9Z9a+uEPnVqSpMNGeloJYMUKePvbYfPmtlsiSQvTvBw5APz1X8Ptt8P27XDqqXPUMEnqsJEfOQC8\n611w7rlwzz1tt0SSFp45D4ck70nyN0meTfL70z9PPxj+4A9gx46ZbKEkaU7DIckpwKeBa4A3ATcm\nuXS651u9Gu66C665ZuEGxJif+Ps5++II++II+2J2zPXIYQ2wu6qeq6qDwAPAdSdzwptugo9/HK66\nqv/vyy/PSDs7w1/8I+yLI+yLI+yL2THX4bAC2Duw/3xTdlJuugm+8Q14+mlYuRJuvBH++I/7ZXv3\nwsGDJ3sFSRoti9puwEy5+OL+B+PGx+GrX4UnnoDPfQ6efx5eeAHOPhvOOgvOPLP/7+mn9//K66mn\n9v+d+HPqqf11jTYMXvfZZ+E735nba3bVrl3w5JNtt6Ib7Isj7IvZMaePsiZ5B7Cxqt7T7K8Hqqru\nnFCve8/XStI8MFOPss51OJwK7ALeDfwIeAK4sap2zlkjJEmTmtNppap6Ncl/AB6hv95xr8EgSd3T\nyU9IS5La1alPSM/UB+TmiyQrkzyWZHuSZ5J8uClfmuSRJLuSPJxkycAxG5LsTrIzydXttX7mJTkl\nyXeTbG32R7IfAJIsSfKl5v62J/n1Ue2P5t62J9mW5PNJFo9KXyS5N8l4km0DZSd870muaPrv2SR3\nTeniVdWJH/pB9f+A84HTgKeBS9tu1yzf83Lgrc322fTXYy4F7gT+c1P++8Anmu3LgKfoTwde0PRX\n2r6PGeyP3wM+B2xt9keyH5p7/CzwgWZ7EbBkFPujeT/4PrC42f8icPOo9AXwG8BbgW0DZSd878C3\ngH/dbD8IXDPZtbs0cpjxD8h1XVXtr6qnm+2XgZ3ASvr3vamptgm4vtleCzxQVYeqag+wm36/zXtJ\nVgK/BfzJQPHI9QNAktcDV1bVfQDNfR5gNPvjH4BXgLOSLAJeB+xjRPqiqr4OvDSh+ITuPcly4Jyq\n+nZT7/6BY46pS+EwKx+Qmy+SXED//xC+CZxXVePQDxBgWVNtYh/tY+H00SeBjwKDi2Cj2A8Aq4Af\nJ7mvmWa7J8mZjGB/VNVLwB8CP6B/Xweq6lFGsC8GLDvBe19B//30sCm9t3YpHEZWkrOBLwMfaUYQ\nE58SWNBPDST5bWC8GUUd7xntBd0PAxYBVwD/vaquAH4CrGfEfi8AklxIf7rxfOBf0R9BvJcR7Ivj\nmJV771I47APeOLC/silb0Jqh8peBP6uqLU3xeJLzmteXAy805fuAXxk4fKH00TuBtUm+D/w58K4k\nfwbsH7F+OOx5YG9VHf5s/F/QD4tR+70A+DXgG1X1YlW9Cvwl8G8Zzb447ETvfVp90qVw+DZwUZLz\nkywG1gFbW27TXPhTYEdVfWqgbCtwS7N9M7BloHxd87TGKuAi+h8knNeq6mNV9caqupD+f/fHqup9\nwFcYoX44rJky2Jvk4qbo3cB2Ruz3orELeEeSM5KEfl/sYLT6IvzLEfUJ3Xsz9XQgyZqmD98/cMyx\ntb0aP2Fl/j30fxl2A+vbbs8c3O87gVfpP5n1FPDdpg9+AXi06YtHgDcMHLOB/lMIO4Gr276HWeiT\nqzjytNIo98Nb6P8P09PA/6D/tNJI9gf9tajtwDb6C7CnjUpfAF8Afgj8jP66yweApSd678DbgWea\n99ZPTeXafghOkjSkS9NKkqSOMBwkSUMMB0nSEMNBkjTEcJAkDTEcJElDDAdJ0hDDQZI05P8Dd6yd\nrJCg/IMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c8d10d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(10, 20, 0.01, 1000)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Или можно было увеличить шаг:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(6.5331862350006843e-22, 3.2138760885179519e-39) 8.53650447624e-43\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10cad1210>]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEACAYAAACznAEdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFMpJREFUeJzt3X+snNV95/H3By5uAjiIpAIndiCmCGpo8wN13TTZqqPQ\nkhAkQ6XI62w3gcCmUqFK1KZobVaqTf8IYauKRN0QKSp1nCqpa4hazJYFr2WmbXabkCYFs9gxVilg\n3HDTqsQRq2hjw3f/mMfx5O79xb3Xz32u5/2SRjzPmTNnzhxG87nnnGfGqSokSTptsTsgSeoGA0GS\nBBgIkqSGgSBJAgwESVLDQJAkAbMIhCT3JBlPsneS+z6R5JUkrx8q25TkYJL9Sa4aKr8iyd4kTyX5\n9MK9BEnSQpjNDGEr8N6JhUlWAb8CPDtUtgZYD6wBrgbuTpLm7s8BN1XVJcAlSf6/NiVJi2fGQKiq\nrwIvTnLXXcCtE8quBbZX1bGqegY4CKxNsgJYXlXfaOp9Ebhuzr2WJC24Oe0hJFkHHKqqJybctRI4\nNHR+uClbCTw/VP58UyZJ6oixV/uAJK8FbmOwXCRJOkW86kAAfgp4C/B4sz+wCvhWkrUMZgQXDNVd\n1ZQdBt48SfmkkvgDS5I0B1WVmWtNbrZLRmluVNX/rqoVVXVRVa1msPzzjqr6LrAT+HdJliVZDVwM\nPFpVLwBHkqxtQuTDwP3TPWFVeati8+bNi96HrtwcC8fCsZj+Nl+zuez0y8D/YnBl0HNJPjLxs5sT\nYbEP2AHsAx4Ebq4TvbwFuAd4CjhYVQ/Nu/eSpAUz45JRVf37Ge6/aML5HcAdk9T7JvCzr7aDkqR2\n+E3ljuv1eovdhc5wLE5wLE5wLBZOFmLdaaElqS72S5K6LAnVwqayJOkUZyBIkgADQZLUMBAkSYCB\nIElqGAiSJMBAkCQ1DARJEmAgSJIaBoIkCTAQJEkNA0GSBBgIkqSGgSBJAgwESVLDQJAkAQaCJKlh\nIEiSAANBktQwECRJgIEgSWrMGAhJ7kkynmTvUNl/SbI/yWNJvpLkdUP3bUpysLn/qqHyK5LsTfJU\nkk8v/EuRJM3HbGYIW4H3TijbBVxeVW8HDgKbAJJcBqwH1gBXA3cnSfOYzwE3VdUlwCVJJrYpSVpE\nMwZCVX0VeHFC2e6qeqU5/RqwqjleB2yvqmNV9QyDsFibZAWwvKq+0dT7InDdAvRfkrRAFmIP4Ubg\nweZ4JXBo6L7DTdlK4Pmh8uebMklSR4zN58FJ/jNwtKr+dIH68yNbtmz50XGv16PX6y30U0jSktbv\n9+n3+wvWXqpq5krJhcADVfXWobIbgI8C76mq/9uUbQSqqu5szh8CNgPPAo9U1ZqmfAPwS1X1G1M8\nX82mX5KkE5JQVZm55uRmu2SU5nb8Sd8H3AqsOx4GjZ3AhiTLkqwGLgYeraoXgCNJ1jabzB8G7p9r\npyVJC2/GJaMkXwZ6wBuSPMfgL/7bgGXA/2guIvpaVd1cVfuS7AD2AUeBm4f+1L8F+ALwGuDBqnpo\ngV+LJGkeZrVk1DaXjCTp1WtryUiSdIozECRJgIEgSWoYCJIkwECQJDUMBEkSYCBIkhqdDYSXX17s\nHkjSaOlsIBw7ttg9kKTR0tlAOHp0sXsgSaPFQJAkAQaCJKnR2UBwD0GS2tXZQHCGIEntMhAkSYCB\nIElqdDYQ3EOQpHZ1NhCcIUhSuwwESRLQ4UBwyUiS2tXZQHCGIEntMhAkSYCBIElqzBgISe5JMp5k\n71DZuUl2JTmQ5OEk5wzdtynJwST7k1w1VH5Fkr1Jnkry6Zme1z0ESWrXbGYIW4H3TijbCOyuqkuB\nPcAmgCSXAeuBNcDVwN1J0jzmc8BNVXUJcEmSiW3+GGcIktSuGQOhqr4KvDih+FpgW3O8DbiuOV4H\nbK+qY1X1DHAQWJtkBbC8qr7R1Pvi0GMmZSBIUrvmuodwXlWNA1TVC8B5TflK4NBQvcNN2Urg+aHy\n55uyKRkIktSusQVqpxaonR+5774t/MM/DI57vR69Xm+hn0KSlrR+v0+/31+w9uYaCONJzq+q8WY5\n6LtN+WHgzUP1VjVlU5VP6ZprtvDrvz7H3knSCJj4x/Ltt98+r/Zmu2SU5nbcTuCG5vh64P6h8g1J\nliVZDVwMPNosKx1JsrbZZP7w0GMm5ZKRJLVrxhlCki8DPeANSZ4DNgOfAu5NciPwLIMri6iqfUl2\nAPuAo8DNVXV8OekW4AvAa4AHq+qh6Z7XQJCkduXE53V3JKnf//3id35nsXsiSUtHEqoqM9ecnN9U\nliQBBoIkqdHZQPCnKySpXZ0NBGcIktQuA0GSBBgIkqRGZwPBPQRJaldnA8EZgiS1y0CQJAEGgiSp\n0dlAcA9BktrV2UBwhiBJ7TIQJEmAgSBJanQ2ENxDkKR2dTYQnCFIUrsMBEkS0OFAcMlIktrV2UBw\nhiBJ7TIQJEmAgSBJanQ2ENxDkKR2dTYQnCFIUrvmFQhJNiV5MsneJF9KsizJuUl2JTmQ5OEk50yo\nfzDJ/iRXTde2gSBJ7ZpzICS5EPgo8I6qeiswBnwQ2AjsrqpLgT3Apqb+ZcB6YA1wNXB3kkzVvoEg\nSe2azwzh+8APgbOSjAGvBQ4D1wLbmjrbgOua43XA9qo6VlXPAAeBtVM17h6CJLVrzoFQVS8CfwA8\nxyAIjlTVbuD8qhpv6rwAnNc8ZCVwaKiJw03ZpJwhSFK7xub6wCQXAb8FXAgcAe5N8mtATag68XxW\nfvCDLWzZMjju9Xr0er25dlWSTkn9fp9+v79g7aVqTp/XJFkP/EpVfbQ5/xDwTuA9QK+qxpOsAB6p\nqjVJNgJVVXc29R8CNlfV1ydpu5LilVfm9qIkaRQloaqm3JudyXz2EA4A70zymmZz+EpgH7ATuKGp\ncz1wf3O8E9jQXIm0GrgYeHSqxqswECSpRXNeMqqqx5N8Efgm8DLw98DngeXAjiQ3As8yuLKIqtqX\nZAeD0DgK3FzTTE/OOGOwj/ATPzHXHkqSXo05LxmdTEnqzDOL8XE4++zF7o0kLQ2LuWR0Up1xhpee\nSlKbOhsIY2NeeipJbepsIBzfQ5AktcNAkCQBHQ8E9xAkqT2dDQT3ECSpXZ0NBJeMJKldBoIkCehw\nIIyNuYcgSW3qbCA4Q5CkdhkIkiSg44HgkpEktaezgeBlp5LUrs4GgktGktQuA0GSBHQ8ENxDkKT2\ndDYQ3EOQpHZ1NhBcMpKkdhkIkiSg44HgHoIktaezgeAegiS1q7OB4JKRJLXLQJAkAfMMhCTnJLk3\nyf4kTyb5+STnJtmV5ECSh5OcM1R/U5KDTf2rpmvbn7+WpHbNd4bwGeDBqloDvA34NrAR2F1VlwJ7\ngE0ASS4D1gNrgKuBu5NkqoadIUhSu+YcCEleB/xiVW0FqKpjVXUEuBbY1lTbBlzXHK8Dtjf1ngEO\nAmunat9AkKR2zWeGsBr4lyRbk3wryeeTnAmcX1XjAFX1AnBeU38lcGjo8Yebskl52akktWtsno+9\nArilqv4uyV0MlotqQr2J57Oye/cWjhyBLVug1+vR6/Xm0VVJOvX0+336/f6CtZeqOX1ek+R84G+r\n6qLm/N8yCISfAnpVNZ5kBfBIVa1JshGoqrqzqf8QsLmqvj5J2/WHf1js3w+f/ezcXpgkjZokVNWU\ne7MzmfOSUbMsdCjJJU3RlcCTwE7ghqbseuD+5ngnsCHJsiSrgYuBR6dq3z0ESWrXfJaMAD4GfCnJ\nGcDTwEeA04EdSW4EnmVwZRFVtS/JDmAfcBS4uaaZnriHIEntmlcgVNXjwL+Z5K5fnqL+HcAds2nb\nn66QpHb5TWVJEmAgSJIanQ0Ef7pCktrV2UBwhiBJ7TIQJEmAgSBJanQ2ENxDkKR2dTYQnCFIUrsM\nBEkS0PFAcMlIktrT2UDwpyskqV2dDQSXjCSpXQaCJAnocCB42akktauzgeAMQZLaZSBIkgADQZLU\n6GwguIcgSe3qbCA4Q5CkdnU2EE4/HV55ZXCTJJ18nQ2ExFmCJLWps4EA7iNIUps6HQjOECSpPfMO\nhCSnJflWkp3N+blJdiU5kOThJOcM1d2U5GCS/UmumqltA0GS2rMQM4SPA/uGzjcCu6vqUmAPsAkg\nyWXAemANcDVwd5JM17BLRpLUnnkFQpJVwPuBPxoqvhbY1hxvA65rjtcB26vqWFU9AxwE1k7XvjME\nSWrPfGcIdwG3AjVUdn5VjQNU1QvAeU35SuDQUL3DTdmUDARJas/YXB+Y5BpgvKoeS9KbpmpNc9+U\ntmzZwpEjcNdd8IEP9Oj1pnsKSRo9/X6ffr+/YO2lak6f1yT5JPAfgGPAa4HlwJ8DPwf0qmo8yQrg\nkapak2QjUFV1Z/P4h4DNVfX1SdququLyy+HP/gx+5mfm1EVJGilJqKpp92anM+clo6q6raouqKqL\ngA3Anqr6EPAAcENT7Xrg/uZ4J7AhybIkq4GLgUenew6XjCSpPXNeMprGp4AdSW4EnmVwZRFVtS/J\nDgZXJB0Fbq4ZpicGgiS1Z0ECoar+Cvir5vhfgV+eot4dwB2zbddAkKT2dPqbyn4PQZLa0+lAcIYg\nSe0xECRJwBIIBJeMJKkdnQ6EsTFnCJLUlk4HgktGktQeA0GSBHQ8ELzsVJLa0+lAcIYgSe0xECRJ\ngIEgSWp0OhDcQ5Ck9nQ6EJwhSFJ7DARJEmAgSJIanQ4E9xAkqT2dDoRly+CHP1zsXkjSaOh0IPzk\nT8I///Ni90KSRkOnA+FNb4J/+qfF7oUkjQYDQZIEGAiSpEanA+H1r4cf/GBwkySdXJ0OhATe+Eb4\nzncWuyeSdOqbcyAkWZVkT5InkzyR5GNN+blJdiU5kOThJOcMPWZTkoNJ9ie5ajbP47KRJLVjPjOE\nY8BvV9XlwC8AtyT5aWAjsLuqLgX2AJsAklwGrAfWAFcDdyfJTE9iIEhSO+YcCFX1QlU91hy/BOwH\nVgHXAtuaatuA65rjdcD2qjpWVc8AB4G1Mz2PgSBJ7ViQPYQkbwHeDnwNOL+qxmEQGsB5TbWVwKGh\nhx1uyqZlIEhSO8bm20CSs4H7gI9X1UtJakKVieezsmXLFgAefxxeeqkH9ObeSUk6BfX7ffr9/oK1\nl6o5fV4PHpyMAf8N+O9V9ZmmbD/Qq6rxJCuAR6pqTZKNQFXVnU29h4DNVfX1Sdqt4/3avRs++UnY\ns2fO3ZSkkZCEqppxb3Yq810y+mNg3/EwaOwEbmiOrwfuHyrfkGRZktXAxcCjMz2BS0aS1I45zxCS\nvBv4a+AJBstCBdzG4EN+B/Bm4FlgfVV9r3nMJuAm4CiDJaZdU7T9oxnC974HF1wA3//+nLopSSNj\nvjOEeS0ZnSzDgVAFZ50F4+OwfPkid0ySOmyxl4xOumSwbOS3lSXp5Op8IID7CJLUhiUTCM4QJOnk\nWjKB4AxBkk4uA0GSBBgIkqSGgSBJAgwESVJjSQTCG984CIQOfodOkk4ZSyIQli+H00/35ysk6WRa\nEoEALhtJ0slmIEiSgCUUCMf3ESRJJ8eSCQRnCJJ0ci2pQPD3jCTp5FlSgeAMQZJOHgNBkgQsoUBY\nuRL+8R/hlVcWuyeSdGpaMoGwejWsWAH337/YPZGkU9OSCYQEfvd34fd+z5+wkKSTYckEAsC6dYP/\nPvDA4vZDkk5FSyoQjs8Sbr/dWYIkLbTWAyHJ+5J8O8lTSf7Tq338tdfCsWPwl395MnonSaOr1UBI\nchrwX4H3ApcDH0zy06+mjdNOG61ZQr/fX+wudIZjcYJjcYJjsXDaniGsBQ5W1bNVdRTYDlz7ahv5\n1V8dzBI+8AF47LEF72On+GY/wbE4wbE4wbFYOG0Hwkrg0ND5803Zq3LaafA3fwPvehe8//1wzTXw\nla/A3r3w0ksL1ldJGilji92BuTr7bPjEJ+CWW2DrVvjCF+DppwdfXjvzzME/qnPmmXDWWbBs2eAf\n2BkbG/w3+fEb/PjxcRPPF8OBA/DNby52L7rBsTjBsTjBsVg4qRYX4pO8E9hSVe9rzjcCVVV3Tqg3\nArsDkrTwqmrOf8q2HQinAweAK4HvAI8CH6yq/a11QpI0qVaXjKrq5SS/CexisH9xj2EgSd3Q6gxB\nktRdnfqm8ny/tLaUJVmVZE+SJ5M8keRjTfm5SXYlOZDk4STnLHZf25LktCTfSrKzOR/JsUhyTpJ7\nk+xv3h8/P8JjsakZg71JvpRk2aiMRZJ7kown2TtUNuVrb8bqYPO+uWo2z9GZQFiIL60tcceA366q\ny4FfAG5pXv9GYHdVXQrsATYtYh/b9nFg39D5qI7FZ4AHq2oN8Dbg24zgWCS5EPgo8I6qeiuDJe8P\nMjpjsZXB5+OwSV97ksuA9cAa4Grg7mTm6yY7Ewgs0JfWlqqqeqGqHmuOXwL2A6sYjMG2pto24LrF\n6WG7kqwC3g/80VDxyI1FktcBv1hVWwGq6lhVHWEExwL4PvBD4KwkY8BrgcOMyFhU1VeBFycUT/Xa\n1wHbm/fLM8BBBp+x0+pSICzIl9ZOBUneArwd+BpwflWNwyA0gPMWr2etugu4FRje5BrFsVgN/EuS\nrc3y2eeTnMkIjkVVvQj8AfAcgyA4UlW7GcGxGHLeFK994ufpYWbxedqlQBCQ5GzgPuDjzUxh4q7/\nKX8VQJJrgPFmxjTdNPeUHwsGyyJXAJ+tqiuA/8NgmWAU3xcXAb8FXAi8icFM4dcYwbGYxrxee5cC\n4TBwwdD5qqZsZDTT4PuAP6mq4/823HiS85v7VwDfXaz+tejdwLokTwN/CrwnyZ8AL4zgWDwPHKqq\nv2vOv8IgIEbxffFzwP+sqn+tqpeBPwfexWiOxXFTvfbDwJuH6s3q87RLgfAN4OIkFyZZBmwAdi5y\nn9r2x8C+qvrMUNlO4Ibm+HrglP9HRKvqtqq6oKouYvA+2FNVHwIeYPTGYhw4lOSSpuhK4ElG8H3B\n4Eut70zymmaD9EoGFx2M0liEH581T/XadwIbmquwVgMXM/gi8PSNd+l7CEnex+CKiuNfWvvUInep\nNUneDfw18ASDaV8BtzH4n7iDQdo/C6yvqu8tVj/bluSXgE9U1bokr2cExyLJ2xhsrp8BPA18BDid\n0RyLWxl8AL4M/D3wH4HljMBYJPky0APeAIwDm4G/AO5lkteeZBNwE3CUwRL0rhmfo0uBIElaPF1a\nMpIkLSIDQZIEGAiSpIaBIEkCDARJUsNAkCQBBoIkqWEgSJIA+H+rBfTGT+X3qQAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c9f5290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(10, 20, 0.1, 100)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Если посмотреть на графики, то видно что начиная с некоторой итерациизначение функции не меняется. То есть скорее всего мы уже нашли оптимальную точку. Так что давайте добавим следующий критерий остановки: если предыдущая и следующая точки отличаются не сильно, то градиентный спуск останавливается. Для этого нужно всего лишь дописать пару строчек:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def grad_descent(x1, x2, step, max_iters):\n",
    "    points = [(x1, x2)]\n",
    "    values = [function(x1, x2)]\n",
    "    for iters in range(max_iters):\n",
    "        x1_new, x2_new = np.array([x1, x2]) - step * grad(x1, x2)\n",
    "        if np.linalg.norm(np.array([x1_new, x2_new]) - np.array([x1, x2])) < 1e-5:\n",
    "            break\n",
    "        x1, x2 = x1_new, x2_new\n",
    "        points.append((x1, x2))\n",
    "        values.append(function(x1, x2))\n",
    "    print iters\n",
    "    return points, values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "26\n",
      "(1.7058172817957805e-05, 9.0071992547409624e-10) 5.81962522209e-10\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10cc5d610>]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAF1lJREFUeJzt3X2wHFWdxvHvkzeSQAgRgWiCkBBgA+JClKgouyNqALGS\nuFSFCCuvZa0bUGu1LBK2ylz+EMTaXcHawioFMSgsBBQTMRsiFUYLVwgCIUBiuFVUIInmgkh4FQjk\nt39032S43NeZnjvd08+naoqeMz3dp2m4z5xz+nQrIjAzs3Ia0eoKmJlZ6zgEzMxKzCFgZlZiDgEz\nsxJzCJiZlZhDwMysxAYMAUnXS+qStKGXz74uabekd9WULZHUKWmTpDk15bMkbZD0hKSrszsEMzOr\n12BaAjcAp/YslDQV+DTwVE3ZTGABMBM4HbhWktKPvw9cFBFHAUdJesc2zcxseA0YAhFxL/B8Lx99\nF/hGj7J5wC0R8WZEbAE6gdmSJgMTIuKBdL0bgfl119rMzDJR15iApLnA1oh4tMdHU4CtNe+3p2VT\ngG015dvSMjMza6FRQ/2CpHHAZSRdQWZmVmBDDgHgCOBw4JG0v38q8JCk2SS//N9Xs+7UtGw7cGgv\n5b2S5BsamZnVISI08Fp7DbY7SOmLiHgsIiZHxPSImEbStXNCRDwDrATOkjRG0jRgBrAuInYAL0ia\nnQbHucCKAQ6kbV9Lly5teR18bD4+H1/7veoxmEtEbwb+j+SKnqclXdDz7zV7A2IjsBzYCKwCFsXe\nml0MXA88AXRGxOq6amxmZpkZsDsoIs4e4PPpPd5fCVzZy3oPAscNtYJmZtY8njHcApVKpdVVaJp2\nPjbw8RVdux9fPVRvP1IzSYo81svMLM8kEU0aGDYzszbkEDAzKzGHgJlZiTkEzMxKzCFgZlZiDgEz\nsxLLbQjs3t3qGpiZtb/chsBLL7W6BmZm7S+3IfDCC62ugZlZ+8ttCOzc2eoamJm1P4eAmVmJOQTM\nzEostyHgMQEzs+bLbQi4JWBm1nwOATOzEnMImJmVmEPAzKzEchsCHhg2M2u+3IaAWwJmZs3nEDAz\nKzGHgJlZiQ0YApKul9QlaUNN2XckbZK0XtLPJO1f89kSSZ3p53NqymdJ2iDpCUlXD7RfjwmYmTXf\nYFoCNwCn9ihbAxwbEccDncASAEnHAAuAmcDpwLWSlH7n+8BFEXEUcJSkntt8m507IWLQx2FmZnUY\nMAQi4l7g+R5ld0dE92Nf7gOmpstzgVsi4s2I2EISELMlTQYmRMQD6Xo3AvP72+/o0fDqq4M+DjMz\nq0MWYwIXAqvS5SnA1prPtqdlU4BtNeXb0rI+HXCAxwXMzJptVCNflvTvwK6I+J+M6rPHrl0dXHEF\nHHQQVCoVKpVK1rswMyu0arVKtVptaBt1h4Ck84HPAKfUFG8HDq15PzUt66u8TzNmdHDOOXDSSfXW\n0MysvfX8gXz55ZcPeRuD7Q5S+kreSKcB3wDmRsTrNeutBBZKGiNpGjADWBcRO4AXJM1OB4rPBVb0\nt0N3B5mZNd+ALQFJNwMV4EBJTwNLgcuAMcCv04t/7ouIRRGxUdJyYCOwC1gUsecan4uBHwNjgVUR\nsbq//ToEzMyaT5HD6zAlxZe+FBx3HCxa1OramJkVgyQiQgOvuVduZwxPnOgJY2ZmzZbbEHB3kJlZ\n8zkEzMxKzCFgZlZiuQ0BjwmYmTVfbkPALQEzs+ZzCJiZlZhDwMysxBwCZmYlltsQGDsWdu+G115r\ndU3MzNpXbkNASloDvkLIzKx5chsC4C4hM7NmcwiYmZVYrkPAE8bMzJor1yHgloCZWXM5BMzMSswh\nYGZWYg4BM7MSy3UIeGDYzKy5ch0CbgmYmTWXQ8DMrMQcAmZmJZbrEPCYgJlZcw0YApKul9QlaUNN\n2SRJayRtlnSXpIk1ny2R1Clpk6Q5NeWzJG2Q9ISkqwdTObcEzMyaazAtgRuAU3uULQbujoijgbXA\nEgBJxwALgJnA6cC1kpR+5/vARRFxFHCUpJ7bfAeHgJlZcw0YAhFxL/B8j+J5wLJ0eRkwP12eC9wS\nEW9GxBagE5gtaTIwISIeSNe7seY7fdpvv+R5Art2DXgcZmZWh3rHBA6OiC6AiNgBHJyWTwG21qy3\nPS2bAmyrKd+WlvVLgv33hxdfrLOWZmbWr1EZbScy2s4eHR0dyYYDVq+ucM45lax3YWZWaNVqlWq1\n2tA2FDHw329JhwG/jIgPpO83AZWI6Eq7eu6JiJmSFgMREVel660GlgJPda+Tli8E/jEi/rWP/UV3\nvWbNgh/+ED74wYaO08ys7UkiIjTwmnsNtjtI6avbSuD8dPk8YEVN+UJJYyRNA2YA69IuoxckzU4H\nis+t+U6/PDhsZtY8A3YHSboZqAAHSnqa5Jf9t4HbJF1I8it/AUBEbJS0HNgI7AIWxd6mxsXAj4Gx\nwKqIWD2YCjoEzMyaZ1DdQcOttjvoggvg5JPhwgtbXCkzs5xrZndQy7glYGbWPA4BM7MScwiYmZVY\n7kPAN5EzM2ue3IeAWwJmZs3jEDAzKzGHgJlZieU+BCZOdAiYmTVL7kPggAM8MGxm1iy5nzH81lsw\nZkzyTIERuY8sM7PWacsZwyNHJg+X8TMFzMyyl/sQAA8Om5k1SyFCwBPGzMyaoxAh4JaAmVlzOATM\nzErMIWBmVmKFCAFPGDMza45ChIAnjJmZNUdhQsAtATOz7DkEzMxKzCFgZlZihQgBTxYzM2uOQoSA\nWwJmZs3RUAhIWiLpcUkbJN0kaYykSZLWSNos6S5JE3us3ylpk6Q5g92PQ8DMrDnqDgFJhwFfBE6I\niA8Ao4DPA4uBuyPiaGAtsCRd/xhgATATOB24VtKgbnnqEDAza45GWgIvAm8A+0oaBYwDtgPzgGXp\nOsuA+enyXOCWiHgzIrYAncDsweyoe0wgh48+MDMrtLpDICKeB/4TeJrkj/8LEXE3cEhEdKXr7AAO\nTr8yBdhas4ntadmARo+GffaBV16pt7ZmZtabUfV+UdJ04N+Aw4AXgNsknQP0/L1e1+/3jo6OPcuV\nSoUDDqiwc2fygBkzM4NqtUq1Wm1oG3U/XlLSAuDTEfHF9P0XgI8ApwCViOiSNBm4JyJmSloMRERc\nla6/GlgaEff3su3oWa9jj4Vbb4X3v7+u6pqZtb3hfrzkZuAjksamA7yfBDYCK4Hz03XOA1akyyuB\nhekVRNOAGcC6we7MN5EzM8te3d1BEfGIpBuBB4G3gIeBHwATgOWSLgSeIrkiiIjYKGk5SVDsAha9\n4+d+P3wTOTOz7NXdHdRMvXUHnX02nHEGnHNOiyplZpZzw90dNKw8V8DMLHsOATOzEitMCPgmcmZm\n2StMCLglYGaWPYeAmVmJOQTMzEqsMCHgyWJmZtkrTAh4spiZWfYKFQJuCZiZZatwIZDDCc5mZoVV\nmBAYOxYkeO21VtfEzKx9FCYEwBPGzMyyVqgQ8LiAmVm2HAJmZiXmEDAzK7FChYAnjJmZZatQIeAJ\nY2Zm2SpcCLglYGaWHYeAmVmJFSoEPCZgZpatQoWAxwTMzLJVuBBwS8DMLDsOATOzEmsoBCRNlHSb\npE2SHpf0YUmTJK2RtFnSXZIm1qy/RFJnuv6coe7PIWBmlq1GWwLXAKsiYibw98AfgcXA3RFxNLAW\nWAIg6RhgATATOB24VpKGsjPfQM7MLFt1h4Ck/YGTI+IGgIh4MyJeAOYBy9LVlgHz0+W5wC3peluA\nTmD2UPbploCZWbYaaQlMA/4i6QZJD0n6gaTxwCER0QUQETuAg9P1pwBba76/PS0btH33hddfhzfe\naKDWZma2x6gGvzsLuDgi/iDpuyRdQT2f/VXXs8A6Ojr2LFcqFSqVCtLey0QPOqjOWpuZtYlqtUq1\nWm1oG4o6n9co6RDg9xExPX3/cZIQOAKoRESXpMnAPRExU9JiICLiqnT91cDSiLi/l21HX/U64ghY\nvRqOPLKuapuZtS1JRMSQxlrr7g5Ku3y2SjoqLfok8DiwEjg/LTsPWJEurwQWShojaRowA1g31P16\nwpiZWXYa6Q4C+Apwk6TRwJPABcBIYLmkC4GnSK4IIiI2SloObAR2AYv6/LnfDw8Om5llp6EQiIhH\ngBN7+ehTfax/JXBlI/t0CJiZZadQM4bBIWBmlqXChYAnjJmZZadwIeCWgJlZdhwCZmYl5hAwMyux\nwoWAny5mZpadwoWAJ4uZmWWnkCHgloCZWTYcAmZmJVa4EPCYgJlZduq+i2gz9XcX0d27YfTo5JkC\nI0cOc8XMzHJsWO8i2iojRsCECfDii62uiZlZ8RUuBMDjAmZmWXEImJmVWCFDwIPDZmbZKGQIeMKY\nmVk2ChsCbgmYmTXOIWBmVmKFDAGPCZiZZaOQIeAxATOzbBQ2BNwSMDNrnEPAzKzEChkCHhMwM8tG\nwyEgaYSkhyStTN9PkrRG0mZJd0maWLPuEkmdkjZJmlPvPj0mYGaWjSxaAl8FNta8XwzcHRFHA2uB\nJQCSjgEWADOB04FrJQ3pbnfd3B1kZpaNhkJA0lTgM8B1NcXzgGXp8jJgfro8F7glIt6MiC1AJzC7\nnv06BMzMstFoS+C7wDeA2pv/HxIRXQARsQM4OC2fAmytWW97WjZkEycm3UG7d9fzbTMz6zaq3i9K\nOgPoioj1kir9rFrXU2s6Ojr2LFcqFSqVvbsYNQrGjYOXX4b9969n62ZmxVetVqlWqw1to+4ni0m6\nAvhn4E1gHDABuAP4EFCJiC5Jk4F7ImKmpMVARMRV6fdXA0sj4v5ett3nk8W6TZ0Kv/89HHpoXdU3\nM2s7w/pksYi4LCLeFxHTgYXA2oj4AvBL4Px0tfOAFenySmChpDGSpgEzgHX17t/jAmZmjau7O6gf\n3waWS7oQeIrkiiAiYqOk5SRXEu0CFg34c78fDgEzs8ZlEgIR8RvgN+nyX4FP9bHelcCVWezTE8bM\nzBpXyBnD4AljZmZZKHQIuCVgZtYYh4CZWYk5BMzMSqywIeCBYTOzxhU2BDwwbGbWuEKHgFsCZmaN\ncQiYmZVYYUPAYwJmZo0rbAh4TMDMrHGFDYHulkD9dx8yM7PChsDYsTByJPztb62uiZlZcRU2BMDj\nAmZmjSp0CPgKITOzxhQ+BDw4bGZWv8KHgFsCZmb1K3QITJoEzzzT6lqYmRVXoUOgUoHVq1tdCzOz\n4lIDj/ltGkmDevzws8/CkUfCn/4E48cPQ8XMzHJMEhGhoXyn0C2Bgw6CE0+EVataXRMzs2IqdAgA\nnHUW3Hprq2thZlZMhe4OAnjuOZg+HbZvh/32a3LFzMxyrHTdQQAHHggnnQS/+lWra2JmVjx1h4Ck\nqZLWSnpc0qOSvpKWT5K0RtJmSXdJmljznSWSOiVtkjQniwMAWLDAXUJmZvWouztI0mRgckSsl7Qf\n8CAwD7gAeC4iviPpUmBSRCyWdAxwE3AiMBW4Gziyt36foXQHATz/PBx+OGzbBhMm1HU4ZmaFN6zd\nQRGxIyLWp8svA5tI/rjPA5alqy0D5qfLc4FbIuLNiNgCdAKz691/rUmT4OSTYeXKLLZmZlYemYwJ\nSDocOB64DzgkIrogCQrg4HS1KcDWmq9tT8sysWABLF+e1dbMzMphVKMbSLuCbge+GhEvS+rZj1NX\nf1NHR8ee5UqlQqVS6Xf9efPgy19Obig3cWK/q5qZtYVqtUq1Wm1oGw1dIippFHAn8L8RcU1atgmo\nRERXOm5wT0TMlLQYiIi4Kl1vNbA0Iu7vZbtDGhPoNm8enHkmnHtu3YdkZlZYrbhE9EfAxu4ASK0E\nzk+XzwNW1JQvlDRG0jRgBrCuwf2/zVlnuUvIzGwoGrk66GPAb4FHSbp8AriM5A/7cuBQ4ClgQUTs\nTL+zBLgI2EXSfbSmj23X1RJ46SWYOhW2bEkGi83MyqSelkDhZwz3dOaZ8NnPwgUXZFwpM7OcK+WM\n4Z58lZCZ2eC1XUvg5ZdhyhR48snklhJmZmXhlgDJTeROPRXuuKPVNTEzy7+2CwHwvYTMzAar7bqD\nAF59Fd77XujsTB48Y2ZWBu4OSo0fD6efDj//eatrYmaWb20ZAuAnjpmZDUZbdgcBvPYavOc9sGkT\nTJ6cUcXMzHLM3UE1xo6FM86An/2s1TUxM8uvtg0B8L2EzMwG0rbdQQCvv550CT32WHK1kJlZO3N3\nUA/77ANz58Ltt7e6JmZm+dTWIQC+l5CZWX/aujsI4I03ki6h9evh0EMz2aSZWS65O6gXY8bA/Pnu\nEjIz603bhwC4S8jMrC9t3x0EsGtXcnXQH/4Ahx2W2WbNzHLF3UF9GD0aPvc5WLoUXnml1bUxM8uP\nUoQAwLe+lbQIjj0WfvELyGEDyMxs2JWiO6jW2rVw8cUwfTp873twxBFN2Y2Z2bBzd9AgnHIKPPII\n/MM/wIc/DJdfntxszsysjEoXApBcNnrppfDQQ7BhQ9JFtGpVq2tlZjb8hr07SNJpwNUkAXR9RFzV\nyzpN6w7qzerVcMklcNxxcPXVvoLIzIop991BkkYA/w2cChwLfF7S3w1nHXpz2mnJTeZOOAFmzYIr\nr0xmGjdLtVpt3sZbrJ2PDXx8Rdfux1eP4e4Omg10RsRTEbELuAWYN8x16NXYsfDNb8IDD8Dvfpc8\nm/jEE+Hss5NLS3/6U7j/fvjrXxvfVzv/h9jOxwY+vqJr9+Orx6hh3t8UYGvN+20kwZAb06fDnXfC\nX/6SPKi++3XnnXuXR4+GI4/c+zr8cJgwIXm28fjxsO++e5e7348dCxpSI83MrPmGOwQK493vTl4f\n/ejbyyPgmWfeHhBr1iST0F59de8/ey6//jqMG5e8XnsNrrsORo58+2vUqHe+l/p/jRjx9vfdBlru\nK5CGWt7TE0/Agw8Obt0i2rzZx1dkQzm+r30NPvGJ5tYnD4Z1YFjSR4COiDgtfb8YiJ6Dw5LyN3nB\nzKwAhjowPNwhMBLYDHwS+DOwDvh8RGwatkqYmdkew9odFBFvSboEWMPeS0QdAGZmLZLL20aYmdnw\nyNWMYUmnSfqjpCckXdrq+mRN0hZJj0h6WNK6VtenUZKul9QlaUNN2SRJayRtlnSXpImtrGMj+ji+\npZK2SXoofZ3WyjrWS9JUSWslPS7pUUlfScvb4vz1cnxfTsvb5fztI+n+9G/J45KuSMuHfP5y0xJI\nJ5I9QTJe8CfgAWBhRPyxpRXLkKQngQ9GxPOtrksWJH0ceBm4MSI+kJZdBTwXEd9Jg3xSRCxuZT3r\n1cfxLQVeioj/amnlGiRpMjA5ItZL2g94kGTOzgW0wfnr5/jOog3OH4Ck8RHxajrW+jvg68Bchnj+\n8tQSyO1EsgyJfP07b0hE3Av0DLR5wLJ0eRkwf1grlaE+jg+S81hoEbEjItanyy8Dm4CptMn56+P4\npqQfF/78AUTEq+niPiR/V56njvOXpz9IvU0km9LHukUVwK8lPSDpi62uTJMcHBFdkPyPCBzc4vo0\nwyWS1ku6rqjdJbUkHQ4cD9wHHNJu56/m+O5Pi9ri/EkaIelhYAdQjYiN1HH+8hQCZfCxiJgFfAa4\nOO1uaHf56G/MzrXA9Ig4nuR/vkJ3K6RdJbcDX01/Mfc8X4U+f70cX9ucv4jYHREnkLTgTpZUoY7z\nl6cQ2A68r+b91LSsbUTEn9N/PgvcQc5umZGRLkmHwJ5+2WdaXJ9MRcSzNbe4/SFwYivr0whJo0j+\nQP4kIlakxW1z/no7vnY6f90i4kVgFfAh6jh/eQqBB4AZkg6TNAZYCKxscZ0yI2l8+qsESfsCc4DH\nWlurTIi397GuBM5Pl88DVvT8QsG87fjS/7G6/RPFPoc/AjZGxDU1Ze10/t5xfO1y/iS9u7srS9I4\n4NPAw9Rx/nJzdRDsedbANeydSPbtFlcpM5Kmkfz6D5JJejcV/fgk3QxUgAOBLmAp8AvgNuBQ4Clg\nQUTsbFUdG9HH8X2CpH95N7AF+JfuPtgikfQx4LfAoyT/TQZwGcks/uUU/Pz1c3xn0x7n7ziSgd/u\ni01+EhH/IeldDPH85SoEzMxseOWpO8jMzIaZQ8DMrMQcAmZmJeYQMDMrMYeAmVmJOQTMzErMIWBm\nVmIOATOzEvt/BJshS/cM6wEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c947290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(10, 20, 0.1, 100)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Теперь градиентный спуск проделал всего 55 итераций."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "По факту нахождение оптимального вектора весов (например, в случае линейных алгоритмов), это тоже оптимизационная задача, но уже мы пытаемся оптимизировать функционал качества. \n",
    "\n",
    "В качестве функционала можно взять упомянутый ранее функционал квадратичных потерь. Пусть мы считаем, что зависимость описывается уравнением $ y = a + b * x_1^2$ и нам необходимо найти коэффициенты a и b. Для этого введем фиктивный признак, везде равный 1. Тогда можно переписать уравнение следующим образом: $ y = a * x_1 + b * x_2^2$.\n",
    "\n",
    "Так как мы оптимизируем функционал и пытаемся найти коэффициенты a и b, необходимо записать градиент Q по этим переменным."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 154,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def function(x1, x2, a, b):\n",
    "    return a * x1 + b * x2 ** 2\n",
    "\n",
    "def Q(x1, x2, a, b, y):\n",
    "    return np.sum((function(x1, x2, a, b) - y) ** 2)\n",
    "\n",
    "def grad_Q(x1, x2, a, b, y):\n",
    "    return np.array([np.sum(2 * (function(x1, x2, a, b) - y) * x1), \n",
    "                     np.sum(2 * (function(x1, x2, a, b) - y) * x2 ** 2)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Отлично, теперь нам будет передаваться в градиентный спуск некоторая выборка, а мы будем по ней подбирать коэффициенты a и b. Небольшой модификацией предыдущего кода, получаем код, решающий нашу задачу:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def grad_descent(x1, x2, y, a0, b0, step, max_iters):\n",
    "    points = [(a0, b0)]\n",
    "    values = [Q(x1, x2, a0, b0, y)]\n",
    "    a, b = a0, b0\n",
    "    for iters in range(max_iters):\n",
    "        a_new, b_new = np.array([a, b]) - step * grad_Q(x1, x2, a, b, y)\n",
    "        if np.linalg.norm(np.array([a_new, b_new]) - np.array([a, b])) < 1e-5:\n",
    "            break\n",
    "        a, b = a_new, b_new\n",
    "        points.append((a, b))\n",
    "        values.append(Q(x1, x2, a, b, y))\n",
    "    print iters\n",
    "    return points, values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Только нужно сгенерировать выборку. Предполагаем, что истинная зависимость $ y = 2 + 3 * x_1^2$ (то есь настоящие a и b равны 2 и 3 соответственно, но об этом никто не знает). И добавим немного шума в ответы."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n = 20\n",
    "x1 = np.ones((1, 20))\n",
    "x2 = np.linspace(0, 10, 20)\n",
    "y = function(x1, x2, 2, 3) + np.random.normal(0, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Запустим наш спуск из случайной начальной точки 0, 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "99\n",
      "(-1.8552849864902168e+292, -1.1704828397605956e+294) inf\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py:5: RuntimeWarning: overflow encountered in square\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10725d810>]"
      ]
     },
     "execution_count": 167,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEGCAYAAACevtWaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE3BJREFUeJzt3X+M5Hd93/Hny74zKpC6wShGtbETcDCNiwGrXNxCknGA\n+EJKHFVVi12RhArqqlyClCi1iRp5UdvUSLaaBIfgo87JRKEmBCUckg1HMNOIJoAh/pVwZ5s4cc4X\nc9StoTVR7hbfu398525ndvZ2Z+9md3Y+fj6k0c73O5/9zvujvXvt597f73cuVYUkqS1nzLoASdL0\nGe6S1CDDXZIaZLhLUoMMd0lqkOEuSQ3a9HBPcluSw0kemGDstUkeSHJvkj9KcunQaz+V5OEkDyX5\nyaH9e5I8OviePxn+Hkl6tshmX+ee5HXA08CHqmrV4E3y/Kp6evD8zcC7quoNSb4T+BJwGRDgy8Bl\nVfXNJHuAvVX1exs6EUnawjZ95V5VnwOeGt6X5CVJ7kpyT5L/keRlg7FPDw17PvDk4PmVwL6q+mZV\nfQPYB+wcGmu7SdKz2lYJwd3Arqp6DfALwG8cfyHJv0vyVeBm4N2D3ecBB4e+/9Bg33G/nOS+JDcn\n2b6xpUvS1jPzcE/yPOCfAB9Nci9wK3Du8der6v1VdRHwc8BvTnDI66vqYuA1wDnAddOvWpK2tpmH\nO10NT1XVZVX16sHjH64w7iPAqwfPDwEXDL12/mAfVXV48HUR2APs2LDKJWmLWjPc17q6Jck1Se4f\nPD6X5BUTvG8GD6rq/wF/keSfDx3z0sHXi4a+558CDw6efwp4Y5KzBydX3zjYR5IXDb4G+AngTyeo\nR5Kasm2CMXuA9wEfOsnrjwI/OLhSZSfwQeDykx0syYeBHnBOkr8CbgD+FfCBJP9hUNMdwAPAriRv\nAI7SnUx9G0BVPZXkP9JdMVPAewYnVgF+O8kL6X553Af82wnmKElNmehSyCQXAp+Y4NLFvwc8WFUv\nnlJ9kqRTMO2e+9uBu6Z8TEnSOk3SlplIkivo2iavm9YxJUmnZirhPjgBuhvYWVVPrTLO//ZJkk5B\nVWU94ydty5y4umXsheQC4GPAW6vqz9c6UFU1+7jhhhtmXoPzc37Ptrk9G+Z3KtZcuZ/k6pazupyu\n3cAvAS8A3j+4/HCxqry2XJJmaM1wr6pr1nj9HcA7plaRJOm0bYU7VJvR6/VmXcKGcn7zq+W5Qfvz\nOxWb+pG/SWoz30+SWpCE2qATqpKkOWK4S1KDDHdJapDhLkkNMtwlNenGG+H++2ddxewY7pKatG8f\nPPnk2uNaZbhLatLRo3DWWbOuYnYMd0lNOnIEnvOcWVcxO4a7pCYZ7pLUIMNdkhpkz12SGuTKXZIa\nZLhLUoMMd0lqkD13SWrMsWOwuGi4S1JTjq/as67/3qIthruk5jzb++1guEtq0LO93w6Gu6QGuXI3\n3CU1yHA33CU16OhRw91wl9ScI0fsuRvukppjW8Zwl9Qgw91wl9Qge+4ThHuS25IcTvLAKmN+Lckj\nSe5L8qrplihJ62PPfbKV+x7gypO9mORHgZdW1fcC1wIfmFJtknRKbMtMEO5V9TngqVWGXAV8aDD2\nC8DZSc6dTnmStH6G+3R67ucBB4e2Dw32SdJM2HP3hKqkBtlzh21TOMYh4MVD2+cP9q1oYWHhxPNe\nr0ev15tCCZK0ZN7bMv1+n36/f1rHSFWtPSj5buATVfWKFV57E/DOqvqxJJcDv1JVl5/kODXJ+0nS\n6bjpJnjiCbj55llXMh1JqKp1fTr9miv3JB8GesA5Sf4KuAE4C6iq2l1VdyZ5U5KvAt8C3rb+0iVp\nevzI3wnCvaqumWDMrumUI0mnb97bMtPgCVVJzTHcDXdJDTLcDXdJDbLnbrhLapArd8NdUoMMd8Nd\nUoP8+AHDXVKD/PgBw11Sg2zLGO6SGmS4G+6SGmTP3XCX1CB77oa7pAbZljHcJTXIcDfcJTXInrvh\nLqlB9twNd0kNsi1juEtqkOFuuEtqkD13w11SY44dg8VF2L591pXMluEuqSnH/6OOZNaVzJbhLqkp\n9ts7hrukpthv7xjukpriNe4dw11SU2zLdAx3SU2xLdMx3CU1xbZMx3CX1BTbMh3DXVJTDPeO4S6p\nKfbcOxOFe5KdSQ4keTjJdSu8fk6Su5Lcl+TBJD899UolaQL23DtrhnuSM4BbgCuBS4Crk7x82bBd\nwH1V9SrgCuDmJNumXawkrcW2TGeSlfsO4JGqeqyqFoE7gKuWjfka8B2D598B/O+q+vb0ypSkyRju\nnUlW1+cBB4e2H6cL/GEfBD6T5K+B5wP/cjrlSdL62HPvTKt18m7g/qq6IslLgU8nubSqnl4+cGFh\n4cTzXq9Hr9ebUgmS1EbPvd/v0+/3T+sYqarVBySXAwtVtXOwfT1QVfXeoTF3Av+5qv7nYPszwHVV\n9aVlx6q13k+STsf73gcPPQS33DLrSqYnCVW1rg8xnqTnfg9wUZILk5wFvAXYu2zMfuANgyLOBV4G\nPLqeQiRpGuy5d9Zsy1TVM0l2AfvofhncVlX7k1zbvVy7gf8C7ElyPxDg31fV/9nIwiVpJfbcOxP1\n3Kvqk8DFy/bdOvT8SeDN0y1NktavhZ77NHiHqqSm2JbpGO6SmmK4dwx3SU2x594x3CU1xZ57x3CX\n1BTbMh3DXVJTbMt0DHdJTXHl3jHcJTXFnnvHcJfUFFfuHcNdUlPsuXcMd0lNceXeMdwlNcWee8dw\nl9QUV+4dw11SU+y5dwx3SU2xLdMx3CU1xbZMx3CX1BTDvWO4S2qKPfeO4S6pGceOweIibN8+60pm\nz3CX1IyjR7uTqcmsK5k9w11SM+y3LzHcJTXDfvsSw11SM7zGfYnhLqkZtmWWGO6SmmFbZonhLqkZ\nrtyXGO6SmmHPfYnhLqkZrtyXTBTuSXYmOZDk4STXnWRML8m9Sf40yWenW6Ykrc2e+5Jtaw1IcgZw\nC/B64K+Be5J8vKoODI05G/h14Eeq6lCSF25UwZJ0Mq7cl0yyct8BPFJVj1XVInAHcNWyMdcAH6uq\nQwBV9eR0y5SktdlzXzJJuJ8HHBzafnywb9jLgBck+WySe5K8dVoFStKkXLkvWbMts47jXAb8MPA8\n4I+T/HFVfXX5wIWFhRPPe70evV5vSiVIerZrpefe7/fp9/undYxJwv0QcMHQ9vmDfcMeB56sqr8F\n/jbJHwKvBFYNd0maplZW7ssXvu95z3vWfYxJ2jL3ABcluTDJWcBbgL3LxnwceF2SM5M8F/h+YP+6\nq5Gk02DPfcmaK/eqeibJLmAf3S+D26pqf5Jru5drd1UdSPIp4AHgGWB3VX1lQyuXpGVaWblPw0Q9\n96r6JHDxsn23Ltu+CbhpeqVJ0vq00nOfBu9QldQMV+5LDHdJzbDnvsRwl9QMV+5LDHdJzbDnvsRw\nl9QM2zJLDHdJzbAts8Rwl9QM2zJLDHdJzXDlvsRwl9QMe+5LDHdJzXDlvsRwl9QMe+5LDHdJzXDl\nvsRwl9QMe+5LDHdJzXDlvsRwl9QMe+5LDHdJzXDlvsRwl9QMe+5LDHdJzXDlvsRwl9QMe+5LDHdJ\nTTh2DBYXYfv2WVeyNRjukppw9GjXb09mXcnWYLhLaoL99lGGu6Qm2G8fZbhLaoIr91GGu6QmeI37\nKMNdUhNsy4wy3CU1wbbMKMNdUhMM91EThXuSnUkOJHk4yXWrjHtNksUk/2x6JUrS2uy5j1oz3JOc\nAdwCXAlcAlyd5OUnGXcj8KlpFylJa7HnPmqSlfsO4JGqeqyqFoE7gKtWGPczwO8CX59ifZI0Edsy\noyYJ9/OAg0Pbjw/2nZDk7wM/UVW/AXjzr6RNZ1tm1LYpHedXgOFe/EkDfmFh4cTzXq9Hr9ebUgmS\nns1aWrn3+336/f5pHSNVtfqA5HJgoap2DravB6qq3js05tHjT4EXAt8C/k1V7V12rFrr/STpVNx+\nO9x9d/e1NUmoqnV1RSZZud8DXJTkQuAJ4C3A1cMDquolQ0XsAT6xPNglaSO1tHKfhjXDvaqeSbIL\n2EfXo7+tqvYnubZ7uXYv/5YNqFOSVmXPfdREPfeq+iRw8bJ9t55k7L+eQl2StC6u3Ed5h6qkJnid\n+yjDXVITXLmPMtwlNcGe+yjDXVITXLmPMtwlNcGe+yjDXVITXLmPMtwlNcGe+yjDXVITXLmPMtwl\nNcGe+yjDXVITXLmPMtwlNcGe+yjDXVITbMuMMtwlNcG2zCjDXVITDPdRhrukJthzH2W4S2qCPfdR\nhrukJtiWGWW4S2qC4T7KcJfUBHvuowx3SU2w5z7KcJc0944dg8VF2L591pVsHYa7pLl39GjXkklm\nXcnWYbhLmnueTB1nuEuae/bbxxnukuaeK/dxhrukuedlkOMMd0lzz5X7OMNd0tyz5z5uonBPsjPJ\ngSQPJ7luhdevSXL/4PG5JK+YfqmStDJX7uPWDPckZwC3AFcClwBXJ3n5smGPAj9YVa8E/hPwwWkX\nKkknY8993CQr9x3AI1X1WFUtAncAVw0PqKrPV9U3B5ufB86bbpmSdHK2ZcZNEu7nAQeHth9n9fB+\nO3DX6RQlSethW2bctmkeLMkVwNuA103zuJK0GsN93CThfgi4YGj7/MG+EUkuBXYDO6vqqZMdbGFh\n4cTzXq9Hr9ebsFRJWllrPfd+v0+/3z+tY6SqVh+QnAk8BLweeAL4InB1Ve0fGnMB8BngrVX1+VWO\nVWu9nySt1+23w913d19blISqWtfHoq25cq+qZ5LsAvbR9ehvq6r9Sa7tXq7dwC8BLwDenyTAYlXt\nWP8UJGn9bMuMm6jnXlWfBC5etu/WoefvAN4x3dIkaTKG+zjvUJU091rruU+D4S5p7nmd+zjDXdLc\nsy0zznCXNPcM93GGu6S5Z899nOEuae7Zcx9nuEuae7Zlxhnukuae4T7OcJc09+y5jzPcJc09e+7j\nDHdJc8+2zDjDXdLcsy0zznCXNPdsy4wz3CXNPdsy4wx3SXPPcB9nuEuae/bcxxnukuaePfdxhruk\nuWdbZpzhLmnuGe7jDHdJc8+e+zjDXdLcs+c+znCXNNeOHYNvfxu2b591JVuL4S5prh092rVkkllX\nsrUY7pLmmv32lRnukuaa/faVGe6S5pqXQa7McJc01wz3lRnukuaaPfeVTRTuSXYmOZDk4STXnWTM\nryV5JMl9SV413TIlaWX23Fe2ZrgnOQO4BbgSuAS4OsnLl435UeClVfW9wLXABzag1i2v3+/PuoQN\n5fzmV8tzO3IEjhzpz7qMLWeSlfsO4JGqeqyqFoE7gKuWjbkK+BBAVX0BODvJuVOtdA60/BcInN88\na3luR47A3/xNf9ZlbDmThPt5wMGh7ccH+1Ybc2iFMZI0dUePwplnzrqKrWfbZr/hm9+82e+4eR56\nCL785VlXsXGc3/xqeW5f/zps2/Qk2/pSVasPSC4HFqpq52D7eqCq6r1DYz4AfLaqPjLYPgD8UFUd\nXnas1d9MkrSiqlrXByxM8vvuHuCiJBcCTwBvAa5eNmYv8E7gI4NfBt9YHuynUpwk6dSsGe5V9UyS\nXcA+uh79bVW1P8m13cu1u6ruTPKmJF8FvgW8bWPLliStZs22jCRp/mzaHaqT3Ag1T5LcluRwkgeG\n9n1nkn1JHkryqSRnz7LGU5Xk/CR3J/mzJA8m+dnB/lbm95wkX0hy72COvzzY38T8oLs/JcmfJNk7\n2G5mbgBJ/jLJ/YOf4RcH+5qYY5Kzk3w0yf7Bn8/vP5W5bUq4T3Ij1BzaQzefYdcDf1BVFwN3A+/e\n9Kqm49vAz1XVJcA/Bt45+Hk1Mb+qOgJcUVWvBi4FfjjJa2lkfgPvAr4ytN3S3ACOAb2qenVV7Rjs\na2WOvwrcWVX/AHglcIBTmVtVbfgDuBy4a2j7euC6zXjvDZ7XhcADQ9sHgHMHz18EHJh1jVOa5+8D\nb2hxfsBzgS8C39fK/IDzgU8DPWDvYF8Tcxua418A5yzbN/dzBP4u8Ocr7F/33DarLTPJjVAt+K4a\nXCVUVV8DvmvG9Zy2JN8NvAr4PN0fribmN2hb3At8DehX1VdoZ37/FfgFYPiEWitzO66ATye5J8nb\nB/tamOP3AE8m2TNoq+1O8lxOYW5+KuTGmuuz1UmeD/wu8K6qeprx+czt/KrqWHVtmfOBH0jSo4H5\nJfkx4HBV3Qesdunx3M1tmddW1WXAm+jahj9AAz8/uisYLwN+fTC/b9F1OtY9t80K90PABUPb5w/2\ntebw8c/USfIi4OszrueUJdlGF+y/VVUfH+xuZn7HVdX/Be4E/hFtzO+1wI8neRT473TnE34L+FoD\nczuhqp4YfP1fdG3DHbTx83scOFhVXxpsf4wu7Nc9t80K9xM3QiU5i+5GqL2b9N4bKYyujvYCPz14\n/lPAx5d/wxz5TeArVfWrQ/uamF+SFx6/2iDJ3wHeCNxLA/Orql+sqguq6iV0f8/urqq3Ap9gzud2\nXJLnDv5VSZLnAT8CPEgbP7/DwMEkLxvsej3wZ5zC3DbtOvckO+nOAh+/EerGTXnjDZLkw3QnrM4B\nDgM30K0gPgq8GHgM+BdV9Y1Z1XiqBleO/CHdX5gaPH6R7sTj7zD/83sFcDvdL+Yz6P51clOSF9DA\n/I5L8kPAz1fVj7c0tyTfA/we3Z/LbcBvV9WNrcwxySuB/wZsBx6luyn0TNY5N29ikqQGeUJVkhpk\nuEtSgwx3SWqQ4S5JDTLcJalBhrskNchwl6QGGe6S1KD/D2ZsRpl/yu+xAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x106f29890>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(x1, x2, y, 0, 0, 0.01, 100)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Получилось что-то явно не то (значение функционала равно +inf). Сократим шаг."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "99\n",
      "(0.0799334813069459, 3.0273948768068668) 27.3532007379\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x10721ba90>]"
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEACAYAAAByG0uxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGyVJREFUeJzt3X+Q3XV97/HnK4kxtkImWAneBBAKweC9FnMltDJXj2gI\n2BlgnCGNdUzQeEcLXp16x15iO2RTaqvMaINzL3FujRAyetOYTk1oM2Sl4XhrByEo3FASw/ZHkCxm\nuZeEddCpks3r/nE+a77Z7K98s57N7r4eMzvz3ff5fL77+X5Zzms/n885ObJNREREHdPGewARETFx\nJUQiIqK2hEhERNSWEImIiNoSIhERUVtCJCIiaht1iEiaJun7kraX7+dI6pS0X9JOSbMrbVdL6pK0\nT9K1lfoiSXskPSNpXaU+U9Lm0ucRSRdUHltZ2u+XtOL0LzkiIsbKqcxEPgnsrXx/O/CQ7cuAXcBq\nAEmXA8uAhcD1wD2SVPqsB1bZXgAskLS01FcBh21fCqwD7irnmgPcAVwJXAWsqYZVRESMr1GFiKT5\nwHuBr1TKNwIby/FG4KZyfAOw2fZR2weALmCxpPOAs2zvLu3ur/SpnmsrcE05Xgp02u61/RLQCVw3\n+suLiIhfptHORP4c+DRQfXv7XNs9ALYPAeeW+jzguUq77lKbBxys1A+W2gl9bPcBvZLOGeZcERFx\nBhgxRCT9NtBj+0lAwzQdy38/ZbifExERZ4gZo2hzNXCDpPcCrwHOkrQJOCRpru2eslT1QmnfDZxf\n6T+/1IaqV/s8L2k6cLbtw5K6gcaAPg8PHKCk/ANgERE12D6tP9pHnInY/oztC2xfDCwHdtn+IPAA\ncEtpthLYVo63A8vLK64uAi4BHitLXr2SFpeN9hUD+qwsxzfT2qgH2AkskTS7bLIvKbXBxpkvmzVr\n1oz7GM6Ur9yL3Ivci+G/xsJoZiJD+RywRdKHgWdpvSIL23slbaH1Sq5XgFt9fLS3AfcBs4Adth8s\n9Q3AJkldwIu0wgrbRyTdCTxOa7lsrVsb7BERcQY4pRCx/W3g2+X4MPCeIdr9GfBng9S/B/yHQeo/\no4TQII/dRyt4IiLiDJN3rE8yjUZjvIdwxsi9OC734rjci7GlsVoXG0+S3NdnpiUSIyJGTRL+ZW+s\nTxR9feM9goiIqWfShMjRo+M9goiIqSchEhERtSVEIiKitoRIRETUlhCJiIjaEiIREVFbQiQiImpL\niERERG0JkYiIqC0hEhERtSVEIiKitoRIRETUlhCJiIjaEiIREVHbpAmR/FPwERHtN2lCJDORiIj2\nGzFEJL1a0qOSnpD0tKQ/LfU1kg5K+n75uq7SZ7WkLkn7JF1bqS+StEfSM5LWVeozJW0ufR6RdEHl\nsZWl/X5JK4YaZ0IkIqL9ZozUwPbPJL3L9k8lTQf+QdLV5eEv2v5itb2khcAyYCEwH3hI0qVufQ7v\nemCV7d2SdkhaansnsAo4bPtSSb8D3AUslzQHuANYBAj4nqRttnsHjjMhEhHRfqNazrL903L46tLn\nSPl+sM/mvRHYbPuo7QNAF7BY0nnAWbZ3l3b3AzdV+mwsx1uBa8rxUqDTdq/tl4BO4BcznqqESERE\n+40qRCRNk/QEcAho2t5bHvq4pCclfUXS7FKbBzxX6d5davOAg5X6wVI7oY/tPqBX0jnDnOskCZGI\niPYb7UzkmO230lqeeoekdwL3ABfbvoJWuHxhDMc12AxnWAmRiIj2G3FPpMr2jyX9LfA229+uPPQX\nwAPluBs4v/LY/FIbql7t83zZdznb9mFJ3UBjQJ+HBxvbli0d/OAHreNGo0Gj0RisWUTElNVsNmk2\nm2N6TrX2u4dpIP0a8IrtXkmvAXYCa4GnbR8qbX4fuNL270q6HPgacBWtpadvAZfatqTvAp8AdgN/\nC3zJ9oOSbgX+ve1bJS0HbrLdv7H+OK2N9Wnl+D+W/ZHqGH3vveaWW8bknkRETAmSsH3KKz9Vo5mJ\nvAHYKEm0nsg32f47SfdLugI4BhwAPgpge6+kLcBe4BXgVh9PqtuA+4BZwA7bD5b6BmCTpC7gRWB5\nOdcRSXfSCg8DawcGSL8sZ0VEtN+IM5GJQJLXrzcf+9h4jyQiYuIYi5lI3rEeERG1JUQiIqK2hEhE\nRNSWEImIiNoSIhERUVtCJCIiakuIREREbQmRiIioLSESERG1JUQiIqK2hEhERNQ2aUKkr2+8RxAR\nMfVMmhDJTCQiov0SIhERUVtCJCIiakuIREREbQmRiIioLSESERG1JUQiIqK2EUNE0qslPSrpCUlP\nS/rTUp8jqVPSfkk7Jc2u9FktqUvSPknXVuqLJO2R9IykdZX6TEmbS59HJF1QeWxlab9f0oqhxpkQ\niYhovxFDxPbPgHfZfivwFuAaSVcDtwMP2b4M2AWsBpB0ObAMWAhcD9wjqf+D4NcDq2wvABZIWlrq\nq4DDti8F1gF3lXPNAe4ArgSuAtZUw6oqIRIR0X6jWs6y/dNy+OrS5whwI7Cx1DcCN5XjG4DNto/a\nPgB0AYslnQecZXt3aXd/pU/1XFuBa8rxUqDTdq/tl4BO4LrBxpgQiYhov1GFiKRpkp4ADgFN23uB\nubZ7AGwfAs4tzecBz1W6d5faPOBgpX6w1E7oY7sP6JV0zjDnOklCJCKi/WaMppHtY8BbJZ0N7JTU\nADyw2RiOSyM3OdG//msHHR2t40ajQaPRGMPhRERMfM1mk2azOabnHFWI9LP9Y0k7gLcBPZLm2u4p\nS1UvlGbdwPmVbvNLbah6tc/zkqYDZ9s+LKkbaAzo8/BgY3vDG46HSEREnGzgH9hr16497XOO5tVZ\nv9a/mS3pNcAS4AlgO3BLabYS2FaOtwPLyyuuLgIuAR4rS169khaXjfYVA/qsLMc309qoB9gJLJE0\nu2yyLym1k2Q5KyKi/UYzE3kDsLE88U8DNtn+u7JHskXSh4Fnab0iC9t7JW0B9gKvALfa7l/qug24\nD5gF7LD9YKlvADZJ6gJeBJaXcx2RdCfwOK3lsrVlg/0kCZGIiPbT8ef3iUuS3/Qms2/feI8kImLi\nkITtU96Drso71iMioraESERE1JYQiYiI2hIiERFRW0IkIiJqmzQh0tc33iOIiJh6Jk2IZCYSEdF+\nCZGIiKgtIRIREbUlRCIiorZJEyIAx46N9wgiIqaWSRMiM2ZkNhIR0W4JkYiIqC0hEhERtSVEIiKi\ntoRIRETUlhCJiIjaEiIREVFbQiQiImobMUQkzZe0S9LTkp6S9F9KfY2kg5K+X76uq/RZLalL0j5J\n11bqiyTtkfSMpHWV+kxJm0ufRyRdUHlsZWm/X9KKocaZEImIaL8Zo2hzFPiU7SclvRb4nqRvlce+\naPuL1caSFgLLgIXAfOAhSZfaNrAeWGV7t6Qdkpba3gmsAg7bvlTS7wB3AcslzQHuABYBKj97m+3e\nky4kIRIR0XYjzkRsH7L9ZDl+GdgHzCsPa5AuNwKbbR+1fQDoAhZLOg84y/bu0u5+4KZKn43leCtw\nTTleCnTa7rX9EtAJ/GLGU5UQiYhov1PaE5H0RuAK4NFS+rikJyV9RdLsUpsHPFfp1l1q84CDlfpB\njofRL/rY7gN6JZ0zzLlOMn16QiQiot1Gs5wFQFnK2gp80vbLku4B/ti2Jf0J8AXgI2M0rsFmOMPq\n6elg/XqYNw8ajQaNRmOMhhIRMTk0m02azeaYnnNUISJpBq0A2WR7G4Dt/1tp8hfAA+W4Gzi/8tj8\nUhuqXu3zvKTpwNm2D0vqBhoD+jw82BgvvLCDD30I3v720VxRRMTUM/AP7LVr1572OUe7nPVVYK/t\nu/sLZY+j3/uAfyzH22ltis+UdBFwCfCY7UO0lqkWSxKwAthW6bOyHN8M7CrHO4ElkmaXTfYlpXaS\n7IlERLTfiDMRSVcDHwCekvQEYOAzwO9KugI4BhwAPgpge6+kLcBe4BXg1vLKLIDbgPuAWcAO2w+W\n+gZgk6Qu4EVgeTnXEUl3Ao+Xn7u2bLCffCEJkYiIttPx5/eJS5Lf8x7zB38AS5aM92giIiYGSdg+\n5T3oqrxjPSIiakuIREREbQmRiIioLSESERG1JUQiIqK2hEhERNSWEImIiNoSIhERUVtCJCIiakuI\nREREbQmRiIioLSESERG1JUQiIqK2hEhERNQ2aUIkn7EeEdF+kyZEMhOJiGi/hEhERNSWEImIiNoS\nIhERUduIISJpvqRdkp6W9JSkT5T6HEmdkvZL2ilpdqXPakldkvZJurZSXyRpj6RnJK2r1GdK2lz6\nPCLpgspjK0v7/ZJWDDXOhEhERPuNZiZyFPiU7TcDvwXcJulNwO3AQ7YvA3YBqwEkXQ4sAxYC1wP3\nSOr/IPj1wCrbC4AFkpaW+irgsO1LgXXAXeVcc4A7gCuBq4A11bCqSohERLTfiCFi+5DtJ8vxy8A+\nYD5wI7CxNNsI3FSObwA22z5q+wDQBSyWdB5wlu3dpd39lT7Vc20FrinHS4FO2722XwI6gesGG+eM\nGdDXN/IFR0TE2DmlPRFJbwSuAL4LzLXdA62gAc4tzeYBz1W6dZfaPOBgpX6w1E7oY7sP6JV0zjDn\nOklmIhER7TdjtA0lvZbWLOGTtl+W5AFNBn5/OjRykxNt397BP/0TdHRAo9Gg0WiM4XAiIia+ZrNJ\ns9kc03OOKkQkzaAVIJtsbyvlHklzbfeUpaoXSr0bOL/SfX6pDVWv9nle0nTgbNuHJXUDjQF9Hh5s\njMuWdfDNb7ZCJCIiTjbwD+y1a9ee9jlHu5z1VWCv7bsrte3ALeV4JbCtUl9eXnF1EXAJ8FhZ8uqV\ntLhstK8Y0GdlOb6Z1kY9wE5giaTZZZN9SamdJMtZERHtN+JMRNLVwAeApyQ9QWvZ6jPA54Etkj4M\nPEvrFVnY3itpC7AXeAW41Xb/UtdtwH3ALGCH7QdLfQOwSVIX8CKwvJzriKQ7gcfLz11bNthPvpCE\nSERE2+n48/vEJckPPGC+/GX4m78Z79FEREwMkrB9ynvQVXnHekRE1JYQiYiI2hIiERFRW0IkIiJq\nS4hERERtCZGIiKht0oRIPmM9IqL9Jk2IZCYSEdF+CZGIiKgtIRIREbUlRCIioraESERE1JYQiYiI\n2hIiERFRW0IkIiJqm1Qh0tc33qOIiJhaJlWIZCYSEdFeCZGIiKht0oRI/7+dNQk+7TciYsIYMUQk\nbZDUI2lPpbZG0kFJ3y9f11UeWy2pS9I+SddW6osk7ZH0jKR1lfpMSZtLn0ckXVB5bGVpv1/SimEv\nZFrr69ixU7n8iIg4HaOZidwLLB2k/kXbi8rXgwCSFgLLgIXA9cA9kvo/BH49sMr2AmCBpP5zrgIO\n274UWAfcVc41B7gDuBK4ClgjafZwA82SVkREe40YIra/AxwZ5CENUrsR2Gz7qO0DQBewWNJ5wFm2\nd5d29wM3VfpsLMdbgWvK8VKg03av7ZeATuAXM57BJEQiItrrdPZEPi7pSUlfqcwQ5gHPVdp0l9o8\n4GClfrDUTuhjuw/olXTOMOcaUkIkIqK9ZtTsdw/wx7Yt6U+ALwAfGaMxDTbDGVFHRwc//zl89rPw\n3vc2aDQaYzSciIjJodls0mw2x/Sc8iheziTpQuAB228Z7jFJtwO2/fny2IPAGuBZ4GHbC0t9OfBO\n27/X38b2o5KmAz+yfW5p07D9sdLny+UcfznIGGybuXNhzx6YO7fezYiImEokYbvWH+79RrucJSoz\nhLLH0e99wD+W4+3A8vKKq4uAS4DHbB+itUy1uGy0rwC2VfqsLMc3A7vK8U5giaTZZZN9SakNKctZ\nERHtNeJylqSvAw3gdZJ+SGtm8S5JVwDHgAPARwFs75W0BdgLvALc6uNTnduA+4BZwI7+V3QBG4BN\nkrqAF4Hl5VxHJN0JPA4YWFs22Ie+mIRIRERbjWo560zXv5x18cXwrW/Br//6eI8oIuLM187lrAkh\nM5GIiPZKiERERG0JkYiIqC0hEhERtSVEIiKitoRIRETUlhCJiIjaEiIREVFbQiQiImqbdCHS1zfe\no4iImDomXYhkJhIR0T4JkYiIqC0hEhERtSVEIiKitoRIRETUlhCJiIjaEiIREVFbQiQiImpLiERE\nRG0jhoikDZJ6JO2p1OZI6pS0X9JOSbMrj62W1CVpn6RrK/VFkvZIekbSukp9pqTNpc8jki6oPLay\ntN8vacVIY50+PSESEdFOo5mJ3AssHVC7HXjI9mXALmA1gKTLgWXAQuB64B5J/R8Cvx5YZXsBsEBS\n/zlXAYdtXwqsA+4q55oD3AFcCVwFrKmG1WAyE4mIaK8RQ8T2d4AjA8o3AhvL8UbgpnJ8A7DZ9lHb\nB4AuYLGk84CzbO8u7e6v9KmeaytwTTleCnTa7rX9EtAJXDfcWBMiERHtVXdP5FzbPQC2DwHnlvo8\n4LlKu+5SmwccrNQPltoJfWz3Ab2SzhnmXENKiEREtNeMMTqPx+g8ABq5yck6Ojr4+78HG97xjgaN\nRmMMhxQRMfE1m02azeaYnrNuiPRImmu7pyxVvVDq3cD5lXbzS22oerXP85KmA2fbPiypG2gM6PPw\nUAPq6Ojgs5+Fn/wEkh8RESdrNE78A3vt2rWnfc7RLmeJE2cI24FbyvFKYFulvry84uoi4BLgsbLk\n1StpcdloXzGgz8pyfDOtjXqAncASSbPLJvuSUhtSlrMiItprxJmIpK/TmhG8TtIPgTXA54BvSPow\n8CytV2Rhe6+kLcBe4BXgVtv9S123AfcBs4Adth8s9Q3AJkldwIvA8nKuI5LuBB6ntVy2tmywD30x\nCZGIiLbS8ef4iUuSbXP33fDP/wxf+tJ4jygi4swnCdu19qH75R3rERFRW0IkIiJqS4hERERtky5E\n+vrGexQREVPHpAuRzEQiItonIRIREbUlRCIioraESERE1JYQiYiI2hIiERFRW0IkIiJqm1Qhks9Y\nj4hor0kVIpmJRES0V0IkIiJqS4hERERtCZGIiKgtIRIREbUlRCIioraESERE1HZaISLpgKT/I+kJ\nSY+V2hxJnZL2S9opaXal/WpJXZL2Sbq2Ul8kaY+kZyStq9RnStpc+jwi6YLhxpMQiYhor9OdiRwD\nGrbfantxqd0OPGT7MmAXsBpA0uXAMmAhcD1wj6T+D4hfD6yyvQBYIGlpqa8CDtu+FFgH3DXcYBIi\nERHtdbohokHOcSOwsRxvBG4qxzcAm20ftX0A6AIWSzoPOMv27tLu/kqf6rm2Au8ebjAJkYiI9jrd\nEDHwLUm7JX2k1Oba7gGwfQg4t9TnAc9V+naX2jzgYKV+sNRO6GO7D3hJ0jlDDSYhEhHRXjNOs//V\ntn8k6fVAp6T9tIKlauD3p0NDPdDR0cG//Ru8/DI0mw0ajcYY/tiIiImv2WzSbDbH9Jyyx+Y5XtIa\n4GXgI7T2SXrKUtXDthdKuh2w7c+X9g8Ca4Bn+9uU+nLgnbZ/r7+N7UclTQd+ZPvcQX62bfPyyzB3\nLvzkJ2NySRERk5okbA/5x/lo1F7OkvQrkl5bjn8VuBZ4CtgO3FKarQS2lePtwPLyiquLgEuAx8qS\nV6+kxWWjfcWAPivL8c20NuqHNGMG9PXVvaKIiDhVp7OcNRf4a0ku5/ma7U5JjwNbJH2Y1ixjGYDt\nvZK2AHuBV4BbfXwadBtwHzAL2GH7wVLfAGyS1AW8CCwf9mKyJxIR0VZjtpw1nvqXs2yYNg2OHQOd\n1gQtImLyG9flrDOR1PpgqixpRUS0x6QKEciSVkREOyVEIiKitkkXIvmc9YiI9pl0IZKZSERE+yRE\nIiKitoRIRETUlhCJiIjaEiIREVFbQiQiImpLiERERG0JkYiIqC0hEhERtSVEIiKitoRIRETUlhCJ\niIjaEiIREVFbQiQiImqbECEi6TpJP5D0jKT/NlzbWbPg2WfbNbKIiKntjA8RSdOA/w4sBd4MvF/S\nm4Zq/4d/CH/0R7BnT7tGeGZpNpvjPYQzRu7FcbkXx+VejK0zPkSAxUCX7WdtvwJsBm4csvFi+NKX\n4IYb4IUX2jbGM0b+Bzku9+K43Ivjci/G1kQIkXnAc5XvD5bakN7/fvjgB+F974Of/eyXOraIiClt\nxngP4Jdl7VrYtw/e8hZ4/eth5kx41atAOt6mejxZdHXBo4+O9yjODLkXx+VeHJd7MbZke7zHMCxJ\nvwl02L6ufH87YNufr7Q5sy8iIuIMZfu0/pyeCCEyHdgPvBv4EfAY8H7b+8Z1YBERceYvZ9nuk/Rx\noJPWHs6GBEhExJnhjJ+JRETEmWsivDprWKfyRsTJRtJ8SbskPS3pKUmfKPU5kjol7Ze0U9Ls8R5r\nO0iaJun7kraX76fkfQCQNFvSNyTtK78fV03V+yFpdbkHeyR9TdLMqXIvJG2Q1CNpT6U25LWXe9VV\nfm+uHc3PmNAhcqpvRJyEjgKfsv1m4LeA28r13w48ZPsyYBewehzH2E6fBPZWvp+q9wHgbmCH7YXA\nbwA/YAreD0kXAv8ZeKvtt9Bawn8/U+de3Evr+bFq0GuXdDmwDFgIXA/cI438GtYJHSKc4hsRJxvb\nh2w/WY5fBvYB82ndg42l2UbgpvEZYftImg+8F/hKpTzl7gOApLOB/2T7XgDbR233MjXvx4+BnwO/\nKmkG8BqgmylyL2x/BzgyoDzUtd8AbC6/LweALlrPscOa6CFyym9EnKwkvRG4AvguMNd2D7SCBjh3\n/EbWNn8OfBqobvJNxfsAcBHw/yTdW5b3/qekX2EK3g/bR4AvAD+kFR69th9iCt6LinOHuPaBz6fd\njOL5dKKHSACSXgtsBT5ZZiQDXy0xqV89Iem3gZ4yKxtu+j2p70PFDGAR8D9sLwJ+QmsJY0r9XgBI\nuhj4feBC4N/RmpF8gCl4L4ZxWtc+0UOkG7ig8v38UpsyyhR9K7DJ9rZS7pE0tzx+HjDZ/xWxq4Eb\nJP0L8L+AayRtAg5NsfvQ7yDwnO3Hy/d/RStUptrvBcDbgH+wfdh2H/DXwNuZmvei31DX3g2cX2k3\nqufTiR4iu4FLJF0oaSawHNg+zmNqt68Ce23fXaltB24pxyuBbQM7TSa2P2P7AtsX0/od2GX7g8AD\nTKH70K8sVTwnaUEpvRt4min2e1HsB35T0qyySfxuWi++mEr3Qpw4Qx/q2rcDy8ur1y4CLqH15u7h\nTz7R3yci6Tpar0TpfyPi58Z5SG0j6WrgfwNP0ZqSGvgMrf/wW2j9VfEssMz2S+M1znaS9E7gv9q+\nQdI5TN378Bu0XmTwKuBfgA8B05mC90PSp2k9afYBTwAfAc5iCtwLSV8HGsDrgB5gDfBN4BsMcu2S\nVgOrgFdoLY93jvgzJnqIRETE+Jnoy1kRETGOEiIREVFbQiQiImpLiERERG0JkYiIqC0hEhERtSVE\nIiKitoRIRETU9v8BOYX7VudoZgIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x107264890>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(x1, x2, y, 0, 0, 0.00001, 100)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Уже лучше, однако кажется что не хватило итераций:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "999\n",
      "(0.34253214066478249, 3.023232531716197) 19.6726346491\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x107373490>]"
      ]
     },
     "execution_count": 169,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEACAYAAACUMoD1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGs5JREFUeJzt3XGMXtV95vHvA64DNOA1TbC7NgQicALZ7BI3mO6yu31D\nahtSCWgliNuoNo1TRQU2aXfVFKdS7ClRmyAlMVUWpCQOGG+yrku3i9NaeErNGykVARNgTbDXzKo1\nwUNssthMmkTKgnn2j/cYX09mPGP7ju/MnecjjXLm995z59wb8z5zzrmvLdtEREScrNOaHkBERLRD\nAiUiImqRQImIiFokUCIiohYJlIiIqEUCJSIiajHuQJF0mqQnJW0u38+W1C9pt6StkmZVjl0laUDS\nLklLKvWFknZIek7S2kp9pqSNpc+jki6ovLaiHL9b0vKTv+SIiJgIxzND+Tiws/L97cDDtt8BbANW\nAUi6DLgJuBS4Frhbkkqfe4CVthcACyQtLfWVwAHblwBrgTvLuWYDnwKuAK4EVleDKyIiJo9xBYqk\n+cAHgK9UytcD60t7PXBDaV8HbLT9mu09wACwSNJc4Gzb28tx91f6VM/1AHB1aS8F+m0P2X4F6Aeu\nGf/lRUTEqTLeGcoXgD8Eqh+rn2N7P4DtfcB5pT4PeKFy3GCpzQP2Vup7S+2oPrYPAUOSzj3GuSIi\nYpIZM1Ak/Rqw3/bTgI5xaJ1/h8uxfk5ERExCM8ZxzFXAdZI+AJwJnC1pA7BP0hzb+8ty1kvl+EHg\n/Er/+aU2Wr3a50VJpwPn2D4gaRDoDOvzyPABSspfSBYRcQJs1/YL/JgzFNuftH2B7bcDy4Bttn8b\n+AZwczlsBfBgaW8GlpUnty4CLgYeL8tiQ5IWlU365cP6rCjtG+lt8gNsBRZLmlU26BeX2kjjzJfN\n6tWrGx/DZPnKvci9yL049lfdxjNDGc1ngE2SPgw8T+/JLmzvlLSJ3hNhrwK3+MjIbwXuA84Atth+\nqNTXARskDQAv0wsubB+UdAfwBL0ltT73NucjImKSOa5Asf1N4JulfQD41VGO+zPgz0aofwd49wj1\nn1ICaYTX7qMXQhERMYnlk/It0+l0mh7CpJF7cUTuxRG5FxNHE7GOdqpJchuuIyLiVJKET+WmfERE\nxHgkUCIiohYJlIiIqEUCJSIiapFAiYiIWiRQIiKiFgmUiIioRQIlIiJqkUCJiIhaJFAiIqIWCZSI\niKhFAiUiImqRQImIiFokUCIiohYJlIiIqEUCJSIiapFAiYiIWowZKJLeJOkxSU9JelbSn5b6akl7\nJT1Zvq6p9FklaUDSLklLKvWFknZIek7S2kp9pqSNpc+jki6ovLaiHL9b0vL6Lj0iIuo0rn8CWNJZ\ntn8i6XTgH4D/Avwq8M+2Pz/s2EuBrwNXAPOBh4FLbFvSY8BttrdL2gLcZXurpN8D3m37FkkfBH7d\n9jJJs4EngIWAgO8AC20PDfuZ+SeAIyKOUyP/BLDtn5Tmm0qfg4fHM8Lh1wMbbb9mew8wACySNBc4\n2/b2ctz9wA2VPutL+wHg6tJeCvTbHrL9CtAPvDETioiIyWNcgSLpNElPAfuAru2d5aXbJD0t6SuS\nZpXaPOCFSvfBUpsH7K3U95baUX1sHwKGJJ17jHNFRMQkM94Zyuu230NvCes/SvoV4G7g7bYvpxc0\nn6txXLVNwSIi4tSYcTwH2/6hpL8F3mv7m5WXvgx8o7QHgfMrr80vtdHq1T4vln2ac2wfkDQIdIb1\neWSksa1Zs+aNdqfTodPpjHRYRMS01e126Xa7E3b+MTflJb0FeNX2kKQzga1AH/Cs7X3lmD8ArrD9\nW5IuA74GXElveervOLIp/23gY8B24G+BP7f9kKRbgH9VNuWXATeMsCl/Wmn/UtlPqY4xm/IREcep\n7k358cxQfhFYL0n03tQ32P57SfdLuhx4HdgDfBTA9k5Jm4CdwKvALZV3+1uB+4AzgC22Hyr1dcAG\nSQPAy8Cycq6Dku6gFyQG+oaHSURETA7jemx4sssMJSLi+DXy2HBERMRYEigREVGLBEpERNQigRIR\nEbVoTaBkTz4iolmtCZSIiGhWAiUiImrRmkDJkldERLNaEygREdGs1gRKZigREc1qTaBERESzEigR\nEVGL1gRKlrwiIprVmkCJiIhmtSZQMkOJiGhWawIlIiKalUCJiIhatCZQsuQVEdGs1gRKREQ0a8xA\nkfQmSY9JekrSs5L+tNRnS+qXtFvSVkmzKn1WSRqQtEvSkkp9oaQdkp6TtLZSnylpY+nzqKQLKq+t\nKMfvlrR8tHFmhhIR0awxA8X2T4H32X4P8K+BqyVdBdwOPGz7HcA2YBWApMuAm4BLgWuBuyWpnO4e\nYKXtBcACSUtLfSVwwPYlwFrgznKu2cCngCuAK4HV1eCKiIjJY1xLXrZ/UppvKn0OAtcD60t9PXBD\naV8HbLT9mu09wACwSNJc4Gzb28tx91f6VM/1AHB1aS8F+m0P2X4F6AeuOa4rjIiIU2JcgSLpNElP\nAfuAru2dwBzb+wFs7wPOK4fPA16odB8stXnA3kp9b6kd1cf2IWBI0rnHONfPyJJXRESzZoznINuv\nA++RdA6wVVIHGP4WXudbusY+5Gif/vQaZpSr6XQ6dDqdGocTETH1dbtdut3uhJ1/XIFymO0fStoC\nvBfYL2mO7f1lOeulctggcH6l2/xSG61e7fOipNOBc2wfkDQIdIb1eWSksf3xH6/hzDOP52oiIqaX\n4b9s9/X11Xr+8Tzl9ZbDG+GSzgQWA08Bm4Gby2ErgAdLezOwrDy5dRFwMfB4WRYbkrSobNIvH9Zn\nRWnfSG+TH2ArsFjSrLJBv7jUIiJikhnPDOUXgfUlBE4DNtj++7KnsknSh4Hn6T3Zhe2dkjYBO4FX\ngVvsN3Y4bgXuA84Atth+qNTXARskDQAvA8vKuQ5KugN4gt6SWl/ZnI+IiElGbsFutiT/+MfmrLOa\nHklExNQhCdvHvWc9mnxSPiIiatGaQGnBRCsiYkprTaBERESzEigREVGL1gRKlrwiIprVmkCJiIhm\nJVAiIqIWrQmULHlFRDSrNYESERHNak2gZIYSEdGs1gRKREQ0K4ESERG1aE2gZMkrIqJZrQmUiIho\nVmsCJTOUiIhmtSZQIiKiWQmUiIioRWsCJUteERHNak2gREREs8YMFEnzJW2T9KykZyT9p1JfLWmv\npCfL1zWVPqskDUjaJWlJpb5Q0g5Jz0laW6nPlLSx9HlU0gWV11aU43dLWj7aODNDiYholjzGO7Gk\nucBc209LejPwHeB64IPAP9v+/LDjLwW+DlwBzAceBi6xbUmPAbfZ3i5pC3CX7a2Sfg94t+1bJH0Q\n+HXbyyTNBp4AFgIqP3uh7aFhP9Mvv2zOPfdkb0dExPQhCduq63xjzlBs77P9dGn/CNgFzDs8nhG6\nXA9stP2a7T3AALCoBNPZtreX4+4Hbqj0WV/aDwBXl/ZSoN/2kO1XgH7gjZlQRERMHse1hyLpQuBy\n4LFSuk3S05K+ImlWqc0DXqh0Gyy1ecDeSn0vR4LpjT62DwFDks49xrl+Rpa8IiKaNWO8B5blrgeA\nj9v+kaS7gT8pS1mfBj4HfKSmcR33FOyzn13DWWf12p1Oh06nU9NQIiLaodvt0u12J+z84woUSTPo\nhckG2w8C2P5B5ZAvA98o7UHg/Mpr80tttHq1z4uSTgfOsX1A0iDQGdbnkZHG+IlPrOEtbxnP1URE\nTE/Df9nu6+ur9fzjXfL6KrDT9l2HC2VP5LDfAL5b2puBZeXJrYuAi4HHbe+jt5S1SJKA5cCDlT4r\nSvtGYFtpbwUWS5pVNugXl1pEREwyY85QJF0FfAh4RtJTgIFPAr8l6XLgdWAP8FEA2zslbQJ2Aq8C\nt/jIo2S3AvcBZwBbbD9U6uuADZIGgJeBZeVcByXdQe9JLwN9ZXM+IiImmTEfG54KJPmll8xb39r0\nSCIipo5T/thwRETEeLQmUFow0YqImNJaEygREdGsBEpERNSiNYGSJa+IiGa1JlAiIqJZrQmUzFAi\nIprVmkCJiIhmJVAiIqIWrQmULHlFRDSrNYESERHNak2gZIYSEdGs1gRKREQ0K4ESERG1aE2gZMkr\nIqJZrQmUiIhoVmsCJTOUiIhmtSZQIiKiWQmUiIioxZiBImm+pG2SnpX0jKSPlfpsSf2SdkvaKmlW\npc8qSQOSdklaUqkvlLRD0nOS1lbqMyVtLH0elXRB5bUV5fjdkpaPNs4seUVENGs8M5TXgP9s+13A\nvwVulfRO4HbgYdvvALYBqwAkXQbcBFwKXAvcLUnlXPcAK20vABZIWlrqK4EDti8B1gJ3lnPNBj4F\nXAFcCayuBldEREweYwaK7X22ny7tHwG7gPnA9cD6cth64IbSvg7YaPs123uAAWCRpLnA2ba3l+Pu\nr/SpnusB4OrSXgr02x6y/QrQD1wz8jjHvtiIiJg4x7WHIulC4HLg28Ac2/uhFzrAeeWwecALlW6D\npTYP2Fup7y21o/rYPgQMSTr3GOeKiIhJZsZ4D5T0Znqzh4/b/pGk4XOCOucIGvuQo33hC2uYVRbD\nOp0OnU6nxuFEREx93W6Xbrc7YecfV6BImkEvTDbYfrCU90uaY3t/Wc56qdQHgfMr3eeX2mj1ap8X\nJZ0OnGP7gKRBoDOszyMjjfH3f38Nb3vbeK4mImJ6Gv7Ldl9fX63nH++S11eBnbbvqtQ2AzeX9grg\nwUp9WXly6yLgYuDxsiw2JGlR2aRfPqzPitK+kd4mP8BWYLGkWWWDfnGpRUTEJDPmDEXSVcCHgGck\nPUVvaeuTwGeBTZI+DDxP78kubO+UtAnYCbwK3GK/sWV+K3AfcAawxfZDpb4O2CBpAHgZWFbOdVDS\nHcAT5ef2lc35iIiYZOQWPB4lyf/0T+bCC5seSUTE1CEJ28e9Zz2afFI+IiJq0ZpAacFEKyJiSmtN\noERERLMSKBERUYvWBEqWvCIimtWaQImIiGa1JlAyQ4mIaFZrAiUiIpqVQImIiFq0JlCy5BUR0azW\nBEpERDSrNYGSGUpERLNaEygREdGsBEpERNSiNYGSJa+IiGa1JlAiIqJZrQmUzFAiIprVmkCJiIhm\nJVAiIqIWYwaKpHWS9kvaUamtlrRX0pPl65rKa6skDUjaJWlJpb5Q0g5Jz0laW6nPlLSx9HlU0gWV\n11aU43dLWn6scWbJKyKiWeOZodwLLB2h/nnbC8vXQwCSLgVuAi4FrgXulqRy/D3AStsLgAWSDp9z\nJXDA9iXAWuDOcq7ZwKeAK4ArgdWSZp3IRUZExMQbM1Bsfws4OMJLGqF2PbDR9mu29wADwCJJc4Gz\nbW8vx90P3FDps760HwCuLu2lQL/tIduvAP3AGzOhnx3nWFcSERET6WT2UG6T9LSkr1RmDvOAFyrH\nDJbaPGBvpb631I7qY/sQMCTp3GOcKyIiJqEZJ9jvbuBPbFvSp4HPAR+paUwjzXzG9MUvruGtb+21\nO50OnU6npuFERLRDt9ul2+1O2PlPKFBs/6Dy7ZeBb5T2IHB+5bX5pTZavdrnRUmnA+fYPiBpEOgM\n6/PIaGO69dY1XHbZ8V9LRMR0MfyX7b6+vlrPP94lL1GZOZQ9kcN+A/huaW8GlpUnty4CLgYet72P\n3lLWorJJvxx4sNJnRWnfCGwr7a3AYkmzygb94lKLiIhJaMwZiqSv05sp/IKk7wGrgfdJuhx4HdgD\nfBTA9k5Jm4CdwKvALfYb2+W3AvcBZwBbDj8ZBqwDNkgaAF4GlpVzHZR0B/AEYKCvbM6PKJvyERHN\nklvwTizJ3/2uede7mh5JRMTUIQnbJ7RvPZJ8Uj4iImrRmkBpwUQrImJKa02gREREs1oTKJmhREQ0\nqzWBEhERzUqgRERELVoTKFnyiohoVmsCJSIimtWaQMkMJSKiWa0JlIiIaFYCJSIiatGaQMmSV0RE\ns1oTKBER0azWBEpmKBERzWpNoERERLMSKBERUYvWBEqWvCIimtWaQImIiGYlUCIiohZjBoqkdZL2\nS9pRqc2W1C9pt6StkmZVXlslaUDSLklLKvWFknZIek7S2kp9pqSNpc+jki6ovLaiHL9b0vJjjTNL\nXhERzRrPDOVeYOmw2u3Aw7bfAWwDVgFIugy4CbgUuBa4W5JKn3uAlbYXAAskHT7nSuCA7UuAtcCd\n5VyzgU8BVwBXAqurwRUREZPLmIFi+1vAwWHl64H1pb0euKG0rwM22n7N9h5gAFgkaS5wtu3t5bj7\nK32q53oAuLq0lwL9todsvwL0A9eMPs6xriQiIibSie6hnGd7P4DtfcB5pT4PeKFy3GCpzQP2Vup7\nS+2oPrYPAUOSzj3GuSIiYhKaUdN56pwfaOxDftaXvrSGv/mbXrvT6dDpdGocUkTE1Nftdul2uxN2\n/hMNlP2S5tjeX5azXir1QeD8ynHzS220erXPi5JOB86xfUDSINAZ1ueR0Qb0u7+7hiuuOMGriYiY\nBob/st3X11fr+ce75CWOnjlsBm4u7RXAg5X6svLk1kXAxcDjZVlsSNKiskm/fFifFaV9I71NfoCt\nwGJJs8oG/eJSi4iISWjMGYqkr9ObKfyCpO8Bq4HPAH8p6cPA8/Se7ML2TkmbgJ3Aq8At9hvb5bcC\n9wFnAFtsP1Tq64ANkgaAl4Fl5VwHJd0BPEFvSa2vbM6PKJvyERHNklvwTizJjz1mFi1qeiQREVOH\nJGyf0L71SPJJ+YiIqEVrAqUFE62IiCmtNYESERHNak2gZIYSEdGs1gRKREQ0K4ESERG1aE2gZMkr\nIqJZrQmUiIhoVmsCJTOUiIhmtSZQIiKiWQmUiIioRWsCJUteERHNak2gREREs1oTKJmhREQ0qzWB\nEhERzUqgRERELVoTKFnyiohoVmsCJSIimtWaQMkMJSKiWScVKJL2SPpfkp6S9HipzZbUL2m3pK2S\nZlWOXyVpQNIuSUsq9YWSdkh6TtLaSn2mpI2lz6OSLjiZ8UZExMQ52RnK60DH9ntsLyq124GHbb8D\n2AasApB0GXATcClwLXC3JJU+9wArbS8AFkhaWuorgQO2LwHWAnee5HgjImKCnGygaIRzXA+sL+31\nwA2lfR2w0fZrtvcAA8AiSXOBs21vL8fdX+lTPdcDwPtHG0iWvCIimnWygWLg7yRtl/SRUptjez+A\n7X3AeaU+D3ih0new1OYBeyv1vaV2VB/bh4BXJJ17kmOOiIgJMOMk+19l+/uS3gr0S9pNL2Sq6pw7\naLQX7r13DY880mt3Oh06nU6NPzYiYurrdrt0u90JO/9JBYrt75f//YGk/wksAvZLmmN7f1nOeqkc\nPgicX+k+v9RGq1f7vCjpdOAc2wdGGsvNN68hGRIRMbrhv2z39fXVev4TXvKSdJakN5f2zwNLgGeA\nzcDN5bAVwIOlvRlYVp7cugi4GHi8LIsNSVpUNumXD+uzorRvpLfJHxERk9DJzFDmAH8tyeU8X7Pd\nL+kJYJOkDwPP03uyC9s7JW0CdgKvArfYb2yl3wrcB5wBbLH9UKmvAzZIGgBeBpaNNphsykdENEtu\nwTuxJG/bZt73vqZHEhExdUjC9qh708crn5SPiIhatCZQIiKiWQmUiIioRWsCJUteERHNak2gRERE\nsxIoERFRi9YESpa8IiKa1ZpAiYiIZrUmUDJDiYhoVmsCJSIimpVAiYiIWrQmULLkFRHRrNYESkRE\nNKs1gZIZSkREs1oTKBER0awESkRE1KI1gZIlr4iIZrUmUCIiollTIlAkXSPpf0t6TtIfjXTM66+f\n6lFFRETVpA8USacBXwSWAu8CflPSO4cfd/DgqR7Z5NTtdpsewqSRe3FE7sURuRcTZ9IHCrAIGLD9\nvO1XgY3A9cMP+sEPTvm4JqX8x3JE7sURuRdH5F5MnKkQKPOAFyrf7y21oyRQIiKaNaPpAdRlwwZ4\n8smmR9G8gQF47LGmRzE55F4ckXtxRO7FxJEn+fO2kn4ZWGP7mvL97YBtf7ZyzOS+iIiIScq26jrX\nVAiU04HdwPuB7wOPA79pe1ejA4uIiKNM+iUv24ck3Qb009vzWZcwiYiYfCb9DCUiIqaGqfCU1zGN\n50OPbSJpvqRtkp6V9Iykj5X6bEn9knZL2ippVqXPKkkDknZJWtLc6Osn6TRJT0raXL6flvcBQNIs\nSX9Zru9ZSVdOx/tRrutZSTskfU3SzOl0HyStk7Rf0o5K7bivX9LCcg+fk7R2XD/c9pT9oheI/wd4\nG/BzwNPAO5se1wRf81zg8tJ+M739pXcCnwU+Uep/BHymtC8DnqK3vHlhuV9q+jpqvB9/APw3YHP5\nflreh3KN9wG/U9ozgFnT7X6U94J/BGaW7/8CWDGd7gPw74HLgR2V2nFfP/AYcEVpbwGWjvWzp/oM\nZVwfemwT2/tsP13aPwJ2AfPpXff6cth64IbSvg7YaPs123uAAXr3bcqTNB/4APCVSnna3QcASecA\n/8H2vQDlOoeYfvfjh8D/A35e0gzgTGCQaXQfbH8LGP53hxzX9UuaC5xte3s57v5Kn1FN9UAZ14ce\n20rShfR+E/k2MMf2fuiFDnBeOWz4PRqkPffoC8AfAtWNwOl4HwAuAv6vpHvLEuCXJJ3FNLsftg8C\nnwO+R++ahmw/zDS7DyM47zivfx6999PDxvXeOtUDZdqS9GbgAeDjZaYy/OmKVj9tIenXgP1ltnas\n5+hbfR8qZgALgf9qeyHwY+B2pt+fi7fTWwZ9G/Av6c1UPsQ0uw/jMCHXP9UDZRC4oPL9/FJrtTKV\nfwDYYPvBUt4vaU55fS7wUqkPAudXurflHl0FXCfpH4H/DlwtaQOwb5rdh8P2Ai/YfqJ8/1f0Ama6\n/bl4L/APtg/YPgT8NfDvmH73Ybjjvf4Tui9TPVC2AxdLepukmcAyYHPDYzoVvgrstH1XpbYZuLm0\nVwAPVurLypMuFwEX0/tw6JRm+5O2L7D9dnr/v2+z/dvAN5hG9+GwspzxgqQFpfR+4Fmm2Z8Leg+p\n/LKkMySJ3n3YyfS7D+LomftxXX9ZFhuStKjcx+WVPqNr+omEGp5ouIbeH6IB4Pamx3MKrvcq4BC9\nJ9qeAp4s9+Bc4OFyL/qBf1Hps4re0xu7gCVNX8ME3JNf4chTXtP5Pvwber9kPQ38D3pPeU27+0Fv\nX+1ZYAe9Deifm073Afg68CLwU3p7Sb8DzD7e6wd+CXimvLfeNZ6fnQ82RkRELab6kldEREwSCZSI\niKhFAiUiImqRQImIiFokUCIiohYJlIiIqEUCJSIiapFAiYiIWvx/Cb2Lu8yfLN8AAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1071acb90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(x1, x2, y, 0, 0, 0.00001, 1000)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9999\n",
      "(1.5262040841834328, 3.0044706267045096) 0.728460931851\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x107610090>]"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEACAYAAAB/BTv2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGdNJREFUeJzt3X+Qldd93/H3R8JIdiwRlFSQgrDlkbBR6lTGFkqrtr6R\nA0jOjCRnRvImmYBiPOPqR+1JO26EO2OWyJPYmrGDMq3UaYwtIHYJoc0INVRsFHTdcUYWyJKKIihs\nJ4MMK4NbgdZRPWMj9Okf9wAPa2AXOLt7l/28Znbm7Pee89zzHMF+9nnOc4VsExERUdNF4z2BiIi4\n8CRcIiKiuoRLRERUl3CJiIjqEi4REVFdwiUiIqobcbhIukjS85I2le+nS+qTtFvSFknTGn2XS+qX\ntEvSokZ9vqQdkvZIWtWoT5W0vox5RtKcxmtLS//dkpac/ylHRMRoO5srl88AOxvfPwA8Zfu9wFZg\nOYCk64C7gHnArcAjklTGPAossz0XmCtpcakvAw7ZvhZYBTxUjjUd+DxwA3AjsKIZYhER0Z1GFC6S\nZgMfBb7aKN8OrCntNcAdpX0bsN72m7b3Av3AAkkzgctsby/91jbGNI+1Ebi5tBcDfbYHbb8O9AG3\njPz0IiJiPIz0yuWPgM8CzY/zz7B9EMD2AeDKUp8F7Gv0Gyi1WcD+Rn1/qZ00xvZRYFDSFWc4VkRE\ndLFhw0XSrwEHbb8I6Axda/5/ZM70PhER0eWmjKDPTcBtkj4KvB24TNI64ICkGbYPlltePyj9B4Cr\nGuNnl9rp6s0xr0q6GLjc9iFJA0BryJinh05QUv4HaRER58D2qPwyP+yVi+3P2Z5j+z1AD7DV9m8D\nTwB3l25LgcdLexPQU54Auxq4BthWbp0NSlpQNviXDBmztLTvpPOAAMAWYKGkaWVzf2GpnWqe+bJZ\nsWLFuM+hW76yFlmLrMWZv0bTSK5cTueLwAZJnwBeofOEGLZ3StpA58myI8C9PnEW9wGPAZcCm20/\nWeqrgXWS+oHX6IQYtg9LehB4js5tt5XubOxHREQXO6twsf0t4FulfQj41dP0+0PgD09R/y7w/lPU\nf0wJp1O89hidQIqIiAkin9C/wLRarfGeQtfIWpyQtTghazE2NNr33caCJF8I5xERMZYk4fHa0I+I\niDhbCZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKqS7hERER1CZeIiKgu4RIR\nEdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKqS7hERER1w4aLpEskPSvpBUkvS/qDUl8hab+k\n58vXLY0xyyX1S9olaVGjPl/SDkl7JK1q1KdKWl/GPCNpTuO1paX/bklL6p16RESMlhH9M8eS3mH7\nR5IuBv4G+DfArwJ/b/srQ/rOA74J3ADMBp4CrrVtSc8C99veLmkz8LDtLZLuAd5v+15JHwc+ZrtH\n0nTgOWA+IOC7wHzbg0PeM//McUTEWRr3f+bY9o9K85Iy5vCxuZ2i++3Aettv2t4L9AMLJM0ELrO9\nvfRbC9zRGLOmtDcCN5f2YqDP9qDt14E+4PgVUkREdKcRhYukiyS9ABwA2rZ3lpful/SipK9KmlZq\ns4B9jeEDpTYL2N+o7y+1k8bYPgoMSrriDMeKiIguNtIrl7dsf4DOba5/IenDwCPAe2xfTyd0vlxx\nXqNymRYREWNjytl0tv1DSX8JfMj2txov/QnwRGkPAFc1XptdaqerN8e8WvZ1Lrd9SNIA0Boy5ulT\nza23t/d4u9Vq0Wq1TtUtImLSarfbtNvtMXmvYTf0Jf08cMT2oKS3A1uAlcDLtg+UPr8L3GD7NyVd\nB3wDuJHOLay/4sSG/neATwPbgb8E/tj2k5LuBf5R2dDvAe44xYb+RaX9wbL/0pxjNvQjIs7SaG7o\nj+TK5ReANZJE5wf8Ott/LWmtpOuBt4C9wKcAbO+UtAHYCRwB7m385L8PeAy4FNhs+8lSXw2sk9QP\nvAb0lGMdlvQgnVAxsHJosERERPcZ0aPI3S5XLhERZ2/cH0WOiIg4GwmXiIioLuESERHVJVwiIqK6\nhEtERFSXcImIiOoSLhERUV3CJSIiqku4REREdQmXiIioLuESERHVJVwiIqK6hEtERFSXcImIiOoS\nLhERUV3CJSIiqku4REREdQmXiIioLuESERHVDRsuki6R9KykFyS9LOkPSn26pD5JuyVtkTStMWa5\npH5JuyQtatTnS9ohaY+kVY36VEnry5hnJM1pvLa09N8taUm9U4+IiNEybLjY/jHwK7Y/APwScLOk\nm4AHgKdsvxfYCiwHkHQdcBcwD7gVeESSyuEeBZbZngvMlbS41JcBh2xfC6wCHirHmg58HrgBuBFY\n0QyxiIjoTiO6LWb7R6V5SRlzGLgdWFPqa4A7Svs2YL3tN23vBfqBBZJmApfZ3l76rW2MaR5rI3Bz\naS8G+mwP2n4d6ANuOaszjIiIMTeicJF0kaQXgANA2/ZOYIbtgwC2DwBXlu6zgH2N4QOlNgvY36jv\nL7WTxtg+CgxKuuIMx4qIiC42ZSSdbL8FfEDS5cAWSS3AQ7tVnJeG73Ky3t7e4+1Wq0Wr1ao4nYiI\nia/dbtNut8fkvUYULsfY/qGkzcCHgIOSZtg+WG55/aB0GwCuagybXWqnqzfHvCrpYuBy24ckDQCt\nIWOePtXcmuESERE/begv3itXrhy19xrJ02I/f2wTXdLbgYXAC8Am4O7SbSnweGlvAnrKE2BXA9cA\n28qts0FJC8oG/5IhY5aW9p10HhAA2AIslDStbO4vLLWIiOhiI7ly+QVgTQmEi4B1tv+67MFskPQJ\n4BU6T4hhe6ekDcBO4Ahwr+1jt8zuAx4DLgU2236y1FcD6yT1A68BPeVYhyU9CDxH57bbyrKxHxER\nXUwnfu5PXJJ8IZxHRMRYkoTts97jHol8Qj8iIqpLuERERHUJl4iIqC7hEhER1SVcIiKiuoRLRERU\nl3CJiIjqEi4REVFdwiUiIqpLuERERHUJl4iIqC7hEhER1SVcIiKiuoRLRERUl3CJiIjqEi4REVFd\nwiUiIqpLuERERHUJl4iIqG7YcJE0W9JWSS9LeknSvyr1FZL2S3q+fN3SGLNcUr+kXZIWNerzJe2Q\ntEfSqkZ9qqT1ZcwzkuY0Xlta+u+WtKTeqUdExGiR7TN3kGYCM22/KOmdwHeB24GPA39v+ytD+s8D\nvgncAMwGngKutW1JzwL3294uaTPwsO0tku4B3m/7XkkfBz5mu0fSdOA5YD6g8t7zbQ8OeU8Pdx4R\nEXEySdjWaBx72CsX2wdsv1jabwC7gFnH5naKIbcD622/aXsv0A8sKCF1me3tpd9a4I7GmDWlvRG4\nubQXA322B22/DvQBx6+QIiKiO53VnoukdwPXA8+W0v2SXpT0VUnTSm0WsK8xbKDUZgH7G/X9nAip\n42NsHwUGJV1xhmNFREQXmzLSjuWW2EbgM7bfkPQI8PvldtcXgC8Dn6w0r7O+TOvt7T3ebrVatFqt\nSlOJiLgwtNtt2u32mLzXsHsuAJKmAP8N+O+2Hz7F6+8CnrD9S5IeAGz7S+W1J4EVwCvA07bnlXoP\n8GHb9xzrY/tZSRcD37d9ZenTsv0vy5j/WI7xZ0PeP3suERFnaVz3XIqvATubwVL2UI75deBvS3sT\n0FOeALsauAbYZvsAndtdCyQJWAI83hiztLTvBLaW9hZgoaRpZXN/YalFREQXG/a2mKSbgN8CXpL0\nAmDgc8BvSroeeAvYC3wKwPZOSRuAncAR4N7GZcV9wGPApcBm20+W+mpgnaR+4DWgpxzrsKQH6Twx\nZmBl2diPiIguNqLbYt0ut8UiIs5eN9wWi4iIGLGES0REVJdwiYiI6hIuERFRXcIlIiKqS7hERER1\nCZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKqS7hERER1CZeIiKgu4RIREdUl\nXCIiorqES0REVJdwiYiI6oYNF0mzJW2V9LKklyR9utSnS+qTtFvSFknTGmOWS+qXtEvSokZ9vqQd\nkvZIWtWoT5W0vox5RtKcxmtLS//dkpbUO/WIiBgtI7lyeRP417Z/EfgnwH2S3gc8ADxl+73AVmA5\ngKTrgLuAecCtwCOSVI71KLDM9lxgrqTFpb4MOGT7WmAV8FA51nTg88ANwI3AimaIRUREdxo2XGwf\nsP1iab8B7AJmA7cDa0q3NcAdpX0bsN72m7b3Av3AAkkzgctsby/91jbGNI+1Ebi5tBcDfbYHbb8O\n9AG3nMuJRkTE2DmrPRdJ7wauB74DzLB9EDoBBFxZus0C9jWGDZTaLGB/o76/1E4aY/soMCjpijMc\nKyIiutiUkXaU9E46VxWfsf2GJA/pMvT786Hhu5yst7f3eLvVatFqtSpOJyJi4mu327Tb7TF5rxGF\ni6QpdIJlne3HS/mgpBm2D5ZbXj8o9QHgqsbw2aV2unpzzKuSLgYut31I0gDQGjLm6VPNsRkuERHx\n04b+4r1y5cpRe6+R3hb7GrDT9sON2ibg7tJeCjzeqPeUJ8CuBq4BtpVbZ4OSFpQN/iVDxiwt7Tvp\nPCAAsAVYKGla2dxfWGoREdHFZJ/5bpakm4D/AbxE59aXgc8B24ANdK44XgHuKpvuSFpO5wmwI3Ru\no/WV+geBx4BLgc22P1PqlwDrgA8ArwE95WEAJN0N/Lvyvl+wvfYUc/Rw5xERESeThO2z3oYY0bEv\nhB/KCZeIiLM3muGST+hHRER1CZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKq\nS7hERER1CZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKqS7hERER1CZeIiKgu\n4RIREdUNGy6SVks6KGlHo7ZC0n5Jz5evWxqvLZfUL2mXpEWN+nxJOyTtkbSqUZ8qaX0Z84ykOY3X\nlpb+uyUtqXPKEREx2kZy5fJ1YPEp6l+xPb98PQkgaR5wFzAPuBV4RJJK/0eBZbbnAnMlHTvmMuCQ\n7WuBVcBD5VjTgc8DNwA3AiskTTuXk4yIiLE1bLjY/jZw+BQv6RS124H1tt+0vRfoBxZImglcZnt7\n6bcWuKMxZk1pbwRuLu3FQJ/tQduvA33A8SukiIjoXuez53K/pBclfbVxRTEL2NfoM1Bqs4D9jfr+\nUjtpjO2jwKCkK85wrIiI6HJTznHcI8Dv27akLwBfBj5ZaU6nuiIaVm9v7/F2q9Wi1WpVmk5ExIWh\n3W7TbrfH5L3OKVxs/5/Gt38CPFHaA8BVjddml9rp6s0xr0q6GLjc9iFJA0BryJinTzenZrhERMRP\nG/qL98qVK0ftvUZ6W0w0rijKHsoxvw78bWlvAnrKE2BXA9cA22wfoHO7a0HZ4F8CPN4Ys7S07wS2\nlvYWYKGkaWVzf2GpRURElxv2ykXSN+lcQfycpO8BK4BfkXQ98BawF/gUgO2dkjYAO4EjwL22XQ51\nH/AYcCmw+dgTZsBqYJ2kfuA1oKcc67CkB4HnAAMry8Z+RER0OZ342T9xSfKFcB4REWNJErbPaZ97\nOPmEfkREVJdwiYiI6hIuERFRXcIlIiKqS7hERER1CZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI\n6hIuERFRXcIlIiKqS7hERER1CZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFR3bDhImm1\npIOSdjRq0yX1SdotaYukaY3Xlkvql7RL0qJGfb6kHZL2SFrVqE+VtL6MeUbSnMZrS0v/3ZKW1Dnl\niIgYbSO5cvk6sHhI7QHgKdvvBbYCywEkXQfcBcwDbgUekaQy5lFgme25wFxJx465DDhk+1pgFfBQ\nOdZ04PPADcCNwIpmiEVERPcaNlxsfxs4PKR8O7CmtNcAd5T2bcB622/a3gv0AwskzQQus7299Fvb\nGNM81kbg5tJeDPTZHrT9OtAH3HIW5xYREePkXPdcrrR9EMD2AeDKUp8F7Gv0Gyi1WcD+Rn1/qZ00\nxvZRYFDSFWc4VkREdLkplY7jSscB0PBdflpvb+/xdqvVotVqVZpORMSFod1u0263x+S9zjVcDkqa\nYftgueX1g1IfAK5q9JtdaqerN8e8Kuli4HLbhyQNAK0hY54+3YSa4RIRET9t6C/eK1euHLX3Gult\nMXHyFcUm4O7SXgo83qj3lCfArgauAbaVW2eDkhaUDf4lQ8YsLe076TwgALAFWChpWtncX1hqERHR\n5Ya9cpH0TTpXED8n6XvACuCLwJ9L+gTwCp0nxLC9U9IGYCdwBLjX9rFbZvcBjwGXApttP1nqq4F1\nkvqB14CecqzDkh4EnqNz221l2diPiIgupxM/+ycuSb4QziMiYixJwvY57XMPJ5/Qj4iI6hIuERFR\nXcIlIiKqS7hERER1CZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKqS7hERER1\nCZeIiKgu4RIREdUlXCIiorqES0REVJdwiYiI6hIuERFRXcIlIiKqO69wkbRX0v+U9IKkbaU2XVKf\npN2Stkia1ui/XFK/pF2SFjXq8yXtkLRH0qpGfaqk9WXMM5LmnM98IyJibJzvlctbQMv2B2wvKLUH\ngKdsvxfYCiwHkHQdcBcwD7gVeESSyphHgWW25wJzJS0u9WXAIdvXAquAh85zvhERMQbON1x0imPc\nDqwp7TXAHaV9G7De9pu29wL9wAJJM4HLbG8v/dY2xjSPtRH4yHnONyIixsD5houBv5K0XdInS22G\n7YMAtg8AV5b6LGBfY+xAqc0C9jfq+0vtpDG2jwKvS7riPOccERGjbMp5jr/J9vcl/QOgT9JuOoHT\nNPT786HTvdDb23u83Wq1aLVaFd82ImLia7fbtNvtMXkv2XV+9ktaAbwBfJLOPszBcsvradvzJD0A\n2PaXSv8ngRXAK8f6lHoP8GHb9xzrY/tZSRcD37d95Sne27XOIyJispCE7dP+0n4+zvm2mKR3SHpn\naf8MsAh4CdgE3F26LQUeL+1NQE95Auxq4BpgW7l1NihpQdngXzJkzNLSvpPOAwIREdHlzue22Azg\nLyS5HOcbtvskPQdskPQJOlcldwHY3ilpA7ATOALc27jcuA94DLgU2Gz7yVJfDayT1A+8BvScx3wj\nImKMVLstNp5yWywi4ux15W2xiIiI00m4REREdQmXiIioLuESERHVJVwiIqK6hEtERFSXcImIiOoS\nLhERUV3CJSIiqku4REREdQmXiIioLuESERHVJVwiIqK6hEtERFSXcImIiOoSLhERUV3CJSIiqku4\nREREdRMiXCTdIul/Sdoj6ffGez4REXFmXR8uki4C/j2wGPhF4DckvW9ov7feGuuZdad2uz3eU+ga\nWYsTshYnZC3GRteHC7AA6Lf9iu0jwHrg9qGdjhwZ83l1pfzFOSFrcULW4oSsxdiYCOEyC9jX+H5/\nqZ3kJz8Zs/lERMQwpoz3BGr52MfgbW8b71mMnDQ6x+3vh23b6h5ztOY6Wo7Nd88eeO658Z1Lt8ha\nnJC1GBuyPd5zOCNJvwz02r6lfP8AYNtfavTp7pOIiOhStkfl18eJEC4XA7uBjwDfB7YBv2F717hO\nLCIiTqvrb4vZPirpfqCPzh7R6gRLRER36/orl4iImHgmwtNiZ3Shf8BS0mxJWyW9LOklSZ8u9emS\n+iTtlrRF0rTGmOWS+iXtkrSoUZ8vaUdZq1XjcT41SLpI0vOSNpXvJ+VaSJom6c/Lub0s6cZJvBbL\nyxrskPQNSVMny1pIWi3poKQdjVq1cy9rub6MeUbSnBFNzPaE/aITjv8beBfwNuBF4H3jPa/K5zgT\nuL6030ln/+l9wJeAf1vqvwd8sbSvA16gc8vz3WV9jl2hPgvcUNqbgcXjfX7nuCa/C/wpsKl8PynX\nAngM+J3SngJMm4xrUf7+/x0wtXz/Z8DSybIWwD8Drgd2NGrVzh24B3iktD8OrB/JvCb6lcuIPmA5\nkdk+YPvF0n4D2AXMpnOea0q3NcAdpX0bnf/4b9reC/QDCyTNBC6zvb30W9sYM2FImg18FPhqozzp\n1kLS5cA/t/11gHKOg0zCtQB+CPwE+BlJU4C3AwNMkrWw/W3g8JByzXNvHmsjnYerhjXRw2VEH7C8\nUEh6N53fUL4DzLB9EDoBBFxZug1dk4FSm0VnfY6ZqGv1R8BngeZm4WRci6uB/yvp6+UW4X+S9A4m\n4VrYPgx8GfgenfMatP0Uk3AtGq6seO7Hx9g+Crwu6YrhJjDRw2XSkPROOr81fKZcwQx9EuOCfzJD\n0q8BB8uV3Jmezb/g14LObY35wH+wPR/4f8ADTM4/F++hc6v0XcA/pHMF81tMwrU4g5rnPqLPxUz0\ncBkAmptLs0vtglIu9TcC62w/XsoHJc0or88EflDqA8BVjeHH1uR09YnkJuA2SX8H/GfgZknrgAOT\ncC32A/tsH/us+X+hEzaT8c/Fh4C/sX2o/Gb9F8A/ZXKuxTE1z/34a+Vzh5fbPjTcBCZ6uGwHrpH0\nLklTgR5g0zjPaTR8Ddhp++FGbRNwd2kvBR5v1HvKEx5XA9cA28ql8aCkBZIELGmMmRBsf872HNvv\nofPfeqvt3waeYPKtxUFgn6S5pfQR4GUm4Z8LOg+5/LKkS8s5fATYyeRaC3HyFUXNc99UjgFwJ7B1\nRDMa7ycdKjwpcQudP1z9wAPjPZ9ROL+bgKN0noR7AXi+nPMVwFPl3PuAn22MWU7nKZBdwKJG/YPA\nS2WtHh7vczvPdfkwJ54Wm5RrAfxjOr9gvQj8VzpPi03WtfgsnXDdQWfz+W2TZS2AbwKvAj+ms+/0\nO8D0WucOXAJsKPXvAO8eybzyIcqIiKhuot8Wi4iILpRwiYiI6hIuERFRXcIlIiKqS7hERER1CZeI\niKgu4RIREdUlXCIiorr/D4WFOfIRHmR6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10739c7d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "points, values = grad_descent(x1, x2, y, 0, 0, 0.00001, 10000)\n",
    "print points[-1], values[-1]\n",
    "plt.plot(values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "График получился не очень репрезентативен. Давайте посмотрим в последнем случае на каждое сотое значение функционала. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[323297.57383659371, 27.461800541681825, 26.474322124700553, 25.522351708096949, 24.604612486151403, 23.71987356484934, 22.866948310976838, 22.044692760579082, 21.252004084648654, 20.487819109983999, 19.751112893237053, 19.04089734623286, 18.356219910723159, 17.69616228079153, 17.059839171199489, 16.446397130020777, 15.85501339397095, 15.28489478489689, 14.735276645947607, 14.205421815998115, 13.694619640950711, 13.202185020588491, 12.727457489702996, 12.269800332261717, 11.82859972742928, 11.403263926296411, 10.993222458211823, 10.597925365653026, 10.216842466610231, 9.8494626434933341, 9.4952931576087316, 9.1538589882863466, 8.8247021957700458, 8.5073813070182869, 8.2014707235895727, 7.9065601508194003, 7.6222540475233425, 7.3481710954877109, 7.083943688036296, 6.8292174369869976, 6.5836506973385776, 6.3469141090476526, 6.1186901552829696, 5.8986727365647234, 5.686566760215797, 5.4820877445768792, 5.2849614374524894, 5.0949234482777008, 4.9117188935104528, 4.7351020547762888, 4.5648360493044935, 4.4006925122154428, 4.2424512902319682, 4.089900146404374, 3.9428344754534685, 3.8010570293485793, 3.6643776527542444, 3.5326130279886137, 3.4055864291539821, 3.2831274851073577, 3.1650719509547276, 3.051261487761797, 2.9415434501860527, 2.8357706817449468, 2.7338013174462925, 2.63549859351546, 2.5407306639643266, 2.4493704237565042, 2.3612953383308062, 2.2763872792550481, 2.1945323657892679, 2.1156208121460778, 2.0395467802435103, 1.9662082377521981, 1.8955068212473267, 1.8273477042811495, 1.7616394701994864, 1.6982939895314009, 1.6372263017879596, 1.578354501511243, 1.5215996284204532, 1.4668855615088179, 1.4141389169480298, 1.3632889496642051, 1.3142674584528251, 1.2670086945056902, 1.2214492732269306, 1.1775280892201385, 1.1351862343323449, 1.0943669186448488, 1.055015394305455, 1.017078882099038, 0.98050650065901157, 0.94524919822390074, 0.91125968684798597, 0.87849237897747279, 0.84690332630753606, 0.81645016083761146, 0.78709203804647498, 0.75878958211068481, 0.73150483309262149]\n"
     ]
    }
   ],
   "source": [
    "print values[::100]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как видно, фукционал действительно уменьшается, то есть все верно (но видим, что даже 10000 итераций оказалось недостаточно)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}