{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "SGNQebQ2VF_v" }, "source": [ "# Phystech@DataScience\n", "\n", "## Регуляризация\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Bot check\n", "\n", "# HW_ID: phds_sem3\n", "# Бот проверит этот ID и предупредит, если случайно сдать что-то не то\n", "\n", "# Status: not final\n", "# Перед отправкой в финальном решении удали \"not\" в строчке выше\n", "# Так бот проверит, что ты отправляешь финальную версию, а не промежуточную" ] }, { "cell_type": "markdown", "metadata": { "id": "3wKI9ax8hHSM" }, "source": [ "**Баллы за задание:**\n", "\n", "* Задача 1 — 20 баллов" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "id": "IbB0CJgQZ6Zu" }, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "from sklearn.metrics import r2_score, mean_squared_error\n", "from sklearn.linear_model import Ridge, Lasso\n", "from sklearn.datasets import load_diabetes\n", "\n", "import warnings\n", "\n", "warnings.simplefilter(action=\"ignore\")" ] }, { "cell_type": "markdown", "metadata": { "id": "twzlgl_shTWc" }, "source": [ "# Задача 1" ] }, { "cell_type": "markdown", "metadata": { "id": "hW9sFI1-hftk" }, "source": [ "В данной задаче мы продолжим работать с датасетом о диабете, с которым вы уже знакомились ранее." ] }, { "cell_type": "markdown", "metadata": { "id": "P-ysW8Yghn9N" }, "source": [ "Загрузим данные" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "v8JJBr3-hUNO" }, "outputs": [], "source": [ "data = load_diabetes()\n", "X, y = data.data, data.target" ] }, { "cell_type": "markdown", "metadata": { "id": "_Qv38ST7hxA-" }, "source": [ "Разбейте данные случайно на две части — обучающую и тестовую в соотношении 80:20." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "dLSDQZdLhlU1" }, "outputs": [], "source": [ "X_train, X_test, y_train, y_test = <...>" ] }, { "cell_type": "markdown", "metadata": { "id": "YzzRpAggi3nl" }, "source": [ "Масштабируйте признаки, с помощью `StandardScaler`\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "eAFCv6Jhh2bl" }, "outputs": [], "source": [ "scaler = StandardScaler()\n", "X_train = <...>\n", "X_test = <...>" ] }, { "cell_type": "markdown", "metadata": { "id": "U1gVgvXx6CTf" }, "source": [ "## Теоретическая справка: L1-регуляризация и градиентный спуск\n" ] }, { "cell_type": "markdown", "metadata": { "id": "QbkSgO0e2AcJ" }, "source": [ "\n", "**Цель регуляризации**:\n", "Борьба с переобучением за счет добавления штрафа за большие значения весов.\n", "В L1-регуляризации штраф пропорционален сумме абсолютных значений весов.\n", "\n", "**Функция потерь**:\n", "$$\n", "L(\\mathbf{w}) = \\underbrace{\\frac{1}{n}\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}_{\\text{MSE}} + \\lambda \\sum_{j=1}^d |w_j|\n", "$$\n", "где:\n", "- $n$ — количество объектов в выборке\n", "- $d$ — количество признаков\n", "- $\\hat{y}_i = \\mathbf{x}_i^T\\mathbf{w} + b$ — предсказание модели ($b$ — свободный коэффициент)\n", "- $\\lambda$ — коэффициент регуляризации\n", "\n", "**Градиенты для обновления параметров**:\\\n", "Для весов:\n", "$$\n", "\\frac{\\partial L}{\\partial w_j} = \\frac{2}{n} \\sum_{i=1}^n x_{ij}(\\hat{y}_i - y_i) + \\lambda \\cdot \\text{sign}(w_j)\n", "$$\n", "\n", "Для свободного коэффициента:\n", "$$\n", "\\frac{\\partial L}{\\partial b} = \\frac{2}{n} \\sum_{i=1}^n (\\hat{y}_i - y_i)\n", "$$\n", "\n", "**Особенности оптимизации**:\n", "1. При $w_j = 0$ функция $|w_j|$ не дифференцируема → используем субградиент:\n", " $$\n", " \\text{sign}(w_j) = \\begin{cases}\n", " 1, & w_j > 0 \\\\\n", " 0, & w_j = 0 \\\\\n", " -1, & w_j < 0\n", " \\end{cases}\n", " $$\n", "\n", "2. L1-штраф применяется только к весам ($w_j$). Градиент для смещения ($b$) содержит только производную от MSE\n", "\n", "3. Обновление параметров происходит по правилу:\n", " $$\n", " w_j = w_{j-1} - \\eta \\frac{\\partial L}{\\partial w_{j-1}}, \\quad b_j = b_{j-1} - \\eta \\frac{\\partial L}{\\partial b_{j-1}}\n", " $$\n", " где $\\eta$ — скорость обучения" ] }, { "cell_type": "markdown", "metadata": { "id": "pKqa_3Y71qov" }, "source": [ "## Реализация L1-регуляризации" ] }, { "cell_type": "markdown", "metadata": { "id": "P6U79M7HjL1F" }, "source": [ "Теперь реализуем Lasso регрессию на практике:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "KYTm-GfckBwW" }, "outputs": [], "source": [ "class MyLassoRegression:\n", " \"\"\"\n", " Класс для реализации Lasso регрессии с использованием градиентного спуска.\n", "\n", " Параметры:\n", " - learning_rate: скорость обучения (по умолчанию 0.01)\n", " - lambda_: коэффициент L1-регуляризации (по умолчанию 0.1)\n", " - n_iters: количество итераций градиентного спуска (по умолчанию 1000)\n", " \"\"\"\n", "\n", " def __init__(self, learning_rate=0.01, lambda_=0.1, n_iters=1000):\n", " self.learning_rate = learning_rate\n", " self.lambda_ = lambda_\n", " self.n_iters = n_iters\n", " self.coef_ = None\n", " self.intercept_ = None\n", "\n", " def fit(self, X, y):\n", " \"\"\"\n", " Обучение модели на тренировочных данных.\n", "\n", " Параметры:\n", " - X: матрица признаков (n_samples, n_features)\n", " - y: вектор целевой переменной (n_samples,)\n", " \"\"\"\n", " n_samples, n_features = X.shape\n", "\n", " # Инициализация параметров\n", " self.coef_ = np.random.normal(scale=0.01, size=n_features)\n", " self.intercept_ = 0\n", "\n", " # Градиентный спуск\n", " for _ in range(self.n_iters):\n", " # Предсказание\n", " y_pred = <...>\n", "\n", " # Вычисление градиентов\n", " dw = <...>\n", " db = <...>\n", "\n", " # Обновление параметров\n", " self.coef_ -= <...>\n", " self.intercept_ -= <...>\n", "\n", " def predict(self, X):\n", " \"\"\"Предсказание целевой переменной для новых данных\"\"\"\n", " return <...>\n", "\n", " def get_coef(self):\n", " \"\"\"Получение коэффициентов модели\"\"\"\n", " return self.coef_" ] }, { "cell_type": "markdown", "metadata": { "id": "ZiFgRpcnkge1" }, "source": [ "Обучите написанную модель и модель `Lasso` из `sklearn.linear_model` на наших данных" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "orGi9naQhtBN" }, "outputs": [], "source": [ "# Создание и обучение кастомной модели\n", "custom_model = MyLassoRegression(learning_rate=0.1, lambda_=0.1, n_iters=5000)\n", "custom_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "rWudKHp8nYSY" }, "outputs": [], "source": [ "# Создание и обучение модели из sklearn\n", "sklearn_model = Lasso(alpha=0.1, max_iter=5000)\n", "sklearn_model.fit(X_train, y_train)" ] }, { "cell_type": "markdown", "metadata": { "id": "iF0LwesInf6G" }, "source": [ "Распечатайте коэффициенты и сравните их с коэффициентами модели из `sklearn`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "b4Rqf0oWne4W" }, "outputs": [], "source": [ "print(\"Кастомная модель коэффициенты:\", custom_model.get_coef())\n", "print(\"Sklearn Lasso коэффициенты:\", sklearn_model.coef_)" ] }, { "cell_type": "markdown", "metadata": { "id": "9ZzY-wijkqF5" }, "source": [ " Сравните качество моделей со на тестовой выборке, используя `mean_squared_error` и `r2_score` из `sklearn.metrics` ." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "i7SKK9xZk3Yu" }, "outputs": [], "source": [ "# Расчет и вывод MSE\n", "custom_pred = custom_model.predict(X_test)\n", "sklearn_pred = sklearn_model.predict(X_test)\n", "\n", "print(\"Кастомная модель MSE:\", mean_squared_error(y_test, custom_pred))\n", "print(\"Sklearn Lasso MSE:\", mean_squared_error(y_test, sklearn_pred), \"\\n\")\n", "\n", "print(\"Кастомная модель R2:\", r2_score(y_test, custom_pred))\n", "print(\"Sklearn Lasso R2:\", r2_score(y_test, sklearn_pred))" ] }, { "cell_type": "markdown", "metadata": { "id": "CvGkaNGco14F" }, "source": [ "## Теоретическая справка: L2-регуляризация (Ridge) и градиентный спуск" ] }, { "cell_type": "markdown", "metadata": { "id": "1gWVPsVX6KOC" }, "source": [ "\n", "**Цель регуляризации**:\n", "Сокращение сложности модели за счет штрафа за большие значения весов.\n", "В L2-регуляризации штраф пропорционален сумме квадратов коэффициентов.\n", "\n", "**Функция потерь**:\n", "$$\n", "L(\\mathbf{w}) = \\underbrace{\\frac{1}{n}\\sum_{i=1}^n (y_i - \\hat{y}_i)^2}_{\\text{MSE}} + \\lambda \\sum_{j=1}^d w_j^2\n", "$$\n", "где:\n", "- $n$ — количество объектов в выборке\n", "- $d$ — количество признаков\n", "- $\\hat{y}_i = \\mathbf{x}_i^T\\mathbf{w} + b$ — предсказание модели\n", "- $\\lambda$ — коэффициент регуляризации\n", "\n", "**Градиент для обновления весов**:\n", "$$\n", "\\frac{\\partial L}{\\partial w_j} = \\frac{2}{n}\\sum_{i=1}^n x_{ij}(\\hat{y}_i - y_i) + 2\\lambda w_j\n", "$$\n", "\\\n", "Заметим, что функция дифференцируема во всех точках → стандартный градиентный спуск" ] }, { "cell_type": "markdown", "metadata": { "id": "B9rz_h9Np-Fu" }, "source": [ "## Реализация L2-регуляризации" ] }, { "cell_type": "markdown", "metadata": { "id": "gMT9PRtmqGNI" }, "source": [ "Настала ваша очередь реализовать свой класс уже для L2-регуляризация" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "692RkTKWqQM-" }, "outputs": [], "source": [ "class MyRidgeRegression:\n", " \"\"\"\n", " Класс для реализации линейной регрессии с L2-регуляризацией (Ridge)\n", "\n", " Параметры:\n", " - learning_rate: скорость обучения (по умолчанию 0.01)\n", " - lambda_: коэффициент L2-регуляризации (по умолчанию 0.1)\n", " - n_iters: количество итераций градиентного спуска (по умолчанию 1000)\n", " \"\"\"\n", "\n", " def __init__(self, learning_rate=0.01, lambda_=0.1, n_iters=1000):\n", " self.learning_rate = learning_rate\n", " self.lambda_ = lambda_\n", " self.n_iters = n_iters\n", " self.coef_ = None\n", " self.intercept_ = None\n", "\n", " def fit(self, X, y):\n", " \"\"\"\n", " Обучение модели с использованием градиентного спуска\n", "\n", " Параметры:\n", " - X: матрица признаков (n_samples, n_features)\n", " - y: вектор целевой переменной (n_samples,)\n", " \"\"\"\n", " n_samples, n_features = X.shape\n", "\n", " # Инициализация параметров\n", " self.coef_ = np.random.normal(scale=0.01, size=n_features)\n", " self.intercept_ = 0\n", "\n", " # Градиентный спуск\n", " for _ in range(self.n_iters):\n", " y_pred = <...>\n", "\n", " # Вычисление градиентов с L2-регуляризацией\n", " dw = <...>\n", " db = <...>\n", "\n", " # Обновление параметров\n", " self.coef_ -= <...>\n", " self.intercept_ -= <...>\n", "\n", " def predict(self, X):\n", " \"\"\"Предсказание целевой переменной для новых данных\"\"\"\n", " return <...>\n", "\n", " def get_coef(self):\n", " \"\"\"Получение коэффициентов модели\"\"\"\n", " return self.coef_" ] }, { "cell_type": "markdown", "metadata": { "id": "NNKp901fA0N-" }, "source": [ "Создайте и обучите кастомную модель и модель `Ridge` из `sklearn`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "id": "pBXdXcwFq_BI" }, "outputs": [], "source": [ "# Создание и обучение кастомной модели\n", "custom_model = MyRidgeRegression(learning_rate=0.1, lambda_=0.5, n_iters=1000)\n", "custom_model.fit(X_train, y_train)\n", "\n", "# Создание и обучение модели Ridge из sklearn\n", "sklearn_model = Ridge(\n", " alpha=0.5, # alpha в sklearn соответствует lambda_ в нашей реализации\n", ")\n", "sklearn_model.fit(X_train, y_train)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "fpI26H1vrPVf" }, "outputs": [], "source": [ "print(\"Кастомная модель коэффициенты:\", custom_model.get_coef())\n", "print(\"Sklearn Ridge коэффициенты:\", sklearn_model.coef_)" ] }, { "cell_type": "markdown", "metadata": { "id": "yU557wszrPVg" }, "source": [ " Сравните качество моделей со на тестовой выборке, используя `mean_squared_error` и `r2_score` из `sklearn.metrics` ." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "g1f-EbQ0rPVg" }, "outputs": [], "source": [ "# Расчет и вывод MSE\n", "custom_pred = custom_model.predict(X_test)\n", "sklearn_pred = sklearn_model.predict(X_test)\n", "\n", "print(\"Кастомная модель MSE:\", mean_squared_error(y_test, custom_pred))\n", "print(\"Sklearn Ridge MSE:\", mean_squared_error(y_test, sklearn_pred), \"\\n\")\n", "\n", "print(\"Кастомная модель R2:\", r2_score(y_test, custom_pred))\n", "print(\"Sklearn Ridge R2:\", r2_score(y_test, sklearn_pred))" ] }, { "cell_type": "markdown", "metadata": { "id": "eRNWIruRnh6o" }, "source": [ "❓ **Вопрос** ❓\n", ">Что вы можете сказать по полученным коэффициентам и метрикам?\n" ] }, { "cell_type": "markdown", "metadata": { "id": "dWaRy7OGntt_" }, "source": [ "**Ответ**:" ] }, { "cell_type": "markdown", "metadata": { "id": "vFsWFLBtrl3X" }, "source": [ "Сделайте выводы по всей задаче." ] }, { "cell_type": "markdown", "metadata": { "id": "2xfXOp6ZtYIA" }, "source": [ "**Вывод:**" ] }, { "cell_type": "markdown", "metadata": { "id": "cxm_uXD6IANq" }, "source": [ "# Визуализация: Градиентный спуск" ] }, { "cell_type": "markdown", "metadata": { "id": "Zwuy_bAww8f8" }, "source": [ "\n", "**Что такое линии уровня?** \n", "Это проекция 3D-поверхности функции потерь на 2D-плоскость. Каждая линия объединяет точки с одинаковым значением функции потерь. Представьте, что:\n", "- Высокие \"горные хребты\" = высокие значения потерь (плохие предсказания)\n", "- \"Долины\" = низкие значения потерь (хорошие решения)\n", "\n", "\n", "\n", "\\\n" ] }, { "cell_type": "markdown", "metadata": { "id": "YEoIVghFZRsH" }, "source": [ "![линии уровня.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAm8AAAFCCAAAAACCh2UhAAAACXBIWXMAABcSAAAXEgFnn9JSAAAgAElEQVR4nOy9Z3hbx5kv/s456JUACALsvTexiWqURIkqlmRJsdxiJ46dXnc3m+xudu/e/929d/e59/6zSTZxuuPEcUscd8mybPVGFZKiSFHsFWAF0Xs7OHM/kJIIAiQBEAQprX4f9IgHwDlzZn7zzjtvG4RhFYEtdqEA7IiHVrMVDxE7MFbz4b7u3zbt/iY6WrTxId+iA9pJ8xECjBGszS5Fqynfxn/UxaUqGZxnk1axEQ8S3Le77OuqmN6msQbZarclOFZTvmFb8XNJU2+bn3pIt+gAD7TFX2pTZI7+wlz5kG+ByEzhOtVxz+SsZhseJNg7pamjEhrUfTWi1W7LAlhNviEWU3dhfH8esYpteKDghdThsZp476ClRLzabVkAqyrfsPYoHEl2+vgPGRcVCHc732ZVCixdrEzWardlAazmSOPJo/T+JMeFdnoVG/EggSE2dKYWkuahxJS1uTtdXb5p37nJVbcfPYHJVWzEAwXf8HSOAqvHM5LWKt8WWU+xC3NX0orjbjTsvfUqEI+Wr0rnOGkuWqujEimwAyWwPV2ObMFqt2QhLMw3PPwJ65GVtFS4WI8V5Pp0dfXCmb/dwIrh+Bvetz6S+6DxjczINE1qz3KKVlUtXwwLN4z+9N9FwidWcET4W3jMvO94pMzZ5930VvFW7mnzgJt+aKK/86DtU4iCp4Y6hnuSctashrKIfLPoTbf3r6BgZogBCMndP13nvQWx45u3Y9RnjdnTYgSs7UypxlZGccKaFdwLz3CULqFUMRwS17TVF7un2VWUMOlBE2/UhX/9SMjrk+28Y32jVzUaIxgW4VuahBp3xK4l1qkJZwyf1k+JUh84vllE+e6Lw0c2zKoo5mv61W1QIBbhW6oETxpiNkHwaOetidhNR6Max2Wt2VUnQrBq9rAbtc8/I539W/3a6Kq2JwgW1t+QJLXJMlQeK0O1o8PiaK7kxuhp9KgVJUqX/t79BaJQoWfEC5l3/tYMeVazOcGwyJLCyOPYR2LVYDwykp/cooqVgKOG7MxMToweFjMgpqIwV3qXblijHouhShwSFuUb1z7sjVE7jCfImk3ouClGj/MOOlgF7Bg9LJaYa8J23Na0rjUBtwjfyEwePW2JTTOcn/bUJyoO3z4doy2DTeVlZ69Zo2iUMNbibZtaYzvUxbZo4gTQTsTEl249+WndRiau3X7sY1ssngeTOhSf+KBtF+bBdZ4odFxyr3Yz/LEI35A4ixgfigXfcPfZ+sM8GthHGj7pj8HzAIYmcL5k6a/dz8CtVx4p2dzcsbaCbxZbU8S5hHE8Js1N/VI2n8aABU+UJsfiefSYCQofbL7hoTfz6vo33j4uyVpLZsbF2sJJ4bjHY+JhSCzjz/yHW5EQi+c51W5GRqxsL6sCevh39JE4Wr7X9KfRtSThFuMbmaKgVLoYNCLmipR2mIpPWbM+7WjA/on7G3kYo+KnHRftq92YOVhsPSWSU/rHdPkxa0vsoBvxpqzZENiogCjdlU0AAKNKbF9LE2sxviFlCuhGfWupudEBntJBomK1W7Gi4G4kZ+YTs4C+X/Q3EOTyrEMxdKLHCp5hO5kuXO1WrCgIxh3xjci1JMgX5RszU+gYccWqKbGDvccuKlirGUwPNhbnW7rIM/bARSUCWEfc4pIH3buwNrEo31CCAk0MU7FqS6yAJ6dBkbyWVpn/Olicb+I8pnbkgeObT6UjMx9s9W3NYvG9iyCbZVQ9cHzzDuiYhWs2Y+7BxuJ842TwvGrzGgsxWDYsox5u3gMX/HZ/YHG+oaQUPDQRo6bEDJMqWr52M5gebCzBt+RsNKF70OTb5IgvK3G1G/FfFEvwTZZGGvsfMAucd0QPWQ/l2+pgCV8HN0fiUq0lf28UYBty8bMe6OCQNYwl+MZIkrh6zbFpSqxg6nEm5Dy09q4OluJbdio9NraWAqiWDayd8CUXMJf+4kOsAJaKHYhLYxqHYpWkFRNQAwaUslbL2z7wWIpv/ByervuB2jC4+w38Yv5qt+K/KpbSY/j5wvFhi2jldnM+DNhHA8ECAAT07NK9gjE0VrUjPj92hZgewg9L8Y1ISpocnYqacxvTXjfCLi+QYi4C8E4ZNG7k0TmwbEsaAJg6TAgAMCNelsRGAC4TBQwuwgw2Ga2gczw6TCseuMpI9w2W3KclZLWNjVdEPj4Y0xjTAAQTAVCdfToT+PRmzN9bzwcY/mGnlQLK7sVx33seAZz96TQAAJD8wm/WALgvHbMiYQIBXEV+EQcAUz4AAiFiGYVQ6VE1kat8aH1bJSzFNxSfzTL2uSL1bjtMU2aT0W7wQu6+eATTP/nESwF4Kcz2lvEBT50dIlgISBYxQ4DR/pmCDl5Kf6CKBNNHf3QCyURAsLf/Sz4C26cdNI4XSmS8xDhehFPA2a9lZq3Rw1f+C2BJ+cbJl43fNvFDFwgYY+y2WjiJDATUuTdUTrvd7aShplSGwGPUk0ImMIQMfpEAAKXt7+YnESCQM2WbmQC4wTQNAIg2mkpLSQBuUbmZslJAW10WLwBo3jiJgccWCFgpTx3gAFAaqyiOhVA44s444JLnP7T2rhaW5BuRpRztN4Wcheyx6zXT2mnNpOIr5QDepqMOkiAJxGSnCxGA4tlMloIDbAWbmxMHACn/6GRwEJAsgsHEGKDgr7wAgLCH4ksBQHCkzObSuMGndddmIACWku/BTocO+9rTd3EA9/1WpUyJlysTpFx2qOLOpKaUGQ9eCtD9gqX5lpraMt6XF2K0v+HYySmjyeymKFl5EQvIgvW2FGGSUCpl56QjAM6hRxADYYKY1cAYc9zmGABI/5x3QiYFmqYBfMBiAoDy23spy5R1zKXilrMAcN97EwwGSyyRStc/mRZaA33qMZyW/nC7sFpY2q8jKjxpub1rUb7RXtu4NVNJAL79ox4aCMSIkxYVMgGY+9f7uAwOg8lABAIAxA6vBhZCfgZCZmEB+NyUk3aQcg4ASt7SobM6bKOAWjLTALBGxU0SMBcVXt4hAztL8nC7sFpYmm/cXOFkj2PB8GvssRkG2geHTPv/RgrAk2tESWmpivj4tAwEgERRNeQjBECy4K4QLPlX9bR2cmx03JYoAQDzKx/wMnPKcqXChQ9ysPdYhDkPYy1XDUvzjZklHx9QLRC/gy3tN6/1a412mog3SQEV/2Q8UcTnc4lYSBDEzc0B2mW3W3XxuQCgv3rDd5Eric+rLa8UL9DeqUF3VuFDvq0aluYbSizsnByuDj5+no9+OOSiEFMsLzqcCADckuKY6uIISD4fgEYIABSPk93TjvGx2x8n/eDZ4A55WjUBaQkP1bdVQwhxOdI8ln6ImqfAYZdTyASgRkYcwvjkmnWZqQlsgNU6/23mqfwnto8Pt7aOaq3qPooJQFnYvHnTxNM3xsp54OpE30cIgW+CPJGm3aCce8mju31FfWQXCzj7TM6yqiTR/HFdFSBOamrtIctkcwfxFBvAc/Ev0roSpZ8uZx6wi4se7MJvaxsh8I3ISdX0z+Ebdg42nrs+hWETC8jyIiDXUoEKxOMpSn3AQgDWD970vl7dUJc9x1it6aGSMh/GWq4eQuh7lJyIRscLZldK7Bo5fazViNnJhWwAINZekW/EmHkpZnaKevLYpYoDO7O5s4zzjQ6j7DVV7/G/GkKZ65L8M7amjbMuVP3RPzebCWl+7dZNa9wrJHg2u+lyt/HsjaNPPB4/Qzh7l4GX/2AX4lrjCIVvzJK4yVu2Gb75bv3mBiSU763LFq+omPDRxHJtKoT8wI6+S6dadRd0hVtn7uUctCeGnemMfZh8KBKjhFD4RubJx3vGFQgAAHHkKUW7duStrGzzGSctSJHEWR7jEBJUFuw4e7pDNksxPDVIycJU37BrctotThc9dLlGBaF0PkrJ75y8tY4EACBK/1mbnbnCK6mvv03Ka+quf2QBq20Y4JXm7B6IK5uRT1THAJmbEp6sMjdZE/Tvyg8XPtxlRAMh9aK44oS5kyLsPj6JBLV4xY99N7yj+XIR/c7NuIYopFFxC/IRAqDtmO/u03FL48P6tffaOzvKXddfMf7DQ7UvGgiJb9yCOHOvzvO25fEyhGJQTdxwcbiuNF1yoW1LNNL2ZqbH4BvoiHjIrcgKL1XGe/MUe6s809aoe8i3aCAkvhEZmar+242/9PIKY2L+EFULZeD1EtzoUZu+9lPCvKWLTp8T+4Z9JAJ6cZcImbc+jUO7ac7D5TQqCK0bUwouTb5zyZCoiM0+Lf7r9nRQaYvWR1FPFPInP5pQM4rvHbJrax8tzfcMSFMW+xlrZ7FEYhji7Hgo3qKC0PgmLBJYP7Qwt+5cqbR0DAAEIjAAIABmKm3vuZx4qCR6m0KiZs/rKr1FnHvnaHdwvvMnU9nnqL5ds3+7J5z3xCmOS5jpGRQndk9fmzp06L7MkF5xTTtshMY3olxu1oJ8f4hBtGGDVnc6yVbTMTGzOAMBAHIPNLUV1Eez5mniYycmDZBwryz5WPOWeNsZe8Md8Tb24uAc6b3ruTsEozUdZz0NefeRBY72URgoGsBOOc0YSBIIBrFG2h8i37IUgxjY8SvVaKy/ZSXU9nYeLyEDAACYqVzn8Ze+nhm9+YkS2ABEUs5dkcl9upTVfXT9pjvm37g9pjlPy76rqBKSMs4f/gh7137QHAavzW53mo3mKS9ojIAsPWPxNCGVEly5MF7E5Ql5zNifHeWPENXguPU37WC/Vb9C2wUiX0Gj13XPxhNxM5Rmxsfjs68pvixe4oehw9tqBuCV3DsOLlFJ9F6qqL/7BOnuuXUV78kDJBTGd/3/joyyNSIhgoOmnWMTk5Pjk1Mml8fjosFFAWA8hACYLERymGy2RJGiTE5OVnKWk767XITIN86mN+xg+eSRkpVpKRIIgJZRqfKZP52TLAVTnnX2xIHo8W3kPROAsOyeGkb6Wi4Xb76XWIsdcwtjszkz9KJ1piQ+N5/ddm3tJhFiu1XT26Eam5x2UxQNQDCYTMTjEEBZ+Szs82DstFI0AMkgOcrE5KziXKUwilv/sBAi38jctGlMt/4leQVjFWk8K2Doy/+e9INSgg2G6B1ObHvvCgYiJefehsfXcnpbDWOcvBPtq355+J4EQ9uemFEeJ/5v61ef5LBIp35tFiXDtEk13HJjzGx2Y4wJoVQs5svFcgXBzOQi9esHS2jrhIeaNBmm7WaTyeY09SKOWJJWXp2bLlgNMReqWSmxtNWHLX9SPrt8F9OSoAebcjUlPifIoyZS7H/+rQ2AzL6XR0s3H6+vJkffLtk9e4GpxHP3pzPcw9rmrgEPx01x49egAQ5Tjt7mG20quxcTDLYyMz05JVmp4DMIkgQgEcQlFFcC9gH20T6ffXp8YnRseETvmZroOsPPqayoyebFPHQx1F6UrHvDR8Dwj/UvJK245xplNjxaiaZ7+QeSonNDWvP6i6OI8LFL7kaN4rYX+Y0DjFbW43fi+hK/NleXng1OQYqqiscFji575ZY1d96WzzR25UzXqB0jriw+p7IkWSoT+MfU0DQNgBgATACIS66gsVWvG2/r7NVZjM0tgtTShtrEuNgGIoTKN7IkrV+UpB75+cgzVSsbiQRAVD7OVE+fwV87HJWyWbTt5qvHdLwMrS6l4q7ANJ7dV/fJb121X0ibHSGEgr6V/OnbxmH1zS1fzllbpiyfY6jj/GWNlWbIUjNKKovj+JylR4WAuLhserdT29N6u3/K1t37aXLtzvJUXkyy6WYQKt9Qbk4/57HJDwx/btt/KFewoi1E8Z/p1jplP6iIRl4ybe//6MMuj3h/xU9RWu7dMSHqCsRHkpyVS5V2YGxMVg2YH12XuZYKsGKvpf3k1S4zTYqSazYUZUsZYQgpQiCQ5+/X9XZfa520dfUcLduyoyCOGSvGhayVyCrP2LzfzXy3t33g5CMNeXErubwgYY3bxY6G8kZZ+j890Wll53/2qY8t7AL53Q/EtQCyAyE0hZGVbgf+Wop+w+6x659eU7sZopTyuspUWQRtI8nk5K1H1C2NnSrj2Wtvb6mvSWbHhnEh8429Saq9Lf5m5RsXplp6jm/eVpEQggCPGIgTBfMqdk+3NV66bUfK+qe3U83WhLJ71o8wepdcU64s7Bht/Piq3svJLttSXbRIJYGlwFAqyp/outrYOd0zdGLT/tpkbiwMjCHzjchP1IwMbWnIrPm43dDaeXJdfXW6kLG2lJq5oH02VcullhEXISvbvy+L2dqN5fd/zCR2Dp45e3MKi9I37CzNYC6PIYgtr1t/8Oalq+qJ95rLdu3MiJy8ISP0EYivvD3RVMvKT204e7Flqqf/TF51ZWkaPwZtjAAeh7qjtaXHQLFTK7ftyOaBr22AVZ6x2s1aLlzqs0ebDUhYtndrniwa4ojgFuU1dF483aVSXTn16LbUFd+Eh843wZZ3rW1GJfCLMnc1X7w5or92I76gtCI7QSog11IcAk1bDbrBm7d6tF4QZVRs2ZDKQwC2W7a4KvnSv17L8I5fPHZtCieW1O0s5kdt8WMo4ivqzzZ2T398c9PhzYoVtsiFzjeyKONW/5ASAPHys+oHmpq7VVPjjXJFen5WeraMFbMdzmLAXsqsGurtUWmmXQQnvbhyQ17CTMNGr4OyaC3tMcMGbbr63vlRWlR1qC5TEFVVi5RsKT144f2OsXdvNBxav7IG/TA0mrR1t0Y7a5gAgFgpSeune24139aMq1s5/KTcjIJCpVCwqmsr9tismp4+9ZDa5KIYwuSS6nWFCXfqTNBdfaigcBVbt2y4Oz/8sNctqNjbUBT9wyNIqSSv5vSpjv6x65/ZX7SSi2oYfJNUvW9uN82uSQQvPbVuaqS1Y3BCo9PdZkoTkrNyM2SyeAFCMTQfAsxUQHdMG3SqgYGJaYOHBmZyclZxdYZyTlUwV4udUx5epsyaAj198s1mE2vd3oNF0ZVtd4AEm0p2v3Oqv03V+NyOFYs7C4tvzOKkvp6puzoQIvlZGRvMo4PtXcMa59R4O5srViSnKBMSE+U8FoMRi901TVFeh3ZyYnpaPTVpdHl8JFuUkFVUlpMu5PnplNpmT9K6tVd6IlR4Ot/4cAQrdj+1XhJOt6IwoiwRKdmQu+3tk8YzIx1PFa6Y5hEG34is3J7h9rk2BUTy+crK/Vq9ur99xGAwG4ZJkiEQi6VJaclJCUIhl89cmXQuDNjrcNhsmokJ1ZjJZLF5fDSQwiRJRll+ilzOn6/00p3DkFO6pgPYFgGe/uSVFpuw9rO7QvNd09jjxF4bjUZMI2Ka5DMJLjOUGu5kwv7K7a+19v7q5hd2ylZojQrHIiWrPT3demhekDdBiEXZVS6HcaSvb1w7rvcatIAYLCZbokiWKaUyvlDMkzBIgrGAgzIM0Bj7fD7K4DSbXTqjRjsxZfB4PRSNCAZHmpyQlJOfKeWxg22WvW06dvX9ujv19b78zjhOe+KZoqVs4JiiXEa9UWvRa7F90gsO9ZAIcxQ8Uh4nkcrj4pmMJQymzNTPrXvtvanTQ93PZK+MSyUcvnHL4zS904FJBQgYAoE8u95t1E9PjQyotTaH00FreglEMMU8vpAbx1PGKcX8JCbBJYHgEkAyUaCJPzDsFANgrw+w0wc+l887aTNrDBqXwWW1uk0eGtM0EGw+T6BIzcyRK2QSDrmQ5jjd6pBV3adntDmafnHKwq19fu8SWhVlM4/2Do7qTGa9jaYBYwBMTyKMEEKIFIol4oSsrDylRLAYkRC3KqX0tRv9vxr5UvWKxJeGwzciq1DT17mAhxsxgCVMA+xyOJ1Tg6rRaZPRZHe4HBhjAMRispkkB3FTOJiTycH8JB5miGZfiCWZaYTBPOkBAKD1npkPXGYKOcftyD1iJ5wTTtrl83q8bgwAiCAInkAkjVOkZGQmcTj8xcNVcfdNyF3b8eALQvfOSx1exTNfyltMpaLdU6P9TbfHbA43BkSQhIgDfCHpmowX+Gx27HR4dRoMiMMVplRVZmYksBbuC0L5haqX39a+1f/N/XHRf5uw+IZSNjZO3Vo8awpxuQA5G3y026jRGvRag15n1FNen8+KKQyoCwGQCBCBgBE3483EHCUTAwAa9nSxMQDyTcyG9dqNFIAPz5whSGMgCILB4rAYMolULpFLZTKljE2QIdQucraNk+syw3jTNQN65JVXxony73xmkVAZn2Pyxo2unmkvhUm2VKRQJCYK5QIklpGqVz5T5jWasdlg1ExNjztcFuNoM1teVF5ZpuQuJOYQa90/Z73Se1078oWk6Nvxw/Io8tclTLRO5C/5PcRkAk+SBRg8TpvNYXVorAata8xG2zzgtAG43BgA3zmm/E5YLYaeO/+585oIALE5AHwuYvFJUSJXLhEmcoR8voAbTjyD/joVv+F+PILS2/GL98zcHX+zdUHhhl0T7ZebB6ZphPipCQV5OYlikVhAzPQgR5pbNPMtymG2Gsb7+wempu22oU8TcjZuLElZyERPJH4t59cXBv5T++WCqDucw7ohKioe7RnICVmTRMBmxwEAYJry0h4frbNgwyTQxmkf0FYLBgDk0M4mBiAdJZ3tVkImwACARCIERIIMQYIcCWUMkkkyIkmkxOM9dNr6sH+2+nA1/sd5t/zIX+UvNLc8uu5r528ZvSRPkpVZui5LyvGz1dL4Tt8yxWIA7HEZe28MDAyapsavStY11OZJFqCcYH/qix/rXlJ9f320IzLC45uy9Jy2vS78+BxEkmwAgEQ8o8diDNhlwwCAnLN8Q+gDw2HJTMYMKeEDACABB81Edi+vSo63SUMWJy79vbUGy8c/b/blv/DMAqfPYudI05lmtRsE2QU1pTnJSyYjIDZbnL7DMdp7q7V/UnfqUnZV/abk4DX2GOv+Of2N4ROmv22I8q4hPIHJLZJpOwzLiAebwxv2bK5fzuzf+LZmw8qcE2lusvLr7rvlFJuO/qydKPvmkbigjKAdA5dOtmowKyVv/dZ8Rai0QEyxuHjvZM/lll7t7a5TtTu3ZgQ92ZPI+rrsN/0XnZ7dgqhKuPD4xi7KmuzpTVuRnR720b6VuC/g7k4qrep+89VjzWsvDzC2fGt3UDMOtvae+vSWGcXl1G6qSA7jtFAAAMTPydquvtHYMjz54bmq3Ttyg7nIkOK5+F+0tPzLxDPSaBIuPL6hlOrmsfb7TFZ4W9VETcgHaq4R4KlXfjnO2/GdumA2Xuwa+uijDiuZWLx9e15cJDo9ISjM29137myv/uzNk4d2ZAZZVZH4MPeXVzp/7Hs2PoqEC7Ox0kqJ/pY2PXrPjwGmb1rE61fClrSCwJpXfj0ufuQ7VcFiNbxjp483GsjE9Ts35US82iGGIqH0kcazLbpzvZf216UEkf/cPXE/O6n6OTwjjx7hwuQbqyh3+vZwylpKH1kKeLjdl1V5P7UYALS/+9WE6PBflwZpNjZd+dO5KUjYuG9r+vIiEFBcdcmecyeaJz5o2vlUTZC4N24dj3l86OfeL0SPcGHyDaWWNo+3V66pDJIl4GwfZ5ausezRpTD90q8mxZ8JSjdn97GjXR5JxaO705dZvh0AELcgbfOp47fUf+7e/2heIHvJyu8Rx4Z/Q3xugeMhw0e4i7+oQqZpOXg/8c3UYhHXCpb+3hqC7Y+/nBQe/m6Qoxax9uxbl4zMssf3FEbnyDLEL8vY8Mnx7usD7c/UBToxGFXf9308/BL/s9EqHBQu31glWROdA2n3z/JE93dSqdX3VWKW/a3fTPA/87clgbtGb++bHw5QybueiqI+isSb8je8dW7qaM+RZzIDAh6IdX9PHht4if+ZMPfACyHcgUCp5S3q9k3RirXA1KwNBDEBAGHvrEGcjJpd29uiJqtS7qfl1HXsx8PMPd8tDqSb49Lvzxi5dU/uSo3q/CHku3PX/+XG7anh52vnb4cRWfE31pO3XxTtiU69xbAbLq38YKpDGzHfaIy9VrfZA4IUHoCv85oNAQBg/sZCBNDVOPMn8CtKOQCuMQsQEo6QDZHHzunazIL199P677nwiz6i7q8DF1OsPfpaizvlwNPromuCBQBWnrL4jROad1RfPCCe78kh133Hcu3Wb4XRKdgTNt/YxTnjN3oiNfmarvVN2vVuvZvO/MZmEoy/es878wHnr3MA0LGfO2abtf1/5QMe+I8+TMh5UlF8QXWEWyQ80OHNK1tztY0WBr71o+t05X/bFDAw9PDLb46hqq/sTVoBczsSb8sq+33XJdXAC6nzCcfa7vz3m+f4stJoRIuEzTeUVn5t/MamkCUG9jodTrfVB8JMPtCX/3XYjjGmAWsfwwBe252KgnwRQoCFDN/MO2EMAHjs6iAAAQTi5n73CBOoMQ2NuDwen80mcIjv7mwfJyoy75/QN6z6RSNV8A+bA8aFuvnixwbR9q/VRUmRmg8y/fn0ly4M/3LsOyXzlDjE3qH/cc+ZlO9Fozpa+IqAZP2Ho1efWppvM1FFmjMDeoPRpnFD9v+uQqAfMROEUCqQsivKSADZV2tna1ombWBhoA/H6WfelV2SDkAUPNlKGw1mM21V6TGA5sWPvEgii5MLMzYXkjA3cGlBTF82ySqjV5V1xaF/7SOn4rv7A0wT3os/vOSUP/t8wYqJaiTal/GLd/RvTX9v03zbr/CQ6Ueat9Oej8ImJXy+Mcsz1D39GYv+ELstOkNyChPwlX/QYJoGAMBWjNC2f55OkQmEPDFTIkQAzM0bZ39BIhoAJT09S7+ZnMK075toq8VmtqgUu5gA1oFB30yhNv43fiAGrOvjK4RBvc134evr9uXU3j++U/eHr+iFLzwe4H13n/6fN+j0b3w+zCiXsFZAxCr7/zJ/pv3E/vd188KlkezpodcnXk5+dPl7hvD5hpSVrVMtWxaM8qUp07iqq7nbtvl/FCKg3CAUCLl8IXNzIQJI/xrFZtxLXfDLocEAyF9JJsRiAAyYdjHYCCD5Sb7VZXjCBRwAACAASURBVHVYXCbKhwBMv/49kVVWk6lULpwF5mge495Hu1Nf48sq1pNfDZAjtj/9cBCVffdwOIIa3/k39LcnUr4l/2nPZcO3n5pnr0RJ39Af7fldetWy7WARbKxF698fbp7IW+A1bE2Xe/qHbRSNBo0AaNO/m+PjJCKxlGQzESAIf4YgQMTM6wueOIideovOqiG28AEcI+PUyEVuUnZJxfYFcq+w9qY1YX00jw1ZUeChV9vpqi+nze9a21v/pmZW/t2+0IKOMPb5fDQ2aUE13U0imRgRJBlaiRck/hz/R+0dP+EcmqclovzPD99ofCU5ablTNwK+MYoK1O23sgN+iQEQ+K78oMsNBEcqyzhcjAAnfSl6MfCIxQKRAu7obdIj3tsGo723+0TiX32bC/cC0+/B19YOBUX3jbHX9NZHzoy/rpyvIVje/D9q9rb/tjEEtQBTVrNFM6IanfQ4HdhtuMkleByOMjUjO14oFoYQG819LOH/XOr5oeexeaKUufUFzdj7xZ8XLnM0IxmLtA1XtDcbJH7XsNcwIM9igs+iJ5UpWfkFRclCjl98ZRQxc09Ow4bpgc7u4WGDTesD8Kom0hPY/j1qb51ibw4QF2sVnlNvGmTP75pPK/Ob/6FmN/xT7VJrGfa5dQN93f1DJo/HSwEQCGMLojEAg8liyTJzSzNzpeylgoA524gfnu/8T/zEvFVBcLj/j9O/y6tf5uSN5Of8ssTeJvVcvvmsquaLXcX/mo6YtX9nzMzIkDNWvogIYkklubvNY0P9vj08wOqfXM3bvClLOKdD8cQNV0bN/bI7xd1/GmQ1PDZfebO889MRVv1SdMO0Y6qruWtA7cAYMbkSDpuRILb1pUtMeq/XZXfZ9P2n+Km5JbUF8iXKQzM30/h81885B/11OJTwdM+ZzteyspY3rpHwjSwq6O+8UXh3Z04b209f6NdjtzEdIPnLePlxC6ECkaRcXuHCHASgb29rP5W3taFCcrc/Pa1ddGHh/eLr1b913lv9xfm5Mc5TP+1nNfz36sWXQs9UV9PZYa2TYIkSkhSJqUlJYoLLHv7N0+VuN2WaGh2dmtJMujo7PpHn11cVJCxayIpdR8LFjl+Kdvqri6j8BVX38YLvLi+hISLxqKy9qL92ePaoGWxpPX2i34niMx9NQYBQ7M814wAAoJSDaFDffPtUw57KOxH/5uvTohrFfbKceq4eNSuerpm3mrrP/bSHsfMfqxajG3YOt5y7Ou4CQXJqQUFmlpLNmPE/O/kJyQBA+yifa1zdO9Azqh8bu5peW1+TsliCOHvj3/gu3vgVz9+DhVj1Hb8w/nn7hmUZzyPiG39jSsfNkdnoFecHv+lwMDMqNtcUrWoQrfJLO5outI229537xlMz7l2suunOXn+fhCJh9bsDnN2Pzlv8qdZfN9HV31q/iIzGzpHzn7ZqfOy0koqKjCTJnJ0onjkhiiCYIJCV7jFMqppaezTdfWerdm9NW2QV4mw1G9svCKXz4u8kT7ef6H0tJ2GBX4WEiPhG5JT09DSVzNDfcq4VMjc9sikhmpljtMPD4IXXNkImK9l//cQVddu5fTN8c9/sZpXkhj4dscNN8lbLNuw4dcZb/sS8Azjpwd+e9RZ9b/siPeEaPv/JdS3EZdatr0lYpDYcQiylsrx+suna9SH12JVNe7emL1ySlLvH9h/9nyi+m+73DZT7TOfgsbonltNFkW035BtPT5/bN3P6rmCbMWH3+sSo+lksQ3qnS1gcrmeak5ZY23xqfMusSNM3WSSbQ7fIO/v1DievKDXG5RJnQHe8PRm3f73/cOCJVz92Zn59z8Iz2Tdx7r3rejq+fHdtcQjV4gmxKGtvZ+PZjumj1zce3qZcqH+R+JD+V+r3FF/2k2WIsX33q1O/X1e4jA4i/yWiX0HzmG3jzGRkZm7dVSVZvo0L37BXz4Y5ORp70zMmXx/KEYX5ZogU565vqJkxEtHtv9UWPxfykb3expupOdY/3cyISuHvcGF65UNqy9dy/Rtrf+ulcdmXv7BgQh62Xfvdyy120YbPf3VPsLPXDFcq5+elIZKfvq4snjLq+tr1wviFKIq4aeYe3WjSvM0WN65tXCtajn8wMr4Bv+eWLXmjddjBIxBHKoxGUet7fMOdp2uqZKKWo9ySsDcfiBRKOQioiTEM731M738sZOdC/7uVG2WS7rfo0lXwR/iu/mws9Rs7/RcJb+NPuniPf2tBdxyt+vN/ntQINn3x67vTg5ZTCcI3AECc5HXVSqdhsqMNkhcKpUNC5fTQtK4s0f9zOXXVaqhZhoMwQr4xHU06VPHm/21bF7WiuPf4Rp/9I7+UR06ccW6LNOFe/c+/nRa92qP4QlWoghdf/i1RKSD1pwwbU5b+drSh+90p5v4v+nvlcPd/XiB2fb9gIXHrbfrxa/1Q8Ny396QuoPsH5xsA4iSWl/P12rFWjVK+wFYEyRL7R6ec6/yy+xFD2dVvEddEboOIcB0kKwtGBn99fthiCDUQLQxgPHnLIiU4SOuK9BbGawP6qV7ILWcBAGC3h8fAbmJRHZPQ3zArEIc0WCN9aOTw3Tjtzj0wjxv6t055858tWWhraj734hWPpP6zdZGs/4R4Q07Nq5cmX1V9ffsCIo5Z9fTwyEdFX/eT9ij1yeapE3u2RaxyRCjfgDNxzdSnZe4/ErXFB9+0r59N3I/L2FXIdDedTz8UqXxjqdpNA0be449wAYDue/NDIsVwgX/HXjPnoHp8N4BClLy7nEm1fxJ/KDXSV4gYmh9fYh5+xt9D6Pn4xQnZV59eyJyj+fWPbuL8b327dhGP5kLyDQCAEOTWiFX6kVaUvgDhGCm228axbP8NPlJ09xhhQ8RGpkj1fO7mpH4rZBxcljHmHigPDS6PnYtZTAQo7XMMEgzNvs0Rl9yVPnG63wrZtSIAwMMv3vAN2qfx1pnP6MFbrnsmKk75jIcGJT5HMsDWaqvOWN6rRAD6xkVv8UH/nTQeeHOYs//JBUyatPrHf9KzN31z1zL856z8v8n8TUvPjya/ETw7ACU813Os/5WKNL+r8c80qS63PBKpgIuUb6i4eoiCjCXcLKGC7jxvI67bfALOtgoECLEBqBvtGw5F7PpElTkDmFlaSgAAHk0/LJ44D9+eHTu6+y/mu6NEx3HTZ94BsQHo7saSJxXLepVIYDo+xW2YZ9M1v3+RLnluAW8l3fqz922Cw9+uWJYVComeyP7pxxO/t321ODgNcr/S3Xfx3a/4CTNy877fjJ/aGumyFinfsHTjJwag6Qh/Pv9uDB4m2B4e7269PDx+LP0H5ZGzmcYYxBtmIu6LyyQejeiz2bNjR9ZX3Ws2JsVzukD/Puef6mLub6VvnvZkPCL1v3bjNXP845uCtwXf/O9nvJIXvpe4XN2ZsyEp/s/6P+r/oSzYgxBj61M/Mb5SWec3DNLH3tWea9ke4aMjlm+wLf8a7j/7bFTs8URhPga25ivSuydUaN+m/2595HTznbkNROZmJgAAIcemxq6nCu4GFQv9JuycjrO9p/p+fewdDOajw5zN8+aW9i0Va8eh4PVBqNZ/O0ulf/2FKAhilP73Ca+ojhPfLwtKBP6TTaf63i3yU2tQ0Y6/DHyyThLs+0sjcjtt1qabrsk/1RRFY3+KEAE0nkl6BgAA2wX1CxvxiCzSxFH1HyeAVXlHoulP2J/MoO/ErmP9OHX3i5iRcvdoC+/Vthca0Cg31uce3T7tyzrk/1Dq/El37jPZQfvW1/K/T3rTvv98NBzDCKd/S/Cz0aPoe0EJh/I/3zHx6c4DftJPfrhx9PyRmsjGPXK+8Xa9NUZffe2bqSvg/qHa+p/YyNAd3VkS0c/x9B8u0SDbMav+6T61H0yyN6fNuhp819+8p79h8ecbZvuA7rpyqIFhPZO2Y3mNDxfO00PkxnnhbWOvj3J2bwg6OLjjP056Mr/3heiUOEAge877W9VR9t8WBQvEZtQ/8kfVm+V+flRy8/rxwQvFkT0/cr6hkuoJ2vYK/UJu1OO1fW1/UboaQTMcmXroG3npZTOQFdUz4lLztqrBN9Hy8TdmO40o/LznHt/Yd+ypePBlIdkIxu5opFmGg5FzbsVuf/Hm/vQSKn0i6O4c97/4qTvnbz8XtdNLUPwLjF8Pfyj8btA4aMWzV7sunn3Gz8Are+SCofHgQgksi2MZXEloOGvB03+Yeq5KFGUVe/yP7/BexeCKKJaUtrX+8QMTgGDWVW+7MJR24QOk35I9+wUiKyvY7/S/f4v3LgZvQkOErY4Q3ss9UDxPlE28a5UcqAj29njyjx/aFV//bBSjrFDCs/bfaP6S+KVgdSyJyoNq3cdbc+ZeY1UXXeq4FpjAEgqWwTfmlpybGPRv9x+oKxVH08dNa+k6wAB0fvhKKbZ2nfuwzY0wpG/hAABgg/1zWed/7zvwxBJDNG3fijAAnaqMqNERw3BGL2jwl6nURzeg5GDQ9hr/8meD/IVnoxojj5TPmd6Y/oN8fhYgAAAIj1y42nQheW6ICkrbeX3qyr6IjKPLWQvzNne542QTzX0fHdyaH1EZ2eBAOd+fzYgJ173gMw9d/OC2hVGk0bNrSmbsuLJ9UubuQkhZagFK/e7MQ1nSJb4YXeCeWzhju79DcvA9k/BAbjDx5jn3qop35CtRsrLfAUr/ou7Dgd+l1wdZp1DBI52aE5sK5rZGsDm7+9ZgRHE0y2EJZ+/x4YQnhy5ONg19sqWuUBHOkS+LAYkjm77eqZ7LV25rQbG18mV9wu5ZkySfD8Bb+kwcEK5OmqrzuopR5a8LeU+1wbpdwYLe6K43usidz0c944woeEFzoe33WcGCt3gNn1xtupI917JM5K3rGW4ti6Rs+LKk0roKtVH0D5tOtE5fbvtoXd2GTM7q5abQzrEbZ5tVFiSvPLDrUz1RWnN/5Mlorrnit/p7raZOWoQHg54po3nrHLXuq+ui/2bMDV+a6Pm05JtBiviiov0d2vN75h46gmSbTxlbDsaab0jWcM7UfOS5LY2ftGk6+k6XbygvlIlWIyCbsuh6b7W0aVzMpMpd27LtF0zsXTFWwyIE3Xcb52/0W5l8zS04Z2cwU6/rzPuWlM/XrUTJGm5D76+m3yjbG4QPgh1vt7e2K+d+wq7I0Hb2J0bA+2XJN1ZtwdVbt/eXZm9pPtc8NjV9NSmzsDpXJuNF/1y5BYGxw6AfbO5Sjdpobk51fW02DxrbfIWbV+T4zqjD2aRlrPMP4jCd1HPqgxW4xl1/HuLu2b8i6z6SPt55rO8PhUFMAkRuXc/olfVztwcotaxN1VYbgYBbFt9QxubW8bMb4/klOfW3m5s69d09F99JSsvKy8yIYy+a4xgdYK/HPDTcN6geN9gxR5m/sbokiQtgPT9BbM5d8adHBdrr1vjNfuOGb1/1Ju0NNpTm41d8lU+nr0y/otzP9d069853gyTRxG39eOjiQT9ribTqqKldlxbw1SWxvF2lqO694YtPSgnEzUipG+u70TSkVatbWGJFclp+frpUyF6Rcg4AAIA9FsNo/8CwasrspTAvI319VX6qgIEAcN8ZZ2LD/VFCFXcP0Ln+znLPuRFUXxqk36imD02JT85PUI0amFufGja/ta028BOiqmKkv7lk7gafVZJ7tWMkAtfS8vhGVFSNqq6X8QEQM05csFUzfLOzT2WdnGxnxUlSszKSlYliPo9AUUx5wjTGDqt1anJieHjUYHZhIPmZOcXrspXi2eQPz+UBVL5xNZJewoena5pR5a9pjt+wixqCGYIMJzpZG3atXDqt6NFTFzrfKwoyURM2XzQ2H5rLNyKz+OrkQCgVdOZheXxD8r0X9Jf3zygbiBEfX1BvUQ32tvfrbJrJHhaLGa8UCZUpSalKPpvNXK6FjvK63Xbt8NTYpMWm0bndXhpxJfG55fk56aI5O+OpCxbRkShbqFYK5m6r0L/8Ju7p8q2rDqKJ062n3PlPL7N8x2JAec/3j310cHPgEzibsq92DSbPncLiIpG1wxa+PX6ZHGBV5TfebEm/u2Mi+XxFtUNvGOrsG9XqTLSxHwGTJxAIpKkZ8YkCEYcrIkgeObvMzn8z/7/vFMzzOShss7us1mnNqEpvt9gdXgCMCYksPiWvOEcq88+Eo6+2+XI23ic1uEY76WT/gG3nDR2jKlhEu/6TQU795pXc/TN3HXt/+GhZYMgwkbbuhurWhrlGaXaBor9DG/yozMWwzHFBafU3phq3zXXHEIRIlFG+z21Sq0fVQ2OTHp/FhAExWAwGK14okiNWPI+pEEulJEjvyu6ZFZf23j2RGMCqB5/BYJnwObUe2miy6l2U10thQATBYinSM1PSUtOk7MD6UubLGsa+2KcgRATvwAgq9t/Z6JrsCZuCGEPo1lPuvN0rK7blT17UfnqgLvADcfXb5jbD3FFmZGf3qweyw7aILFcOCOvev33p1vxEbcRg8KVZNLbrjQZdf9+oxu5xe6w+mABAgBBCAi6PT4DwbhVFQSoXWu0WnndCN3sF22xA250uC8b0rKxjsHhsNicpOSdfHieVCYIrhbjvmjO7/j4pGuLotXEL/ZZTuqefTg+oOAgA5mMDrLralRXb5JZt7wy8Ux1Yy4ZVlNXSrvIbZUnRBf2QN+Z8Iws29KkubwzqgCJAJMrEtNvlcWg1E4ZJo9HuNnhtVqCxzYpni97PALEIoHATwt57Z+4iBIAQEIjgiVhiHl8kS4hXJsl5LA57kde0nRwiNhTEzv63LFh63Ar/5dTTOsneGCTgEw9c8eYfiDh/KETIn72gvdpdGXCdyChtH+1eN9ekKcjmGYecISai0j5EIh8miGXzDeIbTqqvjy9ceAGRPB5AJo1pmjZbnVqP2eydNmHblAdZJygEcIdiCGMmAhYAEEwCADMThZilECKxkhTFsWR8kYgkELHkTnf0slm5Z2XOJY86sH6EkmX78c3Q7RSuD2Kq9lzqZVWt+DFgZM2mDwcbiwNZJC4SWdr96gcz02S6EU1oCpx7pItTzumfTC0IswZRELDWlYzfupK5hDV/pvA4byYiDTs92OOgwWPHAOAbm4m2Jc6Zt8/oc/w0BgAm+CxM8ljA4oZj2nBf7oDSCPbpqwJaNU2k+pdg7umic/KCvPD4BUfS9pXPHEvYfd526WBGwHVWeWrrTc3cUnooMb1/eMGq4X6wXbHknPooR6566du7l803lLT9uuFSfTj7dMTzs55Ts1sEp+bJmYWEWEajNI160a4wTylYNVAqKyvfL07KpxrHwer3061NRNH6ld90k9vzmm51pAboK0R2ZvuUqmjuPFZkkFMaOgRZQF8feEze8ZPDWy+NeiI/B+0uuBtycWOzN/IbMFgsFovFYhAEc/Z/kd/L19ZC5W66P1ynAJ5hGzvbb/FyDNqZ2UEsro7rRl5NRgyalLKRrblqD7wuyuLqBjxzr4iTWfYRZwi3tA+WJfj0qDLnsX+qCaFC+lJA+VuFE+f1y74P4Ll1FiKE4bSasS1opOJahGncI8z1mxz6PpcyJ4g2oLnqSd4Ui003t15qb9MEXucXCi2Dfuxipks8I0GYGQDG5hLC0hlXJGp4PAmiwLe4+mzX9W7f0t9cedA91xyZ2+6bguQ6HVb4n6lsnqAS04Mc69yrRjlFsXDRkfkF9EhP4MRnpkrwhHHudSJRRo05Qrglp1AM2p7sZESQEAW+AaOomjN0zrb8Gy0flvODZE3JfbJbANDpcJK/M2tSjxVBrCHuq1pOSWzSYuXr0dTtINqRLIkYm/bjm0JCa0KpJYU9XhjVFsqxdsATDb5BQr3CfjnC3L2oAo80mpQ7749ASwDAegPhzzePyspMC5JoYW53iatjY8MWrRM5uwO1IxSfQkz48Q1J4gmLZulRxyM//8jZ6srjOhtbosM3ZlUZeetCKKJ1hWG/cgtVbox9Qf4IQekcpMKPRd4xGy8riDNrTI0Ss2MT8UJmZXiHpgOvx6WQ1klq7hWOguGYXJpv9LWfNqp7+CL3LXUONyp8Q8lb482nNcvX9peL8fM6ad2CFc/WHJxaiif3W/zdUw5OamC4OD2kJjJjVLYJJabTk6OBNGIqeb5RP5nCUrIdmhDU9vhqyc31j46d6SotIZfvXwAA4DR8qrl5OW219SbPtRaqaGskWRyrA6+VEvkb6A0an1AZuF3wDFkZ+bGKsBLnsq1DroBeJJRi64h1bnMZMpbLsrSQITYmeTmJzimcn8JZvv8UAABQ5pYWzce7Yl0IYT505yeE27Luj0BLAAC7xsvzcwRisxUlBNld20ZocVqsIqxYGQLHuDuAb6QiTq1zz73ClLBMIaynSFiMgYhTzBz+G53REezIpZuvr/KOwdfe4smtX9UzbsKDz4WFfrXuscGM4oKob1Y1SGNWxJqRInBPBBZORmIePe13mSFnYVcoZjBEEACznu/o8I0o2soZO2GKyr0ihv7MIGdDwf2RdQoAAI5pmuVn7cU2O6ngB1qrzSocH7OIPjJZ5NMGDiWSiMDu705gsbHREKbWHqXVJ25nmu9s76oKOLr7vCu9YaUjdqIJrwOJ/bKeaIMJhIHyDZv1WBK7Kq+CBGTWBdkwCAiX//ZAKEeOUBwMcxElvhHr17PGji3PJLLMs3kt5/qY1cs/YD2GQBiY/ptRu4tUBO53fKMeQha7bRBXQRqC6GWsRK7XMvcyYixdwCNA+kVLuxbvS/AeDeIHWRQ07fPSGACwx26ze7xOm81FYwCg3C73zCchAw+dtiU3xL7Wc+SgbV7wTwynMQZWEG+WnWZlxC4hQ5DO8jgD+x6xCfBnIULYtbjD3toyNG8Uo/Ua5JYtb48cKw1ecDYQPovVSVk1lEdfvJEDMP5uuw/1uvo5kP9cMoDzTDsgWbxEzhJJQi2C477Uy6jdvBKlDlYKlMbOlPmxy6mlmEGMStiNUTROjAoRiATKE8g3gk24prxzO5grRgbtoue9uM9qv5Tn94pRmzaKvSf1Jx8rX/Q7tI/2AYOJAA+9dVNj89hp2rHhxRzA3b/uxQDQBqDcmAygffU4IA6XwxMWPrmVDeClEEkuXiMCD35slNWttkUmLGAvzZL5m3stOE4euOJ4Rj0MUezMPEwRYQ5ix2Ul8/Rmv8tcCeFbfH8qPfDKq08VzHX4RI1vjPU1JzuOFywo4LDT7hgfn5yi1x0QgKfl9yMYgMkEAggAkBS6EbJQQgbOlQEAEgmxz213Y2iV1rDB9WmzNEkRH8fhcBgL6Xjusy24/D6pGXIHCMBfbBEImEE6ELswNyLzG4oovJGbwnQGcdgTHHLe/QgyMKVz3m8Kv/r6HzZtSbg3q6KnFmR+psn4yZPBS2oDNnW09Y5MGMwOvGWjAFB8qYglYMrikbA+BQCK/20a0DHDoxIszwWAxL/a6LVNWjQ2Cy+fBaD75VlunEiUkJqSU5u0wAuOnzbJ96yJwLe5KT+LwuPGQLlYc1YuJwXY4wpQ4JwUBsoV/rshJ+Vyhe9l9HkxUO6Ay14PDZRzbuOcXsDuxduFU54/fbq5NlvCE85kfUWPb8ydBVe7L6QFaFAYEAB1+YddNi8w2WJJGReAWavQc6UsFguYAhIAcYuKAPdrts7muZClBZhyUW6XjpHNBUCSOI9u0odYHPnfPcu7c0s/eJtuQGH9mhBvjsvDoUkVqtXu7Xhz7r7T0kNZzpnm/xo5enyuc+bwW4Kmeo93hc83+obH1/5mADGoJjt1609z10ZLv896ZnrxeYAJD/vo27kFCeX1IoBo8g0l77tlfHezX508TJlHrdJsLtD6MY9cKstITUwvlAEgUVmAnompu8oAAhaw+QCQBwAAsu83TExM60wWq81KA3iHp3hJMqbfsIy/PyneFqyGVeyBvLYQ+eamsdcxJykSHBTQroBfIxcFtCuC+ELkpJz28G2i2I2xxx4gZWk3Bo9tboSIwwu0y7Z4pyNaN+iVxcuFs0cRRXGbzdr1Zlf75dw5DaXNPU0nJ6r+MRuYVYcN+YWZUgHvTpmuMLjBripz2O0m9YiKv40DoHvxYtKWLUVzy8fSLVdxWt3qFESdD27DttBezn3sorfsubn+N+1kk6jh0flDgizqZm7DwfBVMTSgOVx9l86Bi8IC8H14nK54ISCqyyu47Kp4fm5uhXakVbhnz+LtwqpjnCfq8yVMcuaGUeQbyqjrM7Ycumvh91m6G6/2jjo5FgAi68sesURARFSIEAFTJMJUjtXOSCIBjDc6h4daNm4ovnfylaNFz6qI7ECAqAOxQzQKMTgIsQVzoyvtTERw+QF8o0iE2PwIVH8e824mHE2HbFHxMQHYvADdxMsmEMuvuQ4mkLyFDoieAVadmty1NZ2P7giYaJoRZY+fGWi8tWPmxth66/jpASvNy9ggBwB+0XLujBAAyZ4tHB63YVo3MHQ+e+fudZKZQcD9J10Z+9dKFmCo44oAgPb7NgKg6cApSbDBraEj8JygOy4bn2HaJUoQh3Y2skdDMZhBBANNg799eqbWy2L3pIdfNz5WNXfdiSbfiMptQ+qT1TMBNa53XuqyEcqiqk1lUQ7wTvjr3U3XOqfa+s99/vmZV3Ec6yWrau6Tkkh3gRiEx+i3l2WwkcUYqOCzkllegy+S6MLZe/k6LsgVN12PhTYj3UafOCFQmnqn3aR/ZRGXmSYXnwXmE6bPlfs1PKqDJDrw8dSZgzOl/hyNLShp+87qZGFkpnGfi7VADzPSUjY+3nzu0tjNvKeEAAC4+5gjcV94tl7KxVltfjLkPK/OL0KbG0+67UEs+wKCpgKuhg6s/p3ogDL+x9ZvhpS5hinM4gTyjTJ7OAq/IXGasViy6OAShZtK/Ucxqp1O1ta9O/BJkQQAgFs3nLRvizIyBxPt1nf178pb4FNEikXZ268fH5s9f9z+cSdZsz2M+U+7TW0dT2Sssr5HCFnzHJIkB9FUEM8lD3nHlkE499lr30xht1qCdwAAIABJREFU52S/VheklmAgHBMUK0gOCKZoguPv7vUh3uJJPMJt8xOcozvJ4x+7Nnpyez0gBNzHd7AlEbozvR0dEx9zaxZRNxA7LXGbW8wHwBjd+tClOBhGfJivvV37PvnIqm8vEIDXL2UdsZhevStgu8FIYfm0jsjTs8xXfCkM4OZNXKgKxULpmKAkQYLaKa1b4M83nwcvsf8LTKePLt8YG+tf7/tUZpTms5Eg8g5CrAS2fQnlFrEUAACeXq3iox60YWc4L8JWcNwxdIAvBCYPrPq5xUKQRGw3OgMXPZFk2qiNOH8B6ycZXASE0NftCoVv1mlaFOSsIq+R4ijm0hDbdJgbrok9ykqM8tHzo+/cGCn7x5rl3IUsKu7hh2QYx33/oz2rzxn/eDjh1kRhUQ8/MGI65uDKkdsvngcJ+bQhSISPKK1PO1oc6WOwbja9z9cfykvTkxYyPsgZYjYbsPyNHx4nkgQ7U3AxRJlvzJqtb46oabEhtK33AkBobi3CRaEfGVbRxMbts+/htRAi0uNe3FiFUJBAwFUAg4NsxrkdRUjjaK0jsOvE6WAcj/w5Xmq2d9yhOBuoCRsnOVB/ow02JPeva6XzoMUqPwZFtMNcFAeU2CfYVh7BfSOJ70XF2wU+HP/ErMnF9O7X/77Fcfn4HRmBfXOxFjg2F7xElt3sl7IuFmB9kCBpQQayqmNVoMWrtnGTgvBNbyXkfpcpg4eTHO44R9sowKzd+rY3aXe4cbZeF8Yuj40NnCDnmywG2aFPu1H11plp5j77usv8et74/lmV29PRei98AfO2hHBKYCzB5JFOi58dVyojTLpA+cbK5Dl6dcuIXQ7Hh2jpccozgiRRTBkZKby5d6BMXkaQ5J7FEW2+IeXuy+qw5ZSv/aSVaLVb+dxd68P7MeKxQVw/a8h0E5+rpi5c23+n7qhv9JL1Ht9k+WuMb9wEhm3Kz47LU7Jdo54A4ULkZNwemoqUb4jHmwlnQ+IQRhtPqInEzCDb0ykrme63BXSPuaRB9rGLI+pGT0ZBulpzpTa8Q2uRtNBJTFmKRMxwK++6Gsdw6pbZt+DtYDMG6CPb7mya2DvX0/foy1hrdbqYcq5Na5/LLmaywDrqDuAbSknpmFCVRaoSKxJHKAzYTRaFUFqFVo8wMoOUYrJPUcIkP/uWR0vx5xemXxJR5xuRoCTMZ3duDashKDMDwKz5jDxcFQ4PnTCg5DvnzzJE7vam5K13/XWEcO6EXNYWZiWApBLNuHnuDGOlCLRDQQwicSWnzS07IjQwIVlen4EmPaOsDUudcQ0A9jYjPz+Qb1inpuVyP++paZoWrj7fIL72rL7rRGlYggoBgvne4JDgON3mE2y7e6yOu6WtaD3LcqcKNjU8cC8wGnOKY5ajHiLi5T0TfnGUZLpkaNQY6G9mb3lD066P1KDJq2tSuZjm7qoQCgBjzXVaUR4kwkWrxulKv2TZSQOhCHvJiD7f+HUnzlvO7Q7/VFi/eqqhMY9qP64nSnbf6R7XjeaK9eyukS2zQWXezg/uhQNiKX+N8Q3Fy5DGzyACUiVjeiw/QGgQBSmTgz2pERoTGBuOjN5Ov0n9VfbS36UHulF6ED2XntRBkn9y9pSBTA47ojr6fCMKn1YPdr7k3bjwmQxLAgON8dIroPX6y1d96c/fOd3De+2YjB40H0+7o8SyapLmuB056XNuT9M45ADEFUNcEtM6tGmuUhWXyZpS+QJ5pahtG7u6OdIFVfL0Tc20+4vVIbiY3Rd13LIgSSLOITsr3W85pkZNzPQQFmh/rECQhODg9K/Gj6mePJAaaeU/2mTo8eABqSRusRntHv3kT21O+fOPzU4yrG7OEL+qozM23ukFMnlu0Mg9SWc09Xg8fTyxbJUJx87gO/r8zmgRZ3ONfUGO3RNsekvfpI40hhBJt1pcAmEo4nH8siN7YxAWWXrsCVl+S5Zl1BWXGXbW/wrwDcmPdL3jaB5r2l2bzotoRLFlwncIyIlFsi6xa/TqqUvjPsa2p+4ot9haWIOopuR9hfd+FfyILesEdchHj6PVPoaGkSE29vuxi5GjNLRNBvKNXFdxsrs5L+LRCnXfT1/uI4vLAh+Dxwc8ylw/vmlUlCJsc8hK8A1Q+p5rQ76x966WNdRlCiOISyUUouJdAIiz0G9p2/C1s9cnPQCKQ3dPGkHZ2QJ4ci9PvBTHUYKoaBcCFGrq/oqBTJcNqTV+Hvvc3J5uVUHgNEvfdmH60p4VL02sO60Tbgmi5lKd42SB/zI7OUJnJIbdgysSdMjetGXMTRKjE80fVG0uSQo8b24JoMXDDlya7gtN3XqKyfQyN224O+mQEABCiUpZ4vaxA4rPuKkbrJg7q5T5J41t2wPbx9ma09XUuvzjgBaH78YFnFkXRA2y39YK1vmJXY9aR2aEz/8VeQGUsvfSkKjW2DWtbTlevKU6NYEVLT8t7Z0ea73YrbJjfkVcqzZpXwRnqK8diPI4xgE/jxavTDbe+EIg31Depr7hTypXWMAZ/jLFqS0O7FE80eFKKvZT1uzDDkFG2NuFleEbsGrXj7uyH+k4ddvYM3gxLbugvCiew172eeJut67rVvfAqMHLkBbvrDxzhbVhS6gVctYkBPlcQ7d1ri+GUZ4x3t2vCBxyySNnhy7sXVkBR187Tad/JgiJfL29qCjHT2aYuxzpWbE+v34hoJR910euPfHtR2+dbRnV69rZCcmpmVmZyjgJO6zM0zvA4DEap4eHhlXj024fiNPWNaxL7Wh2JO9Li37jYwhGbobu9qCf7y+pomWqcUPgQDJqd788dHJlBZzhrSnW3vVBliLrDS2vws/a6xsagLRAQ+GSWKH5wty0aaL7aGlR7vbhttbbA9aREQaXJ0hNyUxMS1fwSESgkHwJGGMa+5zTI6MTI+pxq91JIVKUU1i1LlPONB5tR7X3y8mTCwClFbSO91XNHTZhzV+mmnVBMqnke872Xajft4IvTJ1tpDIPBCmAjKdvuFP9nQ6enkl2kTx80bFS8jl1//XBY7t3sRKVVU+Oj9y8PGSxmmGQZDDjlRJpujJRGB/HZBMMNnm3/xBB3FNkPD4PRbu9Jp1tQjtkNE7p3RSFgSUQZ9RVZiSJmQjom8etafvva+0NACRFfGv7o3OT1snSLM3VdmXgezGqtw71/qVsBbN81K+Mkburg9zf29yNc4v8qGLtsUjLgxxkuBRWim+sLdvHVG9XywCx2fGlO1+Y7u/snRrTOd3WYSDYHDaDzRYkcsVKXioXAzeOBEAqQ1scBtrgAvCM2qZNrkmLy0O5PC4aACFOSqIytzhfIZsNuTJ8NPj/2jvv+KauNO+fc9UlS+5yt3Hv4IrpHYMJYHoLZDJMCEP6TpLdeWdmd2f389mZnZkMbGZZQkKGAIPpzQSMDTYdbDDGgHsvki2rWrbVy73vH4YE2Ve2LEtXhtzvn9K1dXX0u895zjlPYcyf90p7bwAAeoJvy0OJxe8Wnv5YencWzjLbd2VZ9d2SLU4rrKq+WGqO34ynof77Yu5Ui3AotK0K9Y+2QzxO8z8DVt9pLFqxnAIAAIibW2iqSSttFnQLhEKJwazuR1EAEQgpkAoBYPnQAQAy0z0qAGaJBgDMhKIYhmIAUhEqm+4TGhwUFBLuz6b+kOVielRsiM0NesXNG0ASwtuaqi16RnBmX2q7smHy8GupWcvbu08mOiuvG32Y189Zm4kzolhNuT46y6I6i6m2npIYbsenOE1v1Iy5gq6jqS/8eUilMj2jMfNAf19fl1DU3dWvM5h0JqBVAQAGnjfcFA1eCgCksiCVSWXQuSEBAUGhXJ67G91yIETHG5gzXrmU+uHwM8vk97JftlkwM76j6Ub8cD8Nei0vvfPwVDBO+2dHIDhUDbK24kVZ6EobaGnRFmsD5WOlWwrfjhtx3i/mu6mi8uaNLS+PG4SIlxcAZqPJaDIqFOoeDRApIDAqdBAA2KELYwCAMbxpAPACEbYfx9uLRqNSqTj3aLx/3RyX+8p0ArQOZ+aJtoruqJdegf4r7/ddWoOTT4tM2dLRdjHuTac0ClSduqoPex83gkRQog5aZNmWTPBYn5RqjzPjPL3BjKUNiryp8cPfGSw6EYABFAAUA8Cs1EEAkKOyLT4AYHRPKgAQAQCxHg+HdeX1sJdMf5WK31sBiUlobyy3qApKnx5ZUXNnA84vw85+drgzL3amE9aohtvHJZzNc/FGVFf6FManW3ym8VkHEhtrz/A7cUZyW1Fc/igfr50nAAAACH9IDmMDAADqi+LFMeOivVxqjs919XG7Q/CfWiy9tezluEUYkV0tuzQbr2BA4OaGm48OeCU6/EEzP91fTZn99vBAAQCAMF/hPi/M4tFX3BV7pNs1/E4sew2TN7grT1baXGARw2y9FKs7KvdYYU/O4cSDPT3EWN5skarIXhplunUXp2QzoKa+Fa0v+IfI0ZmNWNs3N83xH0TgzSfGm6Ug0bJUrbmpEpuUadchtDN/M9aKLKQhf3ivYCvYPogDF55RM5YT0+/Y2SCJybD1rkV5ZpiazZJ+L8K7mrlwY0jfqTy5YwWHiQ5fUgVtn4k72QkLpLzsOAslasqa6JOj7JKOU21ExOYA3fe38B7UcYGWndby1+EcK7+SeM316C/qsXiJsyHKfP8WTs8NgPm9udJd+HW+yqGC6z15ROyxbSNuMoLxxn1z0mKLt7DuYo3PNPu8GWfqDVIWLaG3H2txtPUXfdNKm7/41T7J+hH6vFis6pFF9jxMWM7oOd2EczGEYW/nsDv35uOUibMXTH7qWyFvzTbcgjhYY57MPTve4tk2VVQhyTPs8/yd6wP5b04y3Dvf79h/qjt3wxS5+VU/yfoBGD6HIb5m6XVwchOMDwpwS5JTJu+aQ6vafV7jMMH1ntxbz8reGY0rBXV+OUjOsXRd5EVSt+lh9o2/c/UGM9d7ys8/Hk9xxmFgT471uq2y8/GaiLgtDDGWPrZYKyEJa93l+TW46yfq1A9nUZ59cX6snSCtoTz3v3X0BR+k4K55sYrTKq81CRYqQR+VmiLm2uk9O1lv3DWzkNpT3Y78n5LDT2H6homWKz8OYOoiautFmcVrnNxZSPU5Me719Hn/Mp9a86fDYkdYOEx4cHcja+nnOBFQAAAgPlZHmbvcciUqO9vOnBNv5/Ti5D0FOGljqPbatZG7Fo4J45UrOv7PEl6X2RQAADyXhWjuVlsYMxi5lj9QcBt3qQUZcz6dSa376zeisXXsxAMV7NvTwFr4SRZ+trCxqEgfsMbynBSrv24Oz8bdqbMBZ+9hURetY3cef+Kwzs9Y9SEhZd0br1LfyVFBMudSO65ZtvCmz1tKbznZiCsoSJv7m+WMjv/7Y9V4HRXjs//8usttza9n4I8nVndIwFi5wNJ16b8kYszOsnfHmfJ7O//QRiDbq7pDCiaPnvyMVagzRo+HF+29bEj9Nytd4V5RIAu7I9Mkhb387EM396dCCTsRt+AVpISmDLTJ69o9A0csX6a4nxY0wttAU/CHApXvls+TrTjD4r35pin/atkmCr29Rxb63hR7fwBn6w1AD+SxTOxrvVHlC2zSm+b4Qbnv505OUyI88R7h1Db1uaVaRPwgXtonCmlQNL5fhXhNhgJZex3wdxthhhpZb2bhkb9WmMJ+uQun+hYAAADtmW9k3h8tsxzs3i/uIkvetiPSchCn6w1Qg0W1Cnls6Ggzty16Q8v21NE3fejURlmGjqo+Lg0aW7Qjt+pxIJgbo1ysSoy0GCJGYHujvD/RWgNOj9TQdpHoUZeHr3XfYkS9qe/94WAXkv67rTjJOYN3df8vdbRNHw6JLy/cp4z4td3mjQC9QZZ/W3uPforXKPdog96wjt03THN/59TGCaZnDxoKmBHIs99rUonac4GIZ+ezfkaahdWAPJ9GgQikcHG/LYSs+MS+rt66Ryo+z5ozNYLe9PUH/lKq5a3+jwXW4oWxtr9eR2f9dkjTxZ4/P6RtedP+GGPn6w1AL261qIeWNEptBxv0pjiY1x/6m1lOlYHkTpjb9z4ZIP/c7HTi4p2Y1AfigQjLAG3El/FELuAkWDkXh0hoilu3QlTRaPbk4N+pVb0ZOs7tOdONRH30qzir31Fy4Lgq/DdzLN83nPtaE/F5nP0PPBGPMHPu2m7Z8dg144281xbmSd23LnTq2hQV6v1LBwKpmip6LIFbytSMbGH76aQkix+Sk115RHxk0gprggMxHyfn3ZVdLpu3clbAGEbF0Hn7wn0F4M9/e671GA/NxaMK7vohp4ZYw3dKZg5ezLmtEGDfAGD5i9ukooiQEe3FqPbNeGv3M+aqD5wUTv0CWpDhBGtLQOdJ9ltjqwk7Pli8aoHEwzKFHXICJa093aGhVsYNQkbs1EBNn7LmXo2OwqYO85Dx7Jt5oO7i3w4903vOfP+jFOtn0PrCPfWsdR8N8R6VX142pv/beH4BQvQGPb1bBN29UQEjrRlG05v56f/cMs/4bIpzJznI8XicP30J5/G5jGVEVhmBnpqnMm2cxZ4IgD4+TQKRLN7P2rhBxDNpiq9R2dtY9limY9EQSyUM0xuqE987+/WpKr3H9G3vLRqh15qh7ItysOCzZMsPRku+6vb+bNF47D4xcwYlc7u05rqPV6T9YkGbD1zTx+3McPoNq2sNye6mJmXc2ItjjAfusocXavJjQl/WAKRN26GovbX3X0ZwmNiZcYsLS+plNx+GTc2YHOTpZq1RE2ZW9XZVlD8Saii+CYuWxo3k3Zif/d89c8p7qUN0LjjeRske3147QT4KO0e5u/Uc51dh9m5tYcKvz6gmvbvU+TZHK3IPpSkbuTHEBjzByHXPGr9P3Gq518N6o/dvrRc8Pwm1PmyQl5WYc+/OU3Fdw/eBkYnxIQHeDCoFgQAACBEAAMBQs0kvEbc21bR09WOsyMnzZ0aP2DgBrd9fYIx/f94QZfWdLDZE/2J8SUpE+cTuq7oPdZ/k77Cz6xgmOXZaEbB5LQHH9DQPBjDWPgkjOuCJPiunp/tUwgwLmUP3tcr9opOcnSP5TJCbmZRd8eihsLe6ttgzKCzIz9/Xl4sw6VKNXKozmPrlPWKxUNAxoMUYfuFZ6alhI1dIQ1u+zteE/2LlkBAQ072Tct5avPIiY4CwNZj/ds1h6TfgXTtqTgAApEf2d/HWbSciv5m7sLeVcqVx/TjaudgF9NtQf708LyDa8jv6b5UfFX9r3jWi/BFOYmxOc3Vlg0DcLSynsHg8Hovqy+t9puH3Koza/v4+nRmlsMND4zKSw0fr+oG27Dum4L+5YWgAb/2hGjh/6zi9DMK2NEHYduUZ4T7sHf+xT6mY+MhXndyV28MJyJDBtKFbe9UqTgLh6RGUKZvaGi9H7rTY0Icg5JfGE5KDYOfIAY6QxvdNWyVqaGju7O7XSbrNAFApmKkZmswAUKhML4+A0LjIeH/eqP2isKav8hR+b20fOm9KT9w0xG+PGecTT9weExL7kTG/5xt07EYKE3/3Tafbio8SiUjI6j9ZvW6mUBqbTnxAJzu7/oD4WOwyS7eJEv0BdkJ6yPTL0Z42yGT6xuvlYpmku0csN2nVmEHuwULYLLo3PyDYx4fvQ6eOPvTmlr0n5P7b3h16/qi+ckbu/RZ+Ss0YIHJPM+lTeKbrgHJn9NhWqVjb3w+J2Cs+SSVku1/5UIiZSgfW2htQOB78NrVcqj0QlmL50UjMhyBPfFizI3nUAYB0OjcMM+m0Oj2qlIPuc4vioZc7hcFksm3sL2x+su98r89bO4eeGZrLv2th5Ky3N+ztBwjZf3sO4hujapfVSwP88AbO2v6buXLPUan76s9wqmY7A4MkJKapYfaycXSPsBvoxW3o7oKJQypeQ684fZusqcff35YhgJDCcPPw9gmMjOLWrZwTEezr7c5hILZ9HcO9P19UB/1s11DrhjXsLTbP+DRp3I88kXoDiHc8rUne2s4LwfEirOhNe+NPl/v52z5IJMgSM4LZCvbUTFfIDQCKv6le1sFIGVI5HXokMFolHW3ewWPo1gkRRFmWEWyj0AZRF/7XbX3krp8PC+aR7DvdH/XhInsbavwIoXoD0DuR3S7uqKEEc4atGnD1hklP/rFMH/LeTvuya+0A4fkHh/nRXSI3AOgh6hp5e1DcUJV4xPNaJIJGZihrLKstxf3UEeMth11/eHelOfbTrcOqHQ6c3t/jvXMTTunLsUKw3qBbUrCoS/S0i+s/dDsVT2+6sj1/b4YZn2+zFqPlBCBtrD1/Hfnh3EmCFnlLaITlxAUhJz5YIOqqUk4a4RBqGKPF91qC1u/e306Z8a+5wzY5tfl7Wtw2vTf2bgvDIVZvAEBGVJy2U1b31OTrZjmmw/Vm7Dy+u7iXvfy3S/BjwBzOBOhYCT34dd2SzsSgIXYMMsMj+zpktV1+fBs9fzBGvelu/fFcL/ON380ddtBluvPnJ5R5v7KjOvRwiNYbgLSQVHaHvPtRhcGX8/I3GKo3c1f+f59sNoW88+spBGXH9HV6jL0jpqOBgfyaHpEw2W/oxEkPTQIdssYqJIxt65w6Br1h0u/+UKbz3fHb5GHHeGjF7+9j0/7ZSsbgGCFcbwAgnikx2h5l250nBiYP+WHsLPSGGjou/uVgrcZt8W+2OcKM20TD//gGubzmEqSEuVX2diiGx0NT+GleXZKucqmPj40mzma9YboHXxzohFN+vXP4YGMNvy9Go//fEseUSnaB3gBkRmf4Dgz0t959IABMCAedY+yxOpMDAACYWSOrOPbV0Sqt25Qd/5TltPrIw6C3X+ePYbZyFtRIRnVfuy5umKcG3RLClWJF1VN9IMem58JWvRnbD39RrOYt+fdlOJkhzX++ZIj61biDZZ/jCr0BgHinTvNRDQx0PLx5u0Ot0RshAOCROp1t1imlnY/O7T9U2KBzS/z5Z2+MuWH1OGBFtRVhgS7v4gYYiQPV/U2aePdh8yYjIpkqUXQ+6vTwsuU2bdMbJru5+0g7jNz+aSrOlNn2h1Mq//feclSgBNQ7unqRjWDa+pJ7T2QmhMHyjfTne3pyi/sXIb19PeJWkU6PUr2SZi1OYkFC7w5TFl0Pmp7oSacMjVwkFqzny2973Td/jLMJhCpuHLsvR2KW5k4ZKRNwkOYvfpE52jWosvJ8UQfKn/HzOTj5aGjjn05r/H/5rsP2B2C+o+qejB294PFDGQAYBgBA3Fgas5tehWKDbex90tLC2NBhafk231LtHWVEXJRPGm6pR8LAhF8ekXtu24UjOMzYfPpck54+OTcncjTFja43VFVX8H2dgRW3aWU4jnFDG/9ySu3/i12Oc6Gp1QOuG1kYE2AAmEYs79WYNUpIMUEmxc3dK4ANAYOLthKuNgDNUnWP0az2mxTuUr3B4I+RI9I8w67YYSfskB7/8bRTtwXlTQUr5kdbTQW0CXNfQ8nFhn5mzMK1GXg5vcba3edV/jtwFhF2A3GLjBGKrkfSOyAs7GWyU7g+7h4ePoGuak+K6asKpeGTJ/lxmK4u94V1fv2dhLdsVybestAsvXeiVGrixs9elOQzwub0iPYNM8ieFpXWaagB81bN8MbTrf7O/5Zo/N7b4cjNdmIdJHwwDGgL8zMU+rf5LBoEEJqNkOaKjYmBkhs+2XFuLvXdXoC2//1EJyv73Zl4abuYqfVW4SOJkR09beaUYI61nbER9GZUtz2597DJQPPPWjovGPcfqG797bZh0tvv4PTysh9XP8cAAACh9s7VGWsv9HgNhrv0t0go/HDCqin8gLbw0tylhO32jQKc9J7Ptw1X5O8vxgkdgNSYsHl3rz4UVdZfSsyYluw1YjrCMFC1rLq0rK5Xz4zMyJkZhL+z1nt5b6Uh4ZO1js2KnAh6A9rrl1JXe6HYoK2Vn1bMN13zWuFN9C/fXrR41YQpeg5h4JucA1X3VdJcnEcAAmZ02NzHN8o6ekR3QxPSUicFsCk2TQmoWSVse/KkrlMPeZOnZicH4c/HqPD0oXpK5idDkxjGy0TQm/7GxeQ1vs/VBoxXzn6UYRLnMVYT7caxchZNGLkBAKDvBq99ZU/2dP0sAm+Gh4yI0Fl1Dx5UiRuabvCjY6Piw3mMkd0Q1GBQNrY11LRKBzBmWNr0zBhvK9vbpvqD5wTMWR/Pd/RvMAH0pr95NmU9/4f2C91nTWk0Wry2aBrRa8SQwIlVxhC65/AOFLcc6N6RjntjkBbAz8ituV/RKlM0FfNCQyISJ/G5bDb9xSnw83xAAABA9VqVStLS2NTW06fHmIGR6VmTg1jW1Kl58G2B0mPlDtzVyrhwvd4Mxaez1r3UyKi5YjEbAJ5fbUe49T9yCpSJ1o8Lcub7h53qPtG8c5kH/rNH4cVHL5A8fVTTJFZIKxg8rm+wH5/vy/FmQ647ojZpNaZ+FapS9iskMqFQqlLpAcKIik9IS/K3mhgNMOnZw0+N4Zt/FuF4dbhcb8aS0+nrX3LVUJGSiQCA0ORidKL9/sRDTfqAn1dfKm7eGGllEQpp3l6RS7uam2qrxP0KSQuFQqF7sb3YgOdO6X3Wyzf1qVBV30CfwWxCAZXtHZIUHRfjxxnhmNjQcOy4kJKyfQ3fCXsErtabseTQ7M0vZzrqhc+3efViPXFH9RMVCMJ2xO+73fRl5VsL3K1JBNLp7nGGPpmwqkEg7e1X6/owAAGEAMMaAcAwgAEIqVw3b35IfNwkLy5jRB0piw7d7+cuenemU6o6ulhvhmuHFmy2CFNGX7SSwozEny9MPCDwyAnef1FyuebNjVHWI9AghcXyS5hvGujulkhFvb1yVCsxQ7OaSQN0bxbVx93b198nOJj9vNCDdYyNeXldWMiG7THOUYZr9WYszJu/5TVqpeAMIHXK71IOVrX+rXLrghFrhELIYLD9UjBMq1GrgaHfjHSdyok30zh0xI3FYUI4ejAp1nvt2zI1I+2dHGcF8LtUb8bj1vIoAAAIPklEQVTr3y3YaqX0MDYBDj4mCEjw9tSvCuSXK9dsTRolQwoCCAD3xUzYVjotbSwfpH165IwM4+e+O8VptXpcqTd98bElW4bKjf7iyaJ5TazdCRcC6VlhWUeqhV+Vrl0RPob4PDNqHv2iF2C6ttNn68zs5B0rnbjT7kK9GYpPzdg0zLrRwthqFACTlhf0unQAHD8Q+G+ffKqgs7zxVu7ccKekj+mbS/IfqGkRuRtSnfmcu05vupunMzfjZDRGJQr0AKglUZMIv6UJDGTMjM46US4rfDRrw3S+w2OQtT33jz5QQr9ZGxc6t4qsy/SmKbkwbT3edwtdfabBx9xKWfLadJx0CBD4bcgsuvhEfrFi6vzZkUwHDg6mabp1s7wH+KSvzQ5y8p6nq/SmuV6Qtg63RTB9bf8drqFq8Upy920ItJigGYWFNYKuO1PfyArjOkgZpoGW+1cqFICbkrMs2uHnV0Nxkd401y+mrPfBfy/gnSo5Nj3BzkqYrzWctOi5V241Si+XJ86aE813gCNnFNXfvVUnB74xC95IJOAJd43e1NcLMlZbkRsEfr46wHR5IuiEBPJmJK66UfxUVvLgfMasxAieDRXdrIKZ+luf3HncoabxU5fMiSSkiIFL9KYpLpq8ysf610PIqdQqiGdG7II7d592VTVcjUnNiA/wtC9jFjMqu2ofV9bLTcyotFmjFJB2HK7Qm+7W5ZQ15HRpL5CXHv9G+Y3H7aKuMv+ItMzIQPexRt9jBmVny6PHrVIt4h6ZOS81cOQC0g7EBXrTlFzOXGVlMiWxBciJCVvQ8ORhpbi9rdQnPCouPorHtVUyqFYlb2uore/o1UL38MnT06I9CNxYJ15vhuv5aasJjxV/3WAE8DOXN1Y+bBQKBKU8v5CwuEh/by8WMtJxPIqharlc3NDa0i3uM0N2fGxWSmTgyNEijoZwvemvn81cR1q38UPhRIXP2FD/pKpO1CetZbq5BQQHB/r78b2YVAqFBiACAEKhAgBQDDOZzUa9Qizu6hEIxQNqHcpwC01MSon2HT1H38EQrTfD1RPz1hDZCO11huLhET5fJmytfNqtUEjaKRSau4c7l8vn+vhCWggTCCS1AFX3GE0SZa9U3d+nVJrMZozmGRaSlBIV4sl0QTwrwXozXj01aw3uNi+JXVDdOKEZS6Siluau9m6FvqcbA5BBpzMhwqEAvfQZGzVpUVRvMOgxABGEFhAUHBoVHuzlQXVNqTti9Wa88t3SjQ6oAkvyIxAymX7xs40DIolY2C4U9RsMpn4lMGIAACAHACBUQGGwPRjcgPDgUD8+n0t3STL5IITqzVDwXe56pzaf/6lCoTC5AQA1aLQ6nVTWKzTArl4A1c2hnhji5UthBXIDvJlMNpMB7W2Y5yCI1Jvx8rEVGyZShufrBQQUFgsAgKKoEYNGM0Ba925NQyGFAiEVQSbGgQ2BejMW/WPJJlJuTgdBwPMNNQ+mx7gbwjgY4vSmu3pixVpSbkSCYRMu5YgwvekKz89bYyVXgeQnA1Gzurbk3Mx1ZCbWTx6C7Jv66qV5qyeaL0FCPMTYN3VxYVYuKTcSYvSmLv4+fS15qkBCzHyqKr4yPZc8MyUBhOhNffVqRi4ZgEQCABHzqbbkcvrqEYLHSX5KON2+aa5dmklOpiTPcbbe9IWXZqwaS49YktcaJ8+nhuILWatHrCJF8pPCufZNf+ls9ioy3o3kB5xq3wyXz8xdTcqN5EecqTdDwdGFm8kzU5KXcOJ8qr90NHc9Z/TrSH5COE9vustnVqwj5UZigdPmU33hiYVk8DjJEJxl3zRFF3LIaF6SoThJb5qCgjlryaUCyVCcozd1SUHWalJuJMNwit4GCq/OIcMrSXBwxnpBde165kpSbiQ4OEFvqquF6Q7uQk3yuuD4+VR1tShrFRk8ToKLw/WmKriZRca7kVjB0fOptrgwlQweJ7GGg+2bpvDKvOWkdSOxhmPtm/7yxekrSLmRWMWh9q2/qGDuKnIjhMQ6jtSbaH/rR6kublBOMrFxoDzE+wTvp7mgBDHJK4Tj9Nazv+2TVFJuJCPiKL1hPfvbPp0yMWp2kkxcHKQQrOdr4fuk3EhGw0ES6TnY8U4GKTeS0XDIfIoJ/iHcRS4VSEbHEXrDuo50v5lGboSQjI4DVIIJjos2ZZFyI7GB8csE6zjWtS2d5oB7IXn9sd3HR7UaFJg1g32ZAABAJUcBAKjghGQDKTcS27DZvhkEPSr/4C4pNcp/cGGAVbRsZgFMeEy4MYuUG4lt2Ko3/WNRaNPJSL++kpVbB9NKsbaqDQDrON6zOZPAftQkrzY2zqdYa0X8lNiqsmim0vw8mBJDMYh1HpFuIOVGYjM22jdzm28URd6XlezuN4X1/CUNDen+h2jrVHIyJbEZG/UGExk0Q60hlZucSHluEo1yvviI7OeppNxIbMfG+ZQS6gcGmnmpVArlRW7CgJT2d/GbaRRsxD8kIXkZm9cLKFtYHxYM+7oCngfwNraLOaupN1F2jA95cEpiIzbqTXEE2yZo28g211SvGHzFcKOUk3EB0CA9ZyHDefdH8nphm97QqgPhCyrofKT5QfRgOgxWmW9Omh4d6k0xckkPjsRWbNMb5KeFPPD7uPOKeNKcQWMm+Nb021VBTHImJRkTNuot5q89zCCKoDc9cHCzbeC46ssZDp5GISTTpF97bPTfKHwfCEEU9kISyPQ34hwcEQLnaLmO/Y8kEw+bRYMAAH40QOzZjrdFcYC0b45lAm5V2WmknKAMUmwOB6FPOP96wt0QiePwnD7h6qLBCWhzSRwEqmFOtLDr/w8rYdU7ywfEuQAAAABJRU5ErkJggg==)" ] }, { "cell_type": "markdown", "metadata": { "id": "4lB2rj83ZbEW" }, "source": [ "В пространстве 100+ признаков визуализация невозможна, но принципы остаются теми же. Профессионалы мысленно представляют эти ландшафты при настройке моделей!\n", "\n", "Давайте запустим визуализацию и понаблюдаем за \"бегом\" градиента по линиям уровня:" ] }, { "cell_type": "markdown", "metadata": { "id": "p52KUm-k2ioi" }, "source": [ "Реализуем несколько функции для удобства." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "NtZT_yoxEz-H" }, "outputs": [], "source": [ "def generate_polynomial_data(degree, n_samples=100, noise=0.1):\n", " \"\"\"Генерирует синтетические данные для полиномиальной регрессии заданной\n", " степени.\n", "\n", " Возвращает X (без добавления единиц) и y с шумом.\n", " \"\"\"\n", " np.random.seed(0)\n", " X = np.linspace(-1, 1, n_samples)\n", " coef = np.random.randn(degree + 1)\n", " y = np.polyval(coef[::-1], X)\n", " y += np.random.normal(0, noise, y.shape)\n", " return X.reshape(-1, 1), y" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "id": "0ohVXmy5HpMO" }, "outputs": [], "source": [ "def create_mse_loss(X, y):\n", " \"\"\"Создает функцию потерь MSE для данных X и y.\"\"\"\n", " X_b = np.c_[np.ones((len(X), 1)), X] # Добавляем столбец единиц\n", "\n", " def mse_loss(theta):\n", " return np.mean((X_b.dot(theta) - y) ** 2)\n", "\n", " return mse_loss\n", "\n", "\n", "def create_mse_gradient(X, y):\n", " \"\"\"Создает функцию градиента MSE для данных X и y.\"\"\"\n", " X_b = np.c_[np.ones((len(X), 1)), X] # Добавляем столбец единиц\n", " n = len(y)\n", "\n", " def mse_grad(theta):\n", " return (2 / n) * X_b.T.dot(X_b.dot(theta) - y)\n", "\n", " return mse_grad" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "id": "GzXYQcFLHtMA" }, "outputs": [], "source": [ "def gradient_descent(grad_func, start_theta, lr=0.1, n_iters=100):\n", " \"\"\"Выполняет градиентный спуск и возвращает историю значений параметров.\"\"\"\n", " theta = start_theta.copy()\n", " history = [theta.copy()]\n", " for _ in range(n_iters):\n", " grad = grad_func(theta)\n", " theta -= lr * grad\n", " history.append(theta.copy())\n", " return np.array(history)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "id": "juyCjaOfHqP5" }, "outputs": [], "source": [ "def visualize_gradient_descent(\n", " loss_func, grad_func, start_theta=None, lr=0.1, bounds=[[-2, 2], [-2, 2]], n_iters=100\n", "):\n", " \"\"\"Визуализирует путь градиентного спуска на графике линий уровня.\n", "\n", " Параметры:\n", " loss_func: функция потерь, принимающая theta\n", " grad_func: функция градиента, принимающая theta\n", " start_theta: начальная точка (None для случайной)\n", " lr: скорость обучения\n", " bounds: границы для осей [theta0_range, theta1_range]\n", " n_iters: количество итераций\n", " \"\"\"\n", " # Генерация начальной точки\n", " if start_theta is None:\n", " start_theta = np.array(\n", " [\n", " np.random.uniform(bounds[0][0], bounds[0][1]),\n", " np.random.uniform(bounds[1][0], bounds[1][1]),\n", " ]\n", " )\n", "\n", " # Выполнение градиентного спуска\n", " history = gradient_descent(grad_func, start_theta, lr, n_iters)\n", "\n", " # Создание сетки для линий уровня\n", " theta0 = np.linspace(*bounds[0], 100)\n", " theta1 = np.linspace(*bounds[1], 100)\n", " Theta0, Theta1 = np.meshgrid(theta0, theta1)\n", " losses = np.array(\n", " [loss_func([t0, t1]) for t0, t1 in zip(Theta0.ravel(), Theta1.ravel())]\n", " )\n", " losses = losses.reshape(Theta0.shape)\n", "\n", " # Построение графика\n", " plt.figure(figsize=(10, 6))\n", " contour = plt.contour(Theta0, Theta1, losses, levels=20, alpha=0.5)\n", " plt.plot(\n", " history[:, 0],\n", " history[:, 1],\n", " \"r.-\",\n", " markersize=10,\n", " linewidth=1,\n", " label=\"Путь спуска\",\n", " )\n", " plt.scatter(history[-1, 0], history[-1, 1], c=\"green\", s=100, label=\"Финиш\")\n", " plt.scatter(start_theta[0], start_theta[1], c=\"blue\", s=100, label=\"Старт\")\n", " plt.xlabel(\"W1\")\n", " plt.ylabel(\"W2\")\n", " plt.title(f\"Градиентный спуск (lr={lr})\")\n", " plt.legend()\n", " plt.colorbar(contour, label=\"Значение функции потерь\")\n", " plt.grid(True)\n", " plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "s0QijpMZ6diT" }, "source": [ "Обратим внимание, что если не передавать параметр `start_theta` в функцию `visualize_gradient_descent`, то начальные параметры у нас инициализируются случайным образом." ] }, { "cell_type": "markdown", "metadata": { "id": "PT-_CW5e3YNi" }, "source": [ "Напишем пример функции, по которой будем запускать градиентный спуск из точки (-2, -7.5) и понаблюдаем." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "id": "bFxRy48_GDv7" }, "outputs": [], "source": [ "def booth_loss(theta):\n", " \"\"\"Формула функции:\n", "\n", " f(x, y) = (x + 2y - 7)² + (2x + y - 5)²\n", " \"\"\"\n", " return (theta[0] + 2 * theta[1] - 7) ** 2 + (2 * theta[0] + theta[1] - 5) ** 2\n", "\n", "\n", "def booth_grad(theta):\n", " \"\"\"\n", " df/dx = 2x + 10·cos(5x)\n", " df/dy = 2y - 10·sin(5y)\n", " \"\"\"\n", " dx = 2 * (theta[0] + 2 * theta[1] - 7) + 4 * (2 * theta[0] + theta[1] - 5)\n", " dy = 4 * (theta[0] + 2 * theta[1] - 7) + 2 * (2 * theta[0] + theta[1] - 5)\n", " return np.array([dx, dy])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "eYZxTmde3rIC" }, "outputs": [], "source": [ "visualize_gradient_descent(\n", " booth_loss, booth_grad, lr=0.01, start_theta=[-2, -7.5], bounds=[[-10, 10], [-10, 10]]\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "NXsw4aJG3tDE" }, "source": [ "Как видим, все хорошо. Градиент без проблем попал в минимум. Попробуем поменять параметр `lr` и как это скажется на траектории." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "i18GBrw04vsk" }, "outputs": [], "source": [ "visualize_gradient_descent(\n", " booth_loss, booth_grad, lr=0.1, start_theta=[-2, -7.5], bounds=[[-10, 10], [-10, 10]]\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "fKzGeQo85MYN" }, "source": [ "❓ **Вопрос** ❓\n", "> Как вы объясните такую траекторию градиентного спуска?\n", "\\\n", "Чем она отличается от прошлой траектории ?" ] }, { "cell_type": "markdown", "metadata": { "id": "MLK9s3MD51ue" }, "source": [ "**Ответ**" ] }, { "cell_type": "markdown", "metadata": { "id": "D5z4gk7n580N" }, "source": [ "Напишем уже пример функции посложнее, у которой будут локальные минимумы и глобальный в точке (0,0). Также запустим градиентный спуск и понаблюдаем.\n", "\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "id": "cRT68XYNGuJg" }, "outputs": [], "source": [ "def complex_oscillating_loss(theta):\n", " \"\"\"Глобальный минимум в (0,0) с множеством локальных минимумов.\n", "\n", " Формула: f(x,y) = x² + y² + 2*sin(5x) + 2*cos(5y)\n", " \"\"\"\n", " x, y = theta\n", " return x**2 + y**2 + 2 * np.sin(5 * x) + 2 * np.cos(5 * y)\n", "\n", "\n", "def complex_oscillating_grad(theta):\n", " x, y = theta\n", " dx = 2 * x + 10 * np.cos(5 * x)\n", " dy = 2 * y - 10 * np.sin(5 * y)\n", " return np.array([dx, dy])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "HEekFSOF7Ate" }, "outputs": [], "source": [ "visualize_gradient_descent(\n", " loss_func=complex_oscillating_loss,\n", " grad_func=complex_oscillating_grad,\n", " lr=0.05,\n", " bounds=[[-3, 3], [-3, 3]],\n", " n_iters=50,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "rAVrwNPt7UUd" }, "source": [ "Предлагаем попробовать регулировать параметры `lr`, `start_theta` и посмотреть, сможет ли градиент сойтись к глобальному минимуму." ] }, { "cell_type": "markdown", "metadata": { "id": "kcC-mrMd7NvY" }, "source": [ "❓ **Вопрос** ❓\n", "> Как ведут себя траектории в примере со сложной функцией?" ] }, { "cell_type": "markdown", "metadata": { "id": "SVBPWb9Z7NvY" }, "source": [ "**Ответ**" ] }, { "cell_type": "markdown", "metadata": { "id": "ur9PqtsF6r_W" }, "source": [ "# Визуализация: L1-регуляризация, отбор признаков" ] }, { "cell_type": "markdown", "metadata": { "id": "2HsFHBCD7-c2" }, "source": [ "Вы уже услышали на лекции, что L1-регуляризация может занулять некоторые веса модели. Теперь предлагаем посмотреть, как именно зануляется градиент и как его выглядят траектории." ] }, { "cell_type": "markdown", "metadata": { "id": "s_xGN2lc8mpF" }, "source": [ "Напишем несколько вспомогательных функций." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "id": "iCVngRe86vlq" }, "outputs": [], "source": [ "def load_and_prepare_data(feature_indices):\n", " \"\"\"Загружает данные diabetes и подготавливает выбранные фичи.\"\"\"\n", " diabetes = load_diabetes()\n", " X = diabetes.data[:, feature_indices]\n", " y = diabetes.target\n", " feature_names = [diabetes.feature_names[i] for i in feature_indices]\n", " return X, y, feature_names" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "id": "iOnMtcyR69Aq" }, "outputs": [], "source": [ "def plot_lasso_coefficients(X, y, feature_names, alphas=np.logspace(-4, 1, 100)):\n", " \"\"\"Строит график изменения коэффициентов Lasso в зависимости от alpha.\"\"\"\n", " coefficients = []\n", " for alpha in alphas:\n", " model = Lasso(alpha=alpha, fit_intercept=False)\n", " model.fit(X, y)\n", " coefficients.append(model.coef_)\n", "\n", " coefficients = np.array(coefficients)\n", "\n", " plt.figure(figsize=(10, 6))\n", " for i in range(X.shape[1]):\n", " plt.plot(alphas, coefficients[:, i], label=feature_names[i])\n", "\n", " plt.xscale(\"log\")\n", " plt.title(\"Изменение коэффициентов Lasso в зависимости от регуляризации\")\n", " plt.xlabel(\"Alpha (логарифмическая шкала)\")\n", " plt.ylabel(\"Значение коэффициента\")\n", " plt.legend()\n", " plt.axhline(0, color=\"black\", lw=0.5, ls=\"--\")\n", " plt.grid(True)\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "id": "_fVt9az8698i" }, "outputs": [], "source": [ "def calculate_loss_surface(X, y, alpha, w1_range, w2_range):\n", " \"\"\"Рассчитывает поверхность потерь с L1-регуляризацией.\"\"\"\n", " w1 = np.linspace(*w1_range, 100)\n", " w2 = np.linspace(*w2_range, 100)\n", " W1, W2 = np.meshgrid(w1, w2)\n", " loss = np.zeros_like(W1)\n", "\n", " for i in range(W1.shape[0]):\n", " for j in range(W1.shape[1]):\n", " y_pred = X @ [W1[i, j], W2[i, j]]\n", " mse = mean_squared_error(y, y_pred)\n", " loss[i, j] = mse + alpha * (abs(W1[i, j]) + abs(W2[i, j]))\n", "\n", " return W1, W2, loss" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "id": "NmYQOYXq7Fi8" }, "outputs": [], "source": [ "def plot_optimization_path(W1, W2, loss, path, feature_names):\n", " \"\"\"Визуализирует траекторию оптимизации на фоне контурного графика.\"\"\"\n", " plt.figure(figsize=(12, 8))\n", " plt.contour(W1, W2, loss, levels=20)\n", " plt.plot(path[:, 0], path[:, 1], \"ro-\", markersize=3, alpha=0.7)\n", " plt.scatter(path[0, 0], path[0, 1], c=\"blue\", s=100, label=\"Старт\")\n", " plt.scatter(path[-1, 0], path[-1, 1], c=\"red\", s=100, label=\"Финиш\")\n", "\n", " plt.title(\"Траектория градиентного спуска на поверхности потерь\")\n", " plt.xlabel(f\"Коэффициент {feature_names[0]}\")\n", " plt.ylabel(f\"Коэффициент {feature_names[1]}\")\n", " plt.legend()\n", " plt.colorbar(label=\"Значение функции потерь\")\n", " plt.grid(True)\n", " plt.show()" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "id": "GkDu81Li7BMI" }, "outputs": [], "source": [ "def gradient_descent(X, y, initial_weights, alpha, lr=0.001, n_iter=100):\n", " \"\"\"Реализует алгоритм градиентного спуска для Lasso регрессии.\"\"\"\n", " w = initial_weights.copy()\n", " path = [w]\n", "\n", " for _ in range(n_iter):\n", " y_pred = X @ w\n", " grad_mse = 2 / X.shape[0] * X.T @ (y_pred - y)\n", " grad_l1 = alpha * np.sign(w)\n", " grad = grad_mse + grad_l1\n", " w -= lr * grad\n", " path.append(w.copy())\n", "\n", " return np.array(path)" ] }, { "cell_type": "markdown", "metadata": { "id": "u3V42hmY8wDL" }, "source": [ "Использовать будем датасет `diabetes` из `sklearn`, но для облегчения оставим только два признака." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "id": "yQMqgD8e7NjC" }, "outputs": [], "source": [ "FEATURE_INDICES = [4, 1]\n", "X, y, feature_names = load_and_prepare_data(FEATURE_INDICES)" ] }, { "cell_type": "markdown", "metadata": { "id": "bvXZDzGS9aAu" }, "source": [ "Выберем `alpha = 0.3` и посмотрим на градиентный спуск." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "id": "umGpXVoF7W3h" }, "outputs": [], "source": [ "ALPHA = 0.3 # Параметр регуляризации L1\n", "INITIAL_WEIGHTS = [120, 250] # Начальные веса для градиентного спуска\n", "W1_RANGE = [-50, 500] # Границы для первого коэффициента\n", "W2_RANGE = [-50, 250] # Границы для второго коэффициента\n", "LEARNING_RATE = 5 # Скорость обучения\n", "ITERATIONS = 5000 # Количество итераций\n", "\n", "# 2. Расчет поверхности потерь\n", "W1, W2, loss = calculate_loss_surface(X, y, ALPHA, W1_RANGE, W2_RANGE)\n", "\n", "# 3. Оптимизация градиентным спуском\n", "path = gradient_descent(X, y, INITIAL_WEIGHTS, ALPHA, lr=LEARNING_RATE, n_iter=ITERATIONS)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "gmreIhbH7JKh" }, "outputs": [], "source": [ "# 4. Визуализация результатов\n", "plot_optimization_path(W1, W2, loss, path, feature_names)" ] }, { "cell_type": "markdown", "metadata": { "id": "TBxDH6GW9Xc9" }, "source": [ "Видим, что минимум у функции потерь (Loss) находится в окрестности точки (0, 300)." ] }, { "cell_type": "markdown", "metadata": { "id": "X_Yg1IAh9qw2" }, "source": [ "Выберем значние `alpha` достаточно большим и посмотрим на градиентный спуск." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "OgxmsNxR7uus" }, "outputs": [], "source": [ "ALPHA = 20 # Параметр регуляризации L1\n", "INITIAL_WEIGHTS = [100, 100] # Начальные веса для градиентного спуска\n", "W1_RANGE = [-50, 120] # Границы для первого коэффициента\n", "W2_RANGE = [-50, 120] # Границы для второго коэффициента\n", "LEARNING_RATE = 0.1 # Скорость обучения\n", "ITERATIONS = 500 # Количество итераций\n", "\n", "\n", "W1, W2, loss = calculate_loss_surface(X, y, ALPHA, W1_RANGE, W2_RANGE)\n", "\n", "path = gradient_descent(X, y, INITIAL_WEIGHTS, ALPHA, lr=LEARNING_RATE, n_iter=ITERATIONS)\n", "\n", "plot_optimization_path(W1, W2, loss, path, feature_names)" ] }, { "cell_type": "markdown", "metadata": { "id": "pZNiBs_G93po" }, "source": [ "Видим, что минимум находится в окрестности (0, 0), что значит, что из-за больших штрафов нашей модели выгодно занулять все веса." ] }, { "cell_type": "markdown", "metadata": { "id": "WkNiRMCn90ni" }, "source": [ "Выберем значние `alpha` маленьким и посмотрим на градиентный спуск." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "f-gs-upq8FfB" }, "outputs": [], "source": [ "ALPHA = 0.01 # Параметр регуляризации L1\n", "INITIAL_WEIGHTS = [150, 300] # Начальные веса для градиентного спуска\n", "W1_RANGE = [50, 500] # Границы для первого коэффициента\n", "W2_RANGE = [0, 300] # Границы для второго коэффициента\n", "LEARNING_RATE = 5 # Скорость обучения\n", "ITERATIONS = 500 # Количество итераций\n", "\n", "\n", "W1, W2, loss = calculate_loss_surface(X, y, ALPHA, W1_RANGE, W2_RANGE)\n", "\n", "path = gradient_descent(X, y, INITIAL_WEIGHTS, ALPHA, lr=LEARNING_RATE, n_iter=ITERATIONS)\n", "\n", "plot_optimization_path(W1, W2, loss, path, feature_names)" ] }, { "cell_type": "markdown", "metadata": { "id": "FhgH7S41-X0G" }, "source": [ "Видим, что коэффиценты не нулевые и минимум в области (50, 350)" ] }, { "cell_type": "markdown", "metadata": { "id": "o74m4CfcRPY4" }, "source": [ "❓ **Вопрос** ❓\n", "> Как использование L1-регуляризации влияет на область минимума функции потерь и почему это приводит к обнулению части параметров модели, в зависимости от коэффициента регуляризации alpha?" ] }, { "cell_type": "markdown", "metadata": { "id": "vAr3Nh6rSDN4" }, "source": [ "**Ответ:**" ] } ], "metadata": { "colab": { "collapsed_sections": [ "ur9PqtsF6r_W" ], "provenance": [] }, "kernelspec": { "display_name": "stats", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.7" } }, "nbformat": 4, "nbformat_minor": 0 }