{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "intro",
   "metadata": {},
   "source": [
    "# 14　学んだ特徴を新しい課題に生かす\n",
    "\n",
    "FUJIMOTO LAB 深層学習コース。Web教材の図と説明を読んでから実行してください。Pythonの計算はColabの実行環境で行います。\n",
    "\n",
    "**この回の目標**：特徴抽出部の固定と新しい分類部の役割を言える\n",
    "\n",
    "このノートブックは小さな合成データを使用し、元の教材の固定Driveパスや動画を必要としません。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "predict",
   "metadata": {},
   "source": [
    "## 1　まず予想する\n",
    "\n",
    "コードを実行する前に、表示される値や形を予想してください。"
   ]
  },
  {
   "cell_type": "code",
   "id": "demo-one",
   "metadata": {},
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "features = nn.Sequential(nn.Linear(2,8),nn.ReLU())\n",
    "head = nn.Linear(8,2)\n",
    "for parameter in features.parameters():\n",
    "    parameter.requires_grad = False\n",
    "print(sum(p.requires_grad for p in features.parameters()))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "explain-one",
   "metadata": {},
   "source": [
    "**確かめ方**：特徴抽出部のパラメータは更新対象から外れました。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "part-two",
   "metadata": {},
   "source": [
    "## 2　新しい分類部だけ選ぶ\n",
    "\n",
    "固定した部分も推論には使いますが、更新するのは新しい分類部の重みだけです。"
   ]
  },
  {
   "cell_type": "code",
   "id": "demo-two",
   "metadata": {},
   "source": [
    "model = nn.Sequential(features,head)\n",
    "optimizer = torch.optim.SGD(head.parameters(),lr=0.1)\n",
    "print(model(torch.zeros(1,2)).shape)"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "explain-two",
   "metadata": {},
   "source": [
    "**結果を読む**：出力は2クラス分です。ノートブックでは小さな合成データを使い、固定前後を観察します。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "experiment",
   "metadata": {},
   "source": [
    "## 3　値を変えて比べる\n",
    "\n",
    "次のセルでは、表示される値や条件を変えて、何が結果を決めるかを確かめます。長くかかる実験は、少数の合成データで行います。"
   ]
  },
  {
   "cell_type": "code",
   "id": "lab",
   "metadata": {},
   "source": [
    "# 小さな合成課題で、学習した特徴部を固定して新しい頭を学習\n",
    "torch.manual_seed(11)\n",
    "source_x = torch.randn(100,2)\n",
    "source_y = (source_x[:,0]>0).long()\n",
    "source = nn.Sequential(nn.Linear(2,8),nn.ReLU(),nn.Linear(8,2))\n",
    "source_opt = torch.optim.Adam(source.parameters(),lr=0.03)\n",
    "for _ in range(40):\n",
    "    source_opt.zero_grad()\n",
    "    loss = nn.CrossEntropyLoss()(source(source_x),source_y)\n",
    "    loss.backward(); source_opt.step()\n",
    "extractor = nn.Sequential(source[0],source[1])\n",
    "for p in extractor.parameters(): p.requires_grad=False\n",
    "new_head = nn.Linear(8,2)\n",
    "new_model = nn.Sequential(extractor,new_head)\n",
    "target_x = torch.randn(60,2)\n",
    "target_y = ((target_x[:,0]+0.5*target_x[:,1])>0).long()\n",
    "new_opt = torch.optim.Adam(new_head.parameters(),lr=0.03)\n",
    "for _ in range(40):\n",
    "    new_opt.zero_grad()\n",
    "    loss = nn.CrossEntropyLoss()(new_model(target_x),target_y)\n",
    "    loss.backward(); new_opt.step()\n",
    "print('新しい分類部の学習例:',round(loss.item(),3))\n",
    "print('固定した特徴部の重み:',extractor[0].weight.requires_grad)\n"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "extra-guide-1",
   "metadata": {},
   "source": [
    "## 追加実習 1　どの重みを更新した？\n",
    "\n",
    "特徴抽出部は固定、分類部は更新できます。学習前後の比較では、課題の似かよい方にも注意しましょう。\n",
    "\n",
    "**実行前に予想**：何が変わり、何が変わらないでしょうか。"
   ]
  },
  {
   "cell_type": "code",
   "id": "extra-code-1",
   "metadata": {},
   "source": [
    "print('特徴抽出部の更新対象:', sum(p.requires_grad for p in extractor.parameters()))\n",
    "print('新しい分類部の更新対象:', sum(p.requires_grad for p in new_head.parameters()))\n",
    "with torch.no_grad():\n",
    "    accuracy = (new_model(target_x).argmax(1) == target_y).float().mean().item()\n",
    "print('新しい課題の学習用正解率:', round(accuracy, 3))\n",
    "print('注意: これは学習用の結果で、汎化性能ではありません。')\n"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "extra-reflect-1",
   "metadata": {},
   "source": [
    "**確認**：予想と違った点を一つ書き、値を一つ変えて再実行してください。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "reflection",
   "metadata": {},
   "source": [
    "## 自分の言葉で答えよう\n",
    "\n",
    "特徴抽出部を固定すると？\n",
    "\n",
    "- まず予想を書く\n",
    "- コードのどの行が答えを決めるか指す\n",
    "- 条件や値を1つ変えて、予想と実行結果を比べる\n",
    "\n",
    "**ヒント**：固定しても計算には参加しますが、重みを更新しません。"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "14-transfer.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}