{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "intro",
   "metadata": {},
   "source": [
    "# 17　長めの流れを覚えるLSTM\n",
    "\n",
    "FUJIMOTO LAB 深層学習コース。Web教材の図と説明を読んでから実行してください。Pythonの計算はColabの実行環境で行います。\n",
    "\n",
    "**この回の目標**：LSTMの記憶セルと3つのゲートの役割を言える\n",
    "\n",
    "このノートブックは小さな合成データを使用し、元の教材の固定Driveパスや動画を必要としません。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "predict",
   "metadata": {},
   "source": [
    "## 1　まず予想する\n",
    "\n",
    "コードを実行する前に、表示される値や形を予想してください。"
   ]
  },
  {
   "cell_type": "code",
   "id": "demo-one",
   "metadata": {},
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "lstm = nn.LSTM(input_size=1,hidden_size=4,batch_first=True)\n",
    "x = torch.tensor([[[0.0],[0.5],[1.0]]])\n",
    "out,(hidden,cell) = lstm(x)\n",
    "print(out.shape,hidden.shape,cell.shape)"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "explain-one",
   "metadata": {},
   "source": [
    "**確かめ方**：時刻ごとの出力に加え、最後のhiddenと記憶セルcellが返ります。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "part-two",
   "metadata": {},
   "source": [
    "## 2　RNNと同じ窓を使える\n",
    "\n",
    "データの切り出し方はRNNと共通です。変わるのは内部の計算と、記憶セルの扱いです。"
   ]
  },
  {
   "cell_type": "code",
   "id": "demo-two",
   "metadata": {},
   "source": [
    "values = torch.sin(torch.linspace(0,3.14,12))\n",
    "windows = torch.stack([values[i:i+4] for i in range(8)]).unsqueeze(-1)\n",
    "print(windows.shape)"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "explain-two",
   "metadata": {},
   "source": [
    "**結果を読む**：8件・4時刻・1特徴の形です。次の値を正解にして学習できます。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "experiment",
   "metadata": {},
   "source": [
    "## 3　値を変えて比べる\n",
    "\n",
    "次のセルでは、表示される値や条件を変えて、何が結果を決めるかを確かめます。長くかかる実験は、少数の合成データで行います。"
   ]
  },
  {
   "cell_type": "code",
   "id": "lab",
   "metadata": {},
   "source": [
    "# RNNと同じ窓の考え方で、LSTMを小さく学習\n",
    "torch.manual_seed(4)\n",
    "series = torch.sin(torch.linspace(0,12,80))\n",
    "samples = torch.stack([series[i:i+4] for i in range(76)]).unsqueeze(-1)\n",
    "targets = series[4:].unsqueeze(-1)\n",
    "body = nn.LSTM(1,8,batch_first=True)\n",
    "head = nn.Linear(8,1)\n",
    "opt = torch.optim.Adam(list(body.parameters())+list(head.parameters()),lr=0.02)\n",
    "for epoch in range(80):\n",
    "    opt.zero_grad()\n",
    "    states,_ = body(samples[:60])\n",
    "    prediction = head(states[:,-1])\n",
    "    loss = nn.MSELoss()(prediction,targets[:60])\n",
    "    loss.backward(); opt.step()\n",
    "with torch.no_grad():\n",
    "    states,_ = body(samples[60:])\n",
    "    test_loss = nn.MSELoss()(head(states[:,-1]),targets[60:]).item()\n",
    "print('後半を残したテストのMSE:',round(test_loss,4))\n"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "extra-guide-1",
   "metadata": {},
   "source": [
    "## 追加実習 1　予測値と答えを並べる\n",
    "\n",
    "MSEは一つの数字です。実際の予測がどの程度ずれているか、後半の数件を直接見ましょう。\n",
    "\n",
    "**実行前に予想**：何が変わり、何が変わらないでしょうか。"
   ]
  },
  {
   "cell_type": "code",
   "id": "extra-code-1",
   "metadata": {},
   "source": [
    "with torch.no_grad():\n",
    "    state_out, _ = body(samples[60:])\n",
    "    guessed = head(state_out[:, -1])[:, 0]\n",
    "for actual, estimate in zip(targets[60:64, 0], guessed[:4]):\n",
    "    print('答え', round(actual.item(), 3), '予測', round(estimate.item(), 3))\n"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "extra-reflect-1",
   "metadata": {},
   "source": [
    "**確認**：予想と違った点を一つ書き、値を一つ変えて再実行してください。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "reflection",
   "metadata": {},
   "source": [
    "## 自分の言葉で答えよう\n",
    "\n",
    "LSTMの記憶セルを調整するものは？\n",
    "\n",
    "- まず予想を書く\n",
    "- コードのどの行が答えを決めるか指す\n",
    "- 条件や値を1つ変えて、予想と実行結果を比べる\n",
    "\n",
    "**ヒント**：3つのゲートが情報の流れを調整します。"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "17-lstm.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}