Skip to content

Commit

Permalink
Colaboratory を使用して作成しました (#74)
Browse files Browse the repository at this point in the history
  • Loading branch information
@keisuke-umezawa
keisuke-umezawa committed Nov 27, 2018
1 parent 5ebbf88 commit 5e13ac3
Showing 1 changed file with 145 additions and 78 deletions.
223 changes: 145 additions & 78 deletions hands_on_ja/chainer/begginers_hands_on/01_Chainer_basic_tutorial.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
"provenance": [],
"collapsed_sections": [
"7xCCSzxZlde1"
]
],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
Expand Down Expand Up @@ -68,24 +69,24 @@
"metadata": {
"id": "QNxHfX_Blly7",
"colab_type": "code",
"outputId": "767af462-7e3e-4a85-d9ee-2a085722c098",
"outputId": "aa1a5a65-8b78-48e9-ca13-5f4e8245b15b",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1193
"height": 1173
}
},
"cell_type": "code",
"source": [
"!curl https://colab.chainer.org/install | sh -"
],
"execution_count": 1,
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 1379 100 1379 0 0 6629 0 --:--:-- --:--:-- --:--:-- 6598\n",
"\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 1379 100 1379 0 0 10446 0 --:--:-- --:--:-- --:--:-- 10446\n",
"+ apt -y -q install cuda-libraries-dev-9-2\n",
"Reading package lists...\n",
"Building dependency tree...\n",
Expand All @@ -110,7 +111,7 @@
"Get:7 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1710/x86_64 cuda-nvgraph-dev-9-2 9.2.148-1 [30.1 MB]\n",
"Get:8 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1710/x86_64 cuda-npp-dev-9-2 9.2.148-1 [52.0 MB]\n",
"Get:9 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1710/x86_64 cuda-libraries-dev-9-2 9.2.148-1 [2,598 B]\n",
"Fetched 332 MB in 7s (47.5 MB/s)\n",
"Fetched 332 MB in 7s (48.7 MB/s)\n",
"Selecting previously unselected package cuda-cublas-dev-9-2.\n",
"(Reading database ... 22298 files and directories currently installed.)\n",
"Preparing to unpack .../0-cuda-cublas-dev-9-2_9.2.148.1-1_amd64.deb ...\n",
Expand Down Expand Up @@ -170,17 +171,17 @@
"metadata": {
"id": "VePAE4PTmf-p",
"colab_type": "code",
"outputId": "a7c2d627-764d-49d0-ff1e-d693af180137",
"outputId": "82010ac8-c095-4708-ebdc-ca3e043fc35b",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 258
"height": 238
}
},
"cell_type": "code",
"source": [
"!python -c 'import chainer; chainer.print_runtime_info()'"
],
"execution_count": 2,
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
Expand Down Expand Up @@ -210,17 +211,17 @@
},
"cell_type": "markdown",
"source": [
"うまくインストールできていますね。以下のチュートリアルでは、matplotlibを可視化に使いますので、これも同時にインストールしておきましょう。"
"うまくインストールできていますね。以下のチュートリアルでは、pydotをネットワーク構造の可視化に使いますので、これも同時にインストールしておきましょう。"
]
},
{
"metadata": {
"id": "_NGiH6r2c-KZ",
"colab_type": "code",
"outputId": "87d55224-3e3c-4830-a864-9d60eba323bd",
"outputId": "07ba176c-d48b-4944-fcdb-06b390eb4e63",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 122
"height": 102
}
},
"cell_type": "code",
Expand All @@ -243,14 +244,24 @@
}
]
},
{
"metadata": {
"id": "LteyQnjgb_BB",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"最後に以前使用した関数をchutilというパッケージにまとめましたので、ここでインストールしましょう。"
]
},
{
"metadata": {
"id": "H-jjJSw8DmtX",
"colab_type": "code",
"outputId": "ce04beac-559a-4f9f-a404-2dd4313495a7",
"outputId": "9b02c817-7c7e-4fcc-c2a6-fc73e489fe27",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 105
"height": 85
}
},
"cell_type": "code",
Expand Down Expand Up @@ -500,7 +511,7 @@
"metadata": {
"id": "vXXaYZUPrg4U",
"colab_type": "code",
"outputId": "6164a462-b546-4ea0-9c9d-89ab74675b63",
"outputId": "a1a4c18a-333c-4f48-9a2d-bf5d5b94c92f",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 85
Expand Down Expand Up @@ -587,7 +598,9 @@
"* 各convolution層のチャネル数、ノード数を増やす\n",
"* 各convolution層のフィルター数`ksize=3`, ストライド`stride=3`、パッディング`pad=1`に変更\n",
"* convolution層を1層(`conv4`)追加\n",
"* 活性化関数を`ReLU`に変更\n"
"* 活性化関数を`ReLU`に変更\n",
"\n",
"ちなみに、`L.Convolution2D`の`in_channels`や`L.Linear`の第一引数を`None`にすることで明示的にサイズを指定していませんが、実はこれでもちゃんと動きます。なぜかというと、最初の`forward`呼び出し時に、入力されたサイズで動的にメモリを確保してくれるからです。\n"
]
},
{
Expand Down Expand Up @@ -618,7 +631,7 @@
" self.fc5 = L.Linear(None, 1000) # 84 -> 1000\n",
" self.fc6 = L.Linear(None, 10)\n",
"\n",
" def __call__(self, x):\n",
" def forward(self, x):\n",
" h = F.relu(self.conv1(x.reshape((-1, 1, 28, 28))))\n",
" h = F.max_pooling_2d(h, ksize=2, stride=2)\n",
" h = F.relu(self.conv2(h)) # sigmoid -> relu\n",
Expand All @@ -639,7 +652,7 @@
},
"cell_type": "markdown",
"source": [
"\n",
"では、上記変更を加えたモデル`MyConvNet`を訓練してみましょう。\n",
"\n",
"\n"
]
Expand All @@ -656,8 +669,6 @@
},
"cell_type": "code",
"source": [
"enable_cupy = True\n",
"n_epoch = 1\n",
"n_epoch = 20\n",
"batchsize = 128\n",
"\n",
Expand Down Expand Up @@ -698,6 +709,68 @@
}
]
},
{
"metadata": {
"id": "qVbMQ0hyghQU",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"検証用データでも90%の精度を達成しています。これならテスト用データでも目標を達成してそうですね。\n",
"\n",
"以下の関数を実行して確かめてみましょう。"
]
},
{
"metadata": {
"id": "_5FxlxBv0n9p",
"colab_type": "code",
"outputId": "4da38f75-bd19-4374-9508-e3cef3eefa7f",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
}
},
"cell_type": "code",
"source": [
"from chutil.visualize.show import show_test_performance\n",
"\n",
"show_test_performance(classifier_model, test)"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"Test accuracy: 0.91982424\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "lPm8yXMHgxRJ",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"達成できていましたね！パチパチ！\n",
"\n",
"と終わりにしたいところですが、もう少し細かく結果を見てみましょう。"
]
},
{
"metadata": {
"id": "zIImirEIeoEr",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"まずは、`show_graph`を使ってネットワーク構造を表示してみましょう。\n",
"実際にどのようなネットワークができたか確認できますね。"
]
},
{
"metadata": {
"id": "KiBp2v5ygD6n",
Expand Down Expand Up @@ -734,6 +807,20 @@
}
]
},
{
"metadata": {
"id": "8z4qOyPafHRV",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"次に、`show_loss_and_accuracy`を使って、訓練用データと検証用データの損失と精度をplotしてみましょう。\n",
"おそらく以下のことに気づくと思います。\n",
"\n",
"* 訓練用データでは、`epoch`数に伴い、損失は減少・精度は増加し、共に改善している\n",
"* しかし、検証用データでは、`epoch=5`程度で精度は頭打ちになっており、損失も最小値になったあと悪化している"
]
},
{
"metadata": {
"id": "HbKYXpnk0l6q",
Expand Down Expand Up @@ -780,29 +867,42 @@
},
{
"metadata": {
"id": "_5FxlxBv0n9p",
"colab_type": "code",
"outputId": "4da38f75-bd19-4374-9508-e3cef3eefa7f",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
}
"id": "FSn4eo7HhlvN",
"colab_type": "text"
},
"cell_type": "code",
"cell_type": "markdown",
"source": [
"from chutil.visualize.show import show_test_performance\n",
"\n",
"show_test_performance(classifier_model, test)"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"Test accuracy: 0.91982424\n"
],
"name": "stdout"
}
"訓練用データでは精度が順調に伸びるのに、検証用データでは伸びない・・・。このような現象をどこかで聞いたことがあるような気がしますよね。"
]
},
{
"metadata": {
"id": "Zc-kZPtWhQed",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"## 過学習と汎化性能"
]
},
{
"metadata": {
"id": "qMvkkQwLigT3",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"### 過学習とは"
]
},
{
"metadata": {
"id": "xwlTnVjahQqw",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"### Dropout"
]
},
{
Expand Down Expand Up @@ -847,16 +947,11 @@
"metadata": {
"id": "nO84XgsNDiN9",
"colab_type": "code",
"outputId": "b6526c08-2ac0-4aee-f1f2-a24550c60604",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 374
}
"colab": {}
},
"cell_type": "code",
"source": [
"enable_cupy = True\n",
"n_epoch = 20\n",
"n_epoch = 30\n",
"batchsize = 128\n",
"\n",
"model = MyConvNet()\n",
Expand All @@ -866,35 +961,7 @@
" classifier_model, optimizer, train, validation, n_epoch, batchsize)"
],
"execution_count": 0,
"outputs": [
{
"output_type": "stream",
"text": [
"epoch main/loss main/accuracy val/main/loss val/main/accuracy elapsed_time\n",
"\u001b[J1 0.553697 0.794977 0.387361 0.86254 10.9894 \n",
"\u001b[J2 0.352021 0.870724 0.343159 0.873418 21.6753 \n",
"\u001b[J3 0.299985 0.88756 0.281276 0.900218 32.2488 \n",
"\u001b[J4 0.270547 0.901894 0.265802 0.902591 42.9115 \n",
"\u001b[J5 0.256918 0.90643 0.245724 0.911887 53.5615 \n",
"\u001b[J6 0.240187 0.910557 0.24804 0.910008 64.0522 \n",
"\u001b[J7 0.225157 0.91652 0.223051 0.918315 74.6426 \n",
"\u001b[J8 0.216988 0.91867 0.227083 0.914953 85.1441 \n",
"\u001b[J9 0.207234 0.923813 0.216907 0.920293 95.6334 \n",
"\u001b[J10 0.199658 0.925492 0.217243 0.917227 106.077 \n",
"\u001b[J11 0.196208 0.926382 0.230436 0.913667 117.514 \n",
"\u001b[J12 0.19097 0.930047 0.218538 0.919007 129.225 \n",
"\u001b[J13 0.186419 0.930687 0.199121 0.925732 141.079 \n",
"\u001b[J14 0.176123 0.933854 0.202717 0.92415 152.826 \n",
"\u001b[J15 0.171971 0.935322 0.204852 0.921578 164.535 \n",
"\u001b[J16 0.169379 0.936138 0.204297 0.924644 176.419 \n",
"\u001b[J17 0.167468 0.9377 0.215237 0.921282 188.069 \n",
"\u001b[J18 0.163362 0.939618 0.202339 0.92682 199.796 \n",
"\u001b[J19 0.157467 0.940365 0.206196 0.925831 211.327 \n",
"\u001b[J20 0.154054 0.941216 0.20949 0.922567 222.881 \n"
],
"name": "stdout"
}
]
"outputs": []
},
{
"metadata": {
Expand Down Expand Up @@ -1246,4 +1313,4 @@
]
}
]
}
}

0 comments on commit 5e13ac3

Please sign in to comment.