不要怂，就是GAN！AI学藏文.ipynb

{"cells":[{"metadata":{"id":"0EBCBAF2EEEF483A863555F097896CEF","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"### 1. 学会看\n先让AI读懂藏文吧！看acc一步一步往上爬"},{"metadata":{"id":"3E77668050074D1C8EB114AADA38929B","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"stream","text":"Using TensorFlow backend.\n","name":"stderr"}],"source":"from tensorflow import keras\r\nfrom keras import utils\r\nimport numpy as np\r\nfrom sklearn.model_selection import train_test_split\r\nfrom keras.utils import np_utils\r\nfrom keras.models import Sequential\r\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D, MaxPool2D\r\nfrom keras.optimizers import RMSprop\r\nfrom keras.preprocessing.image import ImageDataGenerator\r\nfrom keras.callbacks import ReduceLROnPlateau","execution_count":1},{"metadata":{"id":"253BE808ABF94C09B991D07C44B00ECE","collapsed":false,"scrolled":false,"mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"#### 1.1 全连接神经网络\nMLP test acc: 97.44%"},{"metadata":{"id":"BD8E2BE5BEF4475E8BE0D72A25244A81","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"stream","text":"_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n=================================================================\ndense_33 (Dense)             (None, 784)               615440    \n_________________________________________________________________\ndense_34 (Dense)             (None, 512)               401920    \n_________________________________________________________________\ndropout_15 (Dropout)         (None, 512)               0         \n_________________________________________________________________\ndense_35 (Dense)             (None, 256)               131328    \n_________________________________________________________________\ndropout_16 (Dropout)         (None, 256)               0         \n_________________________________________________________________\ndense_36 (Dense)             (None, 10)                2570      \n=================================================================\nTotal params: 1,151,258\nTrainable params: 1,151,258\nNon-trainable params: 0\n_________________________________________________________________\nTrain on 14214 samples, validate on 3554 samples\nEpoch 1/10\n14214/14214 [==============================] - 5s 345us/step - loss: 0.4274 - acc: 0.8557 - val_loss: 0.1489 - val_acc: 0.9513\nEpoch 2/10\n14214/14214 [==============================] - 3s 245us/step - loss: 0.1498 - acc: 0.9515 - val_loss: 0.1209 - val_acc: 0.9617\nEpoch 3/10\n14214/14214 [==============================] - 3s 244us/step - loss: 0.0965 - acc: 0.9685 - val_loss: 0.1017 - val_acc: 0.9645\nEpoch 4/10\n14214/14214 [==============================] - 3s 246us/step - loss: 0.0593 - acc: 0.9813 - val_loss: 0.0872 - val_acc: 0.9699\nEpoch 5/10\n14214/14214 [==============================] - 4s 248us/step - loss: 0.0361 - acc: 0.9881 - val_loss: 0.0887 - val_acc: 0.9747\nEpoch 6/10\n14214/14214 [==============================] - 3s 243us/step - loss: 0.0343 - acc: 0.9897 - val_loss: 0.1137 - val_acc: 0.9696\nEpoch 7/10\n14214/14214 [==============================] - 3s 241us/step - loss: 0.0347 - acc: 0.9892 - val_loss: 0.1029 - val_acc: 0.9707\nEpoch 8/10\n14214/14214 [==============================] - 4s 251us/step - loss: 0.0261 - acc: 0.9915 - val_loss: 0.0869 - val_acc: 0.9755\nEpoch 9/10\n14214/14214 [==============================] - 3s 243us/step - loss: 0.0158 - acc: 0.9951 - val_loss: 0.1202 - val_acc: 0.9685\nEpoch 10/10\n14214/14214 [==============================] - 3s 244us/step - loss: 0.0221 - acc: 0.9935 - val_loss: 0.0958 - val_acc: 0.9744\n3554/3554 [==============================] - 0s 138us/step\nSummary: Loss over the test dataset: 0.0958, Accuracy: 0.9744\n","name":"stdout"}],"source":"#数据加载\r\ndata = np.load('/home/kesci/input/TibetanMNIST5610/TibetanMNIST.npz')\r\nX, y = data['image'], data['label']  # (17768, 28, 28)\r\nX = X.reshape(17768, 784).astype('float32') / 255\r\ny = np_utils.to_categorical(y, num_classes=10)\r\nX_train, X_test, y_train, y_test = train_test_split(X, y,test_size = 0.2, random_state=2018)\r\n\r\n#MLP\r\nmodel = Sequential()\r\nmodel.add(Dense(784, input_dim=784, activation='relu'))\r\nmodel.add(Dense(512, activation='relu'))\r\nmodel.add(Dropout(0.2))\r\nmodel.add(Dense(256, activation='relu'))\r\nmodel.add(Dropout(0.2))\r\nmodel.add(Dense(10, activation='softmax'))\r\nmodel.compile(optimizer='adam' , loss = \"categorical_crossentropy\", metrics=[\"accuracy\"])\r\nmodel.summary()\r\n\r\nhistory=model.fit(x=X_train, y=y_train,\r\n          batch_size=128,\r\n          epochs=10,\r\n          verbose=1,\r\n          validation_data=(X_test, y_test))\r\nevaluation = model.evaluate(X_test, y_test, verbose=1)\r\nprint('Summary: Loss over the test dataset: %.4f, Accuracy: %.4f' % (evaluation[0], evaluation[1]))","execution_count":18},{"metadata":{"id":"DBB53D6D733048FE8A193A4BC51BBF61","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"#### 1.2 卷积神经网络 \nCNN test acc: 99.16%"},{"metadata":{"id":"BBFB2A3F133949AE8F163673F99D8DFE","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"stream","text":"Train on 14214 samples, validate on 3554 samples\nEpoch 1/10\n - 252s - loss: 0.5506 - acc: 0.8146 - val_loss: 0.1384 - val_acc: 0.9524\nEpoch 2/10\n - 277s - loss: 0.1609 - acc: 0.9466 - val_loss: 0.0729 - val_acc: 0.9761\nEpoch 3/10\n - 278s - loss: 0.1109 - acc: 0.9623 - val_loss: 0.0529 - val_acc: 0.9840\nEpoch 4/10\n - 240s - loss: 0.0874 - acc: 0.9720 - val_loss: 0.0478 - val_acc: 0.9851\nEpoch 5/10\n - 204s - loss: 0.0660 - acc: 0.9783 - val_loss: 0.0407 - val_acc: 0.9871\nEpoch 6/10\n - 214s - loss: 0.0531 - acc: 0.9842 - val_loss: 0.0342 - val_acc: 0.9887\nEpoch 7/10\n - 276s - loss: 0.0513 - acc: 0.9837 - val_loss: 0.0371 - val_acc: 0.9893\nEpoch 8/10\n - 276s - loss: 0.0459 - acc: 0.9860 - val_loss: 0.0411 - val_acc: 0.9868\nEpoch 9/10\n - 269s - loss: 0.0417 - acc: 0.9870 - val_loss: 0.0323 - val_acc: 0.9896\nEpoch 10/10\n - 201s - loss: 0.0341 - acc: 0.9892 - val_loss: 0.0326 - val_acc: 0.9916\n3554/3554 [==============================] - 32s 9ms/step\nSummary: Loss over the test dataset: 0.0326, Accuracy: 0.9916\n","name":"stdout"}],"source":"#CNN\r\nbatch_size = 128\r\nepochs = 10\r\ndata = np.load('/home/kesci/input/TibetanMNIST5610/TibetanMNIST.npz')\r\nX, y = data['image'], data['label']  # (17768, 28, 28)\r\nX = X.reshape(X.shape[0], 28,28,1).astype('float32') / 255\r\nX_train, X_test, y_train, y_test_ = train_test_split(X, y, test_size=0.2, random_state=2018)\r\ny_train = np_utils.to_categorical(y_train, num_classes=10)\r\ny_test = np_utils.to_categorical(y_test_, num_classes=10)\r\n\r\nmodel = Sequential()\r\n\r\nmodel.add(Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \r\n                 activation ='relu', input_shape = (28,28,1)))\r\nmodel.add(Conv2D(filters = 32, kernel_size = (5,5),padding = 'Same', \r\n                 activation ='relu'))\r\nmodel.add(MaxPool2D(pool_size=(2,2)))\r\nmodel.add(Dropout(0.25))\r\n\r\nmodel.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \r\n                 activation ='relu'))\r\nmodel.add(Conv2D(filters = 64, kernel_size = (3,3),padding = 'Same', \r\n                 activation ='relu'))\r\nmodel.add(MaxPool2D(pool_size=(2,2), strides=(2,2)))\r\nmodel.add(Dropout(0.25))\r\n\r\nmodel.add(Flatten())\r\nmodel.add(Dense(256, activation = \"relu\"))\r\nmodel.add(Dropout(0.5))\r\nmodel.add(Dense(10, activation = \"softmax\"))\r\nmodel.compile(optimizer='adam' , loss = \"categorical_crossentropy\", metrics=[\"accuracy\"])\r\n\r\nhistory = model.fit(X_train, y_train, batch_size = batch_size, epochs = epochs, \r\n          validation_data = (X_test, y_test), verbose = 2)\r\n          \r\nevaluation = model.evaluate(X_test, y_test, verbose=1)\r\nprint('Summary: Loss over the test dataset: %.4f, Accuracy: %.4f' % (evaluation[0], evaluation[1]))","execution_count":19},{"metadata":{"id":"021383A1ACC84559AD1F2BDD24434A95","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"#### 1.3 CNN模型调优 \nCNN test acc: 99.24%\n - 数据增强\n - 添加一个学习率衰减器\n \n> 加大epoch数值，精度仍有上升空间，好慢呀"},{"metadata":{"id":"1B145FB94B91452382F7E0A2F9D748EC","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"stream","text":"Epoch 1/10\n - 206s - loss: 0.0290 - acc: 0.9905 - val_loss: 0.0372 - val_acc: 0.9893\nEpoch 2/10\n - 197s - loss: 0.0267 - acc: 0.9913 - val_loss: 0.0365 - val_acc: 0.9893\nEpoch 3/10\n - 230s - loss: 0.0271 - acc: 0.9910 - val_loss: 0.0326 - val_acc: 0.9893\nEpoch 4/10\n - 276s - loss: 0.0253 - acc: 0.9918 - val_loss: 0.0365 - val_acc: 0.9904\nEpoch 5/10\n - 277s - loss: 0.0264 - acc: 0.9907 - val_loss: 0.0299 - val_acc: 0.9921\nEpoch 6/10\n - 244s - loss: 0.0213 - acc: 0.9935 - val_loss: 0.0291 - val_acc: 0.9916\nEpoch 7/10\n - 197s - loss: 0.0238 - acc: 0.9925 - val_loss: 0.0305 - val_acc: 0.9916\nEpoch 8/10\n - 212s - loss: 0.0186 - acc: 0.9937 - val_loss: 0.0360 - val_acc: 0.9904\nEpoch 9/10\n - 276s - loss: 0.0249 - acc: 0.9914 - val_loss: 0.0296 - val_acc: 0.9916\n\nEpoch 00009: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.\nEpoch 10/10\n - 277s - loss: 0.0129 - acc: 0.9961 - val_loss: 0.0281 - val_acc: 0.9924\n3554/3554 [==============================] - 9s 3ms/step\nSummary: Loss over the test dataset: 0.0281, Accuracy: 0.9924\n","name":"stdout"}],"source":"learning_rate_reduction = ReduceLROnPlateau(monitor='val_acc', \r\n                                            patience=3, \r\n                                            verbose=1, \r\n                                            factor=0.5, \r\n                                            min_lr=0.00001)\r\ndatagen = ImageDataGenerator()\r\ndatagen.fit(X_train)\r\n# Fit the model\r\nhistory = model.fit_generator(datagen.flow(X_train,y_train, batch_size=batch_size),\r\n                              epochs = epochs, validation_data = (X_test,y_test),\r\n                              verbose = 2, steps_per_epoch=X_train.shape[0] // batch_size,\r\n                              callbacks=[learning_rate_reduction])\r\nevaluation = model.evaluate(X_test, y_test, verbose=1)\r\nprint('Summary: Loss over the test dataset: %.4f, Accuracy: %.4f' % (evaluation[0], evaluation[1]))","execution_count":20},{"metadata":{"id":"7A259C42F9E14617ADCA88E0EDEB3971","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"### 2. 学会写\n识别是学习的第一步，下面我们GAN起来，看看AI书写的藏文数字长啥样！"},{"metadata":{"id":"AA7697A060DE41729B09FCA8887A9F16","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"自2014年Ian Goodfellow提出生成对抗网络(GAN)的概念后,生成对抗网络变成为了学术界的一个火热的研究热点,Yann LeCun更是称之为”过去十年间机器学习领域最让人激动的点子”.生成对抗网络的简单思路如下,训练一个生成器(Generator,简称G),从随机噪声或者潜在变量(Latent Variable)中生成逼真的的样本,同时训练一个鉴别器(Discriminator,简称D)来鉴别真实数据和生成数据,两者同时训练,直到达到一个纳什均衡,生成器生成的数据与真实样本无差别,鉴别器也无法正确的区分生成数据和真实数据.GAN的结构如图1所示.\n\n![Image Name](https://cdn.kesci.com/upload/image/pj9uwejw6c.png?imageView2/0/w/960/h/960)\n"},{"metadata":{"id":"9994C06D5B984FFB80D3FB7587179313","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"github有一个库使用Keras对21种GAN及其变体作了精简的实现，我们略作修改，看看效果如何\n> https://github.com/eriklindernoren/Keras-GAN"},{"metadata":{"id":"C0BD7C1AAB6E442A8273C098AC44E65E","collapsed":false,"scrolled":false,"mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"#### 2.1 CGAN "},{"metadata":{"id":"427221D85C0F471F8903D465444BCE83","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[],"source":"from keras.layers import Input, Dense, Reshape, Flatten, Dropout, multiply\r\nfrom keras.layers import BatchNormalization, Activation, Embedding, ZeroPadding2D\r\nfrom keras.layers.advanced_activations import LeakyReLU\r\nfrom keras.layers.convolutional import UpSampling2D, Conv2D\r\nfrom keras.models import Sequential, Model\r\nfrom keras.optimizers import Adam\r\nimport matplotlib.pyplot as plt\r\nimport numpy as np\r\nclass CGAN():\r\n    def __init__(self):\r\n        # Input shape\r\n        self.img_rows = 28\r\n        self.img_cols = 28\r\n        self.channels = 1\r\n        self.img_shape = (self.img_rows, self.img_cols, self.channels)\r\n        self.num_classes = 10\r\n        self.latent_dim = 100\r\n\r\n        optimizer = Adam(0.0002, 0.5)\r\n\r\n        # Build and compile the discriminator\r\n        self.discriminator = self.build_discriminator()\r\n        self.discriminator.compile(loss=['binary_crossentropy'],\r\n            optimizer=optimizer,\r\n            metrics=['accuracy'])\r\n\r\n        # Build the generator\r\n        self.generator = self.build_generator()\r\n\r\n        # The generator takes noise and the target label as input\r\n        # and generates the corresponding digit of that label\r\n        noise = Input(shape=(self.latent_dim,))\r\n        label = Input(shape=(1,))\r\n        img = self.generator([noise, label])\r\n\r\n        # For the combined model we will only train the generator\r\n        self.discriminator.trainable = False\r\n\r\n        # The discriminator takes generated image as input and determines validity\r\n        # and the label of that image\r\n        valid = self.discriminator([img, label])\r\n\r\n        # The combined model  (stacked generator and discriminator)\r\n        # Trains generator to fool discriminator\r\n        self.combined = Model([noise, label], valid)\r\n        self.combined.compile(loss=['binary_crossentropy'],\r\n            optimizer=optimizer)\r\n\r\n    def build_generator(self):\r\n\r\n        model = Sequential()\r\n\r\n        model.add(Dense(256, input_dim=self.latent_dim))\r\n        model.add(LeakyReLU(alpha=0.2))\r\n        model.add(BatchNormalization(momentum=0.8))\r\n        model.add(Dense(512))\r\n        model.add(LeakyReLU(alpha=0.2))\r\n        model.add(BatchNormalization(momentum=0.8))\r\n        model.add(Dense(1024))\r\n        model.add(LeakyReLU(alpha=0.2))\r\n        model.add(BatchNormalization(momentum=0.8))\r\n        model.add(Dense(np.prod(self.img_shape), activation='tanh'))\r\n        model.add(Reshape(self.img_shape))\r\n\r\n        model.summary()\r\n\r\n        noise = Input(shape=(self.latent_dim,))\r\n        label = Input(shape=(1,), dtype='int32')\r\n        label_embedding = Flatten()(Embedding(self.num_classes, self.latent_dim)(label))\r\n\r\n        model_input = multiply([noise, label_embedding])\r\n        img = model(model_input)\r\n\r\n        return Model([noise, label], img)\r\n\r\n    def build_discriminator(self):\r\n\r\n        model = Sequential()\r\n\r\n        model.add(Dense(512, input_dim=np.prod(self.img_shape)))\r\n        model.add(LeakyReLU(alpha=0.2))\r\n        model.add(Dense(512))\r\n        model.add(LeakyReLU(alpha=0.2))\r\n        model.add(Dropout(0.4))\r\n        model.add(Dense(512))\r\n        model.add(LeakyReLU(alpha=0.2))\r\n        model.add(Dropout(0.4))\r\n        model.add(Dense(1, activation='sigmoid'))\r\n        model.summary()\r\n\r\n        img = Input(shape=self.img_shape)\r\n        label = Input(shape=(1,), dtype='int32')\r\n\r\n        label_embedding = Flatten()(Embedding(self.num_classes, np.prod(self.img_shape))(label))\r\n        flat_img = Flatten()(img)\r\n\r\n        model_input = multiply([flat_img, label_embedding])\r\n\r\n        validity = model(model_input)\r\n\r\n        return Model([img, label], validity)\r\n\r\n    def train(self, epochs, batch_size=128, sample_interval=50):\r\n\r\n        # Load the dataset\r\n        #(X_train, y_train), (_, _) = mnist.load_data()\r\n        data = np.load('/home/kesci/input/TibetanMNIST5610/TibetanMNIST.npz')\r\n\r\n        # Configure input\r\n        #X_train = (X_train.astype(np.float32) - 127.5) / 127.5\r\n        #X_train = np.expand_dims(X_train, axis=3)\r\n        #y_train = y_train.reshape(-1, 1)\r\n        X, y = data['image'], data['label']  # (17768, 28, 28)\r\n        X_train = X.reshape(X.shape[0], 28,28,1).astype('float32') / 255\r\n        y_train = y.reshape(-1, 1)\r\n        # Adversarial ground truths\r\n        valid = np.ones((batch_size, 1))\r\n        fake = np.zeros((batch_size, 1))\r\n\r\n        for epoch in range(epochs):\r\n\r\n            # ---------------------\r\n            #  Train Discriminator\r\n            # ---------------------\r\n\r\n            # Select a random half batch of images\r\n            idx = np.random.randint(0, X_train.shape[0], batch_size)\r\n            imgs, labels = X_train[idx], y_train[idx]\r\n\r\n            # Sample noise as generator input\r\n            noise = np.random.normal(0, 1, (batch_size, 100))\r\n\r\n            # Generate a half batch of new images\r\n            gen_imgs = self.generator.predict([noise, labels])\r\n\r\n            # Train the discriminator\r\n            d_loss_real = self.discriminator.train_on_batch([imgs, labels], valid)\r\n            d_loss_fake = self.discriminator.train_on_batch([gen_imgs, labels], fake)\r\n            d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)\r\n\r\n            # ---------------------\r\n            #  Train Generator\r\n            # ---------------------\r\n\r\n            # Condition on labels\r\n            sampled_labels = np.random.randint(0, 10, batch_size).reshape(-1, 1)\r\n\r\n            # Train the generator\r\n            g_loss = self.combined.train_on_batch([noise, sampled_labels], valid)\r\n            if epoch % 200 ==0:\r\n                # Plot the progress\r\n                print (\"%d [D loss: %f, acc.: %.2f%%] [G loss: %f]\" % (epoch, d_loss[0], 100*d_loss[1], g_loss))\r\n\r\n            # If at save interval => save generated image samples\r\n            if epoch % sample_interval == 0:\r\n                self.sample_images(epoch)\r\n\r\n    def sample_images(self, epoch, is_save=False):\r\n        r, c = 2, 5\r\n        noise = np.random.normal(0, 1, (r * c, 100))\r\n        sampled_labels = np.arange(0, 10).reshape(-1, 1)\r\n\r\n        gen_imgs = self.generator.predict([noise, sampled_labels])\r\n\r\n        # Rescale images 0 - 1\r\n        gen_imgs = 0.5 * gen_imgs + 0.5\r\n\r\n        fig, axs = plt.subplots(r, c)\r\n        cnt = 0\r\n        for i in range(r):\r\n            for j in range(c):\r\n                axs[i,j].imshow(gen_imgs[cnt,:,:,0], cmap='gray')\r\n                axs[i,j].set_title(\"Digit: %d\" % sampled_labels[cnt])\r\n                axs[i,j].axis('off')\r\n                cnt += 1\r\n        if is_save == True:\r\n            fig.savefig(\"images2/%d.png\" % epoch)\r\n        else:\r\n            plt.show()\r\n        plt.close()\r\n\r\n","execution_count":14},{"metadata":{"id":"CC3A8BCFC06848508E77165B941D19FC","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"stream","text":"_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n=================================================================\ndense_21 (Dense)             (None, 512)               401920    \n_________________________________________________________________\nleaky_re_lu_13 (LeakyReLU)   (None, 512)               0         \n_________________________________________________________________\ndense_22 (Dense)             (None, 512)               262656    \n_________________________________________________________________\nleaky_re_lu_14 (LeakyReLU)   (None, 512)               0         \n_________________________________________________________________\ndropout_7 (Dropout)          (None, 512)               0         \n_________________________________________________________________\ndense_23 (Dense)             (None, 512)               262656    \n_________________________________________________________________\nleaky_re_lu_15 (LeakyReLU)   (None, 512)               0         \n_________________________________________________________________\ndropout_8 (Dropout)          (None, 512)               0         \n_________________________________________________________________\ndense_24 (Dense)             (None, 1)                 513       \n=================================================================\nTotal params: 927,745\nTrainable params: 927,745\nNon-trainable params: 0\n_________________________________________________________________\n_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n=================================================================\ndense_25 (Dense)             (None, 256)               25856     \n_________________________________________________________________\nleaky_re_lu_16 (LeakyReLU)   (None, 256)               0         \n_________________________________________________________________\nbatch_normalization_7 (Batch (None, 256)               1024      \n_________________________________________________________________\ndense_26 (Dense)             (None, 512)               131584    \n_________________________________________________________________\nleaky_re_lu_17 (LeakyReLU)   (None, 512)               0         \n_________________________________________________________________\nbatch_normalization_8 (Batch (None, 512)               2048      \n_________________________________________________________________\ndense_27 (Dense)             (None, 1024)              525312    \n_________________________________________________________________\nleaky_re_lu_18 (LeakyReLU)   (None, 1024)              0         \n_________________________________________________________________\nbatch_normalization_9 (Batch (None, 1024)              4096      \n_________________________________________________________________\ndense_28 (Dense)             (None, 784)               803600    \n_________________________________________________________________\nreshape_3 (Reshape)          (None, 28, 28, 1)         0         \n=================================================================\nTotal params: 1,493,520\nTrainable params: 1,489,936\nNon-trainable params: 3,584\n_________________________________________________________________\n","name":"stdout"},{"output_type":"stream","text":"/opt/conda/lib/python3.5/site-packages/keras/engine/training.py:973: UserWarning: Discrepancy between trainable weights and collected trainable weights, did you set `model.trainable` without calling `model.compile` after ?\n  'Discrepancy between trainable weights and collected trainable'\n","name":"stderr"},{"output_type":"stream","text":"0 [D loss: 0.693387, acc.: 37.89%] [G loss: 0.676710]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9fuat85c.png\">"}},{"output_type":"stream","text":"200 [D loss: 0.515204, acc.: 77.73%] [G loss: 1.695024]\n400 [D loss: 0.671725, acc.: 58.20%] [G loss: 1.061109]\n600 [D loss: 0.604611, acc.: 66.41%] [G loss: 1.114438]\n800 [D loss: 0.509307, acc.: 78.12%] [G loss: 1.131666]\n1000 [D loss: 0.599375, acc.: 66.02%] [G loss: 1.139028]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9fy049hm.png\">"}},{"output_type":"stream","text":"1200 [D loss: 0.471615, acc.: 79.69%] [G loss: 1.905819]\n1400 [D loss: 0.267580, acc.: 89.84%] [G loss: 2.607601]\n1600 [D loss: 0.355972, acc.: 86.72%] [G loss: 2.823330]\n1800 [D loss: 0.370189, acc.: 84.38%] [G loss: 2.798307]\n2000 [D loss: 0.157132, acc.: 94.92%] [G loss: 3.246812]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9g1q8op1.png\">"}},{"output_type":"stream","text":"2200 [D loss: 0.304391, acc.: 90.62%] [G loss: 2.948885]\n2400 [D loss: 0.222136, acc.: 91.80%] [G loss: 2.966596]\n2600 [D loss: 0.174552, acc.: 93.75%] [G loss: 2.849100]\n2800 [D loss: 0.158594, acc.: 94.92%] [G loss: 2.754582]\n3000 [D loss: 0.146312, acc.: 96.09%] [G loss: 2.541392]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9g5g9yt.png\">"}},{"output_type":"stream","text":"3200 [D loss: 0.248664, acc.: 90.62%] [G loss: 3.830889]\n3400 [D loss: 0.268707, acc.: 88.67%] [G loss: 3.673885]\n3600 [D loss: 0.207588, acc.: 93.75%] [G loss: 3.732453]\n3800 [D loss: 0.124569, acc.: 96.48%] [G loss: 2.896123]\n4000 [D loss: 0.126609, acc.: 96.48%] [G loss: 3.584754]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9g97v04g.png\">"}},{"output_type":"stream","text":"4200 [D loss: 0.084122, acc.: 97.27%] [G loss: 3.604610]\n4400 [D loss: 0.103992, acc.: 96.88%] [G loss: 3.801122]\n4600 [D loss: 0.187688, acc.: 92.19%] [G loss: 4.234042]\n4800 [D loss: 0.088149, acc.: 97.27%] [G loss: 2.500763]\n5000 [D loss: 0.075576, acc.: 98.05%] [G loss: 3.907623]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9gcyzbp0.png\">"}},{"output_type":"stream","text":"5200 [D loss: 0.043257, acc.: 98.83%] [G loss: 3.416204]\n5400 [D loss: 0.071402, acc.: 98.05%] [G loss: 3.529195]\n5600 [D loss: 0.059610, acc.: 98.83%] [G loss: 2.267229]\n5800 [D loss: 0.054817, acc.: 98.05%] [G loss: 3.154628]\n6000 [D loss: 0.021207, acc.: 99.61%] [G loss: 3.659095]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9ggol4h2.png\">"}},{"output_type":"stream","text":"6200 [D loss: 0.035089, acc.: 99.61%] [G loss: 4.746068]\n6400 [D loss: 0.065372, acc.: 98.83%] [G loss: 4.548535]\n6600 [D loss: 0.052366, acc.: 98.83%] [G loss: 4.802631]\n6800 [D loss: 0.014006, acc.: 100.00%] [G loss: 1.198764]\n7000 [D loss: 0.050257, acc.: 99.22%] [G loss: 3.755497]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9gkf9460.png\">"}},{"output_type":"stream","text":"7200 [D loss: 0.030736, acc.: 99.22%] [G loss: 3.670065]\n7400 [D loss: 0.035320, acc.: 99.22%] [G loss: 3.552088]\n7600 [D loss: 0.010704, acc.: 99.61%] [G loss: 4.112602]\n7800 [D loss: 0.065062, acc.: 98.83%] [G loss: 4.666056]\n8000 [D loss: 0.030834, acc.: 99.22%] [G loss: 2.369893]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9go56ur8.png\">"}},{"output_type":"stream","text":"8200 [D loss: 0.082850, acc.: 98.44%] [G loss: 5.812540]\n8400 [D loss: 0.069831, acc.: 98.83%] [G loss: 2.156770]\n8600 [D loss: 0.017987, acc.: 99.61%] [G loss: 4.393001]\n8800 [D loss: 0.003687, acc.: 100.00%] [G loss: 3.304473]\n9000 [D loss: 0.023193, acc.: 99.61%] [G loss: 5.492148]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9grwt4hj.png\">"}},{"output_type":"stream","text":"9200 [D loss: 0.011176, acc.: 99.61%] [G loss: 4.429143]\n9400 [D loss: 0.034659, acc.: 99.22%] [G loss: 4.587958]\n9600 [D loss: 0.052863, acc.: 98.83%] [G loss: 2.465764]\n9800 [D loss: 0.057966, acc.: 98.83%] [G loss: 5.312491]\n10000 [D loss: 0.009020, acc.: 100.00%] [G loss: 2.210274]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9gvmet7.png\">"}},{"output_type":"stream","text":"10200 [D loss: 0.024724, acc.: 99.22%] [G loss: 2.497583]\n10400 [D loss: 0.004938, acc.: 100.00%] [G loss: 2.777097]\n10600 [D loss: 0.029476, acc.: 98.44%] [G loss: 3.696933]\n10800 [D loss: 0.093290, acc.: 98.05%] [G loss: 6.178957]\n11000 [D loss: 0.008721, acc.: 100.00%] [G loss: 4.481911]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9gzcl5sg.png\">"}},{"output_type":"stream","text":"11200 [D loss: 0.029163, acc.: 99.61%] [G loss: 4.058701]\n11400 [D loss: 0.042823, acc.: 98.83%] [G loss: 4.776906]\n11600 [D loss: 0.020817, acc.: 99.61%] [G loss: 5.260248]\n11800 [D loss: 0.004055, acc.: 100.00%] [G loss: 3.998247]\n12000 [D loss: 0.036383, acc.: 98.83%] [G loss: 3.967544]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9h329y59.png\">"}},{"output_type":"stream","text":"12200 [D loss: 0.003826, acc.: 100.00%] [G loss: 1.289107]\n12400 [D loss: 0.009029, acc.: 99.61%] [G loss: 1.727201]\n12600 [D loss: 0.033706, acc.: 99.22%] [G loss: 3.783878]\n12800 [D loss: 0.052979, acc.: 98.83%] [G loss: 3.671581]\n13000 [D loss: 0.020131, acc.: 99.61%] [G loss: 3.563331]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9h6tzvm7.png\">"}},{"output_type":"stream","text":"13200 [D loss: 0.028371, acc.: 98.83%] [G loss: 3.731534]\n13400 [D loss: 0.000424, acc.: 100.00%] [G loss: 3.944247]\n13600 [D loss: 0.039124, acc.: 99.22%] [G loss: 2.264724]\n13800 [D loss: 0.038282, acc.: 98.44%] [G loss: 1.750422]\n14000 [D loss: 0.005562, acc.: 100.00%] [G loss: 0.849124]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9haifhnk.png\">"}},{"output_type":"stream","text":"14200 [D loss: 0.030477, acc.: 98.83%] [G loss: 6.087772]\n14400 [D loss: 0.022935, acc.: 99.22%] [G loss: 4.263762]\n14600 [D loss: 0.003880, acc.: 100.00%] [G loss: 3.849062]\n14800 [D loss: 0.007759, acc.: 99.61%] [G loss: 1.155128]\n15000 [D loss: 0.030871, acc.: 98.44%] [G loss: 2.271917]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9he8elqd.png\">"}},{"output_type":"stream","text":"15200 [D loss: 0.015683, acc.: 99.61%] [G loss: 6.847127]\n15400 [D loss: 0.094834, acc.: 96.09%] [G loss: 2.650849]\n15600 [D loss: 0.005842, acc.: 99.61%] [G loss: 2.332110]\n15800 [D loss: 0.007992, acc.: 99.22%] [G loss: 2.717348]\n16000 [D loss: 0.001879, acc.: 100.00%] [G loss: 0.921542]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9hhy1vjf.png\">"}},{"output_type":"stream","text":"16200 [D loss: 0.010797, acc.: 100.00%] [G loss: 0.909980]\n16400 [D loss: 0.009473, acc.: 100.00%] [G loss: 1.912880]\n16600 [D loss: 0.013174, acc.: 99.22%] [G loss: 2.942506]\n16800 [D loss: 0.005672, acc.: 99.61%] [G loss: 3.875148]\n17000 [D loss: 0.030165, acc.: 99.22%] [G loss: 11.210460]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9hloy52s.png\">"}},{"output_type":"stream","text":"17200 [D loss: 0.009672, acc.: 99.61%] [G loss: 3.650989]\n17400 [D loss: 0.032112, acc.: 98.83%] [G loss: 1.075614]\n17600 [D loss: 0.006818, acc.: 100.00%] [G loss: 3.086496]\n17800 [D loss: 0.002379, acc.: 100.00%] [G loss: 7.016960]\n18000 [D loss: 0.017629, acc.: 99.61%] [G loss: 1.718827]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9hpee56o.png\">"}},{"output_type":"stream","text":"18200 [D loss: 0.001037, acc.: 100.00%] [G loss: 0.515290]\n18400 [D loss: 0.013190, acc.: 99.61%] [G loss: 5.891022]\n18600 [D loss: 0.028058, acc.: 98.83%] [G loss: 8.746876]\n18800 [D loss: 0.017850, acc.: 99.61%] [G loss: 6.442552]\n19000 [D loss: 0.015755, acc.: 99.22%] [G loss: 3.640223]\n","name":"stdout"},{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 10 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/CC3A8BCFC06848508E77165B941D19FC/pj9ht4t1jc.png\">"}},{"output_type":"stream","text":"19200 [D loss: 0.017193, acc.: 98.83%] [G loss: 8.695421]\n19400 [D loss: 0.020689, acc.: 99.22%] [G loss: 11.479371]\n19600 [D loss: 0.041356, acc.: 98.83%] [G loss: 4.472672]\n19800 [D loss: 0.030090, acc.: 98.44%] [G loss: 2.904464]\n","name":"stdout"}],"source":"batch_size = 128\r\nepochs = 20000\r\ncgan = CGAN()\r\ncgan.train(epochs=epochs, batch_size=batch_size, sample_interval=1000)#sample_interval 1000epoch输出一次图像\r\n","execution_count":15},{"metadata":{"id":"7280855BAA5D49C6985C09A1F074A9D5","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"可以看见效果不算很好，由于计算效率较低，不再加大epoch实验了"},{"metadata":{"id":"31610AB719EE4E94A8D2542FEAB4C503","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"#### 2.2 其他GAN \n我们可以在上面的库中使用其他GAN进行实验，WGAN效果会比CGAN好一些\n"},{"metadata":{"id":"23E6F6739F6E44798F1F5F01FCAE63D8","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"### 3 搞点创作\n\n#### 3.1 变成彩色\n手动把灰度图变成彩色图"},{"metadata":{"id":"533C45B3B1ED46648050AF45D265B00E","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[],"source":"# 随机抽取风格图片的部分像素作为原图的背景\r\ndef coloful_batch(X_train,y_train, batch_size, style,change_colors=False):\r\n    # Select random batch (WxHxC)\r\n    idx = np.random.choice(X_train.shape[0], batch_size)\r\n    labels = y_train[idx]\r\n    batch_raw = X_train[idx, :, :, 0].reshape((batch_size, 28, 28, 1))\r\n    \r\n    # Resize\r\n    batch_resized = np.asarray([scipy.ndimage.zoom(image, (2.3, 2.3, 1), order=1) for image in batch_raw])\r\n    \r\n    # Extend to RGB\r\n    batch_rgb = np.concatenate([batch_resized, batch_resized, batch_resized], axis=3)\r\n    \r\n    # Make binary\r\n    batch_binary = (batch_rgb > 0.5)\r\n    \r\n    batch = np.zeros((batch_size, 64, 64, 3))\r\n    \r\n    for i in range(batch_size):\r\n        # Take a random crop of the style image (background)\r\n        x_c = np.random.randint(0, style.size[0] - 64)\r\n        y_c = np.random.randint(0, style.size[1] - 64)\r\n        image = style.crop((x_c, y_c, x_c + 64, y_c + 64))\r\n        image = np.asarray(image) / 255.0\r\n\r\n        if change_colors:\r\n            # Change color distribution\r\n            for j in range(3):\r\n                image[:, :, j] = (image[:, :, j] + np.random.uniform(0, 1)) / 2.0\r\n\r\n        # Invert the colors at the location of the number\r\n        image[:, :, 0:3][batch_binary[i]] = 1 - image[:, :, 0:3][batch_binary[i]]\r\n        #image[batch_binary[i]] = 1 - image[batch_binary[i]]\r\n        \r\n        batch[i] = image[:, :, 0:3]\r\n    return batch,labels","execution_count":36},{"metadata":{"id":"5AB1A5DB27164CB9B80A08748BB3A8F9","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 432x288 with 1 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/5AB1A5DB27164CB9B80A08748BB3A8F9/pj9tsofakm.png\">"}}],"source":"import scipy.ndimage\r\nimport numpy as np\r\nimport matplotlib.pyplot as plt\r\nimport PIL\r\nstyle_img = PIL.Image.open('/home/kesci/work/style.jpg') #使用我的桌面壁纸来做风格图片吧~\r\nplt.imshow(style)\r\nplt.axis('off')\r\nplt.show()\r\n","execution_count":29},{"metadata":{"id":"F62F94A2983F45D197ABB1523A4E3B96","collapsed":false,"scrolled":false,"hide_input":false},"cell_type":"code","outputs":[{"output_type":"display_data","metadata":{"needs_background":"light"},"data":{"text/plain":"<Figure size 1080x216 with 20 Axes>","text/html":"<img src=\"https://cdn.kesci.com/rt_upload/F62F94A2983F45D197ABB1523A4E3B96/pj9uj8cs9b.png\">"}}],"source":"data = np.load('/home/kesci/input/TibetanMNIST5610/TibetanMNIST.npz')\r\nX, y = data['image'], data['label']  \r\nX_train = X.reshape(X.shape[0], 28,28,1).astype('float32') / 255\r\ny_train = y.reshape(-1, 1)\r\n#随机查看20张生成的图片\r\nimgs,labels = coloful_batch(X_train,y_train, batch_size = 20, style = style_img)\r\nfig = plt.figure(figsize=(15,3))\r\nfor i in range(20):\r\n    ax = fig.add_subplot(2, 20 // 2, i+1)\r\n    ax.imshow(imgs[i])\r\n    ax.set_title(\"%d\" % labels[i])\r\n    ax.axis('off')\r\n    \r\nplt.tight_layout()\r\nplt.show()\r\n","execution_count":49},{"metadata":{"id":"C25B682ED2EF4CB4932C85BA53A87EFC","collapsed":false,"scrolled":false,"mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"用这种方法得到彩图之后，我们还可以做一些有趣的事情\n- 我们可以再训练一次GAN，这样比起直接拿灰度图去生成效果会好看点。\n- 我们可以再对其进行图像识别，由于揉入了风格图片的像素，相当于提高了识别的难度。"},{"metadata":{"id":"6622D1BF7DE64F37ABB191A10D6670A8","mdEditEnable":false,"hide_input":false},"cell_type":"markdown","source":"#### 3.2 风格迁移\n我们来进行真正的风格迁移吧\n> 待续"}],"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","file_extension":".py","nbconvert_exporter":"python","version":"3.5.5"}},"nbformat":4,"nbformat_minor":0}