前回は、”Training The Dobble Challenge”に従って、 Dobble の CNN モデルをトレーニングして見ようということで、畳込み(アクティベーションは relu)+マックス・プーリングが 3 層とドロップアウト、全結合層、アクティベーション(relu)、全結合層、アクティベーション(softmax)の CNN を学習してテストした。今回は、”Training The Dobble Challenge”の残りをやってみよう。
自分の Web カメラで Dubble を認識してみよう。
pip3 install imutils
dobble_detect_live.py を編集する。
#input_video = 0 # laptop camera
input_video = 1 # USB webcam
python dobble_detect_live.py
を実行したが、カメラ画像は表示される気配はするが、すぐに閉じてしまった。
データの増強をしよう。
今までのデータは、 dobble_deck01_cards_57 〜 dobble_deck10_cards_55 のデータ・ディレクトリの中に、 57 〜 55 個のディレクトリがあって、それぞれ 1 個ずつの画像が入っている。これを save_augmented_images.py で 100 倍に増やす。
画像の変換方法を”Training The Dobble Challenge”から引用する。
Rotation - has already been done on line 177
Horizontal and vertical flip - not good for our dataset (also, currently doing on line 179?)
featurewise_center
samplewise_center
featurewise_std_normalization
zca_epsilon
zca_whitening
shear_range
channel_shift_range
save_augmented_images.py は 1 つのデータ・ディレクトリを処理するので、書き換えながら画像を増やす。
dobble_deck07_cards_55 の内容を示す。
01 のディレクトリの中は、 1 個の画像だ。
python save_augmented_images.py を実行した。
dobble_deck07_cards_55-augmented ディレクトリが生成された。
01 ディレクトリを見ると、 100 個の画像が生成されている。
dobble_deck01_cards_57 〜 dobble_deck10_cards_55 のデータ・ディレクトリについて save_augmented_images.py を実行して、画像を増やした。
次に、 dobble_deck01_cards_57-augmented 〜 dobble_deck10_cards_55-augmented を使用するように、 dobble_tutorial.py を修正して、実行したが、 32 GB 搭載しているメモリを超えて使用してしまった。
そこで半分だけ augmented を使用するように dobble_tutorial.py を修正した。
これで、 dobble_tutorial.py を実行したところ、何とか学習できそうだ。
10 時間程度かかったが、学習が終了した。
精度は 91.666 % だった。
python dobble_test.py を実行した。
こちらのほうが精度が良い。 68.034 % 程度でているが、まだ物足りない。
やはり、全部 augmented 版を使用して学習できると良さそうだが、 64 GB メモリが必要だな。。。
dobble_tutorial.py を実行した時のログを貼っておく。
(base) masaaki@masaaki-H110M4-M01:/media/masaaki/Ubuntu_Disk/AI/dobble_buddy$ python dobble_tutorial.py
PARAMETERS:
Normalized shape of images : 224 x 224
Card Decks : 10 ['dobble_deck01_cards_57-augmented', 'dobble_deck02_cards_55', 'dobble_deck03_cards_55-augmented', 'dobble_deck04_cards_55', 'dobble_deck05_cards_55-augmented', 'dobble_deck06_cards_55', 'dobble_deck07_cards_55-augmented', 'dobble_deck08_cards_55', 'dobble_deck09_cards_55-augmented', 'dobble_deck10_cards_55']
TRAINING/VALIDATION DATA SETS:
Shape of training data (X) is : (22465, 224, 224, 3)
Shape of training data (y) is : (22465,)
Shape of validation data (X) is : (5617, 224, 224, 3)
Shape of validation data (y) is : (5617,)
2021-07-10 17:53:35.693689: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2021-07-10 17:53:36.012876: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3199980000 Hz
2021-07-10 17:53:36.057387: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559761433360 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-07-10 17:53:36.057436: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2021-07-10 17:53:36.281905: I tensorflow/core/common_runtime/process_util.cc:147] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
MODEL SUMMARY:
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 222, 222, 32) 896
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 111, 111, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 109, 109, 64) 18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 54, 54, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 52, 52, 128) 73856
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 26, 26, 128) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 24, 24, 128) 147584
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 12, 12, 128) 0
_________________________________________________________________
dropout (Dropout) (None, 12, 12, 128) 0
_________________________________________________________________
flatten (Flatten) (None, 18432) 0
_________________________________________________________________
dense (Dense) (None, 512) 9437696
_________________________________________________________________
activation (Activation) (None, 512) 0
_________________________________________________________________
dense_1 (Dense) (None, 58) 29754
_________________________________________________________________
activation_1 (Activation) (None, 58) 0
=================================================================
Total params: 9,708,282
Trainable params: 9,708,282
Non-trainable params: 0
_________________________________________________________________
TRAIN MODEL:
Epoch 1/59
702/702 [==============================] - 649s 925ms/step - loss: 3.3797 - val_loss: 2.4933
Epoch 2/59
702/702 [==============================] - 619s 881ms/step - loss: 1.8767 - val_loss: 1.2642
Epoch 3/59
702/702 [==============================] - 620s 883ms/step - loss: 0.9784 - val_loss: 0.6677
Epoch 4/59
702/702 [==============================] - 623s 888ms/step - loss: 0.5961 - val_loss: 0.4000
Epoch 5/59
702/702 [==============================] - 619s 882ms/step - loss: 0.5095 - val_loss: 0.5559
Epoch 6/59
702/702 [==============================] - 621s 884ms/step - loss: 0.4237 - val_loss: 0.2355
Epoch 7/59
702/702 [==============================] - 618s 880ms/step - loss: 0.2760 - val_loss: 0.2104
Epoch 8/59
702/702 [==============================] - 618s 880ms/step - loss: 0.2350 - val_loss: 0.1733
Epoch 9/59
702/702 [==============================] - 617s 879ms/step - loss: 0.2408 - val_loss: 0.1829
Epoch 10/59
702/702 [==============================] - 614s 875ms/step - loss: 0.2283 - val_loss: 0.3288
Epoch 11/59
702/702 [==============================] - 613s 874ms/step - loss: 0.2097 - val_loss: 0.1317
Epoch 12/59
702/702 [==============================] - 616s 877ms/step - loss: 0.1609 - val_loss: 0.1008
Epoch 13/59
702/702 [==============================] - 615s 876ms/step - loss: 0.1632 - val_loss: 0.1554
Epoch 14/59
702/702 [==============================] - 627s 894ms/step - loss: 0.1713 - val_loss: 0.0993
Epoch 15/59
702/702 [==============================] - 622s 886ms/step - loss: 0.1276 - val_loss: 0.2646
Epoch 16/59
702/702 [==============================] - 617s 878ms/step - loss: 0.1852 - val_loss: 0.1097
Epoch 17/59
702/702 [==============================] - 619s 882ms/step - loss: 0.1387 - val_loss: 0.1637
Epoch 18/59
702/702 [==============================] - 617s 878ms/step - loss: 0.1229 - val_loss: 0.1576
Epoch 19/59
702/702 [==============================] - 616s 877ms/step - loss: 0.1321 - val_loss: 0.1307
Epoch 20/59
702/702 [==============================] - 617s 879ms/step - loss: 0.1246 - val_loss: 0.0790
Epoch 21/59
702/702 [==============================] - 614s 875ms/step - loss: 0.1165 - val_loss: 0.0906
Epoch 22/59
702/702 [==============================] - 614s 875ms/step - loss: 0.1205 - val_loss: 0.1210
Epoch 23/59
702/702 [==============================] - 614s 875ms/step - loss: 0.1106 - val_loss: 0.0839
Epoch 24/59
702/702 [==============================] - 614s 875ms/step - loss: 0.0977 - val_loss: 0.0636
Epoch 25/59
702/702 [==============================] - 616s 877ms/step - loss: 0.1171 - val_loss: 0.1314
Epoch 26/59
702/702 [==============================] - 634s 903ms/step - loss: 0.1127 - val_loss: 0.0733
Epoch 27/59
702/702 [==============================] - 635s 904ms/step - loss: 0.1047 - val_loss: 0.0715
Epoch 28/59
702/702 [==============================] - 617s 880ms/step - loss: 0.1182 - val_loss: 0.0712
Epoch 29/59
702/702 [==============================] - 635s 904ms/step - loss: 0.0948 - val_loss: 0.0857
Epoch 30/59
702/702 [==============================] - 618s 881ms/step - loss: 0.1238 - val_loss: 0.0927
Epoch 31/59
702/702 [==============================] - 617s 878ms/step - loss: 0.0966 - val_loss: 0.0701
Epoch 32/59
702/702 [==============================] - 617s 879ms/step - loss: 0.0970 - val_loss: 0.0876
Epoch 33/59
702/702 [==============================] - 617s 880ms/step - loss: 0.1322 - val_loss: 0.0762
Epoch 34/59
702/702 [==============================] - 617s 878ms/step - loss: 0.0835 - val_loss: 0.0815
Epoch 35/59
702/702 [==============================] - 617s 879ms/step - loss: 0.1001 - val_loss: 0.0716
Epoch 36/59
702/702 [==============================] - 616s 878ms/step - loss: 0.1000 - val_loss: 0.0888
Epoch 37/59
702/702 [==============================] - 618s 880ms/step - loss: 0.1183 - val_loss: 0.0640
Epoch 38/59
702/702 [==============================] - 618s 880ms/step - loss: 0.1058 - val_loss: 0.0871
Epoch 39/59
702/702 [==============================] - 617s 878ms/step - loss: 0.1179 - val_loss: 0.0759
Epoch 40/59
702/702 [==============================] - 617s 879ms/step - loss: 0.1015 - val_loss: 0.1003
Epoch 41/59
702/702 [==============================] - 617s 879ms/step - loss: 0.1082 - val_loss: 0.0679
Epoch 42/59
702/702 [==============================] - 617s 880ms/step - loss: 0.0968 - val_loss: 0.0693
Epoch 43/59
702/702 [==============================] - 616s 878ms/step - loss: 0.1161 - val_loss: 0.1391
Epoch 44/59
702/702 [==============================] - 616s 877ms/step - loss: 0.0920 - val_loss: 0.0960
Epoch 45/59
702/702 [==============================] - 618s 880ms/step - loss: 0.0870 - val_loss: 0.1002
Epoch 46/59
702/702 [==============================] - 616s 878ms/step - loss: 0.1016 - val_loss: 0.1532
Epoch 47/59
702/702 [==============================] - 614s 874ms/step - loss: 0.0802 - val_loss: 0.0577
Epoch 48/59
702/702 [==============================] - 615s 876ms/step - loss: 0.0731 - val_loss: 0.0723
Epoch 49/59
702/702 [==============================] - 614s 875ms/step - loss: 0.0901 - val_loss: 0.0977
Epoch 50/59
702/702 [==============================] - 616s 877ms/step - loss: 0.0947 - val_loss: 0.1096
Epoch 51/59
702/702 [==============================] - 615s 876ms/step - loss: 0.0747 - val_loss: 0.0812
Epoch 52/59
702/702 [==============================] - 615s 876ms/step - loss: 0.0922 - val_loss: 0.1069
Epoch 53/59
702/702 [==============================] - 615s 876ms/step - loss: 0.1031 - val_loss: 0.0655
Epoch 54/59
702/702 [==============================] - 615s 877ms/step - loss: 0.0882 - val_loss: 0.0971
Epoch 55/59
702/702 [==============================] - 615s 876ms/step - loss: 0.1150 - val_loss: 0.0693
Epoch 56/59
702/702 [==============================] - 614s 875ms/step - loss: 0.0963 - val_loss: 0.0707
Epoch 57/59
702/702 [==============================] - 615s 877ms/step - loss: 0.0993 - val_loss: 0.0638
Epoch 58/59
702/702 [==============================] - 616s 877ms/step - loss: 0.1011 - val_loss: 0.0984
Epoch 59/59
702/702 [==============================] - 615s 876ms/step - loss: 0.1028 - val_loss: 0.1870
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 222, 222, 32) 896
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 111, 111, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 109, 109, 64) 18496
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 54, 54, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 52, 52, 128) 73856
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 26, 26, 128) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 24, 24, 128) 147584
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 12, 12, 128) 0
_________________________________________________________________
dropout (Dropout) (None, 12, 12, 128) 0
_________________________________________________________________
flatten (Flatten) (None, 18432) 0
_________________________________________________________________
dense (Dense) (None, 512) 9437696
_________________________________________________________________
activation (Activation) (None, 512) 0
_________________________________________________________________
dense_1 (Dense) (None, 58) 29754
_________________________________________________________________
activation_1 (Activation) (None, 58) 0
=================================================================
Total params: 9,708,282
Trainable params: 9,708,282
Non-trainable params: 0
_________________________________________________________________
Shape of test data (X) is : (12, 224, 224, 3)
Shape of test data (y) is : (12, 58)
EVALUATE MODEL:
1/1 [==============================] - 0s 266us/step - loss: 23.8451
./dobble_dataset/dobble_test01_cards : Test Accuracy = 0.9166666666666666
dobble_test.py の実行した時のログを貼っておく。
(base) masaaki@masaaki-H110M4-M01:/media/masaaki/Ubuntu_Disk/AI/dobble_buddy$ python dobble_test.py
2021-07-11 05:01:50.044420: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2021-07-11 05:01:50.136887: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3199980000 Hz
2021-07-11 05:01:50.137389: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563dae8954c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-07-11 05:01:50.137433: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2021-07-11 05:01:50.138320: I tensorflow/core/common_runtime/process_util.cc:147] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
Shape of test data (X) is : (1267, 224, 224, 3)
Shape of test data (y) is : (1267, 58)
EVALUATE MODEL:
40/40 [==============================] - 9s 221ms/step - loss: 4.3736
./dobble_dataset/dobble_test02_cards : Test Accuracy = 0.6803472770323599
0.50% accuracy bound: 0.6716 - 0.6891
0.80% accuracy bound: 0.6636 - 0.6971
0.90% accuracy bound: 0.6589 - 0.7018
0.95% accuracy bound: 0.6547 - 0.7060
0.99% accuracy bound: 0.6465 - 0.7141