【昇腾CANN训练营第二期】【模型营】第二周作业2：使用MobileV2实现手写数字识别（MindSpore）进阶作业低阶实现

张小白引言：这个做法是个比较Low的做法，但是先贴出来吧。没有Low哪有High呢？

回顾下进阶题目：

进阶（完成进阶部分，将会获得额外加分：5分）：将MobileNetV2进行改写，实现数字识别功能，数据集MinistData，截图数字识别的结果。

实操过程：

正当张小白纠结怎么做的时候，@笨笨 同学提出了一个方法：将MNIST数据集转成图片，然后按照 猫狗识别 的方式训练就可以了。

我们来按照这种思路操作一下：

先把 MindSporePetClassification 工程目录下大的数据集移走，然后复制一份，改名为 HandwrittenNumeralRecognition。（手写数字识别）

然后加入 将MNIST数据集转换为 图片的python脚本：convert_mnist_dataset_to_images.py。内容如下（借鉴CHAO融梗XI了笨笨的脚本，当然自己是有改编的地方的。。编剧张小白如是说。。）

from __future__ import absolute_import, division, print_function, unicode_literals

# import matplotlib.pyplot as plt
from matplotlib import image
import tensorflow as tf
import numpy as np
import scipy.misc
import os

if os.path.exists('dataset/MNIST_data') is False:
    os.mkdir('dataset/MNIST_data')
if os.path.exists('dataset/MNIST_data/train') is False:
    os.mkdir('dataset/MNIST_data/train')
if os.path.exists('dataset/MNIST_data/eval') is False:
    os.mkdir('dataset/MNIST_data/eval')
if os.path.exists('dataset/HandWrittenImages') is False:
    os.mkdir('dataset/HandWrittenImages')
if os.path.exists('dataset/HandWrittenImages/train') is False:
    os.mkdir('dataset/HandWrittenImages/train')
if os.path.exists('dataset/HandWrittenImages/eval') is False:
    os.mkdir('dataset/HandWrittenImages/eval')

# 读取MNIST数据集。如果不存在会事先下载
# mnist = input_data.read_data_sets("MNIST_data/",one_hot=True)
mnist = tf.keras.datasets.mnist

(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images, test_images = train_images / 255.0, test_images / 255.0

print(train_labels)
print("ok")
# 把原始图片保存在MNIST_data/raw/文件夹下 如果没有这个文件夹  会自动创建
save_dir = 'dataset/MNIST_data/train/'
if os.path.exists(save_dir) is False:
    os.mkdir(save_dir)

# 把原始图片保存在MNIST_data/raw/文件夹下 如果没有这个文件夹  会自动创建
save_ev_dir = 'dataset/MNIST_data/eval/'
if os.path.exists(save_ev_dir) is False:
    os.mkdir(save_ev_dir)

# 保存图片
for i in range(len(train_images)):
    # 请注意，mnist.train.images[i, :]就表示第i张图片（序号从0开始）
    image_array = train_images[i, :]
    # TensorFlow中的MNIST图片是一个784维的向量，我们重新把它还原为28x28维的图像。
    image_array = image_array.reshape(28, 28)
    # 保存文件的格式为 mnist_train_0.jpg, mnist_train_1.jpg, ... ,mnist_train_19.jpg
    save_train_dir = 'dataset/HandWrittenImages/train/' + str(train_labels[i]) + '/'
    if os.path.exists(save_train_dir) is False:
        os.mkdir(save_train_dir)

    filename = save_train_dir + 'mnist_train_%d.jpg' % i
    # 将image_array保存为图片
    image.imsave(filename, image_array, cmap='gray')  # cmap常用于改变绘制风格，如黑白gray，翠绿色virdidis

# 保存图片
for i in range(len(test_images)):
    # 请注意，mnist.train.images[i, :]就表示第i张图片（序号从0开始）
    image_array = test_images[i, :]
    # TensorFlow中的MNIST图片是一个784维的向量，我们重新把它还原为28x28维的图像。
    image_array = image_array.reshape(28, 28)
    # 保存文件的格式为 mnist_train_0.jpg, mnist_train_1.jpg, ... ,mnist_train_19.jpg
    save_eval_dir = 'dataset/HandWrittenImages/eval/' + str(test_labels[i]) + '/'
    if os.path.exists(save_eval_dir) is False:
        os.mkdir(save_eval_dir)

    filename = save_eval_dir + 'mnist_eval_%d.jpg' % i
    # 将image_array保存为图片
    image.imsave(filename, image_array, cmap='gray')  # cmap常用于改变绘制风格，如黑白gray，翠绿色virdidis

在执行这个脚本之前，先需要在dataset下建个目录：HandWrittenImages（跟PetImages正好对应）

执行了这个脚本之后，会生成dataset下的MINST_data目录，存放的是原始的MNIST数据集；也会生成dataset下的HandWrittenImages下的train和eval目录下的 0~9各10个目录：

随便看下里面的图片：

比如这里有一堆小3

那里有一堆小7：

然后我们修改 src/config.py

这里有2行，num_classes表示分类的个数，map表示实际的分类

# Copyright 2020 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""
network config setting, will be used in train.py and eval.py
"""
from easydict import EasyDict as ed

def set_config(args):
    if not args.run_distribute:
        args.run_distribute = False
    config_cpu = ed({
        "num_classes": 10,
        "image_height": 256,
        "image_width": 256,
        "batch_size": 32,
        "epoch_size": 30,
        "warmup_epochs": 0,
        "lr_init": .0,
        "lr_end": 0.01,
        "lr_max": 0.001,
        "momentum": 0.9,
        "weight_decay": 4e-5,
        "label_smooth": 0.1,
        "loss_scale": 1024,
        "save_checkpoint": True,
        "save_checkpoint_epochs": 1,
        "keep_checkpoint_max": 20,
        "save_checkpoint_path": "./",
        "platform": args.platform,
        "run_distribute": args.run_distribute,
        "activation": "Softmax",
        "map":["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
    })
    return config_cpu

我们查看下 src/dataset.py下的 extract_features方法：

def extract_features(net, dataset_path, config):
    print("start cache feature!")
    features_folder = os.path.join(dataset_path, "features")

    train_dataset = create_dataset(dataset_path=os.path.join(dataset_path, "train"), do_train=True, config=config)
    eval_dataset = create_dataset(dataset_path=os.path.join(dataset_path, "eval"), do_train=False, config=config)

    train_size = train_dataset.get_dataset_size()
    eval_size = eval_dataset.get_dataset_size()
    if train_size == 0:
        raise ValueError("The step_size of dataset is zero. Check if the images count of train dataset is more \
            than batch_size in config.py")
    if os.path.exists(features_folder):
        train_features = np.load(os.path.join(features_folder, f"train_feature.npy"))
        train_labels = np.load(os.path.join(features_folder, f"train_label.npy"))
        eval_features = np.load(os.path.join(features_folder, f"eval_feature.npy"))
        eval_labels = np.load(os.path.join(features_folder, f"eval_label.npy"))
        return (train_features, train_labels, eval_features, eval_labels), train_size
    os.mkdir(features_folder)
    model = Model(net)
    train_feature = []
    train_labels = []
    train_imgs = train_size * config.batch_size
    for i, data in enumerate(train_dataset.create_dict_iterator()):
        image = data["image"]
        label = data["label"]
        feature = model.predict(Tensor(image))
        train_feature.append(feature.asnumpy())
        train_labels.append(label.asnumpy())
        percent = round(i / train_size * 100., 2)
        print(f'training feature cached [{i * config.batch_size}/{train_imgs}] {str(percent)}% ', end='\r', flush=True)
    np.save(os.path.join(features_folder, f"train_feature"), np.array(train_feature))
    np.save(os.path.join(features_folder, f""
                                          f""), np.array(train_labels))
    print(f'training feature cached [{train_imgs}/{train_imgs}] 100%  \ntrain feature cache finished!', flush=True)

    eval_feature = []
    eval_labels = []
    for i, data in enumerate(eval_dataset.create_dict_iterator()):
        image = data["image"]
        label = data["label"]
        feature = model.predict(Tensor(image))
        eval_feature.append(feature.asnumpy())
        eval_labels.append(label.asnumpy())
        percent = round(i / eval_size * 100., 2)
        print(f'evaluation feature cached [{i}/{eval_size}] {str(percent)}% ', end='\r')
    np.save(os.path.join(features_folder, f"eval_feature"), np.array(eval_feature))
    np.save(os.path.join(features_folder, f"eval_label"), np.array(eval_labels))
    print(f'evaluation feature cached [{eval_size}/{eval_size}] 100%  \neval feature cache finished!')
    print(f"saved feature features_folder")
    return (np.array(train_feature), np.array(train_labels), np.array(eval_feature), np.array(eval_labels)), train_size

它会建立  dataset/HandWrittenImages/features 目录，并将图片一个个保存成npy格式。

再看看train.py的训练代码，原来之前会预测一下，之后也会预测一下。可以暂时将前面的预测注释掉。

还有个地方读参数的地方需要修改一下：

src/args.py

然后就开始训练吧：

python train.py

训练代码会生成features的四个npy文件：

训练完毕后，会显示预测的几张图片信息：

X掉后，MindIR也会被导出。

这就完成了 进阶作业。

（全文完，谢谢阅读）

CANN训练营第二期 高玩赛即将开启，请点击：https://bbs.huaweicloud.com/forum/thread-129524-1-1.html

添加下方工作人员微信，添加备注：CANN训练营~ 邀请进群~