MindStudio制作MindSpore TBE算子(三)算子测试(ST测试)

举报
塞恩斯 发表于 2025/02/11 15:44:00 2025/02/11
1.5w+ 0 0
【摘要】 MindStudio制作MindSpore TBE算子(三)算子测试(ST测试)

需要真实Ascend设备支持,无法在虚拟环境进行
在上一节中,成功制作了Mindspore的Add算子,具体可以查看MindStudio制作MindSpore TBE算子(一)算子制作
并在MindStudio制作MindSpore TBE算子(二)算子测试中介绍了算子的UT测试,本节内容将详细介绍算子ST测试,即算子的实现验证。
建议参考以下内容一起食用:
算子代码实现
MindSpore框架TBE算子开发全流程

制作Mindspore的TBE流程中用于测试的是test_xxx_impl.py和xxx_case_timestation.json文件。
image.png

ST测试

ST测试(System Testing,系统测试) 目标是确保算子不仅在孤立的环境中能够正确运行,还能与其他组件(如数据管道、模型训练框架等)协同工作,并满足性能和功能要求。与单元测试(UT)不同,ST测试关注的是系统级的功能、性能、可靠性和用户体验。

  1. 右击对应的算子工程文件,选择New Case中的ST case;![在这里插入图片描述]
    image.png

  2. 选择对应的算子,点击OK
    image.png

  3. 编写测试样例信息
    image.png

image.png

image.png

image.png

点击执行后,可能出现报错:

bug 1:ST测试无法解析正确Python环境

Add_custom ST case start running...
[INFO] ===========================================================================Generate acl_op.json===========================================================================
2025-02-11 14:20:25  (198485) - [INFO] [STEP1] [case_design.py] Start to parser testcase json.
2025-02-11 14:20:25  (198485) - [INFO] Start to create sub test cases for /home/science/MindstudioProjects/Add_custom/testcases/st/addcustom/Addcustom_case_20250211135538.json.
2025-02-11 14:20:25  (198485) - [WARNING] There is no expect output function in the case json, if no need to compare output data, ignore.
2025-02-11 14:20:25  (198485) - [INFO] Create 1 sub test cases for Test_Addcustom_001.
2025-02-11 14:20:25  (198485) - [INFO] Process finished!
2025-02-11 14:20:25  (198485) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:20:25  
[INFO] ===========================================================================Generate TestCase===========================================================================
2025-02-11 14:20:27  (198496) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py successfully.
2025-02-11 14:20:27  (198496) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/pytest.ini successfully.
2025-02-11 14:20:27  (198496) - [INFO] Operator test code files for specified test cases generated successfully.
2025-02-11 14:20:27  (198496) - [INFO] [STEP1] [data_generator.py] Generate data for testcase.
2025-02-11 14:20:27  (198496) - [INFO] Start to generate the input data for Test_Addcustom_001_case_1_float16.
2025-02-11 14:20:27  (198496) - [INFO] Generate data for testcase in /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/run/out/test_data/data.
2025-02-11 14:20:27  (198496) - [INFO] Process finished!
2025-02-11 14:20:27  (198496) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:20:28  
ST run command is cd /home/science/MindstudioProjects/Add_custom/testcases/st/out && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/mindspore/impl && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/op_proto && export PYTHONPATH=$PYTHONPATH:/home/science/Ascend/ascend-toolkit/8.0.0/atc/python/site-packages && python3 -m pytest -s --color=no addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
ERROR: not found: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
============================= test session starts ==============================
(no name '/home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16' in any of [<Module test_addcustom.py>])
platform linux -- Python 3.12.7, pytest-7.4.4, pluggy-1.0.0

rootdir: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src
configfile: pytest.ini
plugins: anyio-4.2.0
collected 0 items / 1 error

==================================== ERRORS ====================================
______________________ ERROR collecting test_addcustom.py ______________________
ImportError while importing test module '/home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/home/science/anaconda3/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
addcustom/src/test_addcustom.py:5: in <module>
    import mindspore.nn as nn
E   ModuleNotFoundError: No module named 'mindspore'
=========================== short test summary info ============================
ERROR addcustom/src/test_addcustom.py
=============================== 1 error in 0.07s ===============================

Run MindSpore St Failed.

报错信息显示,缺乏MindSpore安装包,这里也是和之前类似的问题,应该使用mindspore环境进行调试开发,但是这里使用的是base环境。
这是因为MindStudio是在conda的base环境中启动的,这里需要重新在conda mindspore环境中其中mindstudio。

conda activate mindstudio
bash MindStudio.sh

bug 2:缺乏Ascend环境,无法执行

继续执行后出现新的报错,

Add_custom ST case start running...
[INFO] ===========================================================================Generate acl_op.json===========================================================================
2025-02-11 14:59:06  (4459) - [INFO] [STEP1] [case_design.py] Start to parser testcase json.
2025-02-11 14:59:06  (4459) - [INFO] Start to create sub test cases for /home/science/MindstudioProjects/Add_custom/testcases/st/addcustom/Addcustom_case_20250211135538.json.
2025-02-11 14:59:06  (4459) - [WARNING] There is no expect output function in the case json, if no need to compare output data, ignore.
2025-02-11 14:59:06  (4459) - [INFO] Create 1 sub test cases for Test_Addcustom_001.
2025-02-11 14:59:06  (4459) - [INFO] Process finished!
2025-02-11 14:59:06  (4459) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:59:06  
[INFO] ===========================================================================Generate TestCase===========================================================================
2025-02-11 14:59:07  (4485) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py successfully.
2025-02-11 14:59:07  (4485) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/pytest.ini successfully.
2025-02-11 14:59:07  (4485) - [INFO] Operator test code files for specified test cases generated successfully.
2025-02-11 14:59:07  (4485) - [INFO] [STEP1] [data_generator.py] Generate data for testcase.
2025-02-11 14:59:07  (4485) - [INFO] Start to generate the input data for Test_Addcustom_001_case_1_float16.
2025-02-11 14:59:07  (4485) - [INFO] Generate data for testcase in /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/run/out/test_data/data.
2025-02-11 14:59:07  (4485) - [INFO] Process finished!
2025-02-11 14:59:07  (4485) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:59:08  
ST run command is cd /home/science/MindstudioProjects/Add_custom/testcases/st/out && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/mindspore/impl && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/op_proto && export PYTHONPATH=$PYTHONPATH:/home/science/Ascend/ascend-toolkit/8.0.0/atc/python/site-packages && python3 -m pytest -s --color=no addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
ERROR: found no collectors for /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
============================= test session starts ==============================

platform linux -- Python 3.9.18, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src
configfile: pytest.ini
collected 0 items / 1 error

==================================== ERRORS ====================================
______________________ ERROR collecting test_addcustom.py ______________________
addcustom/src/test_addcustom.py:11: in <module>
    context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", device_id=0)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/_checkparam.py:1367: in wrapper
    return func(*args, **kwargs)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:1875: in set_context
    ctx.set_device_target(kwargs['device_target'])
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:495: in set_device_target
    self.set_param(ms_ctx_param.device_target, target)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:187: in set_param
    self._context_handle.set_param(param, value)
E   RuntimeError: Unsupported device target Ascend. This process only supports one of the ['CPU']. Please check whether the Ascend environment is installed and configured correctly, and check whether current mindspore wheel package was built with "-e Ascend". For details, please refer to "Device load error message".
E   
E   ----------------------------------------------------
E   - Device load error message:
E   ----------------------------------------------------
E   Load dynamic library: libmindspore_ascend.so.2 failed. libascend_hal.so: cannot open shared object file: No such file or directory
E   
E   ----------------------------------------------------
E   - C++ Call Stack: (For framework developers)
E   ----------------------------------------------------
E   mindspore/core/utils/ms_context.cc:287 SetDeviceTargetFromInner
=========================== short test summary info ============================
ERROR addcustom/src/test_addcustom.py - RuntimeError: Unsupported device targ...
=============================== 1 error in 0.85s ===============================

Run MindSpore St Failed.

核心报错信息为:

==================================== ERRORS ====================================
______________________ ERROR collecting test_addcustom.py ______________________
addcustom/src/test_addcustom.py:11: in <module>
    context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", device_id=0)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/_checkparam.py:1367: in wrapper
    return func(*args, **kwargs)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:1875: in set_context
    ctx.set_device_target(kwargs['device_target'])
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:495: in set_device_target
    self.set_param(ms_ctx_param.device_target, target)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:187: in set_param
    self._context_handle.set_param(param, value)
E   RuntimeError: Unsupported device target Ascend. This process only supports one of the ['CPU']. Please check whether the Ascend environment is installed and configured correctly, and check whether current mindspore wheel package was built with "-e Ascend". For details, please refer to "Device load error message".
addcustom/src/test_addcustom.py:11: in <module>
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", device_id=0)

这里我的执行环境是虚拟机,没有ascend设置,执行设置设置到Ascend中,导致出现此问题。因此,尝试去解决需要修改代码。

(mindspore) science@science-2204:~/MindstudioProjects$ find . -name test_addcustom.py
./Add_custom/testcases/st/out/addcustom/src/test_addcustom.py

如果去修改test_addcustom.py,发现是无法解决此问题的,每次重新点击运行后依然会覆盖此文件。

context.set_context(mode=context.GRAPH_MODE, device_target="CPU", device_id=0)

https://www.hiascend.com/doc_center/source/zh/mindstudio/600/msug/msug_000132.html
后查阅资料,发现只能在真实环境中验证算子功能正确性,奈何现有MindStudio最新版本砍掉远程连接功能,无法链接云服务器,只能在此戛然而止,欠各位看官一篇完整报告,后期补上。

【声明】本内容来自华为云开发者社区博主,不代表华为云及华为云开发者社区的观点和立场。转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息,否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@huaweicloud.com
  • 点赞
  • 收藏
  • 关注作者

作者其他文章

评论(0

抱歉,系统识别当前为高风险访问,暂不支持该操作

    全部回复

    上滑加载中

    设置昵称

    在此一键设置昵称,即可参与社区互动!

    *长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。

    *长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。