MindStudio制作MindSpore TBE算子(三)算子测试(ST测试)
【摘要】 MindStudio制作MindSpore TBE算子(三)算子测试(ST测试)
需要真实Ascend设备支持,无法在虚拟环境进行
在上一节中,成功制作了Mindspore的Add算子,具体可以查看MindStudio制作MindSpore TBE算子(一)算子制作,
并在MindStudio制作MindSpore TBE算子(二)算子测试中介绍了算子的UT测试,本节内容将详细介绍算子ST测试,即算子的实现验证。
建议参考以下内容一起食用:
算子代码实现
MindSpore框架TBE算子开发全流程
制作Mindspore的TBE流程中用于测试的是test_xxx_impl.py和xxx_case_timestation.json文件。
ST测试
ST测试(System Testing,系统测试) 目标是确保算子不仅在孤立的环境中能够正确运行,还能与其他组件(如数据管道、模型训练框架等)协同工作,并满足性能和功能要求。与单元测试(UT)不同,ST测试关注的是系统级的功能、性能、可靠性和用户体验。
-
右击对应的算子工程文件,选择New Case中的ST case;![在这里插入图片描述]
-
选择对应的算子,点击OK
-
编写测试样例信息
点击执行后,可能出现报错:
bug 1:ST测试无法解析正确Python环境
Add_custom ST case start running...
[INFO] ===========================================================================Generate acl_op.json===========================================================================
2025-02-11 14:20:25 (198485) - [INFO] [STEP1] [case_design.py] Start to parser testcase json.
2025-02-11 14:20:25 (198485) - [INFO] Start to create sub test cases for /home/science/MindstudioProjects/Add_custom/testcases/st/addcustom/Addcustom_case_20250211135538.json.
2025-02-11 14:20:25 (198485) - [WARNING] There is no expect output function in the case json, if no need to compare output data, ignore.
2025-02-11 14:20:25 (198485) - [INFO] Create 1 sub test cases for Test_Addcustom_001.
2025-02-11 14:20:25 (198485) - [INFO] Process finished!
2025-02-11 14:20:25 (198485) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:20:25
[INFO] ===========================================================================Generate TestCase===========================================================================
2025-02-11 14:20:27 (198496) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py successfully.
2025-02-11 14:20:27 (198496) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/pytest.ini successfully.
2025-02-11 14:20:27 (198496) - [INFO] Operator test code files for specified test cases generated successfully.
2025-02-11 14:20:27 (198496) - [INFO] [STEP1] [data_generator.py] Generate data for testcase.
2025-02-11 14:20:27 (198496) - [INFO] Start to generate the input data for Test_Addcustom_001_case_1_float16.
2025-02-11 14:20:27 (198496) - [INFO] Generate data for testcase in /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/run/out/test_data/data.
2025-02-11 14:20:27 (198496) - [INFO] Process finished!
2025-02-11 14:20:27 (198496) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:20:28
ST run command is cd /home/science/MindstudioProjects/Add_custom/testcases/st/out && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/mindspore/impl && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/op_proto && export PYTHONPATH=$PYTHONPATH:/home/science/Ascend/ascend-toolkit/8.0.0/atc/python/site-packages && python3 -m pytest -s --color=no addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
ERROR: not found: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
============================= test session starts ==============================
(no name '/home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16' in any of [<Module test_addcustom.py>])
platform linux -- Python 3.12.7, pytest-7.4.4, pluggy-1.0.0
rootdir: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src
configfile: pytest.ini
plugins: anyio-4.2.0
collected 0 items / 1 error
==================================== ERRORS ====================================
______________________ ERROR collecting test_addcustom.py ______________________
ImportError while importing test module '/home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/home/science/anaconda3/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
addcustom/src/test_addcustom.py:5: in <module>
import mindspore.nn as nn
E ModuleNotFoundError: No module named 'mindspore'
=========================== short test summary info ============================
ERROR addcustom/src/test_addcustom.py
=============================== 1 error in 0.07s ===============================
Run MindSpore St Failed.
报错信息显示,缺乏MindSpore安装包,这里也是和之前类似的问题,应该使用mindspore环境进行调试开发,但是这里使用的是base环境。
这是因为MindStudio是在conda的base环境中启动的,这里需要重新在conda mindspore环境中其中mindstudio。
conda activate mindstudio
bash MindStudio.sh
bug 2:缺乏Ascend环境,无法执行
继续执行后出现新的报错,
Add_custom ST case start running...
[INFO] ===========================================================================Generate acl_op.json===========================================================================
2025-02-11 14:59:06 (4459) - [INFO] [STEP1] [case_design.py] Start to parser testcase json.
2025-02-11 14:59:06 (4459) - [INFO] Start to create sub test cases for /home/science/MindstudioProjects/Add_custom/testcases/st/addcustom/Addcustom_case_20250211135538.json.
2025-02-11 14:59:06 (4459) - [WARNING] There is no expect output function in the case json, if no need to compare output data, ignore.
2025-02-11 14:59:06 (4459) - [INFO] Create 1 sub test cases for Test_Addcustom_001.
2025-02-11 14:59:06 (4459) - [INFO] Process finished!
2025-02-11 14:59:06 (4459) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:59:06
[INFO] ===========================================================================Generate TestCase===========================================================================
2025-02-11 14:59:07 (4485) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py successfully.
2025-02-11 14:59:07 (4485) - [INFO] Content appended to /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/pytest.ini successfully.
2025-02-11 14:59:07 (4485) - [INFO] Operator test code files for specified test cases generated successfully.
2025-02-11 14:59:07 (4485) - [INFO] [STEP1] [data_generator.py] Generate data for testcase.
2025-02-11 14:59:07 (4485) - [INFO] Start to generate the input data for Test_Addcustom_001_case_1_float16.
2025-02-11 14:59:07 (4485) - [INFO] Generate data for testcase in /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/run/out/test_data/data.
2025-02-11 14:59:07 (4485) - [INFO] Process finished!
2025-02-11 14:59:07 (4485) - [INFO] The st report saved in: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/st_report.json.
2025-02-11 14:59:08
ST run command is cd /home/science/MindstudioProjects/Add_custom/testcases/st/out && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/mindspore/impl && export PYTHONPATH=$PYTHONPATH:/home/science/MindstudioProjects/Add_custom/op_proto && export PYTHONPATH=$PYTHONPATH:/home/science/Ascend/ascend-toolkit/8.0.0/atc/python/site-packages && python3 -m pytest -s --color=no addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
ERROR: found no collectors for /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src/test_addcustom.py::test_Addcustom_001_case_1_float16
============================= test session starts ==============================
platform linux -- Python 3.9.18, pytest-8.3.4, pluggy-1.5.0
rootdir: /home/science/MindstudioProjects/Add_custom/testcases/st/out/addcustom/src
configfile: pytest.ini
collected 0 items / 1 error
==================================== ERRORS ====================================
______________________ ERROR collecting test_addcustom.py ______________________
addcustom/src/test_addcustom.py:11: in <module>
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", device_id=0)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/_checkparam.py:1367: in wrapper
return func(*args, **kwargs)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:1875: in set_context
ctx.set_device_target(kwargs['device_target'])
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:495: in set_device_target
self.set_param(ms_ctx_param.device_target, target)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:187: in set_param
self._context_handle.set_param(param, value)
E RuntimeError: Unsupported device target Ascend. This process only supports one of the ['CPU']. Please check whether the Ascend environment is installed and configured correctly, and check whether current mindspore wheel package was built with "-e Ascend". For details, please refer to "Device load error message".
E
E ----------------------------------------------------
E - Device load error message:
E ----------------------------------------------------
E Load dynamic library: libmindspore_ascend.so.2 failed. libascend_hal.so: cannot open shared object file: No such file or directory
E
E ----------------------------------------------------
E - C++ Call Stack: (For framework developers)
E ----------------------------------------------------
E mindspore/core/utils/ms_context.cc:287 SetDeviceTargetFromInner
=========================== short test summary info ============================
ERROR addcustom/src/test_addcustom.py - RuntimeError: Unsupported device targ...
=============================== 1 error in 0.85s ===============================
Run MindSpore St Failed.
核心报错信息为:
==================================== ERRORS ====================================
______________________ ERROR collecting test_addcustom.py ______________________
addcustom/src/test_addcustom.py:11: in <module>
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", device_id=0)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/_checkparam.py:1367: in wrapper
return func(*args, **kwargs)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:1875: in set_context
ctx.set_device_target(kwargs['device_target'])
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:495: in set_device_target
self.set_param(ms_ctx_param.device_target, target)
/home/science/anaconda3/envs/mindspore/lib/python3.9/site-packages/mindspore/context.py:187: in set_param
self._context_handle.set_param(param, value)
E RuntimeError: Unsupported device target Ascend. This process only supports one of the ['CPU']. Please check whether the Ascend environment is installed and configured correctly, and check whether current mindspore wheel package was built with "-e Ascend". For details, please refer to "Device load error message".
addcustom/src/test_addcustom.py:11: in <module>
context.set_context(mode=context.GRAPH_MODE, device_target="Ascend", device_id=0)
这里我的执行环境是虚拟机,没有ascend设置,执行设置设置到Ascend中,导致出现此问题。因此,尝试去解决需要修改代码。
(mindspore) science@science-2204:~/MindstudioProjects$ find . -name test_addcustom.py
./Add_custom/testcases/st/out/addcustom/src/test_addcustom.py
如果去修改test_addcustom.py,发现是无法解决此问题的,每次重新点击运行后依然会覆盖此文件。
context.set_context(mode=context.GRAPH_MODE, device_target="CPU", device_id=0)
https://www.hiascend.com/doc_center/source/zh/mindstudio/600/msug/msug_000132.html
后查阅资料,发现只能在真实环境中验证算子功能正确性,奈何现有MindStudio最新版本砍掉远程连接功能,无法链接云服务器,只能在此戛然而止,欠各位看官一篇完整报告,后期补上。
【声明】本内容来自华为云开发者社区博主,不代表华为云及华为云开发者社区的观点和立场。转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息,否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱:
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
作者其他文章
评论(0)