飞桨x昇腾生态适配方案:13_API离线推理

举报
RaceSnail 发表于 2025/05/18 11:46:08 2025/05/18
【摘要】 ais_bench提供的python API可供使能基于昇腾硬件的离线模型(.om模型)推理。具体介绍可参考[API_GUIDE](https://gitee.com/ascend/tools/blob/master/ais-bench_workload/tool/ais_bench/API_GUIDE.md#api%E7%AE%80%E4%BB%8B)下面列举几个常用的API推理场景使用方...

ais_bench提供的python API可供使能基于昇腾硬件的离线模型(.om模型)推理。具体介绍可参考[API_GUIDE](https://gitee.com/ascend/tools/blob/master/ais-bench_workload/tool/ais_bench/API_GUIDE.md#api%E7%AE%80%E4%BB%8B)
下面列举几个常用的API推理场景使用方法。

静态API推理

单个om推理

api_1.py

import cv2
import numpy as np
from ais_bench.infer.interface import InferSession
from ais_bench.infer.common.utils import logger_print


def infer_api_static():
    device_id = 0
    model_path = '/home/w30067200/paddle_test/PaddleOCR/inference/om/inference.om'
    image_path = '/home/w30067200/paddle_test/PaddleOCR/test/general_ocr_002.png'
    image = cv2.imread(image_path)

    # create session of om model for inference
    session = InferSession(device_id, model_path)

    # create new numpy data according inputs info
    shape0 = session.get_inputs()[0].shape
    print(shape0)
    height, width = shape0[2], shape0[3]
    resized_image = cv2.resize(image, (width, height))
    image_array = np.array(resized_image).astype(np.float32)

    feeds = [image_array]

    # execute inference, inputs is ndarray list and outputs is ndarray list
    outputs = session.infer(feeds, mode='static')
    print(outputs[0].shape)

    np.set_printoptions(threshold=np.inf)
    print(outputs1[0])

    # cv2.imwrite('output.png', outputs1[0])
    logger_print("outputs: %s" % outputs)

    # free model resource and device context of session
    session0.free_resource()


infer_api_static()

多个om推理

这里需要注意的是需要将前一个om推理结果输出的shape转化为下一个om推理模型输入的shape:
api_2.py

import cv2
import numpy as np
from ais_bench.infer.interface import InferSession
from ais_bench.infer.common.utils import logger_print


def infer_api_static():
    device_id = 0
    model_path0 = './det.om'
    model_path = './rec.om'
    image_path = './general_ocr_002.png'
    image = cv2.imread(image_path)

    # create session of om model for inference
    session0 = InferSession(device_id, model_path0)
    session = InferSession(device_id, model_path)

    # create new numpy data according inputs info
    shape0 = session0.get_inputs()[0].shape
    shape1 = session.get_inputs()[0].shape
    print("shape0:",shape0)
    print("shape1:",shape1)
    height, width = shape0[2], shape0[3]
    resized_image = cv2.resize(image, (width, height))
    image_array = np.array(resized_image).astype(np.float32)

    feeds = [image_array]

    # execute inference, inputs is ndarray list and outputs is ndarray list
    outputs = session0.infer(feeds, mode='static')
    print("outputs[0].shape:",outputs[0].shape)
    # det.om推理结果输出的shape [1, 1, 640, 480] 转化为下一个rec.om推理模型输入的shape[1, 3, 48, 320]
    height, width = shape1[2], shape1[3]
    arr = np.repeat(outputs[0], 3, axis=1)
    arr = arr.transpose(0, 2, 3, 1)  # [1, 224, 224, 3]

    resized = cv2.resize(arr[0], (width, height), interpolation=cv2.INTER_LINEAR)


    resized = resized[np.newaxis, ...].transpose(0, 3, 1, 2)  # [1, 3, 48, 680]

    '''
    resized_image1 = cv2.resize(arr, (width, height))
    image_array1 = np.array(resized_image1).astype(np.float32)
    '''
    feeds = [resized]
    print("feeds[0].shape",feeds[0].shape)
    # execute inference, inputs is ndarray list and outputs is ndarray list
    outputs1 = session.infer(feeds, mode='static')
    np.set_printoptions(threshold=np.inf)
    # print(outputs1[0].shape)

    # cv2.imwrite('output.png', outputs1[0])
    # logger_print("outputs1: %s" % outputs1)

    # free model resource and device context of session
    session0.free_resource()


infer_api_static()

执行结果如下所示:
01_静态API推理结果

动态API推理

多个om推理

import onnx
import numpy as np
from ais_bench.infer.interface import InferSession



def infer_api_dymshape():
    model_path0 = './model_dest_linux_x86_64.om'
    model_path1 = './mode_loop_input2_i_cond_linux_x86_64.om'

    # Load the model
    device_id = 1
    session0 = InferSession(device_id, model_path0)
    session1 = InferSession(device_id, model_path1)

    # 假设 batch 大小为 4
    batch = 4

    # 生成全 1 的 input1
    input1 = np.ones((batch, 16, 32), dtype=np.float32)

    # 生成全 2 的 input2
    input2 = np.full((batch, 16, 32), 2, dtype=np.float32)

    # print("input1 的形状:", input1.shape)
    # print("input2 的形状:", input2.shape)

    input = [input1, input2]
    # 执行model_dest_linux_x86_64.om推理
    output0 = session0.infer(input, mode='dymshape')
    input3 = output0[0]
    cnts = output0[1]

    for i in range(cnts):
        inputs = {"x.13": input3, "input2": input2}
        # 执行mode_loop_input2_i_cond_linux_x86_64.om 推理
        input3 = session1.infer(inputs, mode='dymshape')
    print("input3", input3)

    # free model resource and device context of session
    session0.free_resource()
    session1.free_resource()

infer_api_dymshape()

执行结果:
02_动态API推理结果

【声明】本内容来自华为云开发者社区博主,不代表华为云及华为云开发者社区的观点和立场。转载时必须标注文章的来源(华为云社区)、文章链接、文章作者等基本信息,否则作者和本社区有权追究责任。如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱: cloudbbs@huaweicloud.com
  • 点赞
  • 收藏
  • 关注作者

评论(0

0/1000
抱歉,系统识别当前为高风险访问,暂不支持该操作

全部回复

上滑加载中

设置昵称

在此一键设置昵称,即可参与社区互动!

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。

*长度不超过10个汉字或20个英文字符,设置后3个月内不可修改。