性能工具之 Locust 工具关联与参数化

前言

不同的压力工具在参数化的实现逻辑上也会不同，但是参数化必须依赖业务逻辑，而不是工具中能做到什么功能。所以在参数化之前，我们必须分析真实业务逻辑中如何使用数据，再在工具中选择相对应的组合参数的方式去实现。

参数化

在 Locust 工具中有怎么使用参数化完成工作，在开展工作开始前，先了解 Python 中的一个 Queue 类，queue它是一个队列数据结构是先进先出的数据结构，具体原理大家自己查询即可。

Queue种类：

FIFO：
Queue.Queue(maxsize=0)
FIFO即First in First Out,先进先出。Queue提供了一个基本的FIFO容器，使用方法很简单,maxsize是个整数，指明了队列中能存放的数据个数的上限。一旦达到上限，插入会导致阻塞，直到队列中的数据被消费掉。如果maxsize小于或者等于0，队列大小没有限制。

LIFO
Queue.LifoQueue(maxsize=0)
LIFO即Last in First Out,后进先出。与栈的类似，使用也很简单,maxsize用法同上。

priority
class Queue.PriorityQueue(maxsize=0)
构造一个优先队列。maxsize用法同上。

基本方法：

• Queue.Queue(maxsize=0) FIFO， 如果maxsize小于1就表示队列长度无限
• Queue.LifoQueue(maxsize=0) LIFO， 如果maxsize小于1就表示队列长度无限
• Queue.qsize() 返回队列的大小
• Queue.empty() 如果队列为空，返回True,反之False
• Queue.full() 如果队列满了，返回True,反之False
• Queue.get([block[, timeout]]) 读队列，timeout等待时间
• Queue.put(item, [block[, timeout]]) 写队列，timeout等待时间
• Queue.queue.clear() 清空队列

队列 queue 多应用在多线程应用中，多线程访问共享变量。对于多线程而言，访问共享变量时，队列queue是线程安全的。

有以上简单基础知识后，直接上 Locust 脚本怎么编写参数化，直接仿照即可写出参数脚本，下面是简单一个参数化代码。

import csv
import os, requests
import queue
from locust import TaskSet, task, HttpUser
from requests.packages.urllib3.exceptions import InsecureRequestWarning

# 禁用安全请求警告
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)


def fnReadData():
    f = open("uuid.text", "r") #读取参数文件
    data = []                  #声明空列表
    data = csv.reader(f)      #通过 csv读取文件内容
    s = queue.Queue()         #实例化一个queue对象
    for each in data:         #循环读取open里面的数据
        for key in each:
            try:
                s.put_nowait(key)    #put到队列中
            except queue.Full:
                print("Queue overflow")
    f.close()
    return s


class MyBlogs(TaskSet):
    # 访问我的博客首页
    @task(1)
    def get_blog(self):
        # 定义请求头
        header = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"}
        data = self.user.queueData.get()
        req = self.client.get("/357712148/%s" % data, headers=header, verify=False)
        if req.status_code == 200:
            print("success")
        else:
            print("fails")


class httpGet(HttpUser):
    tasks = [MyBlogs]
    min_wait = 3000  # 单位为毫秒
    max_wait = 6000  # 单位为毫秒
    queueData = fnReadData()  # 队列实例化


if __name__ == "__main__":
    #通过好
    os.system("locust -f lcome.py --host=https://blog.com  --headless -u 1 -r 1 -t 1s")

参数化文件：

运行结果：

[2021-04-25 13:39:51,536] liwen.local/INFO/locust.main: Run time limit set to 1 seconds
[2021-04-25 13:39:51,536] liwen.local/INFO/locust.main: Starting Locust 1.4.4
[2021-04-25 13:39:51,536] liwen.local/INFO/locust.runners: Spawning 1 users at the rate 1 users/s (0 users already running)...
[2021-04-25 13:39:51,536] liwen.local/INFO/locust.runners: All users spawned: httpGet: 1 (1 total running)
[2021-04-25 13:39:51,537] liwen.local/INFO/root: Terminal was not a tty. Keyboard input disabled
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

success
success
success
[2021-04-25 13:39:52,536] liwen.local/INFO/locust.main: Time limit reached. Stopping Locust.
[2021-04-25 13:39:52,537] liwen.local/INFO/locust.runners: Stopping 1 users
[2021-04-25 13:39:52,537] liwen.local/INFO/locust.runners: 1 Users have been stopped, 0 still running
[2021-04-25 13:39:52,537] liwen.local/INFO/locust.main: Running teardowns...
[2021-04-25 13:39:52,537] liwen.local/INFO/locust.main: Shutting down (exit code 0), bye.
[2021-04-25 13:39:52,537] liwen.local/INFO/locust.main: Cleaning up runner...
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /357712148/2525651                                             1     0(0.00%)  |     238     238     238     238  |    1.30    0.00
 GET /357712148/2532241                                             1     0(0.00%)  |     210     210     210     210  |    1.30    0.00
 GET /357712148/2562906                                             1     0(0.00%)  |     320     320     320     320  |    1.30    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         3     0(0.00%)  |     256     210     320     240  |    3.89    0.00

Response time percentiles (approximated)
 Type     Name                                                              50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|
 GET      /357712148/2525651                                                240    240    240    240    240    240    240    240    240    240    240      1
 GET      /357712148/2532241                                                210    210    210    210    210    210    210    210    210    210    210      1
 GET      /357712148/2562906                                                320    320    320    320    320    320    320    320    320    320    320      1
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|
 None     Aggregated                                                        240    240    320    320    320    320    320    320    320    320    320      3

关联

相对于关联都是获取响应结果来做入参，关联理论请参考关联和断言：一动一静，核心都是在取数据 这里面理论已经讲解很清楚，现在看看 locust 怎么关联数据。

打开项目工程 关键内容如下：

/**
     * 需要关联的数据
     *
     * @param id
     * @return
     */
    @GetMapping("/associated/{id}")
    @ResponseBody
    public R associated(@PathVariable Integer id) {
        HashMap<String, Object> map = new HashMap<>();
        if (StringUtils.isEmpty(id)) {
            return R.error();
        } else if (id == 1) {
            HttpHeaders headers = new HttpHeaders();
            //根据自己的需要动态添加你想要的content type
            headers.add(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
            map.put("des", "关联数据");
            map.put("rew", 6666);
            map.put("header", headers);
            return R.ok().put("data", map);
        } else {
            return R.error().put("data", map);
        }
    }

    /**
     * 需要关联的数据
     *
     * @param id
     * @return
     */
    @GetMapping("/associated/data/{id}")
    @ResponseBody
    public R associatedData(@PathVariable Integer id) {
        HashMap<String, Object> map = new HashMap<>();
        if (StringUtils.isEmpty(id)) {
            return R.error();
        } else if (id == 6666) {
            map.put("des", "关联数据成功");
            map.put("rew", 8888);
            return R.ok().put("data", map);
        } else {
            return R.error().put("data", map);
        }
    }

locust 脚本参考：

  def get_param(self):
        '''获取参数'''
        response = self.client.get("/associated/1")
        print("Response json:", type(response.json()))  # 判断类型
        print("Response json:", response.json())
        res = response.json()  # 转换字典
        return res['data']['rew']  # 获取参数

    @task(1)
    def request_param(self):
        '''关联参数请求'''
        id = self.get_param()
        response = self.client.get("/associated/data/%s" % id)
        print("Response json:", type(response.json()))  # 判断类型
        print("Response json:", response.json())

结果：

[2021-04-25 22:52:25,837] liwen.local/INFO/locust.main: Run time limit set to 1 seconds
[2021-04-25 22:52:25,837] liwen.local/INFO/locust.main: Starting Locust 1.4.4
[2021-04-25 22:52:25,837] liwen.local/INFO/locust.runners: Spawning 1 users at the rate 1 users/s (0 users already running)...
[2021-04-25 22:52:25,838] liwen.local/INFO/locust.runners: All users spawned: webTestDunShan: 1 (1 total running)
[2021-04-25 22:52:25,838] liwen.local/INFO/root: Terminal was not a tty. Keyboard input disabled
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)  |       0       0       0       0  |    0.00    0.00

Response json: <class 'dict'>
Response json: {'msg': 'success', 'code': 0, 'data': {'des': '关联数据', 'rew': 6666, 'header': {'Content-Type': ['application/json']}}}
Response json: <class 'dict'>
Response json: {'msg': 'success', 'code': 0, 'data': {'des': '关联数据成功', 'rew': 8888}}
.....
略
....
Response json: {'msg': 'success', 'code': 0, 'data': {'des': '关联数据成功', 'rew': 8888}}
Response json: <class 'dict'>
Response json: {'msg': 'success', 'code': 0, 'data': {'des': '关联数据', 'rew': 6666, 'header': {'Content-Type': ['application/json']}}}
[2021-04-25 22:52:26,705] liwen.local/INFO/locust.main: Time limit reached. Stopping Locust.
[2021-04-25 22:52:26,705] liwen.local/INFO/locust.runners: Stopping 1 users
[2021-04-25 22:52:26,705] liwen.local/INFO/locust.runners: 1 Users have been stopped, 0 still running
[2021-04-25 22:52:26,705] liwen.local/INFO/locust.main: Running teardowns...
[2021-04-25 22:52:26,705] liwen.local/INFO/locust.main: Shutting down (exit code 0), bye.
[2021-04-25 22:52:26,705] liwen.local/INFO/locust.main: Cleaning up runner...
 Name                                                          # reqs      # fails  |     Avg     Min     Max  Median  |   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /associated/1                                                406     0(0.00%)  |       0       0       5       1  |  468.35    0.00
 GET /associated/data/6666                                        405     0(0.00%)  |       1       0       7       1  |  467.20    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                       811     0(0.00%)  |       0       0       7       1  |  935.55    0.00

Response time percentiles (approximated)
 Type     Name                                                              50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100% # reqs
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|
 GET      /associated/1                                                       1      1      1      1      1      1      1      1      5      5      5    406
 GET      /associated/data/6666                                               1      1      1      1      1      1      1      2      7      7      7    405
--------|------------------------------------------------------------|---------|------|------|------|------|------|------|------|------|------|------|------|
 None     Aggregated

总结

Locust 的参数化与关联需要自己写相应代码才能完成，但对性能工程师来说学习 Python 代码是不可绕过去的事情。上面是简单的参数化与关联代码希望对大家有帮助，