前言

ES的查询效率算是比较高的，但是from+size的分页查询方式只能查到一万条，并且随着分页到后面，执行效率越低。

Scroll滚动查询的方式可以查询大量数据，并能保证查询数据结果稳定。对于后台批量数据来说非常有用。

查询

第一次查询

第一次查询和通常的_search查询基本一致，只需要在后面加上?scroll=1m,1m代表一分钟，参考的时间格式如下

GET bbs/_search?scroll=1m
{
   "size": 200
}

  

 

  
1
  
2
  
3
  
4
 

返回结果除了和正常查询结果基本一致之外，增加了返回值_scroll_id

{
  "_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAFwRFnpvalZqbDZEVEFPQWptS0wyZjYzTXcAAAAAAABcEhZ6b2pWamw2RFRBT0FqbUtMMmY2M013AAAAAAAAXBMWem9qVmpsNkRUQU9Bam1LTDJmNjNNdwAAAAAAAFwUFnpvalZqbDZEVEFPQWptS0wyZjYzTXcAAAAAAABcFRZ6b2pWamw2RFRBT0FqbUtMMmY2M013",
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 5001,
    "max_score" : 1.0,
    "hits" : [
      {
（省略。。。。。。）

  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
 

滚动请求

得到第一次请求的_scroll_id之后，就可以在设定的有效时间内，使用这个_scroll_id完成滚动查询。

 GET /_search/scroll
{
  "scroll":"10m",
   "scroll_id": "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAFsTFnpvalZqbDZEVEFPQWptS0wyZjYzTXcAAAAAAABbFxZ6b2pWamw2RFRBT0FqbUtMMmY2M013AAAAAAAAWxQWem9qVmpsNkRUQU9Bam1LTDJmNjNNdwAAAAAAAFsVFnpvalZqbDZEVEFPQWptS0wyZjYzTXcAAAAAAABbFhZ6b2pWamw2RFRBT0FqbUtMMmY2M013"
}
 

  

 

  
1
  
2
  
3
  
4
  
5
  
6
 

滚动查询得到的结果和第一次请求的结果一致，返回的_scroll_id也是一致的

如果请求翻页的结果已经翻完，返回的结果也是一致的，只是hits里面没有数据了，可以根据这个判断数据已经刷完。

{
  "_scroll_id" : "DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAF17FnpvalZqbDZEVEFPQWptS0wyZjYzTXcAAAAAAABdeRZ6b2pWamw2RFRBT0FqbUtMMmY2M013AAAAAAAAXXwWem9qVmpsNkRUQU9Bam1LTDJmNjNNdwAAAAAAAF16FnpvalZqbDZEVEFPQWptS0wyZjYzTXcAAAAAAABdfRZ6b2pWamw2RFRBT0FqbUtMMmY2M013",
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 5001,
    "max_score" : 1.0,
    "hits" : [ ]
  }
}


  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
  
17
 

如果查询的_scroll_id已经超时，那么就会返回错误码

{
  "error" : {
    "root_cause" : [
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [24047]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [24051]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [24048]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [24049]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [24050]"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : -1,
        "index" : null,
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [24047]"
        }
      },
      {
        "shard" : -1,
        "index" : null,
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [24051]"
        }
      },
      {
        "shard" : -1,
        "index" : null,
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [24048]"
        }
      },
      {
        "shard" : -1,
        "index" : null,
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [24049]"
        }
      },
      {
        "shard" : -1,
        "index" : null,
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [24050]"
        }
      }
    ],
    "caused_by" : {
      "type" : "search_context_missing_exception",
      "reason" : "No search context found for id [24050]"
    }
  },
  "status" : 404
}


  

 

  
1
  
2
  
3
  
4
  
5
  
6
  
7
  
8
  
9
  
10
  
11
  
12
  
13
  
14
  
15
  
16
  
17
  
18
  
19
  
20
  
21
  
22
  
23
  
24
  
25
  
26
  
27
  
28
  
29
  
30
  
31
  
32
  
33
  
34
  
35
  
36
  
37
  
38
  
39
  
40
  
41
  
42
  
43
  
44
  
45
  
46
  
47
  
48
  
49
  
50
  
51
  
52
  
53
  
54
  
55
  
56
  
57
  
58
  
59
  
60
  
61
  
62
  
63
  
64
  
65
  
66
  
67
  
68
  
69
  
70
  
71
  
72
  
73
  
74
  
75
  
76
  
77
  
78
 

清理scroll

我们可以主动清理scroll,释放es压力。

DELETE /_search/scroll
{
  "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}

  

 

  
1
  
2
  
3
  
4
 

总结

优点

• 可以查询大量数据
• 稳定分页不会数据重复
• 可以超出分页的一万条限制

缺点

• 不能跨页请求
• 不支持重试请求

参考资料

• https://www.elastic.co/guide/en/elasticsearch/reference/current/scroll-api.html#scroll-api
• https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#scroll-search-results
• https://www.elastic.co/guide/en/elasticsearch/reference/current/clear-scroll-api.html

文章来源: coderfix.blog.csdn.net，作者：小雨青年，版权归原作者所有，如需转载，请联系作者。

原文链接：coderfix.blog.csdn.net/article/details/113945699