白话Elasticsearch17-深度探秘搜索技术之match_phrase query 短语匹配搜索
概述
继续跟中华石杉老师学习ES，第17篇
课程地址： https://www.roncoo.com/view/55
官网
https://www.elastic.co/guide/en/elasticsearch/reference/current/full-text-queries.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query-phrase.html
近似匹配
假设content字段中有2个语句
java is my favourite programming language, and I also think spark is a very good big data system.
java spark are very related, because scala is spark's programming language and scala is also based on jvm like java.
- 1
- 2
- 3
- 4
使用match query , 搜索java spark ,DSL 大致如下
{
"match": {
"content": "java spark"
}
}
- 1
- 2
- 3
- 4
- 5
content 被拆分为两个单词 java 和 spark去匹配，所以如上两个doc都能被查询出来。
match query，只能搜索到包含java和spark的document，但是不知道java和spark是不是离的很近. 包含java或包含spark，或包含java和spark的doc，都会被查询出来。我们其实并不知道哪个doc，java和spark距离的比较近。
如果我们希望搜索java spark，中间不能插入任何其他的字符， 这个时候match就无能为力了 。
再比如 ， 如果我们要尽量让java和spark离的很近的document优先返回，要给它一个更高的relevance score，这就涉及到了proximity match，近似匹配.
例子
假设要实现两个需求：
- java spark，就靠在一起，中间不能插入任何其他字符，就要搜索出来这种doc
- java spark，但是要求，java和spark两个单词靠的越近，doc的分数越高，排名越靠前
要实现上述两个需求，用match做全文检索，是搞不定的，必须得用proximity match，近似匹配
phrase match：短语匹配
proximity match：近似匹配
这里我们要学习的是phrase match，就是仅仅搜索出java和spark靠在一起的那些doc，比如有个doc，是java use’d spark，不行。必须是比如java spark are very good friends，是可以搜索出来的。
match phrase query，就是要去将多个term作为一个短语，一起去搜索，只有包含这个短语的doc才会作为结果返回。
不像是match query，java spark，java的doc也会返回，spark的doc也会返回。
match query
为了做比对，我们先看下match query的查询结果
GET /forum/article/_search
{
"query": {
"match": {
"content": "java spark"
}
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
返回结果
{
"took": 40,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1.8166281,
"hits": [
{
"_index": "forum",
"_type": "article",
"_id": "5",
"_score": 1.8166281,
"_source": {
"articleID": "DHJK-B-1395-#Ky5",
"userID": 3,
"hidden": false,
"postDate": "2019-05-01",
"tag": [
"elasticsearch"
],
"tag_cnt": 1,
"view_cnt": 10,
"title": "this is spark blog",
"content": "spark is best big data solution based on scala ,an programming language similar to java spark",
"sub_title": "haha, hello world",
"author_first_name": "Tonny",
"author_last_name": "Peter Smith",
"new_author_last_name": "Peter Smith",
"new_author_first_name": "Tonny"
}
},
{
"_index": "forum",
"_type": "article",
"_id": "2",
"_score": 0.7721133,
"_source": {
"articleID": "KDKE-B-9947-#kL5",
"userID": 1,
"hidden": false,
"postDate": "2017-01-02",
"tag": [
"java"
],
"tag_cnt": 1,
"view_cnt": 50,
"title": "this is java blog",
"content": "i think java is the best programming language",
"sub_title": "learned a lot of course",
"author_first_name": "Smith",
"author_last_name": "Williams",
"new_author_last_name": "Williams",
"new_author_first_name": "Smith"
}
}
]
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
可以看到单单包含java的doc也返回了，不是我们想要的结果 。
match phrase query
为了演示match phrase query的功能，我们先调整一下测试数据
POST /forum/article/5/_update
{
"doc": {
"content":"spark is best big data solution based on scala ,an programming language similar to java spark"
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
将id=5的doc的content设置为恰巧包含java spark这个短语 。
GET /forum/article/_search
{
"query": {
"match_phrase": {
"content": "java spark"
}
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
返回结果
{
"took": 47,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.4302213,
"hits": [
{
"_index": "forum",
"_type": "article",
"_id": "5",
"_score": 1.4302213,
"_source": {
"articleID": "DHJK-B-1395-#Ky5",
"userID": 3,
"hidden": false,
"postDate": "2019-05-01",
"tag": [
"elasticsearch"
],
"tag_cnt": 1,
"view_cnt": 10,
"title": "this is spark blog",
"content": "spark is best big data solution based on scala ,an programming language similar to java spark",
"sub_title": "haha, hello world",
"author_first_name": "Tonny",
"author_last_name": "Peter Smith",
"new_author_last_name": "Peter Smith",
"new_author_first_name": "Tonny"
}
}
]
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
从结果中可以看到只有包含java spark这个短语的doc才返回，只包含java的doc不会返回
term position
分词后，每个单词就是一个term
分词后 ， es还记录了 每个field的位置。
举个例子 两个doc 如下：
hello world, java spark doc1
hi, spark java doc2
建立倒排索引后
| 分词 | 文档(位置) | 文档(位置 |
|---|---|---|
| hello | doc1(1) | - |
| wolrd | doc1(1) | |
| java | doc1(2) | doc2(2) |
| spark | doc1(3) | doc2(1) |
| hi | doc2(0) |
可以通过如下API来看下
GET _analyze
{
"text": "hello world, java spark",
"analyzer": "standard"
}
- 1
- 2
- 3
- 4
- 5
返回：
{
"tokens": [
{
"token": "hello",
"start_offset": 0,
"end_offset": 5,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "world",
"start_offset": 6,
"end_offset": 11,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "java",
"start_offset": 13,
"end_offset": 17,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "spark",
"start_offset": 18,
"end_offset": 23,
"type": "<ALPHANUM>",
"position": 3
}
]
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
通过position 可以看到位置信息 。
match_phrase的基本原理
理解下索引中的position，match_phrase
两个doc 如下
hello world, java spark doc1
hi, spark java doc2
- 1
- 2
| 分词 | 文档(位置) | 文档(位置 |
|---|---|---|
| hello | doc1(1) | - |
| wolrd | doc1(1) | |
| java | doc1(2) | doc2(2) |
| spark | doc1(3) | doc2(1) |
| hi | doc2(0) |
java spark , 采用match phrase来查询
-
首先 java spark 被拆成 java和spark ，分别取索引中查找
java 出现在 doc1(2) doc2(2) spark 出现在 doc1(3) doc2(1)- 1
- 2
要找到每个term都在的一个共有的那些doc，就是要求一个doc，必须包含每个term，才能拿出来继续计算
doc1 --> java和spark --> spark position恰巧比java大1 --> java的position是2，spark的position是3，恰好满足条件
doc1符合条件
doc2 --> java和spark --> java position是2，spark position是1，spark position比java position小1，而不是大1 --> 光是position就不满足，那么doc2不匹配 .
文章来源: artisan.blog.csdn.net，作者：小小工匠，版权归原作者所有，如需转载，请联系作者。
原文链接：artisan.blog.csdn.net/article/details/97617008
- 点赞
- 收藏
- 关注作者
评论（0）