python复习之【difflib文本相似性】

Python 库提供了检测difflib模块中两个序列之间差异的工具。由于文本本身是一个序列（字符序列），我们可以应用提供的函数来检测字符串中的相似性。

1.difflib.SequenceMatcher计算字符串之间的相似性（从 0 到 1）

s1 = 'Today the weather is nice'
s2 = 'Today the weater is nice'
s3 = 'Yesterday the weather was nice'
s4 = 'Today my dog ate steak'

import difflib
difflib.SequenceMatcher(None, s1, s2, False).ratio()

0.9795918367346939

difflib.SequenceMatcher(None, s1, s3, False).ratio()

0.8

difflib.SequenceMatcher(None, s1, s4, False).ratio()

0.46808510638297873

2..get_matching_blocks()返回匹配位置

.get_matching_blocks()返回匹配位置，忽略字符时，可以适用maketrans

a = 'aaaaaaaaaaaaaXaaaaaaaaaa'
b = 'X'

difflib.SequenceMatcher(None, a, b, False).get_matching_blocks()

[Match(a=13, b=0, size=1), Match(a=24, b=1, size=0)]

difflib.SequenceMatcher(lambda c: c=='a', a, b, False).get_matching_blocks()

[Match(a=13, b=0, size=1), Match(a=24, b=1, size=0)]

discardmap = str.maketrans({"a": None})
difflib.SequenceMatcher(None, a.translate(discardmap), b.translate(discardmap), False).ratio()

1.0

3.拼写建议

• 如果拼写错误，返回正确拼写
• 如果拼写无误，火车查不到，则为正确。

# 引入单词词表
dictionary = {'ability', 'able', 'about', 'above', 'accept',    
              'according', 
              'account', 'across', 'act', 'action', 'activity', 
              'actually', 
              'add', 'address', 'administration', 'admit', 'adult', 
              'affect', 
              'after', 'again', 'against', 'age', 'agency', 
              'agent', 'ago', 
              'agree', 'agreement', 'ahead', 'air', 'all', 'allow',  
              'almost', 
              'alone', 'along', 'already', 'also', 'although', 
              'always', 
              'American', 'among', 'amount', 'analysis', 'and', 
              'animal', 
              'another', 'answer', 'any', 'anyone', 'anything', 
              'appear', 
              'apply', 'approach', 'area', 'argue', 
              'arm', 'around', 'arrive', 
              'art', 'article', 'artist', 'as', 'ask', 'assume', 
              'at', 'attack', 
              'attention', 'attorney', 'audience', 'author',  
              'authority', 
              'available', 'avoid', 'away', 'baby', 'back', 'bad', 
              'bag', 
              'ball', 'bank', 'bar', 'base', 'be', 'beat', 
              'beautiful', 
              'because', 'become'}

# 相似度比较
import difflib

def suggest(phrase):
    changes = 0
    words = phrase.split()
    for idx, w in enumerate(words):
        if w not in dictionary:
            changes += 1
            matches = difflib.get_close_matches(w, dictionary)
            if matches:
                words[idx] = matches[0]
    return changes, ' '.join(words)

suggest('assume anu answer')

(1, 'assume any answer')

suggest('beautiful dog')

(1, 'beautiful dog')