Python爬虫实战(二):爬取天涯帖子(只看楼主)
【摘要】 先上代码
#coding=utf-8import requestsfrom bs4 import Tagfrom bs4 import BeautifulSoup def getHtml(url): page = requests.get(url) html =page.text ...
先上代码
-
#coding=utf-8
-
import requests
-
from bs4 import Tag
-
from bs4 import BeautifulSoup
-
-
-
def getHtml(url):
-
page = requests.get(url)
-
html =page.text
-
return html
-
-
def getText(html):
-
get_text = Tag.get_text
-
soup = BeautifulSoup(html, 'html.parser')
-
-
author_info = soup.find_all('div', class_='atl-info')
-
listauthor = [x.get_text() for x in author_info]
-
-
list_info = soup.find_all('div', class_='bbs-content')
-
listtext = [x.get_text() for x in list_info]
-
-
global i
-
if i > 1:
-
listtext = [""] + listtext
-
-
for x in range(len(listauthor)):
-
if "楼主" in listauthor[x]:
-
print (listtext[x].strip())
-
-
if __name__=='__main__':
-
for i in range(1,6):
-
url = ("http://bbs.tianya.cn/post-feeling-4286798-%s.shtml" % str(i))
-
html = getHtml(url)
-
getText(html)
-
-
-
-
-
-
刚学Python不到一个月,代码写的有点乱,以后优化。
文章来源: blog.csdn.net,作者:悦来客栈的老板,版权归原作者所有,如需转载,请联系作者。
原文链接:blog.csdn.net/qq523176585/article/details/77836244
【版权声明】本文为华为云社区用户转载文章,如果您发现本社区中有涉嫌抄袭的内容,欢迎发送邮件进行举报,并提供相关证据,一经查实,本社区将立刻删除涉嫌侵权内容,举报邮箱:
cloudbbs@huaweicloud.com
- 点赞
- 收藏
- 关注作者
评论(0)