在进行文本分析、提取关键词时,新闻评论等文本通常是中英文及其他语言的混杂,若不加处理直接分析,结果往往差强人意。
# 编程之家 (jb51.cc)
s = "China's Legend Holdings will split its several business arms to go public on stock markets,the group's president Zhu Linan said on Tuesday.该集团总裁朱利安周二表示,中国联想控股将分拆其多个业务部门在股市上市。"
result = "".join(i for i in s if ord(i) < 256)
print(result)
# End 编程之家 jb51.cc
out:
China's Legend Holdings will split its several business arms to go public on stock markets,the group's president Zhu Linan said on Tuesday.
2、unicode编码识别
# 编程之家 (jb51.cc)
import re
s = "China's Legend Holdings will split its several business arms to go public on stock markets,the group's president Zhu Linan said on Tuesday.该集团总裁朱利安周二表示,中国联想控股将分拆其多个业务部门在股市上市。"
uncn = re.compile(r'[\u0061-\u007a,\u0020]')
en = "".join(uncn.findall(s.lower()))
print(en)
# End 编程之家 jb51.cc
out:
chinas legend holdings will split its several business arms to go public on stock markets,the groups president zhu linan said on tuesday
中文的编码范围是:\u4e00-\u9fa5,相应的[^\u4e00-\u9fa5]可匹配非中文。
匹配英文时,需要将空格[\u0020]加入,不然单词之间没空格了。
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。