解决Scrapy中xpath用到中文报错问题

问题描述

1	links = sel.xpath('//i[contains(@title,"置顶")]/following-sibling::a/@href').extract()

报错：ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL bytes or control characters

方法一：将整个xpath语句转成Unicode

1	links = sel.xpath(u'//i[contains(@title,"置顶")]/following-sibling::a/@href').extract()

方法二：xpath语句用已转成Unicode的title变量

1 2	title = u"置顶" links = sel.xpath('//i[contains(@title,"%s")]/following-sibling::a/@href' %(title)).extract()

方法三：直接用xpath中变量语法($符号加变量名)$title, 传参title即可

1	links = sel.xpath('//i[contains(@title,$title)]/following-sibling::a/@href', title="置顶").extract()

方法四：从__future__模块导入unicode_literals，使用Python3的字符串默认编码是Unicode的特性

1	from __future__ import unicode_literals #放在文件首行