在资源管理器中访问路径：%appdata%
新建pip文件夹，在pip文件夹内新建文件：pip.ini
文件内容：
|
1 2 3 4 |
[global] timeout = 600 index-url = https://pypi.tuna.tsinghua.edu.cn/simple trusted-host = pypi.tuna.tsinghua.edu.cn |
pip
在Mac中用pip安装Scrapy
遇到了两个大问题，第一个在这里已经解决。
第二个问题是：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
creating build/temp.macosx-10.11-intel-2.7/src/lxml cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/usr/include/libxml2 -Isrc/lxml/includes -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.11-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace In file included from src/lxml/lxml.etree.c:323: src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found #include "libxml/xmlversion.h" ^ 1 error generated. Compile failed: command 'cc' failed with exit status 1 creating tmp cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInitb43sL5.c -o tmp/xmlXPathInitb43sL5.o /tmp/xmlXPathInitb43sL5.c:1:10: fatal error: 'libxml/xpath.h' file not found #include "libxml/xpath.h" ^ 1 error generated. ********************************************************************************* Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed? Perhaps try: xcode-select --install ********************************************************************************* error: command 'cc' failed with exit status 1 ---------------------------------------- Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-PmvVPX/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-uPgqEz-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-PmvVPX/lxml |
我使用了这个文章里的解决办法：
|
1 2 3 4 5 6 7 8 9 10 |
MacBook-Pro:~ EverybodyLies$ sudo STATIC_DEPS=true pip install lxml Password: The directory '/Users/EverybodyLies/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/EverybodyLies/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Collecting lxml Downloading lxml-3.5.0.tar.gz (3.8MB) 100% |████████████████████████████████| 3.8MB 120kB/s Installing collected packages: lxml Running setup.py install for lxml ... done Successfully installed lxml-3.5.0 |
成功安装lxml后，再运行：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
MacBook-Pro:~ EverybodyLies$ sudo pip install Scrapy The directory '/Users/EverybodyLies/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/EverybodyLies/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Collecting Scrapy Downloading Scrapy-1.0.5-py2-none-any.whl (291kB) 100% |████████████████████████████████| 294kB 257kB/s Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): queuelib in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.8.0 in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): lxml in /Library/Python/2.7/site-packages (from Scrapy) Collecting Twisted>=10.0.0 (from Scrapy) Downloading Twisted-15.5.0.tar.bz2 (3.1MB) 100% |████████████████████████████████| 3.1MB 105kB/s Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): service-identity in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): cryptography>=0.7 in /Library/Python/2.7/site-packages (from pyOpenSSL->Scrapy) Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Twisted>=10.0.0->Scrapy) Requirement already satisfied (use --upgrade to upgrade): characteristic>=14.0.0 in /Library/Python/2.7/site-packages (from service-identity->Scrapy) Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules in /Library/Python/2.7/site-packages (from service-identity->Scrapy) Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /Library/Python/2.7/site-packages (from service-identity->Scrapy) Requirement already satisfied (use --upgrade to upgrade): enum34 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy) Requirement already satisfied (use --upgrade to upgrade): ipaddress in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy) Requirement already satisfied (use --upgrade to upgrade): setuptools>=1.0 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy) Requirement already satisfied (use --upgrade to upgrade): idna>=2.0 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy) Requirement already satisfied (use --upgrade to upgrade): cffi>=1.4.1 in /Library/Python/2.7/site-packages (from cryptography>=0.7->pyOpenSSL->Scrapy) Requirement already satisfied (use --upgrade to upgrade): pycparser in /Library/Python/2.7/site-packages (from cffi>=1.4.1->cryptography>=0.7->pyOpenSSL->Scrapy) Installing collected packages: Twisted, Scrapy Running setup.py install for Twisted ... done Successfully installed Scrapy-1.0.5 Twisted-15.5.0 |
安装成功。
在OS X 10.11 (El Capitan)中用pip安装Scrapy遇到无法更新six包的错误
在OSX 10.11 (El Capitan) 中用pip安装Scrapy时遇到了这样的问题：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
MacBook-Pro:~ EverybodyLies$ sudo pip install Scrapy The directory '/Users/EverybodyLies/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. The directory '/Users/EverybodyLies/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag. Collecting Scrapy Downloading Scrapy-1.0.5-py2-none-any.whl (291kB) 100% |████████████████████████████████| 294kB 1.5MB/s Collecting cssselect>=0.9 (from Scrapy) Downloading cssselect-0.9.1.tar.gz Collecting queuelib (from Scrapy) Downloading queuelib-1.4.2-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Scrapy) Collecting w3lib>=1.8.0 (from Scrapy) Downloading w3lib-1.13.0-py2.py3-none-any.whl Collecting lxml (from Scrapy) Downloading lxml-3.5.0.tar.gz (3.8MB) 100% |████████████████████████████████| 3.8MB 106kB/s Collecting Twisted>=10.0.0 (from Scrapy) Downloading Twisted-15.5.0.tar.bz2 (3.1MB) 100% |████████████████████████████████| 3.1MB 127kB/s Collecting six>=1.5.2 (from Scrapy) Downloading six-1.10.0-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): service-identity in /Library/Python/2.7/site-packages (from Scrapy) Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from Twisted>=10.0.0->Scrapy) Requirement already satisfied (use --upgrade to upgrade): characteristic>=14.0.0 in /Library/Python/2.7/site-packages (from service-identity->Scrapy) Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules in /Library/Python/2.7/site-packages (from service-identity->Scrapy) Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /Library/Python/2.7/site-packages (from service-identity->Scrapy) Requirement already satisfied (use --upgrade to upgrade): setuptools in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from zope.interface>=3.6.0->Twisted>=10.0.0->Scrapy) Installing collected packages: cssselect, queuelib, six, w3lib, lxml, Twisted, Scrapy Running setup.py install for cssselect ... done Found existing installation: six 1.4.1 DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project. Uninstalling six-1.4.1: Exception: Traceback (most recent call last): File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 209, in main status = self.run(options, args) File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 317, in run prefix=options.prefix_path, File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 725, in install requirement.uninstall(auto_confirm=True) File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 752, in uninstall paths_to_remove.remove(auto_confirm) File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 115, in remove renames(path, new_path) File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 266, in renames shutil.move(old, new) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move copy2(src, real_dst) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2 copystat(src, dst) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat os.chflags(dst, st.st_flags) OSError: [Errno 1] Operation not permitted: '/tmp/pip-2O_yX_-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info' |
去网上查找解决方法，看到了这个：“OSError: [Errno 1] Operation not permitted” when installing Scrapy in OSX 10.11 (El Capitan) (System Integrity Protection)，发现问题一模一样，于是就按照回答开始尝试。
按照最高票回答的方法尝试手动更新six：
|
1 2 3 4 5 6 |
MacBook-Pro:~ EverybodyLies$ sudo -H pip install --ignore-installed six Password: Collecting six Using cached six-1.10.0-py2.py3-none-any.whl Installing collected packages: six Successfully installed six-1.4.1 |
然而并没有成功更新！于是去看了：Source: github.com/pypa/pip/issues/3165
按照里面的某些讨论进行尝试：
|
1 |
MacBook-Pro:~ EverybodyLies$ sudo -H pip install Scrapy --upgrade --ignore-installed six |
的确没有再次报错，但是我觉得并没有解决这个问题（后来检查日志应该是更新成功了）。
另一个同学在尝试中使用easy_install升级six，解决了这个问题，我也进行了这种尝试：
|
1 2 3 4 5 6 7 8 9 |
MacBook-Pro:~ EverybodyLies$ sudo easy_install "six>=1.5.2" Password: Searching for six>=1.5.2 Best match: six 1.10.0 Adding six 1.10.0 to easy-install.pth file Using /Library/Python/2.7/site-packages Processing dependencies for six>=1.5.2 Finished processing dependencies for six>=1.5.2 |
非常顺利就能完成six的升级。
一些思考：
在解决这个问题的过程中我一开始觉得是10.11中SIP的问题，而且很多文章也说是SIP的问题。
但是后来另一个同学用easy_install很容易的就升级了six，我突然意识到可能是six这个包本来是用easy_install安装的，因此pip才无法卸载six。