您现在的位置是: 网站首页> 学习笔记 学习笔记
scrapydweb部署爬虫时报错NotADirectoryError: [Errno 20] Not a directory: '/tmp/
2022-07-12 [scrapy] 3629人已围观
在scrapydweb上部署爬虫时报错,具体信息如下:
Oops! Something went wrong.
Fail to deploy project, got status: error
{
"auth": null,
"message": "See details below",
"node_name": "583a69ced86b",
"status": "error",
"status_code": 200,
"url": "http://192.168.255.128:6800/addversion.json",
"when": "2022-07-12 09:47:09"
}
Details
Traceback (most recent call last):
File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/site-packages/scrapyd/runner.py", line 46, in <module>
main()
File "/usr/local/lib/python3.6/site-packages/scrapyd/runner.py", line 43, in main
execute()
File "/usr/local/lib/python3.6/site-packages/scrapy/cmdline.py", line 114, in execute
settings = get_project_settings()
File "/usr/local/lib/python3.6/site-packages/scrapy/utils/project.py", line 68, in get_project_settings
settings.setmodule(settings_module_path, priority='project')
File "/usr/local/lib/python3.6/site-packages/scrapy/settings/__init__.py", line 287, in setmodule
module = import_module(module)
File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
File "/tmp/littlebee-2022-07-12T09_43_26-0s96tbil.egg/littlebee/settings.py", line 162, in <module>
File "/usr/local/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/usr/local/lib/python3.6/os.py", line 210, in makedirs
makedirs(head, mode, exist_ok)
File "/usr/local/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
NotADirectoryError: [Errno 20] Not a directory: '/tmp/littlebee-2022-07-12T09_43_26-0s96tbil.egg/littlebee'
先定位出问题的地方,如图
是因为settings中配置日志文件路径时使用了
os.path.dirname(os.path.abspath(__file__))
而在scrapyd-client中有说不能这样用,需要替换用法; You should avoid using file in your project code as it doesn't play well with eggs. Consider using pkgutil.get_data instead. Instead of:
open(os.path.join(os.path.abspath(os.path.dirname(__file__)), "tools/json/test.json")) # BAD
Use:
pkgutil.get_data("projectname", "tools/json/test.json")
所以按这个修改就可以了。
相关文章
文章评论
暂无评论添加评论
点击排行
本栏推荐
标签云
热评文章
- django使用qq邮箱发送邮件
- mysql8设置数据库远程连接
- pip修改下载源为国内源
- win10看不到win7共享的文件夹的解决方法
- SQLyog连接 Mysql 8.0.11 报error no.1251- Client does not support authentic...
- 使用Oracel Net Nanager配置Oracle数据库远程访问
- 将anaconda的下载源切换为国内的源
- Python+selenium+firefox设置代理IP
- selenium+firefox+js实现动态设置firefox浏览器代理IP
- scrapy文件下载(高新技术企业认定网)
- Python调用JS代码
- Chrome浏览器的overrides的使用
站点信息
- 建站时间:2021-01-01
- 网站程序:Django 3.1.2
- 文章统计:53篇
- 文章评论:31条
- 统计数据: