scrapy 运行日志
**************ProxyMiddleware not pass************171.38.66.23:9999
2017-01-26 23:05:38 [selenium.webdriver.remote.remote_connection] DEBUG: POST
http://127.0.0.1:57234/wd/hub/session {"desiredCapabilities": {"browserName": "phantomjs", "version": "", "platform": "ANY", "javascriptEnabled": true}, "requiredCapabilities": {}}
2017-01-26 23:05:38 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2017-01-26 23:05:38 [selenium.webdriver.remote.remote_connection] DEBUG: POST
http://127.0.0.1:57234/wd/hub/session/e5c0aeb0-e3d8-11e6-a629-15e39efe8c5a/url {"url": "Matweb Engineering Materials List", "sessionId": "e5c0aeb0-e3d8-11e6-a629-15e39efe8c5a"}
phantonjs 中间件代码:
def process_request(self, request, spider):
driver = webdriver.PhantomJS(executable_path=r"/Users/apple/phantomjs-2.1.1-macosx/bin/phantomjs")
driver.get(request.url)
body = driver.page_source
print ("访问"+request.url)
return HtmlResponse(driver.current_url, body=body, encoding='utf-8', request=request)
每次执行到 selenium.webdriver.remote.remote_connection 。 都会卡上 20s , 请问这是为什么?