Pip search fails with private index url

[j-bennet]

Environment

• pip version: 19.2.2
• Python version: 3.6.5
• OS: macOS Mojave

Description

I am searching for a package in custom repository. The repository is hosted with SSL and protected by basic auth. pip search command fails with 403 Client Error as follows:

--- ~ » pip search -i "https://user:[email protected]/cogtree/" -vvv engicore
Found credentials in url for pypi.cogtree.com
Starting new HTTPS connection (1): pypi.cogtree.com:443
https://pypi.cogtree.com:443 "POST /cogtree/ HTTP/1.1" 403 162
ERROR: HTTP error 403 while getting https://user:[email protected]/cogtree/
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 188, in main
    status = self.run(options, args)
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/commands/search.py", line 48, in run
    pypi_hits = self.search(query, options)
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/commands/search.py", line 65, in search
    hits = pypi.search({'name': query, 'summary': query}, 'or')
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/download.py", line 1010, in request
    response.raise_for_status()
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_vendor/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
pip._vendor.requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://pypi.cogtree.com/cogtree/

I also tried adding simple to the URL, which gives me a 405:

--- ~ » pip search -i "https://user:[email protected]/cogtree/simple/" -vvv engicore                                                                                                                                                  2 ↵
Found credentials in url for pypi.cogtree.com
Starting new HTTPS connection (1): pypi.cogtree.com:443
https://pypi.cogtree.com:443 "POST /cogtree/simple/ HTTP/1.1" 405 166
ERROR: HTTP error 405 while getting https://user:[email protected]/cogtree/simple/
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 188, in main
    status = self.run(options, args)
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/commands/search.py", line 48, in run
    pypi_hits = self.search(query, options)
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/commands/search.py", line 65, in search
    hits = pypi.search({'name': query, 'summary': query}, 'or')
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_internal/download.py", line 1010, in request
    response.raise_for_status()
  File "/Users/irina/.pyenv/versions/3.6.5/lib/python3.6/site-packages/pip/_vendor/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
pip._vendor.requests.exceptions.HTTPError: 405 Client Error: Not Allowed for url: https://pypi.cogtree.com/cogtree/simple/

Expected behavior

pip search to not fail.

I am able to browse the repository using the web browser. This also returns a valid response:

--- ~ » ipython
In [1]: import requests

In [2]: requests.get('https://user:[email protected]/cogtree/')
Out[2]: <Response [200]>

[uranusjr]

The search command uses a non-standard XML-RPC API (see #395). cogtree.com (I don’t know anything about this website) likely does not have it implemented.

[xavfernandez]

Hello, we'd need to know what software is behind pypi.cogtree.com (devpi, etc) to help.

But as @uranusjr said, pip search uses a specific endpoint, usually different than /simple.
For instance for pypi.org: pip install uses https://pypi.org/simple/ but pip search uses https://pypi.org/pypi/.

[j-bennet]

Yes, apparently pypi.cogtree.com is a simple mirror created with pip2pi:

https://github.com/dan-blanchard/pip2pi

It would not have the endpoint pip search is looking for.

I'm closing the issue.