File: //lib/python2.7/site-packages/urllib3/poolmanager.pyc
�
i�$ec @@ s- d d l m Z d d l Z d d l Z d d l Z d d l m Z d d l m Z m Z m
Z
d d l m Z m
Z
m Z m Z m Z d d l m Z d d l m Z d d l m Z d d
l m Z d d l m Z d d l m Z d
d d g Z e j e � Z d d d d d d d d d f Z! d d d d d d d d d! d" d# d$ d% d& d' d( d) d* d+ d, d- d. d/ d0 d1 d2 f Z" e j# d3 e" � Z$ d d4 f Z% e j# d5 e% � Z& d6 � Z' i e j( e' e$ � d7 6e j( e' e$ � d8 6Z) i e d7 6e d8 6Z* d
e f d9 � � YZ+ d e+ f d: � � YZ, d; � Z- d S(< i ( t absolute_importNi ( t RecentlyUsedContainer( t HTTPConnectionPoolt HTTPSConnectionPoolt port_by_scheme( t LocationValueErrort
MaxRetryErrort ProxySchemeUnknownt ProxySchemeUnsupportedt URLSchemeUnknown( t six( t urljoin( t RequestMethods( t connection_requires_http_tunnel( t Retry( t parse_urlt PoolManagert ProxyManagert proxy_from_urlt key_filet cert_filet cert_reqst ca_certst ssl_versiont ca_cert_dirt ssl_contextt key_passwordt server_hostnamet
key_schemet key_hostt key_portt key_timeoutt key_retriest
key_strictt key_blockt key_source_addresst key_key_filet key_key_passwordt
key_cert_filet
key_cert_reqst key_ca_certst key_ssl_versiont key_ca_cert_dirt key_ssl_contextt key_maxsizet key_headerst
key__proxyt key__proxy_headerst key__proxy_configt key_socket_optionst key__socks_optionst key_assert_hostnamet key_assert_fingerprintt key_server_hostnamet PoolKeyt use_forwarding_for_httpst ProxyConfigc C@ s | j � } | d j � | d <| d j � | d <xG d D]? } | | k r; | | d k r; t | | j � � | | <q; q; W| j d � } | d k r� t | � | d <n x1 t | j � � D] } | j | � | d | <q� Wx* | j
D] } | | k r� d | | <q� q� W| | � S(
s�
Create a pool key out of a request context dictionary.
According to RFC 3986, both the scheme and host are case-insensitive.
Therefore, this function normalizes both before constructing the pool
key for an HTTPS request. If you wish to change this behaviour, provide
alternate callables to ``key_fn_by_scheme``.
:param key_class:
The class to use when constructing the key. This should be a namedtuple
with the ``scheme`` and ``host`` keys at a minimum.
:type key_class: namedtuple
:param request_context:
A dictionary-like object that contain the context for a request.
:type request_context: dict
:return: A namedtuple that can be used as a connection pool key.
:rtype: PoolKey
t schemet hostt headerst _proxy_headerst _socks_optionst socket_optionst key_( s headerss _proxy_headerss _socks_optionsN( t copyt lowert Nonet frozensett itemst gett tuplet listt keyst popt _fields( t key_classt request_contextt contextt keyt socket_optst field( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt _default_key_normalizerO s
! t httpt httpsc B@ s� e Z d Z d Z d Z d d d � Z d � Z d � Z d d � Z d � Z
d d d d � Z d � Z d d
� Z
d d � Z d � Z d
� Z d � Z e d � Z RS( s$
Allows for arbitrary requests while transparently keeping track of
necessary connection pools for you.
:param num_pools:
Number of connection pools to cache before discarding the least
recently used pool.
:param headers:
Headers to include with all requests, unless other headers are given
explicitly.
:param \**connection_pool_kw:
Additional parameters are used to create fresh
:class:`urllib3.connectionpool.ConnectionPool` instances.
Example::
>>> manager = PoolManager(num_pools=2)
>>> r = manager.request('GET', 'http://google.com/')
>>> r = manager.request('GET', 'http://google.com/mail')
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> len(manager.pools)
2
i
c K@ sD t j | | � | | _ t | � | _ t | _ t j � | _ d S( N( R t __init__t connection_pool_kwR t poolst pool_classes_by_schemet key_fn_by_schemeR@ ( t selft num_poolsR; RU ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyRT � s
c C@ s | S( N( ( RY ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt __enter__� s c C@ s | j � t S( N( t cleart False( RY t exc_typet exc_valt exc_tb( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt __exit__� s
c C@ s� | j | } | d k r+ | j j � } n x d D] } | j | d � q2 W| d k r| x! t D] } | j | d � q_ Wn | | | | � S( s�
Create a new :class:`urllib3.connectionpool.ConnectionPool` based on host, port, scheme, and
any additional pool keyword arguments.
If ``request_context`` is provided, it is provided as keyword arguments
to the pool class used. This method is used to actually create the
connection pools handed out by :meth:`connection_from_url` and
companion methods. It is intended to be overridden for customization.
R9 R: t portRR N( s schemes hosts port( RW RB RU R@ RI t SSL_KEYWORDS( RY R9 R: Rb RL t pool_clsRN t kw( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt _new_pool� s
c C@ s | j j � d S( s�
Empty our store of pools and direct them all to close.
This will not affect in-flight connections, but they will not be
re-used after completion.
N( RV R\ ( RY ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyR\ � s RR c C@ sz | s t d � � n | j | � } | p- d | d <| sY t j | d j � d � } n | | d <| | d <| j | � S( s�
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the host, port, and scheme.
If ``port`` isn't given, it will be derived from the ``scheme`` using
``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is
provided, it is merged with the instance's ``connection_pool_kw``
variable and used to create the new connection pool, if one is
needed.
s No host specified.RR R9 iP Rb R: ( R t _merge_pool_kwargsR RE RA t connection_from_context( RY R: Rb R9 t pool_kwargsRL ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt connection_from_host� s
c C@ sV | d j � } | j j | � } | s7 t | � � n | | � } | j | d | �S( s�
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the request context.
``request_context`` must at least contain the ``scheme`` key and its
value must be a key in ``key_fn_by_scheme`` instance variable.
R9 RL ( RA RX RE R t connection_from_pool_key( RY RL R9 t pool_key_constructort pool_key( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyRh � s c
C@ sy | j j �g | j j | � } | r) | S| d } | d } | d } | j | | | d | �} | | j | <Wd QX| S( s
Get a :class:`urllib3.connectionpool.ConnectionPool` based on the provided pool key.
``pool_key`` should be a namedtuple that only contains immutable
objects. At a minimum it must have the ``scheme``, ``host``, and
``port`` fields.
R9 R: Rb RL N( RV t lockRE Rf ( RY Rm RL t poolR9 R: Rb ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyRk s
c C@ s4 t | � } | j | j d | j d | j d | �S( s�
Similar to :func:`urllib3.connectionpool.connection_from_url`.
If ``pool_kwargs`` is not provided and a new pool needs to be
constructed, ``self.connection_pool_kw`` is used to initialize
the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs``
is provided, it is used instead. Note that if a new pool does not
need to be created for the request, the provided ``pool_kwargs`` are
not used.
Rb R9 Ri ( R Rj R: Rb R9 ( RY t urlRi t u( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt connection_from_url s c C@ sq | j j � } | rm xU | j � D]D \ } } | d k r\ y | | =Wqf t k
rX qf Xq" | | | <q" Wn | S( s
Merge a dictionary of override values for self.connection_pool_kw.
This does not modify self.connection_pool_kw and returns a new dict.
Any keys in the override dictionary with a value of ``None`` are
removed from the merged dictionary.
N( RU R@ RD RB t KeyError( RY t overridet base_pool_kwargsRN t value( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyRg / s
c C@ s- | j d k r t St | j | j | j � S( s�
Indicates if the proxy requires the complete destination URL in the
request. Normally this is only needed when not using an HTTP CONNECT
tunnel.
N( t proxyRB R] R
t proxy_configR9 ( RY t
parsed_url( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt! _proxy_requires_url_absolute_formC s c C@ s^ | j d k s | d k r d S| j j d k r5 d St j rZ | j j rZ t d � � n d S( s�
Validates that were not attempting to do TLS in TLS connections on
Python2 or with unsupported SSL implementations.
RS Nsf Contacting HTTPS destinations through HTTPS proxies 'via CONNECT tunnels' is not supported in Python 2( Rw RB R9 R
t PY2Rx R7 R ( RY t
url_scheme( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt$ _validate_proxy_scheme_url_selectionP s c K@ s4 t | � } | j | j � | j | j d | j d | j �} t | d <t | d <d | k rv | j j � | d <n | j | � r� | j
| | | � } n | j
| | j | � } | o� | j � } | s� | St
| | � } | j d k r� d } n | j d � } t | t � s.t j | d | �} n | j r�| j | � r�t t j | d � � }
x= |
D]2 } | j � | j k rg| d j | d � qgqgWn y" | j | | d | d
| �} Wn+ t k
r�| j r�| j � � n | SX| | d <| | d <t j d | | � | j � | j
| | | � S(
sN
Same as :meth:`urllib3.HTTPConnectionPool.urlopen`
with custom cross-host redirect logic and only sends the request-uri
portion of the ``url``.
The given ``url`` parameter must be absolute, such that an appropriate
:class:`urllib3.connectionpool.ConnectionPool` can be chosen for it.
Rb R9 t assert_same_hostt redirectR; i/ t GETt retriest responset _pools Redirecting %s -> %sN(! R R} R9 Rj R: Rb R] R; R@ Rz t urlopent request_urit get_redirect_locationR t statusRE t
isinstanceR t from_intt remove_headers_on_redirectt is_same_hostRG R
t iterkeysRA RI RB t incrementR t raise_on_redirectt
drain_connt logt info( RY t methodRp R Re Rq t connR� t redirect_locationR� R; t header( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyR� a sJ $
"
N( t __name__t
__module__t __doc__RB Rw Rx RT R[ Ra Rf R\ Rj Rh Rk Rr Rg Rz R} t TrueR� ( ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyR � s
c B@ sP e Z d Z d d d d e d � Z d d d d � Z d d � Z e d � Z RS( s>
Behaves just like :class:`PoolManager`, but sends all requests through
the defined proxy, using the CONNECT method for HTTPS URLs.
:param proxy_url:
The URL of the proxy to be used.
:param proxy_headers:
A dictionary containing headers that will be sent to the proxy. In case
of HTTP they are being sent with each request, while in the
HTTPS/CONNECT case they are sent only once. Could be used for proxy
authentication.
:param proxy_ssl_context:
The proxy SSL context is used to establish the TLS connection to the
proxy when using HTTPS proxies.
:param use_forwarding_for_https:
(Defaults to False) If set to True will forward requests to the HTTPS
proxy to be made on behalf of the client instead of creating a TLS
tunnel via the CONNECT method. **Enabling this flag means that request
and response headers and content will be visible from the HTTPS proxy**
whereas tunneling keeps request and response headers and content
private. IP address, target hostname, SNI, and port are always visible
to an HTTPS proxy even when this flag is disabled.
Example:
>>> proxy = urllib3.ProxyManager('http://localhost:3128/')
>>> r1 = proxy.request('GET', 'http://google.com/')
>>> r2 = proxy.request('GET', 'http://httpbin.org/')
>>> len(proxy.pools)
1
>>> r3 = proxy.request('GET', 'https://httpbin.org/')
>>> r4 = proxy.request('GET', 'https://twitter.com/')
>>> len(proxy.pools)
3
i
c
K@ s t | t � r. d | j | j | j f } n t | � } | j d k r[ t | j � � n | j s� t j | j d � } | j d | � } n | | _
| p� i | _ | | _ t
| | � | _ | j
| d <| j | d <| j | d <t t | � j | | | � d S(
Ns
%s://%s:%iRR RS iP Rb t _proxyR< t
_proxy_config( s https https( R� R R9 R: Rb R R R RE t _replaceRw t
proxy_headerst proxy_ssl_contextR8 Rx t superR RT (
RY t proxy_urlRZ R; R� R� R7 RU Rw Rb ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyRT � s&
RR c C@ sb | d k r. t t | � j | | | d | �St t | � j | j j | j j | j j d | �S( NRS Ri ( R� R Rj Rw R: Rb R9 ( RY R: Rb R9 Ri ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyRj � s
c C@ sI i d d 6} t | � j } | r/ | | d <n | rE | j | � n | S( s�
Sets headers needed by proxies: specifically, the Accept and Host
headers. Only sets headers not provided by the user.
s */*t Acceptt Host( R t netloct update( RY Rp R; t headers_R� ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt _set_proxy_headers� s
c K@ sw t | � } t | j | j | j � sU | j d | j � } | j | | � | d <n t t | � j
| | d | | �S( s@ Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.R; R ( R R
Rw Rx R9 RE R; R� R� R R� ( RY R� Rp R Re Rq R; ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyR� s
N(
R� R� R� RB R] RT Rj R� R� R� ( ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyR � s &
c K@ s t d | | � S( NR� ( R ( Rp Re ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyR s (. t
__future__R t collectionst functoolst loggingt _collectionsR t connectionpoolR R R t
exceptionsR R R R R t packagesR
t packages.six.moves.urllib.parseR t requestR t
util.proxyR
t
util.retryR t util.urlR t __all__t getLoggerR� R� Rc t _key_fieldst
namedtupleR6 t _proxy_config_fieldsR8 RQ t partialRX RW R R R ( ( ( s7 /usr/lib/python2.7/site-packages/urllib3/poolmanager.pyt <module> sz ( 5� s