a [f D@sUdZddlZddlZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z mZmZmZmZddlmZddlmZmZddlmZddlmZdd lmZdd lmZdd lm Z e rdd lm!Z!dd l"m#Z#dgZ$e ee%e&fZ'e e&gee&effZ(e)dej*Z+e)dZ,e)dZ-ej.ej/ej0ej1ej2ej3ej4ej5ej6ej7ej8ej9ej:ej;gZd<ej?ej@ejAgZBee dej=fe>d<ddeBDZCGdddZDGdddZEdBe&eedeejeFeeDdddZGe&e'dd d!ZHdCeEeejeDd"d#d$ZIdDee&e%edeejeedd%d&d'ZJdEeEeejedeeeeDd(d)d*ZKGd+d,d,ZLede(d-d.d/ZMe&ee&e&fd0d1d2ZNGd3d4d4eOZPejQd5d6d7ZRe'e'd8d9d:ZSe'e'd8d;d<ZTe'e'd8d=d>ZUe&eee&e&fd?d@dAZVdS)Fz Requirements file parsing N)Values) TYPE_CHECKINGAnyCallableDictIterableIteratorListOptionalTuple) cmdoptions)InstallationErrorRequirementsFileParseError) SearchScope) PipSession)raise_for_status) auto_decode)get_url_scheme)NoReturn) PackageFinderparse_requirementsz^(http|https|file):z (^|\s+)#.*$z#(?P\$\{(?P[A-Z0-9_]+)\}).SUPPORTED_OPTIONSSUPPORTED_OPTIONS_REQcCsg|]}t|jqS)strdest).0orr>/usr/lib/python3.9/site-packages/pip/_internal/req/req_file.py Nrc @s8eZdZdeeeeeeeefeeddddZdS)ParsedRequirementN) requirement is_editable comes_from constraintoptions line_sourcereturncCs(||_||_||_||_||_||_dSN)r"r#r$r&r%r')selfr"r#r$r%r&r'rrr__init__Rs zParsedRequirement.__init__)NN) __name__ __module__ __qualname__rboolr rrr+rrrrr!Qsr!c@s$eZdZeeeeeddddZdS) ParsedLineN)filenamelinenoargsoptsr%r(cCsZ||_||_||_||_|r0d|_d|_||_n&|jrPd|_d|_|jd|_nd|_dS)NTFr)r1r2r4r%is_requirementr#r"Z editables)r*r1r2r3r4r%rrrr+dszParsedLine.__init__)r,r-r.rintrr/r+rrrrr0csr0Fr)r1sessionfinderr&r%r(c csFt|}t||}|||D]"}t||||d}|dur|VqdS)aqParse a requirements file and yield ParsedRequirement instances. :param filename: Path or url of requirements file. :param session: PipSession instance. :param finder: Instance of pip.index.PackageFinder. :param options: cli options. :param constraint: If true, parsing a constraint file rather than requirements file. )r&r8r7N)get_line_parserRequirementsFileParserparse handle_line) r1r7r8r&r% line_parserparserZ parsed_line parsed_reqrrrr~s )contentr(cCs,t|dd}t|}t|}t|}|S)zxSplit, filter, and join lines, and return a line iterator :param content: the content of the requirements file )start) enumerate splitlines join_linesignore_commentsexpand_env_variables)r@ lines_enumrrr preprocesss rI)liner&r(cCsd|jrdnd|j|j}|js&J|jrBt|j|j||jdS|rTt ||j i}t D],}||j j vr\|j j |r\|j j |||<q\d|jd|j}t|j|j||j||dSdS)Nz{} {} (line {})z-cz-r)r"r#r$r%line  of )r"r#r$r%r&r') formatr%r1r2r5r#r!r"r Zcheck_install_build_globalr4SUPPORTED_OPTIONS_REQ_DEST__dict__)rJr&Zline_comes_fromZ req_optionsrr'rrrhandle_requirement_lines8  rP)r4r1r2r8r&r7r(cs8r4|jr|j_|jr4jfdd|jD|r4|j}|j}|jrT|jg}|jdurbg}|jrt||j|jr|jd}tj tj |} tj | |} tj | r| }|||r||t||d} | |_|jr||jr||r4|jp gD]$} d|d|} |j| | dqdS) Nc3s|]}|jvr|VqdSr))features_enabled)rfr&rr sz%handle_option_line..Tr) find_links index_urlsrKrL)source)require_hashesrQextendrUrV index_urlno_indexZextra_index_urlsospathdirnameabspathjoinexistsappendZupdate_index_urlsr search_scopepreZset_allow_all_prereleases prefer_binaryZset_prefer_binaryZ trusted_hostsZadd_trusted_host)r4r1r2r8r&r7rUrVvalueZreq_dirZrelative_to_reqs_filerchostrWrrSrhandle_option_linesL       rh)rJr&r8r7r(cCs4|jrt||}|St|j|j|j|||dSdS)aHandle a single parsed requirements line; This can result in creating/yielding requirements, or updating the finder. :param line: The parsed line to be processed. :param options: CLI options. :param finder: The finder - updated by non-requirement lines. :param session: The session - updated by non-requirement lines. Returns a ParsedRequirement object if the line is a requirement line, otherwise returns None. For lines that contain requirements, the only options that have an effect are from SUPPORTED_OPTIONS_REQ, and they are scoped to the requirement. Other options from SUPPORTED_OPTIONS may be present, but are ignored. For lines that do not contain requirements, the only options that have an effect are from SUPPORTED_OPTIONS. Options from SUPPORTED_OPTIONS_REQ may be present, but are ignored. These lines may contain multiple options (although our docs imply only one is supported), and all our parsed and affect the finder. N)r5rPrhr4r1r2)rJr&r8r7r?rrrr<s r<c@s`eZdZeeddddZeeee dddZ eeee ddd Z eeee dd d Z dS) r:N)r7r=r(cCs||_||_dSr))_session _line_parser)r*r7r=rrrr+<szRequirementsFileParser.__init__)r1r%r(ccs|||EdHdS)z*Parse a given file, yielding parsed lines.N)_parse_and_recurse)r*r1r%rrrr;DszRequirementsFileParser.parseccs|||D]}|js|jjs&|jjr|jjr@|jjd}d}n|jjd}d}t|rjtj ||}n t|st j t j ||}|||EdHq |Vq dS)NrFT) _parse_filer5r4 requirements constraints SCHEME_REsearchurllibr;urljoinr\r]r`r^rk)r*r1r%rJZreq_pathZnested_constraintrrrrkHs(     z)RequirementsFileParser._parse_and_recursec cst||j\}}t|}|D]l\}}z||\}} Wn>tyt} z&d|d| j} t| WYd} ~ n d} ~ 00t|||| |VqdS)NzInvalid requirement:  )get_file_contentrirIrjOptionParsingErrormsgrr0) r*r1r%_r@rH line_numberrJargs_strr4ervrrrrlgs z"RequirementsFileParser._parse_file) r,r-r.r LineParserr+rr/rr0r;rkrlrrrrr:;s  r:)r8r(cs ttttfdfdd }|S)NrJr(csJt}|}d|_r j|_t|\}}|t||\}}||fSr)) build_parserZget_default_valuesrZZformat_controlbreak_args_options parse_argsshlexsplit)rJr>defaultsryZ options_strr4rwr8rr parse_line~s z#get_line_parser..parse_line)rr r)r8rrrrr9}sr9r|cCsf|d}g}|dd}|D]2}|ds6|dr<qRq|||dqd|d|fS)zBreak up the line into an args and options string. We only want to shlex (and then optparse) the options, not the args. args can contain markers which are corrupted by shlex.  N-z--r)r startswithrbpopr`)rJtokensr3r&tokenrrrr~s    r~c@seZdZeddddZdS)ruN)rvr(cCs ||_dSr))rvr*rvrrrr+szOptionParsingError.__init__)r,r-r.rr+rrrrrusru)r(cCsJtjdd}tt}|D]}|}||qttdddd}||_|S)z7 Return a parser for parsing requirement lines F)Zadd_help_optionr)r*rvr(cSs t|dSr))rurrrr parser_exitsz!build_parser..parser_exit)optparse OptionParserrrZ add_optionrrexit)r>Zoption_factoriesZoption_factoryoptionrrrrr}s  r})rHr(ccsd}g}|D]\}}|dr(t|rvt|r:d|}|rj|||dusTJ|d|fVg}q||fVq |s~|}||dq |r|dusJ|d|fVdS)zJoins a line ending in '' with the previous line (except when following comments). The joined line takes on the index of the first line. N\r)endswith COMMENT_REmatchrbr`strip)rHZprimary_line_numberZnew_linerxrJrrrrEs$      rEccs4|D]*\}}td|}|}|r||fVqdS)z1 Strips comments and filter empty lines. rN)rsubr)rHrxrJrrrrFs   rFccsL|D]B\}}t|D]$\}}t|}|s.q|||}q||fVqdS)aReplace all environment variables that can be retrieved via `os.getenv`. The only allowed format for environment variables defined in the requirement file is `${MY_VARIABLE_1}` to ensure two things: 1. Strings that contain a `$` aren't accidentally (partially) expanded. 2. Ensure consistency across platforms for requirement files. These points are the result of a discussion on the `github pull request #3514 `_. Valid characters in variable names follow the `POSIX standard `_ and are limited to uppercase letter, digits and the `_` (underscore). N) ENV_VAR_REfindallr\getenvreplace)rHrxrJZenv_varZvar_namerfrrrrGs  rG)urlr7r(c Cst|}|dvr.||}t||j|jfSz:t|d}t|}Wdn1s\0YWn2ty}zt d|WYd}~n d}~00||fS)aGets the content of a file; it may be a filename, file: URL, or http: URL. Returns (location, content). Content is unicode. Respects # -*- coding: declarations on the retrieved files. :param url: File path or url. :param session: PipSession instance. )httphttpsfilerbNz"Could not open requirements file: ) rgetrrtextopenrreadOSErrorr )rr7schemeZresprRr@excrrrrts   .$rt)NNF)N)NNN)NNN)W__doc__rr\rer urllib.parserqrtypingrrrrrrr r r Zpip._internal.clir Zpip._internal.exceptionsr rZ!pip._internal.models.search_scoperZpip._internal.network.sessionrZpip._internal.network.utilsrZpip._internal.utils.encodingrZpip._internal.utils.urlsrrZ"pip._internal.index.package_finderr__all__r6rZ ReqFileLinesr{compileIrorrrZZextra_index_urlr[rnrmZeditablerUZ no_binaryZ only_binaryrerXrdZ trusted_hostZuse_new_featurerZOption__annotations__Zinstall_optionsZglobal_optionshashrrNr!r0r/rrIrPrhr<r:r9r~ Exceptionrurr}rErFrGrtrrrrs ,            1 ? ,B