o cď@s^dZddlZddlZddlZddlZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd lm Z dd lm Z dd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZertddlmZzddlmZWn eydZYnweeZGdddZ de!de!fddZ"de!de!fddZ#defddZ$dS)z@ApacheParser is a member object of the ApacheConfigurator class.N) Collection)Dict)Iterable)List)Mapping)Optional)Pattern)Set)Tuple) TYPE_CHECKING)Union)errors)os) apache_util) constants)ApacheConfigurator)Augeasc @seZdZUdZedZeed<hdZ e e ed< dyde dd d e d e e d fd df ddZde d dfddZd eeee ffddZd e e fddZdzddZdee d dfddZdee d dfddZd e d!e d dfd"d#Zd$e d dfd%d&Zdzd'd(Zdzd)d*Zdzd+d,Zdzd-d.Zdzd/d0Zdzd1d2Z d3e d4e d ee fd5d6Z!d7e d8e d4ee d dfd9d:Z"d7e d;e d e fdd?Z$d7e%e d8e%e d4eee e fd dfd@dAZ&d7e%e dBe d4eee e fd dfdCdDZ'd7e dEe d dfdFdGZ(d{dHe dIe%e d ee fdJdKZ)  Ld|d8e dHe%e dIe%e dMed ee f dNdOZ*dPe d e%e fdQdRZ+d e fdSdTZ,d3ee d ee fdUdVZ-dPe dWe e e.e fd efdXdYZ/dHe d e fdZd[Z0dHe%e d e%e fd\d]Z1d^e d e fd_d`Z2dae d dfdbdcZ3dde%e d efdedfZ4dde%e d efdgdhZ5dde die6e ee fd efdjdkZ7dae d e eeffdldmZ8dae d dfdndoZ9dpe d dfdqdrZ:dzdsdtZ;d edS)} ApacheParseraClass handles the fine details of parsing the Apache Configuration. .. todo:: Make parsing general... remove sites-available etc... :ivar str root: Normalized absolute path to the server root directory. Without trailing slash. :ivar set modules: All module names that are currently enabled. :ivar dict loc: Location to place directives, root - configuration origin, default - user config file, name - NameVirtualHost, z \$\{[^ \}]*}arg_var_interpreter>*?[\] fnmatch_charsroot configuratorr vhostrootversion.returnNcCs||_t|_|stdi|_i|_i|_t j ||_ d| i|_||jd||||j|t|j|_|r^|t j |d|jjjdSdS)NzApache plugin support requires libaugeas0 and augeas-lenses version 1.2.0 or higher, please make sure you have you have those installed.r/)r init_augeasaugcheck_aug_versionr NotSupportedErrormodules parser_paths variablesrpathabspathr_find_config_rootloc parse_fileupdate_runtime_variablesstandardize_excl parse_modulesupdate_set_locationscopydeepcopyexisting_pathsoptions vhost_files)selfrrr r!r;L/opt/certbot/lib/python3.10/site-packages/certbot_apache/_internal/parser.py__init__2s, zApacheParser.__init__lensc Csv|jd}|D]0}|j|d}|r8||vr8d|dt|d|j|d|j|d}t|qdS) zVerify Augeas can parse all of the lens files. :param str lens: lens to check for errors :raises .errors.PluginError: If there has been an error in parsing with the specified lens. /augeas//errorz/lensz@There has been an error in parsing the file {0} on line {1}: {2} z/line/messageN)r%matchgetformatlenr PluginError)r:r> error_filesr+ lens_pathmsgr;r;r<check_parsing_errorsas   z!ApacheParser.check_parsing_errorscCsP|jddz|jd}Wnty|jdYdSw|jd|S)z Checks that we have recent enough version of libaugeas. If augeas version is recent enough, it will support case insensitive regexp matchingz/test/path/testing/argaRgUMeNTz,/test//*[self::arg=~regexp('argument', 'i')]z /test/pathF)r%setrC RuntimeErrorremove)r:matchesr;r;r<r&{s   zApacheParser.check_aug_versionc Cs|jd}|jdd|jd}z|jWnttfy1||d|j_ t dw|jd||jd}t}|rV|D]}| |j|ddqF|S) aLists files that have modified Augeas DOM but the changes have not been written to the filesystem yet, used by `self.save()` and ApacheConfigurator to check the file state. :raises .errors.PluginError: If there was an error in Augeas, in an attempt to save the configuration, or an error creating a checkpoint :returns: `set` of unsaved files z /augeas/savenoopr?z-Error saving files, check logs for more info.z/augeas/events/savedrAN) r%rDrMrCsaverNIOError_log_save_errorsr save_notesr rGadd)r: save_stateex_errs save_paths save_filesr+r;r;r< unsaved_filess&    zApacheParser.unsaved_filescCs*|r|jjd7_|jdSdS)zMakes sure that all Augeas dom changes are written to files to avoid loss of configuration directives when doing additional augeas parsing, causing a possible augeas.load() resulting dom reset z (autosave)N)r\rrVrSr:r;r;r<ensure_augeas_statesz ApacheParser.ensure_augeas_stater[cCsnd|j_|jd}z|jWn ty||w|r5|D] }|jd|q#|jdSdS)zSaves all changes to the configuration files. save() is called from ApacheConfigurator to handle the parser specific tasks of saving. :param list save_files: list of strings of file paths that we need to save. rRr?z/files/N) rrVr%rCrSrTrUrOload)r:r[rYsfr;r;r<rSs   zApacheParser.saverYc sfdd|jdD}|D]}td|j||dt|d|j|dqtdd d d |D|jj rGd |jj dSd dS)zfLog errors due to bad Augeas save. :param list ex_errs: Existing errors before save csg|]}|vr|qSr;r;).0erYr;r< sz1ApacheParser._log_save_errors..r?zError %s saving %s: %sr@rArBzUnable to save files: %s.%sz, css$|] }|dt|dVqdS)r@rAN)rF)raerrr;r;r< s"z0ApacheParser._log_save_errors..z Save Notes: rRN) r%rCloggerdebugrDrFerrorjoinrrV)r:rYnew_errsrer;rcr<rUszApacheParser._log_save_errors main_configinc_pathcCsh|td|s2td|t||t|d|tj|}tj |}|j |g |dSdS)zAdd Include for a new configuration file if one does not exist :param str main_config: file path to main Apache config file :param str inc_path: path of file to include IncludezAdding Include %s to %sN) find_dircase_irgrh get_aug_pathadd_dirrr+dirnamebasenamer7 setdefaultappend)r:rlrmnew_dirnew_filer;r;r< add_includes  zApacheParser.add_includemod_namecCsH|d|jvrd|j|d<d|d|jvr"d|jd|d<dSdS)z%Shortcut for updating parser modules._moduleNmod_z.c)r()r:rzr;r;r<add_mods zApacheParser.add_modcCsi|_||dS)zgReset the loaded modules list. This is called from cleanup to clear temporarily loaded modules.N)r(update_modulesr2r]r;r;r< reset_moduless zApacheParser.reset_modulesc Csi}|d}t|}d}t||krRt|}t||D]/\}}||}||}|rA|rA|||<||tj|ddd<qt d|ddqt||ks|j |dS)zIterates on the configuration until no new modules are loaded. ..todo:: This should be attempted to be done with a binary to avoid the iteration issue. Else... parse and enable mods at same time. LoadModuleNcz8Could not read LoadModule directive from Augeas path: %srA) roiterrFzipget_argrr+rtrgrhr(r3) r:modsrPiterator prev_size match_namematch_filenamerz mod_filenamer;r;r<r2s&       zApacheParser.parse_modulescCs|||dS)zAUpdate Includes, Defines and Includes from httpd config dump dataN)update_definesupdate_includesr~r]r;r;r<r0"s z%ApacheParser.update_runtime_variablescCst|jjj|_dS)z>Updates the dictionary of known variables in the configurationN)r parse_definesrr8get_defines_cmdr*r]r;r;r<r)szApacheParser.update_definescCsD|d}t|jjj}|r|D]}||s||qdSdS)z>Get includes from httpd process, and add them to DOM if neededrnN)rorparse_includesrr8get_includes_cmdparsed_in_currentr/)r:_rPir;r;r<r-s   zApacheParser.update_includescCs,t|jjj}|D] }||q dS)z:Get loaded modules from httpd process, and add them to DOMN)rr2rr8get_modules_cmdr}strip)r:rPmodr;r;r<r~;szApacheParser.update_modulesrPargscCsg}|dkr!t|D]\}}|dr|||ddq |St|D]0\}}|d|rU|t|dksE||dd|dsU|||dtd| q%|S)aFilter out directives with specific number of arguments. This function makes the assumption that all related arguments are given in order. Thus /files/apache/directive[5]/arg[2] must come immediately after /files/apache/directive[5]/arg[1]. Runs in 1 linear pass. :param str matches: Matches of all directives with arg nodes :param int args: Number of args you would like to filter :returns: List of directives that contain # of arguments. (arg is stripped off) /argN/arg[%d]) enumerateendswithrvrF)r:rPrfilteredrrCr;r;r<filter_args_numBs"   zApacheParser.filter_args_num aug_conf_path directivecCs||d}|j|ddd|d}|j||t|dkr.|j|d|dd St|D]\}}|jd ||df|q2d S) aAdds directive and value to IfMod ssl block. Adds given directive and value along configuration path within an IfMod mod_ssl.c block. If the IfMod block does not exist in the file, it is created. :param str aug_conf_path: Desired Augeas config path to add directive :param str directive: Directive you would like to add, e.g. Listen :param args: Values of the directive; list of str (eg. ["443"]) :type args: list z mod_ssl.cargrFz directive[1]rrrz %s/arg[%d]N) get_ifmodr%insertrMrFr)r:rrr if_mod_pathnvh_pathrrr;r;r<add_dir_to_ifmodsslas  z ApacheParser.add_dir_to_ifmodsslrcCs6|jd||f}|s|||S|dddS)aReturns the path to and creates one if it doesn't exist. :param str aug_conf_path: Augeas configuration path :param str mod: module ie. mod_ssl.c :param bool beginning: If the IfModule should be created to the beginning of augeas path DOM tree. :returns: Augeas path of the requested IfModule directive that pre-existed or was created during the process. The path may be dynamic, i.e. .../IfModule[last()] :rtype: str z%s/IfModule/*[self::arg='%s']rr)r%rC create_ifmod rpartition)r:rrif_modsr;r;r<r{s  zApacheParser.get_ifmodcCs>d|}d|}|j|dd|}|j|||S)a>Creates a new and returns its path. :param str aug_conf_path: Augeas configuration path :param str mod: module ie. mod_ssl.c :returns: Augeas path of the newly created IfModule directive. The path may be dynamic, i.e. .../IfModule[last()] :rtype: str z{}/IfModule[last() + 1]z{}/IfModule[last()]/argrRz{}/IfModule[last()]/)rEr%rM)r:rrc_path c_path_argretpathr;r;r<rs  zApacheParser.create_ifmodcCsl|r|nd}|j|d|t|tr+t|dD]\}}|jd||f|qdS|j|d|dS)aAppends directive to the end fo the file given by aug_conf_path. .. note:: Not added to AugeasConfigurator because it may depend on the lens :param str aug_conf_path: Augeas configuration path to add directive :param str directive: Directive to add :param args: Value of the directive. ie. Listen 443, 443 is arg :type args: list or str rRz/directive[last() + 1]rz%s/directive[last()]/arg[%d]z/directive[last()]/argN)r%rM isinstancelistr)r:rrrrvaluer;r;r<rrs   zApacheParser.add_dirrscCs|r|nd}|d}|j|r|j|ddn|j|d|j||t|trCt|dD]\}}|j|d||q1dS|j|d|dS) a)Adds the directive to the beginning of defined aug_conf_path. :param str aug_conf_path: Augeas configuration path to add directive :param str dirname: Directive to add :param args: Value of the directive. ie. Listen 443, 443 is arg :type args: list or str rRz /directive[1]rTrrrN)r%rDrrMrrr)r:rrsr first_dirrrr;r;r<add_dir_beginnings   zApacheParser.add_dir_beginningcommentcCs|j|d|dS)zAdds the comment to the augeas path :param str aug_conf_path: Augeas configuration path to add directive :param str comment: Comment content z/#comment[last() + 1]N)r%rM)r:rrr;r;r< add_commentszApacheParser.add_commentrstartcCsR|st|j}|jd|}g}|D]}|j|}|r&||vr&||q|S)aFinds a comment with specified content from the provided DOM path :param str arg: Comment content to search :param str start: Beginning Augeas path to begin looking :returns: List of augeas paths containing the comment content :rtype: list z%s//*[label() = '#comment'])rqrr%rCrDrv)r:rrcommentsresultsr c_contentr;r;r< find_commentss    zApacheParser.find_commentsTexcludec Cs|s t|jd}dt|tdtdf}|jd||f}|r'||}|dur.d}ndt|}g}|D]2} |j| } | d vrY|| ||| | | d|| |krj||j| |q8|S) aFinds directive in the configuration. Recursively searches through config files to find directives Directives should be in the form of a case insensitive regex currently .. todo:: arg should probably be a list .. todo:: arg search currently only supports direct matching. It does not handle the case of variables or quoted arguments. This should be adapted to use a generic search for the directive and then do a case-insensitive self.get_arg filter Note: Augeas is inherently case sensitive while Apache is case insensitive. Augeas 1.0 allows case insensitive regexes like regexp(/Listen/, "i"), however the version currently supported by Ubuntu 0.10 does not. Thus I have included my own case insensitive transformation by calling case_i() on everything to maintain compatibility. :param str directive: Directive to look for :param arg: Specific value directive must have, None if all should be considered :type arg: str or None :param str start: Beginning Augeas path to begin looking :param bool exclude: Whether or not to exclude directives based on variables and enabled modules :rtype list rz(%s)|(%s)|(%s)rnIncludeOptionalz$%s//*[self::directive=~regexp('%s')]Nrz/*[self::arg=~regexp('%s')])includeincludeoptional) rqr.rpr%rC exclude_dirsrDlowerextendro_get_include_pathr) r:rrrrregexrP arg_suffixordered_matchesrCdir_r;r;r<ros6!    zApacheParser.find_dirrCc Csr|j|}|s dS|d}tj|}|D]}z|||j|dd}Wqty6t d|w|S)zUses augeas.get to get argument value and interprets result. This also converts all variables and parameters appropriately. N'"rrzError Parsing variable: %s) r%rDrrrfindallreplacer*KeyErrorr rG)r:rCrr*varr;r;r<r?s    zApacheParser.get_argcCst|jdS)z@ Returns the Augeas path of root configuration. r)rqr.r]r;r;r<get_root_augpath\szApacheParser.get_root_augpathcCsNd|jfd|jfg}g}|D]}|D] }|||snq||q|S)z>Exclude directives that are not loaded into the configuration.ifmoduleifdefine)r(keysr* _pass_filterrv)r:rPfilters valid_matchesrCfilter_r;r;r<rbs  zApacheParser.exclude_dirsrcCs|}||d}|dkrG|d|}|j|d|d}|dr3|dd|dvr2dSn||dvr;dS||d|}|dksd S) zDetermine if directive passes a filter. :param str match: Augeas path :param list filter: list of tuples of form [("lowercase if directive", set of relevant parameters)] rrr#Nr!rFT)rfindr%rD startswith)r:rCrmatch_llast_match_idx end_of_if expressionr;r;r<rps   zApacheParser._pass_filtercCs@|d}|dstjtj|j|}|Stj|}|S)zEnsure paths are consistent and absolute :param str arg: Argument of directive :returns: Standardized argument path :rtype: str rr#)rrrr+normpathrjr)r:rr;r;r<standard_path_from_server_roots  z+ApacheParser.standard_path_from_server_rootcCs|durdS||}tj|r|tj|dn|||d}t|D]\}}tdd|Dr@d| |||<q*d|}t |S)a@Converts an Apache Include directive into Augeas path. Converts an Apache Include directive argument into an Augeas searchable path .. todo:: convert to use os.path.join() :param str arg: Argument of Include directive :returns: Augeas path string :rtype: str Nrr#css|]}|tjvVqdSN)rr)racharr;r;r<rfsz1ApacheParser._get_include_path..z* [label()=~regexp('%s')]) rrr+isdirr/rjsplitrany fnmatch_to_rerq)r:r split_argidxrr;r;r<rs     zApacheParser._get_include_pathclean_fn_matchcCst|ddS)aMethod converts Apache's basic fnmatch to regular expression. Assumption - Configs are assumed to be well-formed and only writable by privileged users. https://apr.apache.org/docs/apr/2.0/apr__fnmatch_8h_source.html :param str clean_fn_match: Apache style filename match, like globs :returns: regex suitable for augeas :rtype: str r)fnmatch translate)r:rr;r;r<rszApacheParser.fnmatch_to_refilepathcCs\||\}}||r*|jd|}|s,|r|||||jdSdSdS)zParse file with Augeas Checks to see if file_path is parsed by Augeas If filepath isn't parsed, the file is added and Augeas is reloaded :param str filepath: Apache config file path z&/augeas/load/Httpd['%s' =~ glob(incl)]N)_check_path_actionsr^r%rC_remove_httpd_transform_add_httpd_transformr_)r:ruse_new remove_oldinc_testr;r;r<r/s   zApacheParser.parse_filefilepcC|sdS|||jS)a<Checks if the file path is parsed by current Augeas parser config ie. returns True if the file is found on a path that's found in live Augeas configuration. :param str filep: Path to match :returns: True if file is parsed in existing configuration tree :rtype: bool F)_parsed_by_parser_pathsr)r:rr;r;r<r zApacheParser.parsed_in_currentcCr)a[Checks if the file path is parsed by existing Apache config. ie. returns True if the file is found on a path that matches Include or IncludeOptional statement in the Apache configuration. :param str filep: Path to match :returns: True if file is parsed in existing configuration tree :rtype: bool F)rr7rr;r;r<parsed_in_originalrzApacheParser.parsed_in_originalpathsc Cs:|D]}||D]}t|tj||rdSqqdS)znHelper function that searches through provided paths and returns True if file path is found in the setTF)rrr+rj)r:rr directoryfilenamer;r;r<rs z$ApacheParser._parsed_by_parser_pathscCsdz!tj|}|jtj|}d|vrd}nd}|dk}W||fSty1d}d}Y||fSw)aLDetermine actions to take with a new augeas path This helper function will return a tuple that defines if we should try to append the new filepath to augeas parser paths, and / or remove the old one with more narrow matching. :param str filepath: filepath to check the actions for rFT)rr+rtr)rsr)r:rnew_file_matchexisting_matchesrrr;r;r<rs   z ApacheParser._check_path_actionscCsd|jtj|}tj|}|D]}|d|}|jd|}|j|dq|j|dS)z[Remove path from Augeas transform :param str filepath: filepath to remove r#z!/augeas/load/Httpd/incl [. ='%s']rN)r)rr+rsr%rCrOpop)r:rremove_basenamesremove_dirnamename remove_path remove_incr;r;r<r8s  z$ApacheParser._remove_httpd_transforminclcCs|jd}|r|j|ddd|jd|n|jdd|jd|z|jtj|tj |Wd St yStj |g|jtj|<Yd Sw) a"Add a transform to Augeas. This function will correctly add a transform to augeas The existing augeas.add_transform in python doesn't seem to work for Travis CI as it loads in libaugeas.so.0.10.0 :param str incl: filepath to include for transform z /augeas/load/Httpd/incl [last()]rrFz/augeas/load/Httpd/incl[last()]z/augeas/load/Httpd/lensz Httpd.lnsz/augeas/load/Httpd/inclN) r%rCrrMr)rr+rsrvrtr)r:r last_includer;r;r<rGs    z!ApacheParser._add_httpd_transformcCszddddddddd |jd |jd |jd |jd |jd|jdg}t|dD] \}}|jd||q(|jdS)aaStandardize the excl arguments for the Httpd lens in Augeas. Note: Hack! Standardize the excl arguments for the Httpd lens in Augeas Servers sometimes give incorrect defaults Note: This problem should be fixed in Augeas 1.0. Unfortunately, Augeas 0.10 appears to be the most popular version currently. z*.augnewz *.augsavez *.dpkg-distz *.dpkg-bakz *.dpkg-newz *.dpkg-oldz *.rpmsavez*.rpmnewz*~z /*.augsavez/*~z /*/*augsavez/*/*~z/*/*/*.augsavez/*/*/*~rz/augeas/load/Httpd/excl[%d]N)rrr%rMr_)r:exclrexcludedr;r;r<r1cs zApacheParser.standardize_exclcCsD|jd}tj|jd}tj|r|}|}n|}|}|||dS)zSet default location for directives. Locations are given as file_paths .. todo:: Make sure that files are included rz ports.conf)defaultlistenr)r.rr+rjrisfile)r:rtemprrr;r;r<r4s   zApacheParser._set_locationscCsHgd}|D]}tjtj|j|rtj|j|Sqtd)z(Find the Apache Configuration Root file.)z apache2.confz httpd.confzconf/httpd.confz!Could not find configuration root)rr+rrjrr NoInstallationError)r:locationrr;r;r<r-s  zApacheParser._find_config_root)r)r"Nr)NNT)?__name__ __module__ __qualname____doc__recompilerr__annotations__rr strr intr=rKr boolrr&r\r^rrSrUryr}rr2r0rrr~rrrrrrrrrrrorrrrrrrrr/rrrrrrrr1rr4r-r;r;r;r<r"s     / '           O"."   rstringr"cCsdddt|DS)a`Returns case insensitive regex. Returns a sloppy, but necessary version of a case insensitive regex. Any string should be able to be submitted and the string is escaped and then made case insensitive. May be replaced by a more proper /i once augeas 1.0 is widely supported. :param str string: string to make case i regex rRcss4|]}|rd||dn|VqdS)rrN)isalphaupperr)rarr;r;r<rfs  zcase_i..)rjrescape)rr;r;r<rps rp file_pathcCsd|S)zTReturn augeas path for full filepath. :param str file_path: Full filepath z/files%sr;)rr;r;r<rqsrqcCs*tstdttjtjtjBtjBdS)z' Initialize the actual Augeas instance zProblem in Augeas installation)loadpathflags)rr r rAUGEAS_LENS_DIRNONENO_MODL_AUTOLOAD ENABLE_SPANr;r;r;r<r$s r$)%rr5rloggingrtypingrrrrrrrr r r r certbotr certbot.compatrcertbot_apache._internalrr%certbot_apache._internal.configuratorraugeasr ImportError getLoggerr rgrrrprqr$r;r;r;r<sN