o f @s2dZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZddlmZddlmZddlmZdd lmZdd lmZdd lmZmZmZmZmZm Z m!Z!m"Z"m#Z#ddl$Z$dd l%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+ddl,m-Z-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6m7Z7m8Z8m9Z9m:Z:zddl;mZd?Zd ddAeRd7e_fdBdCZed7ee<fdDdEZfeSfdegdFeRdGe_dHe!e>fdIdJZheehddKZieehdLdKZjeSfdHe!e>fdMdNZkeSfdHe!e>fdOdPZldQdRZmddFeRfdTdUZndVeRd7eeofdWdXZp @ @ SddVeRdYegdZegd[egd7egf d\d]ZqdeVjWd@d@dSd@fdAeRdFeeRd^eVdYegd_egd[egd`egd7egfdadbZrGdcddddZsdddedfeRdge_dheRdieeUdjeeUd7e"e_etff dkdlZudme_d7eeTfdndoZv ddpe_dme_dqee_d7e^fdrdsZweVjWd@dfdpe_dFeRd^eVdZegdqee_d7egf dtduZxdvdwZydxdyZzdzeRd{e_d7ee_fd|d}Z{dzeRd~eRd7e_fddZ|d7e_fddZ}deRd~eRfddZ~deRfddZdeRd7e_fddZddFeRd~eRd7e_fddZdeQd7e_fddZdeQd7e_fddZddeQdFeeRd7e_fddZd7eRfddZded7e_fddZd7e_fddZd7eRfddZeVjWfd^eVd7eRfddZdddZddZd7e e_ee]ffddZddZddZe@dkreedSdS)zFschema.py: Set of module functions for processing cloud-config schema.N) defaultdict)Iterable)suppress)deepcopy)Enum)EACCES)partial)chain) TYPE_CHECKING DefaultDictList NamedTupleOptionalTupleTypeUnioncast)featuresimporter lifecyclesafeyaml)read_cfg_paths)INCLUSION_TYPES_MAPtype_from_starts_with)Paths)DataSourceNotFoundException)mkdtemp)errorget_modules_from_dirload_text_file load_yaml write_file)ValidationErrorz!versions.schema.cloud-config.jsonzschema-cloud-config-v1.jsonzschema-network-config-v1.jsonzschema-network-config-v2.jsontruefalsenull)TFNap {name} {title_underbar} {title} .. tab-set:: {prefix3}.. tab-item:: Summary {description} {prefix6}**Internal name:** ``{id}`` {prefix6}**Module frequency:** {frequency} {prefix6}**Supported distros:** {distros} {prefix6}{activate_by_schema_keys} {prefix3}.. tab-item:: Config schema {property_doc} {prefix3}.. tab-item:: Examples {prefix6}:: {examples} z5{prefix}* **{prop_name}:** ({prop_type}){description}zJ{prefix}* Each object in **{prop_name}** list supports the following keys: deprecatedcCsg|] }|ddkr|qS)r#).0tr(r(9/usr/lib/python3/dist-packages/cloudinit/config/schema.py osr,) NotRequired TypedDictc@sjeZdZUeed<eed<eed<eed<ejeed<ejeeefed<eed<e eeed<d S) MetaSchemaidnametitle descriptiondistrosexamples frequencyactivate_by_schema_keysN) __name__ __module__ __qualname__str__annotations__typingr rdictr-r(r(r(r+r/ys r/cs&eZdZdedeffdd ZZS)SchemaDeprecationErrormessageversionc stj|fi|||_dSN)super__init__rA)selfr@rAkwargs __class__r(r+rDs zSchemaDeprecationError.__init__)r8r9r:r;rD __classcell__r(r(rGr+r?s r?c@s,eZdZUeed<eed<defddZdS) SchemaProblempathr@returncCs|jd|jS)N: )rKr@rEr(r(r+formatszSchemaProblem.formatN)r8r9r:r;r<rOr(r(r(r+rJs rJc@s eZdZdZdZdZdZdZdS) SchemaTypeaSupported schema types are either cloud-config or network-config. Vendordata and Vendordata2 format adheres to cloud-config schema type. Cloud Metadata is unique schema to each cloud platform and likely will not be represented in this enum. z cloud-confignetwork-configznetwork-config-v1znetwork-config-v2N)r8r9r:__doc__ CLOUD_CONFIGNETWORK_CONFIGNETWORK_CONFIG_V1NETWORK_CONFIG_V2r(r(r(r+rPs rPlatestc@s(eZdZdZdZdZdZdZddZdS) InstanceDataTypez-Types of instance data provided to cloud-initz user-datarQz vendor-dataz vendor2-datacC|jSrBvaluerNr(r(r+__str__szInstanceDataType.__str__N) r8r9r:rRUSERDATArT VENDORDATA VENDOR2DATAr\r(r(r(r+rXs rXc@s&eZdZUeed<eed<eed<dS)InstanceDataPart config_type schema_type config_pathN)r8r9r:rXr<rPr;r(r(r(r+r`s  r`c@seZdZUeed<eed<dS)UserDataTypeAndDecodedContent userdata_typecontentN)r8r9r:r;r<r(r(r(r+rds  rd, prefix separatorschema_problemsrirjrLcCs(|tdd|}|r||}|S)NcSs|SrB)rO)pr(r(r+sz)_format_schema_problems..)joinmap)rkrirj formattedr(r(r+_format_schema_problemss rqcsFeZdZdZ  d deedeeffdd Zdefdd ZZ S) SchemaValidationErrorz.handle_problemszCloud config schema errors: rw"Cloud config schema deprecations: N)rsrtrCrD)rErsrtr}rGr|r+rDs  zSchemaValidationError.__init__rLcCs t|jSrB)boolrsrNr(r(r+ has_errorss z SchemaValidationError.has_errors)NN) r8r9r:rRrSchemaProblemsrDrrrIr(r(rGr+rrs"rrc@seZdZdZdS)"SchemaValidationInvalidHeaderErrorz>Raised when no valid header is declared in the user-data file.N)r8r9r:rRr(r(r(r+r srcCs@zddlm}Wn tyYdSw|j|dpt|tfS)zWTYPE_CHECKER override allowing bytes for string type For jsonschema v. 3.0.0+ r)Draft4ValidatorFstring) jsonschemar ImportError TYPE_CHECKERis_type isinstancebytes)checkerinstancerr(r(r+is_schema_byte_strings  rFconfigcsPdtffdd }|r|ngd}dt||}dd}||S)zcombine description with new/changed/deprecated message deprecated/changed/new keys require a _version key (this is verified in a unittest), a _description key is optional keycsr|sdS|dd}|dd|d}|d|d|}r1d|Sd |d S) Nru _description_versionz z in version .  z **)get capitalizestrip)rkey_descriptionvmsgannotaterr(r+format_message%s   z:_add_deprecated_changed_or_new_msg..format_message)r&changednewrur3)r;rnrorrstrip)rr filter_keyr filter_keyschanged_new_deprecatedr3r(rr+"_add_deprecated_changed_or_new_msgs   rcCsg}d}t|to d|v}|D]\}|rO|d|jdididgvr+|gSt|dr?|jdddkr>||q|jrN|jddkrN||qt|j|kr\||qt|j|krkt|j}|g}q|S)zReturn the best_match errors based on the deepest match in the json_path This is useful for anyOf and oneOf subschemas where the most-specific error tends to be the most appropriate. rtype propertiesenum json_pathN) rr>schemarhasattrrappendrKlen)errorsr best_matches path_depthrerrr(r(r+cloud_init_deepest_matches?s2      rrr error_typeccs2|rt|d|gd}|||ddVdSdS)zJsonschema validator for `deprecated` items. It yields an instance of `error_type` if deprecated that must be handled, otherwise the instance is consider faulty. T)rrdeprecated_versiondevelN)rr) _validatorr& _instancerrrrr(r(r+r`s r)rrc#sddlm}g}g}d}t|D]U\} } t|j|| | d} ttfdd| } ttfdd| } | s>|| n:t|trad|vrad | d d vrad |d| d vrad }| EdH|| q|sn||Vt d|f|dV|EdHdS)aJsonschema validator for `anyOf`. It treats occurrences of `error_type` as non-errors, but yield them for external processing. Useful to process schema annotations, as `deprecated`. Cloud-init's network schema under the `config` key has a complexity of allowing each list dict item to declare it's type with a `type` key which can contain the values: bond, bridge, nameserver, physical, route, vlan. This schema 'flexibility' makes it hard for the default jsonschema.exceptions.best_match function to find the correct schema failure because it typically returns the failing schema error based on the schema of greatest match depth. Since each anyOf dict matches the same depth into the network schema path, `best_match` just returns the first set of schema errors, which is almost always incorrect. To find a better schema match when encountering schema validation errors, cloud-init network schema introduced schema $defs with the prefix `anyOf_type_`. If the object we are validating contains a 'type' key, and one of the failing schema objects in an anyOf clause has a name of the format anyOf_type_XXX, raise those schema errors instead of calling best_match. r) best_matchF schema_pathc t| SrBrerr(r+rm z_anyOf..c t|SrBrrrr(r+rm r anyOf_type$refru anyOf_type_TNz.%r is not valid under any of the given schemas)context) jsonschema.exceptionsr enumeraterydescendfilterextendrr>rr") validatoranyOfr_schemarr all_errorsall_deprecationsskip_best_matchindex subschemaall_errserrs deprecationsr(rr+_anyOfxs>     rc#st|}g}g}|D]4\}} tj| |d} ttfdd| } ttfdd| } | s:| } || n|| q t|EdHfdd|D}|rn|| dd d |D}td |fVdS|EdHdS) zJsonschema validator for `oneOf`. It treats occurrences of `error_type` as non-errors, but yield them for external processing. Useful to process schema annotations, as `deprecated`. rcrrBrrrr(r+rmrz_oneOf..crrBrrrr(r+rmrNcs g|] \}}|r|qSr()is_valid)r)is)rrr(r+r,s z_oneOf..rgcs|]}t|VqdSrB)repr)r)rr(r(r+ z_oneOf..z%r is valid under each of %s) rryrrrrrrnr")roneOfrrr subschemasrrrrrrr first_valid more_validreprsr()rrrr+_oneOfs4       rc Csddlm}m}ddlm}t|j}ddi|dd<i}t|dr2d |d <|j dt }d |i}n |j }t t f|d<d |i}t|j}t|t<t|d <t|d<t|d<|d||dd|}ddd} | |_||fS)zGet metaschema validator and format checker Older versions of jsonschema require some compatibility changes. @returns: Tuple: (jsonschema.Validator, FormatChecker) @raises: ImportError when jsonschema is not present r)r FormatChecker)createrrrlabelrFadditionalProperties type_checker default_typesrrrdraft4) meta_schema validatorsrANc[s$tdd|||}t|dduS)zgOverride version of `is_valid`. It does ignore instances of `SchemaDeprecationError`. cSs t|t SrB)rr?rr(r(r+rm%sz.is_valid..N)r iter_errorsnext)rErr__rr(r(r+rs  z*get_jsonschema_validator..is_validr(rB)rrrjsonschema.validatorsrr META_SCHEMArrredefiner DEFAULT_TYPESr;rr> VALIDATORS_validator_deprecatedDEPRECATED_KEY_validator_changedrrr) rrrrvalidator_kwargsrtypesrcloudinitValidatorrr(r(r+get_jsonschema_validators<       rTc Cs|ddlm}z||WdS|y=}z#|r,ttddd|jD|jgd|t d|WYd}~dSd}~ww) a Validate provided schema meets the metaschema definition. Return strict Validator and FormatChecker for use in validation @param validator: Draft4Validator instance used to validate the schema @param schema: schema to validate @param throw: Sometimes the validator and checker are required, even if the schema is invalid. Toggle for whether to raise SchemaValidationError or log warnings. @raises: ImportError when jsonschema is not present @raises: SchemaValidationError when the schema is invalid r) SchemaError.cSg|]}t|qSr(r;r)rlr(r(r+r,Iz3validate_cloudconfig_metaschema..rszGMeta-schema validation failed, attempting to validate config anyway: %sN) rr check_schemarrrJrnrKr@LOGwarning)rrthrowrrr(r(r+validate_cloudconfig_metaschema1s& rnetwork_configcCs d|vr |ddS|dS)z6Return the version of the network schema when present.networkrAr)r r(r(r+network_schema_versionTs r strictr log_detailsc Csbz ddlm}m}WntytdYdSwt}tj |d}t |}d|vr0d|i}t |} t || dd|} g} z| |Wn%|ym} z| td j| j| jd d | jWYd } ~ nd } ~ wwtj|ryt|| r|r|rt | \} }tt|| || d t| |rt| tjjddd}ntjjd}t |dS)ajOn systems with netplan, validate network_config schema for file Leverage NetplanParser for error annotation line, column and detailed errors. @param network_config: Dict of network configuration settings validated against @param strict: Boolean, when True raise SchemaValidationErrors instead of logging warnings. @param annotate: Boolean, when True, print original network_config_file content with error annotations @param log_details: Boolean, when True logs details of validation errors. If there are concerns about logging sensitive userdata, this should be set to False. @return: True when schema validation was performed. False when not on a system with netplan and netplan python support. @raises: SchemaValidationError when netplan's parser raises NetplanParserExceptions. r)NetplanParserExceptionParserz.rrcSrr(rrr(r(r+r, rz/validate_cloudconfig_schema..rz#.*\('(?P.*)' was unexpected\)r1rz"Deprecated cloud-config provided: rwrrrhr)&cloudinit.net.netplanr0rPrTr rVrU get_schemar,rrrrrrxrrnrKrrrematchr@rr?rArshould_log_deprecationrDEPRECATION_INFO_BOUNDARYrrJrqinfor&rrr[r)rrrbr r-rr.netplan_availablenetwork_versionrrrrrinfo_deprecations schema_errorrK prop_matchr@detailsr(r(r+validate_cloudconfig_schemas #               rBc@seZdZdededefddZededeedefd d Zd e fd d Z e d deedeedeede dede f ddZ deedededeefddZ de de defddZdS)! _Annotator cloudconfigoriginal_content schemamarkscCs||_||_||_dSrB) _cloudconfig_original_content _schemamarks)rErDrErFr(r(r+rDLs z_Annotator.__init__r2rfrLcCsd|}d|d|dS)Nr# z: ------------- rv)rn)r2rfbodyr(r(r+ _build_footerVs z_Annotator._build_footerrkcCsztt}|D]4\}}td|}|r"|\}}|t||n d}||j|||dur:dj|||d}q|S)Nz&format-l(?P\d+)\.c(?P\d+).*zLine {line} column {col}: {msg})rrr) rryr7r8groupsintrrIrO)rErkerrors_by_linerKrr8rrr(r(r+_build_errors_by_line[s   z _Annotator._build_errors_by_linerur{labelsfooterr label_prefixcCsB|D]}||}|||d|d||d7}q|S)NrJrMr2)r)r{rQrRrrSproblemrr(r(r+ _add_problemsks    z_Annotator._add_problemslinesrOdeprecations_by_linec sg}g}g}d}d}t|dD]8\} } || } || } | s| rBg} j| | ||dd}j| | ||dd}|| dd| q|| q|tfddtd dd |fd |ff|S) Nr2E)rSDz # ,cs j|SrB)rLseqrNr(r+rmrz._Annotator._annotate_content..cSs t|dS)Nr2)rr[r(r(r+rmrErrors Deprecations)rrUrrnrror)rErVrOrWannotated_content error_footerdeprecation_footer error_indexdeprecation_index line_numberrrrrQr(rNr+_annotate_contentzsD    z_Annotator._annotate_contentrsrtcCsF|s|s|jS|jd}||}||}||||}d|S)Nr)rHsplitrPrern)rErsrtrVrOrWr_r(r(r+rs    z_Annotator.annotateN)ru)r8r9r:r>r;rD staticmethodr rLrrPrNrUrerr(r(r(r+rCKsT    ,rCrsrtrDrErFrsrtcCst||||p g|p gS)aReturn contents of the cloud-config file annotated with schema errors. @param cloudconfig: YAML-loaded dict from the original_content or empty dict if unparsable. @param original_content: The contents of a cloud-config file @param schemamarks: Dict with schema marks. @param schema_errors: Instance of `SchemaProblems`. @param schema_deprecations: Instance of `SchemaProblems`. @return Annotated schema )rCr)rDrErFrsrtr(r(r+r$s  r$rfc Csbddlm}||vr gSg}t|dD]\}}||r.|td|d||dq|S)aAnnotate and return schema validation errors in merged cloud-config.txt When merging multiple cloud-config parts cloud-init logs an error and ignores any user-data parts which are declared as #cloud-config but cannot be processed. the handler.cloud_config module also leaves comments in the final merged config for every invalid part file which begin with MERGED_CONFIG_SCHEMA_ERROR_PREFIX to aid in triage. r)MERGED_PART_SCHEMA_ERROR_PREFIXr2zformat-lz.c1zIgnored invalid user-data: )cloudinit.handlers.cloud_configrir splitlines startswithrrJreplace)rfrirline_numrr(r(r+)process_merged_cloud_config_part_problemss"    rorcinstance_data_pathc Cs:ddlm}m}m}m}t|}d}|dkrnz||||}WnI|y3} z tt|dg| d} ~ w|yN} ztdt | dd WYd} ~ nd} ~ w|yg} ztt | dd WYd} ~ nd} ~ wwd }t|}|s| d \} } } t t|d |d | dd t g|dkrtd|dt||S)a Return tuple of user-data-type and rendered content. When encountering jinja user-data, render said content. :return: UserDataTypeAndDecodedContent :raises: SchemaValidationError when non-jinja content found but header declared ## template: jinja. :raises JinjaSyntaxParsingException when jinja syntax error found. :raises JinjaLoadError when jinja template fails to load. r)JinjaLoadErrorJinjaSyntaxParsingException NotJinjaErrorrender_jinja_payload_from_filez format-l1.c1z text/jinja2zRDetected type '{user_data_type}' from header. But, content is not a jinja templateNz&Failed to render templated user-data. Tsys_exitz format-l2.c1rz!Unrecognized user-data header in z: "z%". Expected first line to be one of: rgtext/cloud-configzUser-data type 'z.' not currently evaluated by cloud-init schema)!cloudinit.handlers.jinja_templaterqrrrsrtrrrrJrr; partitionrrnUSERDATA_VALID_HEADERSr#rd) rcrfrprqrrrsrtuser_data_typeschema_positionr header_liner*r(r(r+&_get_config_type_and_rendered_userdatasb     r~c Csddlm}t|}|std|j|fdS|tjfvr$t|j|}nt|||}|j dvr1dS|j }t |} z|rCt |\} } nt|} i} Wnetjy} zXd} }d}t| drjt| drjt| d}nt| d ryt| d ryt| d }|r|jd} |jd}| td j| |d d |t| t| }|rtti|i|jd || d} ~ wwt| ts|st|jd|d|tjkr| d| stddSt | }|dkrtj!}t"| d|drdS|rtddSn |dkrtj#}t$|}zt%| ||dddstd|jdWdSWdSty]} z:| &r)| | j7} |r9tt| || | | j'dn| j'rIt(| j'ddd}t|| rRt| d | WYd} ~ dSd} ~ ww)aValidate cloudconfig file adheres to a specific jsonschema. @param config_path: Path to the yaml cloud-config file to parse, or None to default to system userdata from Paths object. @param schema: Dict describing a valid jsonschema to validate against. @param schema_type: One of SchemaType.NETWORK_CONFIG or CLOUD_CONFIG @param annotate: Boolean set True to print original config file with error annotations on the offending lines. @param instance_data_path: Path to instance_data JSON, used for text/jinja rendering. :return: True when validation was performed successfully :raises SchemaValidationError containing any of schema_errors encountered. :raises RuntimeError when config_path does not exist. rr/z,Empty '%s' found at %s. Nothing to validate.F)rQrwr2N context_mark problem_markrrzFile {0} is not valid YAML. {1}rrz is not a YAML dict.r z:Skipping network-config schema validation on empty config.r1T)r r rzSSkipping network-config schema validation for version: 2. No netplan API available.)rrbr r.z Skipping z2 schema validation. Jsonschema dependency missing.rhr~rgrh))r5r0rr#r[rPrTrdr~rerfrorr"yaml safe_load YAMLErrorrgetattrrrrrJrOr;rrr$rsrr> RuntimeErrorrr rVr,rUr6rBrrtrq)rcrrbrrpr<decoded_contentdecoded_configrfrrDr+rrrmarkr?r=r@r(r(r+validate_cloudconfig_file1s              "     rcCs|dkrdS|dkr dSdS)zProvide a sorting weight for documentation of property types. Weight values ensure 'array' sorted after 'object' which is sorted after anything else which remains unsorted. arrayr1objectr2rr(rZr(r(r+_sort_property_orders rccs>|D]}t|trt|ttfst|EdHq|VqdSrB)rrr;r_flatten)xsxr(r(r+rs r property_dict multi_keycCsng}||iD](}|trq|dr$|dd|dgDq|dr0||dqtt|S)NrcS g|] }dt||dqS`` _YAML_MAPr)r) enum_valuer(r(r+r,sz,_collect_subschema_types..r)rrrrryr)rrproperty_typesrr(r(r+_collect_subschema_typess     rdefsc Csnt|||dg}t|ts|g}|dr"dd|dD}n|dr0|t|dn |dr=|t|dt|dkrH|d}n |jtd d |}|d i}|dg}t|tsg|g}t |}t |di|diD]}t ||}|r|d krqx| |qx|rt|dkr|d |dS|jtd dd |d} |d | S|pd S)zNReturn a string representing a property type from a given jsonschema. rrcSrrrr)kr(r(r+r,sz&_get_property_type..rrr2rr4/items UNDEFINEDz of ())flatten_schema_refsrrryrrrsortrrnrr _get_property_typer) rrr property_typersub_property_typesprune_undefinedsub_itemsub_typesub_property_docr(r(r+rsB                  rcCs>|}tdd|}tdd||}tdd||}|S)aParse description from the meta in a format that we can better display in our docs. This parser does three things: - Guarantee that a paragraph will be in a single line - Guarantee that each new paragraph will be aligned with the first paragraph - Proper align lists of items @param description: The original description in the meta. @param prefix: The number of spaces used to align the current description z (\S)\n(\S)z\1 \2z\n\nz\n\n{}z\n( +)-z\n{}-)r7subrO)r3rilist_paragraphr(r(r+_parse_descriptions  rsrc_cfgcCsd|vr|ddd}|||d|vrUd|dvr2|dddd}|d||d|dvrU|ddD]}d|vrT|ddd}|||q>t|dg|dg|dgD]}d|vr}|ddd}|||qgdS) zEFlatten schema: replace $refs in src_cfg with definitions from $defs.rz#/$defs/rurrrallOfN)poprmupdater r)rr reference sub_schemar(r(r+r's.     rcCs,|dd}|s dS|D]}||q dS)zFlatten schema: Merge allOf. If a schema as allOf, then all of the sub-schemas must hold. Therefore it is safe to merge them. rN)rr)r sub_schemasrr(r(r+flatten_schema_all_ofAs  r prop_configcCsdtdtdtfdd}|di}|di}g}g}||||t||D]}||||q&dt||}|r>d |}|S) aGReturn accumulated property description. Account for the following keys: - top-level description key - any description key present in each subitem under anyOf or allOf Order and deprecated property description after active descriptions. Add a trailing stop "." to any description not ending with ":". r descriptionsdeprecated_descriptionscSsBtt|jdr|t|dS|dr|t|dSdS)N)rchanged_version new_versionr3)anyrorrr)rrrr(r(r+assign_descriptionsYs z6_get_property_description..assign_descriptionsrrrr)r>ryrr rn)rrrrrrrr3r(r(r+_get_property_descriptionNs$     r c s|d}g}ddurdSfdddD}fdd|D}|D]}|D]\}} t| |t| | ddur>q)t| } | d |} |tj|| t| |dt | |d | d } | rt| || d sr| d r|t j|| d|t | ||d| dgD] } | d s| d r|t j|| d|t | ||dqd | vsd | vr|t | ||dq)q#d |S)zDReturn restructured text describing the supported schema properties. hiddenTrucs$g|]}dvs|dvr|qS)rr(r)rrr(r+r,~s z%_get_property_doc..)rpatternPropertiescsg|]}|iqSr(r rrr(r+r,sr)ri prop_namer3 prop_typerrr)rirrrirrv) rrrrrrSCHEMA_PROPERTY_TMPLrOrrSCHEMA_LIST_ITEM_TMPL_get_property_docrn)rrri new_prefixr property_keysproperty_schemas prop_schemaprop_keyrr3rr alt_schemar(rr+rxsz        5rmetacCst}tj|jd}|d}|sdSd}|D]3}t|tr;|dr,d|dd}nd}|ttj||d}t |d d}|d|7}q|S) zAReturn restructured text describing the meta examples if present. module-docsr5rucommentrJrfiler) rrrKrndocs_dirrrr>rtextwrapindentrf)rpathsmodule_docs_dirr5 rst_contentexamplerindented_linesr(r(r+ _get_exampless"  rcCs2|dsdSddd|dD}d|dS)Nr7rurgcss|] }d|dVqdS)rNr(rr(r(r+rs  z3_get_activate_by_schema_keys_doc..z**Activate only on keys:** rv)rrn)r schema_keysr(r(r+ _get_activate_by_schema_keys_docs   rc Cs|durt}|r |stdt|}hd}dh}d}||r*d||}n |||r7d||}|r=t|tt|}d|d<d |d <d |d <t t t |d d |d <| di}| |drv| |di}t t|}t |drz t||d d|d<Wntytdd|d<Ynw| ddsd|d<t t|d |d<|dsd|d<d|d|d<tdd|d|d<t||d<tjdi|}|S)zReturn reStructured text rendering the provided metadata. @param meta: Dict of metadata to render. @param schema: Optional module schema, if absent, read global schema. @raise KeyError: If metadata lacks an expected key. Nz"Expected non-empty meta and schema>r0r1r2r4r5r6r3r7ruz(Missing required keys in module meta: {}z3Additional unexpected keys found in module meta: {}property_headerz prefix6rprefix3r3z$defsr0rr property_docz3Unable to render property_doc due to invalid schemaz+ No schema definitions for this moduler5z$ No examples for this modulergr4r-r1title_underbarr()r6 ValueErrorrzkeysrOKeyErrorr>rrrrr;rrvaluesrAttributeErrorrrrrnr7rrSCHEMA_DOC_TMPL) rrr required_keys optional_keys error_message meta_copyrtemplater(r(r+ get_meta_docsh          rcCstjtjt}t|SrB)rrKdirnameabspath__file__r) configs_dirr(r(r+ get_modules(srrequested_modulesc Csd}ttdg}t|t|}|r&tdt|d|ddt}t }t |D]E}d|vs:||vrut |dgdg\}}|rut |d } || jd rn| j|| jd i|t| j|pkd7}q0|| jpsd7}q0|S) zaLoad module docstrings Docstrings are generated on module load. Reduce, reuse, recycle. ruallz+Invalid --docs value {}. Must be one of: {}rgTruzcloudinit.configrrr0)ryrrrz differencerrOrnget_module_docsr6rxr find_module import_modulerrrrrR) rdocs all_modules invalid_docs module_docsrmod_namemod_locsr*modr(r(r+load_doc-s4  rcCstjtjtjtdS)Nschemas)rrKrnrrrr(r(r(r+get_schema_dirOsrcCsFt}i}tj|jd}t|dD] }|tt|q|S)z with documentation inforz /*/data.yaml) rrrKrnrglobrr r)rmod_docsrmod_docr(r(r+rSs rc Cs\tjtt|d}d}z tt|}W|Stt fy-t d|j |iYSw)ziReturn jsonschema for a specific type. Return empty schema when no specific schema file exists. rWNz.r2z;Expected one of --config-file, --system or --docs argumentsTruz[WARNING: The --schema-type parameter is inapplicable when either --system or --docs presentz;Invalid flag combination. Cannot use --annotate with --docsN) config_filersystemrrrrbr#r)argsexclusive_argsr(r(r+_assert_exclusive_argss   r%c Csdtdtdtdtfdd}ztdd}Wn3ttfy7}z|jtkr,td t}nWYd }~nd }~wt yGt}t d Ynw|j rO|j }nt d kr[|d }n|d}g}|jr|jrnt|j}ntj}|tjkrztj}ntj}|t|||jnUt d krtddd||dd}|ttjtj|ttjtj||ddttjtj||ddttjtj|dpdg} | D]} | jrt j| jr|| qt j|d jstd|d jdddd||fS)aReturn appropriate instance-data.json and instance data parts Based on command line args, and user permissions, determine the appropriate instance-data.json to source for jinja templates and a list of applicable InstanceDataParts such as user-data, vendor-data and network-config for which to validate schema. Avoid returning any InstanceDataParts when the expected config_path does not exist. :return: A tuple of the instance-data.json path and a list of viable InstanceDataParts present on the system. rprimary_path_keyraw_fallback_path_keyrLcSs||pd}tt-t|js(||pd}t|jr0|WdSWd|SWd|S1s;wY|S)akGet processed data path when non-empty of fallback to raw data path. - When primary path and raw path exist and are empty, prefer primary path. - When primary path is empty but the raw fallback path is non-empty, this indicates an invalid and ignored raw user-data was provided and cloud-init emitted a warning and did not process unknown raw user-data. In the case of invalid raw user-data header, prefer raw_fallback_path_key so actionable sensible warnings can be reported to the user about the raw unparsable user-data. ruN) get_ipathrFileNotFoundErrorrstatst_size)rr&r'primary_datapathraw_pathr(r(r+get_processed_or_fallback_paths     zBget_config_paths_from_args..get_processed_or_fallback_pathtrust)fetch_existing_datasourcez=Using default instance-data/user-data paths for non-root userNzEdatasource not detected, using default instance-data/user-data paths.rrinstance_data_sensitivezNUnable to read system userdata or vendordata as non-root user. Try using sudo.Tru cloud_config userdata_rawvendor_cloud_configvendordata_rawvendor2_cloud_configvendordata2_rawr ruz Config file z does not existz Error: {}fmtrv)rr;rr r errnorrrrrrrgetuidrr!rbrPrSrTrXr]rr`rr^r_r(rcrKr) r#r.rrrp config_filesrbinstancedata_type userdata_filesupplemental_config_files data_partr(r(r+get_config_paths_from_argss             rAc Cst|t}|jrtt|jdSt|\}}d}tt|dk}|r5tdddd|Dd}g}t |dD]\}} d } |rTtd |d | j d | j d | j t jkr`t| j } n|} z t| j | | j |j|} Wn^ty} z'|jst|d| j d| j tt| |dd|| j WYd} ~ q.rFrrz at :zInvalid rz Error: {} )r9z Valid schema csrrBr)r)rr(r(r+rrzError: Invalid schema: {} Tr8)r%r6rr#rrArrrnrrarcrbrPrTrrrrrr;rrr!)r1r#rrpr<nested_output_prefixmulti_config_output error_typesidxrBperformed_schema_validation cfg_schemarcfgr(r(r+handle_schema_argsGs    rKcCst}td|dS)zDTool to validate schema of a cloud-config file or print schema docs.rr)rrK parse_argsrr(r(r+mainsrM__main__)FN)T)FFTrB)r)rRrrr loggingrr7r sysr collectionsrcollections.abcr contextlibrcopyrrrr:r functoolsr itertoolsr r=r r r r rrrrrr cloudinitrrrrcloudinit.cmd.develrcloudinit.handlersrrcloudinit.helpersrcloudinit.sourcesrcloudinit.temp_utilsrcloudinit.utilrrrr r!rr"r Exception getLoggerr8rVERSIONED_USERDATA_SCHEMA_FILEUSERDATA_SCHEMA_FILENETWORK_CONFIG_V1_SCHEMA_FILENETWORK_CONFIG_V2_SCHEMA_FILErrrrrrxrrztyping_extensionsr-r.r/r>r?rJrrPrSrTrUrVr rXr`rdr;rqrrrrrrrrrrrrrrrrNr r,rBrCrr$ror~rrrrrrrrrrrrrrryrrrr6rr%rArKrMexitr(r(r(r+s        ,             ) #'   H *L#  c p   ! H  , *F J" D }C