o fg7@sdZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z m Z ddl m Z m Z mZmZmZmZmZddlmZddlmZddlmZmZddlmZdd lmZmZmZeej e!e"Z#Gd d d eZ$e$d d e$dde$ddgZ%e$dde$dde$dde$dde$dde$dde$dde$d d!e$d"d#e$d$d%e$d&d%e$d'd(g Z& dcd)eej'd*ej'fd+d,Z(d-ee)d.ej*d/e)d*ee)fd0d1Z+d-ee)d.ej*d/e)d*dfd2d3Z,d4ej*d5ej*d6e-d*dfd7d8Z.d9ej*d6e-d*dfd:d;Z/d9ej*d*dfdd?Z1d@e e)e fd*eej*fdAdBZ2e*dCfdDej*d*eej*fdEdFZ3dGej*d*eej*fdHdIZ4dJej*d*eej*fdKdLZ5d9ej*d@e e)e fdJej*dGej*d6e-d*df dMdNZ6e*dOe*dPdQfdRe)d@e e)e fdJej*dGej*d6e-d*df dSdTZ7dUe8d*dfdVdWZ9  Q XdddRe)dUe8dYe-dZe-d*df d[d\Z:d]e)d^ej;d*e8fd_d`ZdSdS)ezGDefine 'collect-logs' utility and handler to include in cloud-init cmd.N)datetimetimezone)AnyDictIteratorList NamedTupleOptionalcast)log)Init)ProcessExecutionErrorsubp)tempdir)copyget_config_logfiles write_filec@seZdZUeed<eed<dS) ApportFilepathlabelN)__name__ __module__ __qualname__str__annotations__rr:/usr/lib/python3/dist-packages/cloudinit/cmd/devel/logs.pyrs  rz(/var/log/installer/autoinstall-user-dataAutoInstallUserDataz/autoinstall.yamlAutoInstallYAMLz'/etc/cloud/cloud.cfg.d/99-installer.cfgInstallerCloudCfgz//var/log/installer/ubuntu_desktop_installer.logUdiLogz-/var/log/installer/subiquity-server-debug.logSubiquityServerDebugz-/var/log/installer/subiquity-client-debug.logSubiquityClientDebugz%/var/log/installer/curtin-install.log CurtinLogz0/var/log/installer/subiquity-curtin-install.confCurtinInstallConfigz8/var/log/installer/curtin-install/subiquity-initial.confCurtinConfigInitialz:/var/log/installer/curtin-install/subiquity-curthooks.confCurtinConfigCurtHooksz8/var/log/installer/curtin-install/subiquity-extract.confCurtinConfigExtractz=/var/log/installer/curtin-install/subiquity-partitioning.confCurtinConfigPartitioningz(/var/log/installer/curtin-error-logs.tar CurtinErrorz$/var/log/installer/curtin-errors.tarz(/var/log/installer/block/probe-data.json ProbeDataparserreturncCsh|s tjddd}|jdddddd d |jd d d dd|jddddddd|jdddddd|S)a2Build or extend and arg parser for collect-logs utility. @param parser: Optional existing ArgumentParser instance representing the collect-logs subcommand which will be extended to support the args of this utility. @returns: ArgumentParser with proper argument configuration. z collect-logsz)Collect and tar all cloud-init debug info)prog descriptionz --verbosez-vcountr verbosityzBe more verbose.)actiondefaultdesthelpz --tarfilez-tzcloud-init.tar.gzzOThe tarfile to create containing all collected logs. Default: cloud-init.tar.gz)r2r4z--include-userdataz-uF store_trueuserdataz?DEPRECATED: This is default behavior and this flag does nothing)r2r1r3r4z--redact-sensitivez-rzRedact potentially sensitive data from logs. Sensitive data may include passwords or keys in user data and root read-only files.)r2r1r4)argparseArgumentParser add_argument)r+rrr get_parserOsF    r:cmd file_pathmsgc Cs|jjdddzt|j}Wn"ty1}zt|t|td|d}WYd}~|Sd}~wwt||td||j |S)zCHelper which runs a command and writes output or error to filename.Tparentsexist_okcollecting %s failed.Ncollected %s to file '%s') parentmkdirrstdoutr rrLOGdebugstem)r;r<r=outputerrr_write_command_output_to_files  rKc Cs|jjdddz|d}tj|||dWdn1s!wYWn tyG}zt|t|t d|WYd}~dSd}~wwt d||j dS)zHelper which runs a command and writes output or error to filename. `subprocess.call` is invoked directly here to stream output to the file. Otherwise memory usage can be high for large outputs. Tr>w)rEstderrNrArB) rCrDopen subprocesscallOSErrorrrrFrGrH)r;r<r=frJrrr_stream_command_output_to_files rSrout_dirinclude_sensitivecCsd|r*|s|jtj@r"|jdddt||td|dStd|dStd|dS)z-Collect a file into what will be the tarball.Tr>zcollected file: %sz#sensitive file %s was not collectedzfile %s did not existN) is_filestatst_modeS_IROTHrDrrFrGtrace)rrTrUrrr _collect_files r[log_dircCs|tD]}t||jddj}tt|j|ddq|r:tD]}t||jddj}tt|j|ddq dSdS)z'Obtain subiquity logs and config files.NT)rU)INSTALLER_APPORT_FILESpathlibPathrrCr[ INSTALLER_APPORT_SENSITIVE_FILES)r\rUsrc_filedestination_dirrrr_collect_installer_logss"  rdcCs@tddg|ddd}tgd|ddd}|s|pd }d Sd S) z8Include cloud-init version and dpkg version in the logs. cloud-initz --versionversionzcloud-init --versionr;r<r=)z dpkg-queryz--showz-f=${Version} rez dpkg-versionz dpkg versionz not-availableN)rK)r\rfdpkg_verrrr_collect_version_infos ricCsL|r tdg|dddtgd|dddtgdt|d d dd S) z0Include dmesg and journalctl output in the logs.dmesgz dmesg.txtz dmesg outputrg) journalctlz--boot=0-o short-precisez journal.txtzsystemd journal of current boot)rkz --boot=-1rlrmzjournal-previous.txtz systemd journal of previous bootN)rSr_r`)r\rUrrr_collect_system_logss   rnlog_cfgccs t|D]}t|VqdS)z7Get paths for cloud-init.log and cloud-init-output.log.N)rr_r`)rorrrr_get_cloudinit_logss rpz /etc/cloud etc_cloud_dirc#s6|d|ddgfdd|dDEdHdS)zGet paths for all files in /etc/cloud. Excludes: /etc/cloud/keys because it may contain non-useful sensitive data. /etc/cloud/templates because we already know its contents keys templatesz99-installer.cfgc3s(|]}|jvr|jvr|VqdSN)namerC).0rignorerr sz!_get_etc_cloud..z**/*Nglob)rqrrwr_get_etc_clouds  r| cloud_dirc Cs:t|d|d|ddd|dD|dS)zkGet paths for files in /var/lib/cloud. Skip user-provided scripts, semaphores, and old instances. zdata/*z handlers/*zseed/*css|] }|r|VqdSrt)rVrvprrrrysz%_get_var_lib_cloud..z instance/*zinstance/handlers) itertoolschainr{r}rrr_get_var_lib_cloudsrrun_dircCsdd|dDS)zGet all paths under /run/cloud-init except for hook-hotplug-cmd. Note that this only globs the top-level directory as there are currently no relevant files within subdirectories. css|] }|jdkr|VqdS)zhook-hotplug-cmdN)rur~rrrry*sz_get_run_dir..*rzrrrr _get_run_dir$srcCst|t||t||t|D]}t||t|jddqt t t |dt |dD]}t||t|jd|q2dS)z8Collect all cloud-init logs into the provided directory./TrrN)rirnrdrpr[r_r`rC relative_torrr|rr)r\rorr}rUlogfilerrr_collect_logs_into_tmp_dir-s(   rz/run/cloud-initz/var/lib/cloudTtarfilec Cstj|}ttjd}t |d*}t ||}t |||||dt dd|d|t||ddgWd n1sCwYtd |d S) aCollect all cloud-init logs and tar them up into the provided tarfile. :param tarfile: The path of the tar-gzipped file to create. :param log_cfg: The cloud-init base configuration containing logging cfg. :param run_dir: The path to the cloud-init run directory. :param cloud_dir: The path to the cloud-init cloud directory. :param include_sensitive: Boolean, true means include sensitive data. zcloud-init-logs-%Y-%m-%d)dir)r\rorr}rUtarczfz-CrNzWrote %s)osrabspathrnowrutcdatestrftimerr_r`rrrreplacerFinfo)rrorr}rUdir_nametmp_dirr\rrr collect_logsNs.   rr0cCs^t|dkr tj}n |dkrtj}ntj}t|t}| t dt |dS)zSet up the logger for CLI use. The verbosity controls which level gets printed to stderr. By default, DEBUG and TRACE are hidden. rr]z %(message)sN) r reset_loggingloggingINFODEBUGTRACErFsetLevel StreamHandler setFormatter Formatter addHandler)r0levelhandlerrrr _setup_loggerws rFredact_sensitiveinclude_userdatacCs|t|tdkrtd|rtdtgd}|t||j t |j j t |j j| d|ssJ$    & 9            $  ) !