job_archiver Class Reference

Archive the paths associated with a single job. More...

#include <archiver.h>

Collaboration diagram for job_archiver:

[legend]List of all members.

Public Types
enum	archiving_status {   status_pending, status_processing, status_retry_later, status_fatal_error,   status_error, status_completed, status_done }
Public Member Functions
	job_archiver (const job *a_job)
	C'tor.
const std::string	prefix (void)
	Generate a job prefix string.
const std::string	id (void)
	Generate a job id string.
void	clear (void)
	Clear the job archiver and return it to it's initial state.
void	end (void)
	End any processes handling this job.
const archiving_status	status (void)
	Return the processing status of this job archiver.
void	start (void)
	Begin processing.
void	process (void)
	Parent processor for a job.
single_job_report	report (void) const
	Return the job report for this job.
Private Member Functions
void	mf_do_chores (void)
	Child processor for a job.
void	mf_process_report (const std::string &a_str)
void	mf_process_child_io (bool a_finalize)
	Process I/O from the child.
void	mf_process_rsync_io (execute &a_exec, uint16 a_timeout, uint64 &a_files_total, uint64 &a_files_xferd, uint64 &a_size_total, uint64 &a_size_xferd, bool &a_overflow_detected)
	Process I/O from rsync.
void	mf_parse_rsync_io (std::string a_str, uint64 &a_files_total, uint64 &a_files_xferd, uint64 &a_size_total, uint64 &a_size_xferd)
	Parse I/O from rsync.
void	mf_trim_string (std::string &a_str)
	Trim off all non-digit leading and trailing characters from a string.
void	mf_parse_report (const std::string &a_str)
Private Attributes
const job *	m_job
archiving_status	m_status
timer	m_timer
execute	m_exec
bool	m_success
estring	m_io_out
estring	m_io_err
pid_t	m_child_pid
job_path_report	m_jpr
single_job_report	m_jr
std::string	m_error_msg

---

Detailed Description

Archive the paths associated with a single job.

Definition at line 36 of file archiver.h.

---

Member Enumeration Documentation

enum job_archiver::archiving_status

Enumeration values: status_pending  status_processing  status_retry_later  status_fatal_error  status_error  status_completed  status_done  Definition at line 38 of file archiver.h.

---

Constructor & Destructor Documentation

job_archiver::job_archiver (  const job *  a_job  )

C'tor. Set a job to be assiciated with this job archiver and initialize it's processing status to "pending". Definition at line 121 of file archiver.cc. References clear(), m_job, m_status, and status_pending. Here is the call graph for this function:

---

Member Function Documentation

void job_archiver::clear (  void   )

Clear the job archiver and return it to it's initial state. End any processes handling this job and return the job archiver to it's "pending" state. Definition at line 163 of file archiver.cc. References job_path_report::clear(), single_job_report::clear(), end(), m_child_pid, m_error_msg, m_exec, m_jpr, m_jr, m_status, m_success, execute::my_pid(), and status_pending. Referenced by job_archiver(). Here is the call graph for this function:

void job_archiver::end (  void   )

End any processes handling this job. If any child processes are handling this job, terminate them. Erase any pending I/O for the now defunct child. Set our processing status to "done". Definition at line 180 of file archiver.cc. References execute::child_running(), execute::clear(), execute::kill_child(), logger, configuration_manager::logging_manager, m_child_pid, m_exec, m_io_err, m_io_out, m_status, m_timer, prefix(), status_done, timer::stop(), and log_manager::write(). Referenced by clear(). Here is the call graph for this function:

const std::string job_archiver::id (  void   )

Generate a job id string. Definition at line 146 of file archiver.cc. References job::generate_job_id(), m_job, and prefix(). Here is the call graph for this function:

void job_archiver::mf_do_chores (  void   )  [private]

Child processor for a job. For each path in this job: Create the directory heiararchy for this job in the archive Until done or until out of retrys: Choose a hardlink source, if applicable and available Construct the command line to pass to rsync Spawn rsync Process I/O sent back from rsync Process exit code or signal number returned from rsync Generate and submit a report to the report manager Definition at line 378 of file archiver.cc. References archiver, config, job::connection_local, ERROR, job::excludes, execute::exec(), exists(), rstat::exit(), rsync_behavior::fail, job::generate_archive_path(), job::generate_source_path(), job::includes, is_timestamp(), logger, m_error_msg, m_job, m_success, mf_process_rsync_io(), mk_dirhier(), mk_relative_path(), rsync_behavior::ok, subdirectory::path(), job::paths, reform_path(), relative_path(), rsync_behavior::retry_without_hardlinks, job::rsync_behavior, job::rsync_connection, job::rsync_hardlink, configuration_manager::rsync_local_path(), job::rsync_options, job::rsync_remote_path, job::rsync_retry_count, job::rsync_timeout, log_manager::set_error_logging(), rstat::signal(), timer::start(), timer::stop(), configuration_manager::timestamp(), TRY_nomem, vault_manager::vault(), vaulter, archive_manager::working_archive_path(), and writable(). Referenced by start(). Here is the call graph for this function:

void job_archiver::mf_parse_report (  const std::string &  a_str  )  [private]

void job_archiver::mf_parse_rsync_io (  std::string  a_str, uint64 &  a_files_total, uint64 &  a_files_xferd, uint64 &  a_size_total, uint64 &  a_size_xferd )  [private]

Parse I/O from rsync. Search for special output from rsync to tell us something about the number and size of files and files transfered. Definition at line 727 of file archiver.cc. References err_unknown, ERROR_INSTANCE, mf_trim_string(), and error::push_back(). Referenced by mf_process_rsync_io(). Here is the call graph for this function:

void job_archiver::mf_process_child_io (  bool  a_finalize  )  [private]

Process I/O from the child. While there is I/O to be read, read and parse it. When the end of a line is reached write that line to the log file. If a_finalize is true, the flush the child I/O buffer string. Definition at line 636 of file archiver.cc. References config, execute::err_eof(), execute::err_read(), execute::err_ready(), configuration_manager::io_poll_interval(), logger, configuration_manager::logging_rsync, m_child_pid, m_exec, m_io_err, m_io_out, mf_process_report(), execute::out_eof(), execute::out_read(), execute::out_ready(), prefix(), and log_manager::write(). Referenced by process(). Here is the call graph for this function:

void job_archiver::mf_process_report (  const std::string &  a_str  )  [private]

Definition at line 621 of file archiver.cc. References single_job_report::add_report(), m_jpr, and m_jr. Referenced by mf_process_child_io(). Here is the call graph for this function:

void job_archiver::mf_process_rsync_io (  execute &  a_exec, uint16  a_timeout, uint64 &  a_files_total, uint64 &  a_files_xferd, uint64 &  a_size_total, uint64 &  a_size_xferd, bool &  a_overflow_detected )  [private]

Process I/O from rsync. If there is I/O from rsync to be read, read it and then send it through the parser. Definition at line 849 of file archiver.cc. References config, timer::duration_secs(), configuration_manager::io_poll_interval(), m_error_msg, mf_parse_rsync_io(), vault_manager::overflow(), timer::start(), timer::stop(), TRY_nomem, and vaulter. Referenced by mf_do_chores(). Here is the call graph for this function:

void job_archiver::mf_trim_string (  std::string &  a_str  )  [private]

Trim off all non-digit leading and trailing characters from a string. Definition at line 713 of file archiver.cc. Referenced by mf_parse_rsync_io().

const std::string job_archiver::prefix (  void   )

Generate a job prefix string. Create a string to uniquely identify this job to be used in the log file to uniquely identify this job Definition at line 134 of file archiver.cc. References m_job. Referenced by end(), id(), mf_process_child_io(), process(), and start().

void job_archiver::process (  void   )

Parent processor for a job. Check for I/O from the child. Check the child's status to see if it's still running, has exited with an exit code, or has exited from a signal. If the child sis not exit normally (i.e. exit from a signal or exit with a non-zero exit code) then check the vault for overflow. If the vault has exceeded it's overflow threshold then that could be the cause for the child's failure, in which case we reschedule the child to be processed again later. If the job is finished (whether successful or not), update the job archiver's status to "completed". Definition at line 300 of file archiver.cc. References execute::child_exit_code(), execute::child_exited_normally(), execute::child_running(), execute::child_signal_no(), execute::child_signaled(), timer::duration(), logger, m_exec, m_status, m_timer, mf_process_child_io(), prefix(), status_completed, status_error, timer::stop(), and log_manager::write(). Here is the call graph for this function:

single_job_report job_archiver::report (  void   )  const

Return the job report for this job. Definition at line 360 of file archiver.cc. References m_jr.

void job_archiver::start (  void   )

Begin processing. Attempt to fork a child process to handle this job. If unsuccessful then retry again later. The child then calls mf_do_chores() to handle the actual processing, while the parent updates the job archiver's status from "pending" to "processing" and begins a timer to measure the duration of the job process. Definition at line 212 of file archiver.cc. References execute::child_pid(), err_unknown, execute::exit(), execute::fork(), job::generate_job_id(), single_job_report::id(), execute::is_child(), logger, m_child_pid, m_exec, m_job, m_jr, m_status, m_success, m_timer, mf_do_chores(), prefix(), execute::reroute_stdio(), timer::start(), status_processing, status_retry_later, error::str(), and log_manager::write(). Here is the call graph for this function:

const job_archiver::archiving_status job_archiver::status (  void   )

Return the processing status of this job archiver. Definition at line 198 of file archiver.cc. References m_status.

---

Member Data Documentation

pid_t job_archiver::m_child_pid [private]

Definition at line 67 of file archiver.h. Referenced by clear(), end(), mf_process_child_io(), and start().

std::string job_archiver::m_error_msg [private]

Definition at line 70 of file archiver.h. Referenced by clear(), mf_do_chores(), and mf_process_rsync_io().

execute job_archiver::m_exec [private]

Definition at line 63 of file archiver.h. Referenced by clear(), end(), mf_process_child_io(), process(), and start().

estring job_archiver::m_io_err [private]

Definition at line 66 of file archiver.h. Referenced by end(), and mf_process_child_io().

estring job_archiver::m_io_out [private]

Definition at line 65 of file archiver.h. Referenced by end(), and mf_process_child_io().

const job* job_archiver::m_job [private]

Definition at line 60 of file archiver.h. Referenced by id(), job_archiver(), mf_do_chores(), prefix(), and start().

job_path_report job_archiver::m_jpr [private]

Definition at line 68 of file archiver.h. Referenced by clear(), and mf_process_report().

single_job_report job_archiver::m_jr [private]

Definition at line 69 of file archiver.h. Referenced by clear(), mf_process_report(), report(), and start().

archiving_status job_archiver::m_status [private]

Definition at line 61 of file archiver.h. Referenced by clear(), end(), job_archiver(), process(), start(), and status().

bool job_archiver::m_success [private]

Definition at line 64 of file archiver.h. Referenced by clear(), mf_do_chores(), and start().

timer job_archiver::m_timer [private]

Definition at line 62 of file archiver.h. Referenced by end(), process(), and start().

---

The documentation for this class was generated from the following files:

• archiver.h
• archiver.cc

---