job_archiver Class Reference

Archive the paths associated with a single job. More...

#include <archiver.h>

Collaboration diagram for job_archiver:

Collaboration graph
[legend]
List of all members.

Public Types

enum  archiving_status {
  status_pending, status_processing, status_retry_later, status_fatal_error,
  status_error, status_completed, status_done
}

Public Member Functions

 job_archiver (const job *a_job)
 C'tor.
const std::string prefix (void)
 Generate a job prefix string.
const std::string id (void)
 Generate a job id string.
void clear (void)
 Clear the job archiver and return it to it's initial state.
void end (void)
 End any processes handling this job.
const archiving_status status (void)
 Return the processing status of this job archiver.
void start (void)
 Begin processing.
void process (void)
 Parent processor for a job.
single_job_report report (void) const
 Return the job report for this job.

Private Member Functions

void mf_do_chores (void)
 Child processor for a job.
void mf_process_report (const std::string &a_str)
void mf_process_child_io (bool a_finalize)
 Process I/O from the child.
void mf_process_rsync_io (execute &a_exec, uint16 a_timeout, uint64 &a_files_total, uint64 &a_files_xferd, uint64 &a_size_total, uint64 &a_size_xferd, bool &a_overflow_detected)
 Process I/O from rsync.
void mf_parse_rsync_io (std::string a_str, uint64 &a_files_total, uint64 &a_files_xferd, uint64 &a_size_total, uint64 &a_size_xferd)
 Parse I/O from rsync.
void mf_trim_string (std::string &a_str)
 Trim off all non-digit leading and trailing characters from a string.
void mf_parse_report (const std::string &a_str)

Private Attributes

const jobm_job
archiving_status m_status
timer m_timer
execute m_exec
bool m_success
estring m_io_out
estring m_io_err
pid_t m_child_pid
job_path_report m_jpr
single_job_report m_jr
std::string m_error_msg

Detailed Description

Archive the paths associated with a single job.

Definition at line 36 of file archiver.h.


Member Enumeration Documentation

enum job_archiver::archiving_status
 

Enumeration values:
status_pending 
status_processing 
status_retry_later 
status_fatal_error 
status_error 
status_completed 
status_done 

Definition at line 38 of file archiver.h.


Constructor & Destructor Documentation

job_archiver::job_archiver const job a_job  ) 
 

C'tor.

Set a job to be assiciated with this job archiver and initialize it's processing status to "pending".

Definition at line 121 of file archiver.cc.

References clear(), m_job, m_status, and status_pending.

Here is the call graph for this function:


Member Function Documentation

void job_archiver::clear void   ) 
 

Clear the job archiver and return it to it's initial state.

End any processes handling this job and return the job archiver to it's "pending" state.

Definition at line 163 of file archiver.cc.

References job_path_report::clear(), single_job_report::clear(), end(), m_child_pid, m_error_msg, m_exec, m_jpr, m_jr, m_status, m_success, execute::my_pid(), and status_pending.

Referenced by job_archiver().

Here is the call graph for this function:

void job_archiver::end void   ) 
 

End any processes handling this job.

If any child processes are handling this job, terminate them. Erase any pending I/O for the now defunct child. Set our processing status to "done".

Definition at line 180 of file archiver.cc.

References execute::child_running(), execute::clear(), execute::kill_child(), logger, configuration_manager::logging_manager, m_child_pid, m_exec, m_io_err, m_io_out, m_status, m_timer, prefix(), status_done, timer::stop(), and log_manager::write().

Referenced by clear().

Here is the call graph for this function:

const std::string job_archiver::id void   ) 
 

Generate a job id string.

Definition at line 146 of file archiver.cc.

References job::generate_job_id(), m_job, and prefix().

Here is the call graph for this function:

void job_archiver::mf_do_chores void   )  [private]
 

Child processor for a job.

For each path in this job:

  • Create the directory heiararchy for this job in the archive
  • Until done or until out of retrys:
    • Choose a hardlink source, if applicable and available
    • Construct the command line to pass to rsync
    • Spawn rsync
    • Process I/O sent back from rsync
    • Process exit code or signal number returned from rsync
  • Generate and submit a report to the report manager

Definition at line 378 of file archiver.cc.

References archiver, config, job::connection_local, ERROR, job::excludes, execute::exec(), exists(), rstat::exit(), rsync_behavior::fail, job::generate_archive_path(), job::generate_source_path(), job::includes, is_timestamp(), logger, m_error_msg, m_job, m_success, mf_process_rsync_io(), mk_dirhier(), mk_relative_path(), rsync_behavior::ok, subdirectory::path(), job::paths, reform_path(), relative_path(), rsync_behavior::retry_without_hardlinks, job::rsync_behavior, job::rsync_connection, job::rsync_hardlink, configuration_manager::rsync_local_path(), job::rsync_options, job::rsync_remote_path, job::rsync_retry_count, job::rsync_timeout, log_manager::set_error_logging(), rstat::signal(), timer::start(), timer::stop(), configuration_manager::timestamp(), TRY_nomem, vault_manager::vault(), vaulter, archive_manager::working_archive_path(), and writable().

Referenced by start().

Here is the call graph for this function:

void job_archiver::mf_parse_report const std::string &  a_str  )  [private]
 

void job_archiver::mf_parse_rsync_io std::string  a_str,
uint64 &  a_files_total,
uint64 &  a_files_xferd,
uint64 &  a_size_total,
uint64 &  a_size_xferd
[private]
 

Parse I/O from rsync.

Search for special output from rsync to tell us something about the number and size of files and files transfered.

Definition at line 727 of file archiver.cc.

References err_unknown, ERROR_INSTANCE, mf_trim_string(), and error::push_back().

Referenced by mf_process_rsync_io().

Here is the call graph for this function:

void job_archiver::mf_process_child_io bool  a_finalize  )  [private]
 

Process I/O from the child.

While there is I/O to be read, read and parse it. When the end of a line is reached write that line to the log file. If a_finalize is true, the flush the child I/O buffer string.

Definition at line 636 of file archiver.cc.

References config, execute::err_eof(), execute::err_read(), execute::err_ready(), configuration_manager::io_poll_interval(), logger, configuration_manager::logging_rsync, m_child_pid, m_exec, m_io_err, m_io_out, mf_process_report(), execute::out_eof(), execute::out_read(), execute::out_ready(), prefix(), and log_manager::write().

Referenced by process().

Here is the call graph for this function:

void job_archiver::mf_process_report const std::string &  a_str  )  [private]
 

Definition at line 621 of file archiver.cc.

References single_job_report::add_report(), m_jpr, and m_jr.

Referenced by mf_process_child_io().

Here is the call graph for this function:

void job_archiver::mf_process_rsync_io execute a_exec,
uint16  a_timeout,
uint64 &  a_files_total,
uint64 &  a_files_xferd,
uint64 &  a_size_total,
uint64 &  a_size_xferd,
bool &  a_overflow_detected
[private]
 

Process I/O from rsync.

If there is I/O from rsync to be read, read it and then send it through the parser.

Definition at line 849 of file archiver.cc.

References config, timer::duration_secs(), configuration_manager::io_poll_interval(), m_error_msg, mf_parse_rsync_io(), vault_manager::overflow(), timer::start(), timer::stop(), TRY_nomem, and vaulter.

Referenced by mf_do_chores().

Here is the call graph for this function:

void job_archiver::mf_trim_string std::string &  a_str  )  [private]
 

Trim off all non-digit leading and trailing characters from a string.

Definition at line 713 of file archiver.cc.

Referenced by mf_parse_rsync_io().

const std::string job_archiver::prefix void   ) 
 

Generate a job prefix string.

Create a string to uniquely identify this job to be used in the log file to uniquely identify this job

Definition at line 134 of file archiver.cc.

References m_job.

Referenced by end(), id(), mf_process_child_io(), process(), and start().

void job_archiver::process void   ) 
 

Parent processor for a job.

Check for I/O from the child. Check the child's status to see if it's still running, has exited with an exit code, or has exited from a signal. If the child sis not exit normally (i.e. exit from a signal or exit with a non-zero exit code) then check the vault for overflow. If the vault has exceeded it's overflow threshold then that could be the cause for the child's failure, in which case we reschedule the child to be processed again later.

If the job is finished (whether successful or not), update the job archiver's status to "completed".

Definition at line 300 of file archiver.cc.

References execute::child_exit_code(), execute::child_exited_normally(), execute::child_running(), execute::child_signal_no(), execute::child_signaled(), timer::duration(), logger, m_exec, m_status, m_timer, mf_process_child_io(), prefix(), status_completed, status_error, timer::stop(), and log_manager::write().

Here is the call graph for this function:

single_job_report job_archiver::report void   )  const
 

Return the job report for this job.

Definition at line 360 of file archiver.cc.

References m_jr.

void job_archiver::start void   ) 
 

Begin processing.

Attempt to fork a child process to handle this job. If unsuccessful then retry again later. The child then calls mf_do_chores() to handle the actual processing, while the parent updates the job archiver's status from "pending" to "processing" and begins a timer to measure the duration of the job process.

Definition at line 212 of file archiver.cc.

References execute::child_pid(), err_unknown, execute::exit(), execute::fork(), job::generate_job_id(), single_job_report::id(), execute::is_child(), logger, m_child_pid, m_exec, m_job, m_jr, m_status, m_success, m_timer, mf_do_chores(), prefix(), execute::reroute_stdio(), timer::start(), status_processing, status_retry_later, error::str(), and log_manager::write().

Here is the call graph for this function:

const job_archiver::archiving_status job_archiver::status void   ) 
 

Return the processing status of this job archiver.

Definition at line 198 of file archiver.cc.

References m_status.


Member Data Documentation

pid_t job_archiver::m_child_pid [private]
 

Definition at line 67 of file archiver.h.

Referenced by clear(), end(), mf_process_child_io(), and start().

std::string job_archiver::m_error_msg [private]
 

Definition at line 70 of file archiver.h.

Referenced by clear(), mf_do_chores(), and mf_process_rsync_io().

execute job_archiver::m_exec [private]
 

Definition at line 63 of file archiver.h.

Referenced by clear(), end(), mf_process_child_io(), process(), and start().

estring job_archiver::m_io_err [private]
 

Definition at line 66 of file archiver.h.

Referenced by end(), and mf_process_child_io().

estring job_archiver::m_io_out [private]
 

Definition at line 65 of file archiver.h.

Referenced by end(), and mf_process_child_io().

const job* job_archiver::m_job [private]
 

Definition at line 60 of file archiver.h.

Referenced by id(), job_archiver(), mf_do_chores(), prefix(), and start().

job_path_report job_archiver::m_jpr [private]
 

Definition at line 68 of file archiver.h.

Referenced by clear(), and mf_process_report().

single_job_report job_archiver::m_jr [private]
 

Definition at line 69 of file archiver.h.

Referenced by clear(), mf_process_report(), report(), and start().

archiving_status job_archiver::m_status [private]
 

Definition at line 61 of file archiver.h.

Referenced by clear(), end(), job_archiver(), process(), start(), and status().

bool job_archiver::m_success [private]
 

Definition at line 64 of file archiver.h.

Referenced by clear(), mf_do_chores(), and start().

timer job_archiver::m_timer [private]
 

Definition at line 62 of file archiver.h.

Referenced by end(), process(), and start().


The documentation for this class was generated from the following files:
Generated on Fri Jun 23 16:57:08 2006 for rvm by  doxygen 1.4.2