cncopagent - backup agent dispatcher

Overview

The cncopagent is responsibile for orchestrating the actions of running a backup job. It manages the details of running the appropriate backup agent, submitting finish reports to the cncop network, pruning backup logs, re-running backups when it detects a correctible error and running pre/post backup commands. It also ensures only one instance of a specific job runs at a time, and can optionally transmit ad-hoc email messages when a job finishes.

Backup agents

The work of backups are performed by individual agents. The main job of cncopagent is to run the assigned agent with the appropriately formatted command line. The details of which agent it runs are determined by the jobtype setting in the BackupJob database.

Job types

The current list of supported jobtypes is

Job Type	Agent	Description
CCS	backupagentnq and backupagentdq	Carroll-Net Cloud Backup
CIFS	backupagentcifs	Windows File System Backup
FTP FTPS	backupagentftp	FTP Server Backup and FTP over TLS Backup
MSSQL	backupagentmssql	Microsoft SQL Server
SFTP	backupagentsftp	FTP over SSH Backup
RSYNC	backupagentrsync	Rsync Backup
WIMG	backupagentwimg	Windows Image Backup

Job re-run

Each agent can finish with one of three possible states; 1) OK, 2) ERROR or 3) RE-RUN. In the case of OK or ERROR, the cncopagent will consider the job finished, but if an agent finishes with RE-RUN cncopagent will consult the job’s maximum retry setting and if retries are not exhausted, it will re-run the agent. This design allows backups to recover from a variety of transient error conditions (consult the documentation for each agent for an explanation of what it considers transient errors).

Agent log maintenance

Each individual backup agent creates logs of the actions it took. These detailed records provide critical information to know what an agent did and why. To prevent an excessive accumulation of logs, cncopagent will prune the oldest logs. The default is to keep 90 days of logs. Users can change the number of log days kept by setting days_of_logs in the [defaults] section of the Config/cncop3.ini file.

The log files created by the cncopagent are written to the Logs directory. Each invocation of the cncopagent will create a unique logfile who’s name is formatted as CA-YY-MM-DD_HHMM-NNN.log. The component YY-MM-DD is the date, HHMM is the time of day (24-hour) using the local time of the machine. The last three digit suffix is an optional integer that will be sequentially increased to ensure uniqueness (eg: 001, 002, 003,…).

The cncopagent log files are useful when researching why a job may not have run. If an agent experiences a catastrophic failure, error messages are printed to it’s standard error output before the failure. This can provide vital diagnostic information when other traditional research avenues fail to provide insight.

To watch log files are they are being written, use the watcher utility.

Submit finish reports to the cncop network

Each backup agent creates it’s own job finished reports that are written the Spool/Messages/ directory. After each agent is run, cncopagent will invoke cncopreporter to submit job finish reports to the cncop network. This style of store and forward reporting ensures that even if the Internet is temporarily unavailable, job finished reports will be reliably submitted to the cncop network for job reporting.

Pre/Post backup commands

Each job can be configured to run an external command before the job runs and after a job runs. This system is useful when preparatory steps are required to run a job; eg: dumping a database before backing it up, mounting a remote share, etc…

cncopagent is responsible for running these commands. The commands can be any arbitrary program or script the user choose. For security, the program or script must be located within the cncop application directory (under the Bin/ directory).

Each job has three settings that can be used to select when these external commands are run. They are 1) Run before job, 2) Run after successful completion and 3) Run after failed completion.

Prerun exceptions

The stdout and stderr of the prerun command are capture to the cncopagent log. If the prerun command exits with anything other than zero, the first 2,048 bytes of stderr will be submitted as a jobexception to the cncop network. The jobexception will be associated with the backup session that the prerun was run before.

Single job instance protection

cncopagent will ensure that only one instance of a specific jobid runs at a time. If a second instance tries to start, cncopagent will detect the duplicate request and skip the job.

Ad-hoc job finished email notifications

cncopagent can optionally be invoked with a request to send email notifications when a job is finished. This one-shot notification provides a mechanism for an administrator who wishes to monitor a job session.

These one-shot notifications are sent directly from the host running the cncopagent and requires valid mail settings in the Config/cncop3.ini. To use this option, the mail_server setting must be configured.

Note

This notification is in addition to the JobFinished reports configured in the backup settings of each individual job. It’s intended as an ad-hoc notification system when one wishes to be notified about a specific instance of a job run.

Command line options

usage: cncopagent  [-?] [-d] [-c CONFIG] [-a] [-t TYPE] [--email EMAIL] [job [job ...]]

Positional arguments

job

Jobid or Jobtitle of backup job(s) to run. Caller must specify one of --alljobs, --type TYPE or list individual job(s).

Optional argument:

?, -h, --help

Display help and exit

-d, --debug

Generate diagnotic logging. The output is directed to the Spool/Logs/ folder and can be viewed using the cncop watcher scripts.

-c CONFIG, --config CONFIG

Alternate configuration file.

-a, --alljobs

Run all backup job(s), sorted alphabetically by Jobtitle. On a Healthy Server, all PULL jobs will be run before PUSH jobs.

-t TYPE, --type TYPE

Run all job(s) with the specified jobtype (case insensitive)

--email EMAIL

After job completes, send a completion email message to EMAIL. Note: this option requires the mail server settings have been configured in the Config/cncop3.ini file.