cncopagent - backup agent dispatcher
Contents
cncopagent
- backup agent dispatcher#
Overview#
The cncopagent
is responsibile for orchestrating the actions of running a
backup job. It manages the details of running the appropriate backup agent,
submitting finish reports to the cncop network, pruning backup logs, re-running
backups when it detects a correctible error and running pre/post backup
commands. It also ensures only one instance of a specific job runs at a time,
and can optionally transmit ad-hoc email messages when a job finishes.
Backup agents#
The work of backups are performed by individual agents. The main job of
cncopagent
is to run the assigned agent with the appropriately formatted
command line. The details of which agent it runs are determined by the
jobtype
setting in the BackupJob database.
Job types#
The current list of supported jobtypes
is
Job Type |
Agent |
Description |
---|---|---|
CCS |
Carroll-Net Cloud Backup |
|
CIFS |
|
Windows File System Backup |
FTP FTPS |
|
FTP Server Backup and FTP over TLS Backup |
MSSQL |
|
Microsoft SQL Server |
SFTP |
|
FTP over SSH Backup |
RSYNC |
|
Rsync Backup |
WIMG |
|
Windows Image Backup |
Job re-run#
Each agent can finish with one of three possible states; 1) OK, 2) ERROR or 3)
RE-RUN. In the case of OK or ERROR, the cncopagent
will consider the job
finished, but if an agent finishes with RE-RUN cncopagent
will consult the
job’s maximum retry
setting and if retries are not exhausted, it will
re-run the agent. This design allows backups to recover from a variety of
transient error conditions (consult the documentation for each agent for an
explanation of what it considers transient errors).
Agent log maintenance#
Each individual backup agent creates logs of the actions it took. These
detailed records provide critical information to know what an agent did and
why. To prevent an excessive accumulation of logs, cncopagent
will prune
the oldest logs. The default is to keep 90 days of logs. Users can change the
number of log days kept by setting days_of_logs
in the [defaults]
section of the Config/cncop3.ini
file.
The log files created by the cncopagent
are written to the Logs
directory. Each invocation of the cncopagent
will create a unique logfile
who’s name is formatted as CA-YY-MM-DD_HHMM-NNN.log
. The component
YY-MM-DD
is the date, HHMM
is the time of day (24-hour) using the local
time of the machine. The last three digit suffix is an optional integer that
will be sequentially increased to ensure uniqueness (eg: 001, 002, 003,…).
The cncopagent
log files are useful when researching why a job may not have
run. If an agent experiences a catastrophic failure, error messages are printed
to it’s standard error output before the failure. This can provide vital
diagnostic information when other traditional research avenues fail to provide
insight.
To watch log files are they are being written, use the watcher
utility.
Submit finish reports to the cncop network#
Each backup agent creates it’s own job finished reports that are written the
Spool/Messages/
directory. After each agent is run, cncopagent
will
invoke cncopreporter
to submit job finish reports to the cncop network.
This style of store and forward reporting ensures that even if the Internet is
temporarily unavailable, job finished reports will be reliably submitted to the
cncop network for job reporting.
Pre/Post backup commands#
Each job can be configured to run an external command before the job runs and after a job runs. This system is useful when preparatory steps are required to run a job; eg: dumping a database before backing it up, mounting a remote share, etc…
cncopagent
is responsible for running these commands. The commands can be
any arbitrary program or script the user choose. For security, the program or
script must be located within the cncop application directory (under the
Bin/
directory).
Each job has three settings that can be used to select when these external commands are run. They are 1) Run before job, 2) Run after successful completion and 3) Run after failed completion.
Prerun exceptions#
The stdout and stderr of the prerun command are capture to the cncopagent
log.
If the prerun command exits with anything other than zero, the first 2,048
bytes of stderr will be submitted as a jobexception to the cncop network. The
jobexception will be associated with the backup session that the prerun was run
before.
Single job instance protection#
cncopagent
will ensure that only one instance of a specific jobid
runs
at a time. If a second instance tries to start, cncopagent
will detect the
duplicate request and skip the job.
Ad-hoc job finished email notifications#
cncopagent
can optionally be invoked with a request to send email
notifications when a job is finished. This one-shot notification provides
a mechanism for an administrator who wishes to monitor a job session.
These one-shot notifications are sent directly from the host running the
cncopagent
and requires valid mail settings in the
Config/cncop3.ini
. To use this option, the mail_server
setting
must be configured.
Note
This notification is in addition to the JobFinished reports configured in the backup settings of each individual job. It’s intended as an ad-hoc notification system when one wishes to be notified about a specific instance of a job run.
Command line options#
usage: cncopagent [-?] [-d] [-c CONFIG] [-a] [-t TYPE] [--email EMAIL]
[job [job ...]]
Positional arguments#
-
job
#
Jobid or Jobtitle of backup job(s) to run. Caller must specify one of
--alljobs
,--type TYPE
or list individual job(s).
Optional argument:#
-
?
,
-h
,
--help
#
Display help and exit
-
-d
,
--debug
#
Generate diagnotic logging. The output is directed to the
Spool/Logs/
folder and can be viewed using the cncop watcher scripts.
-
-c
CONFIG
,
--config
CONFIG
# Alternate configuration file.
-
-a
,
--alljobs
#
Run all backup job(s), sorted alphabetically by
Jobtitle
. On a Healthy Server, all PULL jobs will be run before PUSH jobs.
-
-t
TYPE
,
--type
TYPE
# Run all job(s) with the specified jobtype (case insensitive)
-
--email
EMAIL
# After job completes, send a completion email message to
EMAIL
. Note: this option requires the mail server settings have been configured in theConfig/cncop3.ini
file.