Operations, Maintenance, and Monitoring¶
The long-running processes in the nowcast system framework, the message broker, the manager, and the scheduler, are managed by the circus process manager tool.
Start the nowcast system with:
$ source activate /misc/data1/dlatornell/nowcast-sys/nowcast-env (/misc/data1/dlatornell/nowcast-sys/nowcast-env)$ circusd --daemon $NOWCAST_CONFIG/circus.ini
circusd monitors the long-running processes and restarts them if they crash or are shutdown accidentally.
circusctl is the command-line interface for interacting with the processes that are running under circusd. Start it with:
$ source activate /misc/data1/dlatornell/nowcast-sys/nowcast-env (/misc/data1/dlatornell/nowcast-sys/nowcast-env)$ circusctl
See the circusctl man page, or use the help command within circusctl to get information on the available commands. A few that are useful:
list to get a comma-separated list of the processes that circusd is managing
status to see their status
stop to stop a process; e.g. stop scheduler
start to start a stopped process; e.g. start scheduler
restart to stop and restart a process; e.g. restart scheduler
signal hup to send a HUP signnal to a process, which will cause it to reload its configuration from the
NOWCAST_YAMLfile that the process was started with; e.g. signal hup manager. This is the way to communicate nowcast system configuration changes to the long-running processes.
quit to stop all of the processes and shutdown circusd
Use ctrl-c to exit from circusctl.