The GroundWork Distributed Monitoring Agent (GDMA) was developed primarily as a scaling method for GroundWork Monitor installations. In simple terms, it is an agent that runs independently of the Nagios scheduler in GroundWork Monitor, and forwards the state of the monitored services on individual nodes to the GroundWork server using the NSCA protocol. The Nagios instance in GroundWork Monitor processes these results as passive checks. The configuration information for which services to run on which hosts, the monitoring thresholds, frequency, etc., are hosted on the GroundWork server with http or https and are retrieved by the GDMAs.

Auto Registration overview

The GDMA installer asks for an Automated Agent Registration username/password at install time. (The defaults presented to the user are blank, but gdma/gdma will work in a fresh GroundWork Monitor install out of the box. Changing the password on the server is recommended for customers before deploying GDMA. If both values are given, then the auto-registration is enabled on that GDMA client. If one or both are blank, then the auto-registration is disabled and the old auto-configuration mechanism will still be in play, albeit perhaps at a slower pace than before (see the hour delay noted below).

This new gdma user on the server has limited (generally read-only) privileges for accessing most server functions, and those restrictions should be left in place. It is intended that this username be used only for this Automated Agent Registration purpose, partly so that if any trouble arises with it, the account can be disabled without impacting any other aspect of the monitoring.

The GDMA client installer also asks for host and service profile names. Both are optional. The installer suggests a standard default value for the host profile, which happens to be the same as the initial default we have configured on the server side. That default is gdma-linux-host.xml or equivalent for other platforms (gdma-windows-host.xmlgdma-solaris-host.xmlgdma-aix-host.xml). If no service profile is specified, then no extra service profile will be applied beyond any service profiles referenced directly by the host profile.

The new values requested by the GDMA client installer are stored on the client in gdma/config/gdma_auto.conf:

Auto_Register_Host_Profile = ""
Auto_Register_Pass = ""
Auto_Register_Service_Profile = ""
Auto_Register_User = ""

These settings can be adjusted after installation, if need be. Like anything else in the gdma_auto.conf file, modifying values there will in due course cause the client to recognize a configuration change and attempt to reload its own configuration — now in up to three files: gdma_auto.confgdma_override.conf, and the gwmon_<hostname>.conf file.

The gdma/config/gdma_override.conf file is new with the GDMA 2.3.0 release. It is dynamically created as the place where the poller will store the hostname returned by the server, after a successful auto-registration request. The poller uses a Forced_Hostname option to save this centrally-agreed-upon hostname. This gets picked up by the spooler as well, so both poller and spooler know the registered name of the host. Any use of the Forced_Hostname option in the gdma_auto.conf file will be overridden by the value in the gdma_override.conf file, if the latter file is present.

When the client makes an auto-registration call to the server, it passes along a small amount of information about the client system, including  the configured Auto_Register_Host_Profile and Auto_Register_Service_Profile values. The foundation/scripts/ script on the server is called to validate the host attributes, add the host to Monarch, build externals for the host (based on whatever externals were attached to the host profile, or to the services attached to the service profile(s)), and return the final hostname. Building externals means that the associated host and service externals are collected together, processed via possible macro substitution, and written to a location where the client can pick them up.

In practice, a green system operates in the following way. An un-configured poller will reach for a local copy of the host-config file, find none, and go to sleep for 10 minutes (configured in gdma_auto.conf as the Poller_Proc_Interval). On waking up, when it again sees it still cannot find its config file, if it has auto-registration credentials, it will do the auto-registration thing; otherwise, it will fall back to the old auto-configure protocol. All the auto-registration thing does is to make a call to the server, and look at the results. If the results include a hostname, the poller stores the hostname in the gdma_override.conf file. At that moment, it still doesn't have a host config file in hand. On the next cycle, it should pull the externals file from the GroundWork server, save it locally, and start monitoring the client machine.

The spooler, in its own cycle time, eventually sees that the configuration files have changed, and re-reads them. Then it will see the forced hostname and adapt to it.

When an auto-registration request comes into the server, it first tries to look up in Monarch to see if the host is already registered. There is a special Perl package installed (foundation/scripts/ that implements a pass of possible host-attribute recoding before this lookup. If that lookup fails, a second possible recoding (presumably with different logic this time) and lookup is attempted. If both passes fail and the package has not concluded that the attributes are totally invalid, the host will be added to Monarch.

The second pass of recoding allows IP = hostname lookups based on specific addresses listed in the server config/ file, for cases where DNS just isn't operating right on the client and there's no way to fix that easily. 

Auto-registered hosts will begin monitoring as soon as they receive their own host configuration file from the server. They will then start sending on the results of those checks to the GroundWork server. At this point, however, Nagios won't know of the existence of such hosts, and it will casually drop those results on the floor (probably with a notice to that effect in the nagios/var/nagios.log file). Results from such hosts will continue to be ignored until the administrator (or perhaps some automated process) runs a Commit operation.

Additional Information

  • The config/ file on the server contains a number of configuration options to control the server-side scripting.
  • The module on the server contains documentation regarding the internal routines, which can help both with understanding its actions and for making possible modifications: /usr/local/groundwork/perl/bin/perldoc /usr/local/groundwork/foundation/scripts/

Related articles