This page provides a quick-study guide to configuring GDMA Auto Setup. Background and detailed reference material are available in sibling pages.

Client-side Configuration

The initial versions of GDMA where Auto-Setup was available did not provide for automatic configuration of this feature on the client; manual intervention was necessary. However, to take advantage of a variety of improvements and bug fixes in GDMA overall, you should be using GDMA 2.7.1 or later, where enabling Auto-Setup is supported during the installation process. Here, we will focus mainly on that context.

GDMA Auto-Setup is not enabled by default; because locally-tailored server-side instructions and trigger files are involved, using it is a deliberate choice. Also, it uses the same user credentials as GDMA Auto-Registration. Therefore, the following unattended-mode command-line install options (or their equivalent option-file forms) are needed to make Auto-Setup available on the client.

--gdma_autoregistration_username 'my_gdma_autoregistration_username'
--gdma_autoregistration_password 'my_gdma_autoregistration_password'
--gdma_auto_configuration autosetup

Equivalent choices are available in text-mode and UI-mode GDMA-client installation interactions. You must provide the user/pass values in order for the choice of enabling Auto-Setup to appear.

Behind the scenes, enabling Auto-Setup modifies the following settings in gdma/config/gdma_auto.conf on the client, to values as shown. (The Auto_Register_User and Auto_Register_Pass values must be tuned as appropriate for your site. Some other settings are also involved, but they remain unchanged so we will not discuss them here.)

Auto_Register_Attempts = "never"
Auto_Register_Host_Profile = "none"
Auto_Register_Pass = "gdma"
Auto_Register_User = "gdma"
Enable_Auto_Setup = "on"
Auto_Register_Attempts is changed from its typical value of fibonacci to never instead, to disable that mode of automatically establishing an initial configuration for the GDMA client. Similarly, Auto_Register_Host_Profile is set to none so as to prevent any possible application of this field should Auto-Setup be mistakenly disabled at some point in the future.

When installing GDMA 2.6.X on a client, where you did not have the opportunity to enable Auto-Setup from the get-go during the install process, there was a danger that Auto-Registration would kick in and establish an undesired initial client configuration before you had a chance to set up Auto-Setup. There are ways around that, but the simple approach is to just not use such an older release. Use GDMA 2.7.1 or later, instead.

The following setting is also highly recommended now, established in gdma/config/gdma_auto.conf and not overridden in host externals:

Enable_Local_Logging = "on"
That's because as of GDMA 2.6.1 (and fully fixed in 2.7.1), logging will automatically auto-rotate each logfile when it reaches a certain size, so there is no longer any danger of infinite growth. Having the logfiles always available makes problem diagnosis easier, as most often you won't need to run the GDMA processes manually to see what failed. You can just look at the end of the appropriate logfile. (That may be especially useful now on Windows, if you have GDMA run as a special non-admin user that you cannot log in as to run the daemons manually.) In terms of Auto-Setup, the discovery results themselves will be sent to the server, but there may be some additional amount of detail logged on the client about resources found on the machine but not matched by your discovery instructions.

Note: There are some issues with controlling Enable_Local_Logging in host externals if you are sharing host externals between Windows GDMA 2.7.1-or-later machines and earlier releases on this platform. See GDMA 2.7.1 Release Notes for more information.

While developing new Auto-Setup instructions, you might also want to shorten tests of successive versions on one GDMA machine of interest with:

Poller_Proc_Interval = "60"
instead of the default client cycle time:
Poller_Proc_Interval = "600"
With a one-minute polling interval like that, you won't need to waste so much time waiting for the GDMA client to cycle around and look for updated instructions and trigger files. Once your development has settled down and you are no longer doing dry runs, you can put back the longer polling interval for production monitoring.

Once gdma/config/gdma_auto.conf is configured, if you had disabled starting the GDMA daemons automatically at install time, you can start them using the appropriate platform-specific command, see Docker Commands.

Setting Up Service Externals

Starting with GroundWork Monitor 7.2.0, building externals now supports several new macros that may be referenced within externals. These may be used in combination with Auto-Setup facilities to make service externals definitions almost completely boilerplate. A typical service external definition might look like this:

Check_$BASESERVICEDESC$[$INSTANCE$]_Enable="ON"
Check_$BASESERVICEDESC$[$INSTANCE$]_Service="$SERVICEDESC$"
Check_$BASESERVICEDESC$[$INSTANCE$]_Command="check_train -t $INSTANCESUFFIX$ -w $ARG2$ -c $ARG3$"
Check_$BASESERVICEDESC$[$INSTANCE$]_Check_Interval="1"
Check_$BASESERVICEDESC$[$INSTANCE$]_Timeout="30"
In such a service externals definition, it is primarily the _Command line that must still be tuned for the particular service check to be run. And even that definition can take advantage of some of the new macro support. For exmple, in the sample externals above, $INSTANCESUFFIX$ refers to the service instance name as defined in Monarch, for a multiple-instance service, with a possible leading underscore omitted. Also, argument references like $ARG2$ and $ARG3$ refer to either externals arguments defined at the host-service level (for a single-instance-on-host service), or at the service-instance level (for a multiple-instance-on-host service). Especially when using GDMA Auto-Setup, you should strive to make your service externals as general as possible, and use such argument substitution to fill in the details. With Auto-Setup in particular, the values substituted for these macros can be derived automatically from the auto-discovery results, greatly reducing the manual work involved. We'll have more to say about that in later sections of this document.

Effectively, as of GroundWork Monitor 7.2.0, all service externals should be converted over to this kind of boilerplate definition, whether or not Auto-Setup is in use. That will simplify maintenance going forward.

Creating Instructions Files

Instructions files consist of a single heading line specifying the file format version:

format_version = "1.0"
followed by some number of sensor definitions. Blank lines in the file are ignored. There is no convention for continuation lines. Comments in the file start with the # character and run to the end of the line. Instructions files are subject to one pass of backslash-escape interpretation as they are read, which means you must double any backslashes you wish to be present as literal characters in directive values. Depending on the nature of how the directive value is to be processed, further doubling may be required to have the value interpreted as you wish by the final code. For instance, a pattern directive value is a Perl regular expression, so to get a backslash recognized as a literal character in the regex, it will need to be doubled for that purpose.

(Because handling of backslashes that way can be so clumsy on Windows, some sensors allow forward slashes to be used in pathnames on Windows platforms. We will have more to say on that in the discussion of those sensors, in the configuration-reference material on GDMA Auto-Setup.)

Most directive values should be double-quoted, to guard against any misinterpretation when the value is parsed as the instructions file is read:

instance_ext_args = "$MATCHED1$!40!60"
Occasionally, it may be necessary to include a double-quoted sub-value within an overall double-quoted directive value.
resource = " "C:\\\\program files/outlook*" "
That looks odd, but such nesting is fully supported. In the Configuration Reference for Auto Setup page, we describe the situations where this may be useful.

Sensor definitions come in three flavors, depending on the type of Monarch object the sensor is to configure (host profile, service profile, or individual service).

<host "my label 1">
    type = "..."
    ...
    host_profile = "my-host-profile-name"
</host>

<service "my label 2">
    type = "..."
    ...
    service_profile = "my-service-profile-name"
</service>

<service "my label 3">
    type = "..."
    ...
    service = "my-service-name"
</service>
You will want exactly one host_profile sensor to match on each GDMA client, to define all host-level setup. It is perfectly acceptable to include multiple host_profile sensors in a single instructions file, but you must ensure that their matching behavior is mutually exclusive.

You can have as many service_profile and service sensors as you wish match on each GDMA client, depending on the monitoring requirements. service_profile sensors are provided for generality, but they are less useful than original envisaged. Using them effectively requires that you create service profiles within Monarch that contain only a single service, which leads to a somewhat pointless proliferation of service profiles. It's generally better to just use service sensors instead, and skip the intermediate level of configuration.

As noted, each sensor definition refers to a single object in Monarch to be configured (a host profile, service profile, or service). That means you must provide the initial definition of the host profile, service profile, or generic service for the server-side discovery processing to customize and fold into the setup for the GDMA client. Those object definitions must be in place before discovery results are processed, or the processing will generate an error and fail to change the host's configuration.

In GroundWork Monitor 7:

  • Host profiles are defined under Configuration > Profiles > Host profiles.
  • Service profiles are defined under Configuration > Profiles > Service profiles.
  • Generic services are defined under Configuration > Services, not attached to an individual host.

In GroundWork Monitor 8:

  • Host profiles are defined under Configuration > Nagios Monitoring > Profiles > Host profiles.
  • Service profiles are defined under Configuration > Nagios Monitoring > Profiles > Service profiles.
  • Generic services are defined under Configuration > Nagios Monitoring > Services, not attached to an individual host.

To ease your own maintenance burden, you will want to create a small set of general-purpose instructions files that each applies to many GDMA client machines. If a given service only runs on a few GDMA machines, it can still be processed on other GDMA machines. The resource filtering and pattern matching in the sensor definition should be sufficient to exclude the sensor from consideration on those other clients.

You can find some sample instructions files Examples of Auto Setup, and detailed reference material on the available sensors in the document Configuration Reference for Auto Setup.

The autosetup tool installs instructions files in a special location where the GDMA clients know to pick them up, using a special protocol to avoid race conditions. In order that autosetup be able to recognize an instructions file so it can put it in the proper location, the name of each of your instructions files must consist of at least one character followed by _instructions to form the complete filename. Typically, you would name each master instructions file based on the class of machines it is meant to configure, such as:

backup_server_instructions
compute_node_instructions
database_server_instructions
file_server_instructions 
eb_server_instructions
You get to choose the boundaries between classes, so you get to decide how many such files you need. Typically, you would want to have separate classes for each type of OS platform (e.g., Linux vs. Windows), because sensor definitions are unlikely to be usefully shared across platforms.
linux_database_instructions
windows_database_instructions
Aside from the likely sensibility of separating OS platforms, you may be able to collapse all of your instructions into simple general OS-dependent instructions files, and allow sensor matching (or not) to do all the work of distinguishing classes, if that fits your monitoring needs:
linux_instructions
windows_instructions

Creating Trigger Files

Trigger files consist of just a few possible directives. Here is the canonical dry-run trigger file:

# Dry-run Auto_Setup discovery trigger.
last_step = "test_configuration"
if_duplicate = "optimize"
soft_error_reporting = "ignore"
change_policy = "non_destructive"
Details on the supported values for those directives are provided in the document Configuration Reference for Auto Setup, with implementation limitations. To start out, you probably only need two trigger files, one for dry-run testing and one for live-action deployment of configuration changes. Sample trigger files are provided in Examples of Auto Setup.

The autosetup tool installs trigger files in a special location where the GDMA clients know to pick them up, using a special protocol to avoid race conditions. In order that autosetup be able to recognize a trigger file so it can put it in the proper location, the name of each of your trigger files must consist of at least one character followed by _trigger to form the complete filename. Beyond that, the filename can be whatever you want, though for everyone's sanity, it should describe the particular purpose of the trigger:

dry_run_trigger live_action_trigger

Invoking the autosetup Command

In GroundWork 7, the command to run autosetup is:

/usr/local/groundwork/gdma/bin/autosetup
In GroundWork 8, the command to run autosetup is:
# Situate yourself in the proper directory before running autosetup.
cd /path/to/gw8
docker-compose exec monarch /usr/local/groundwork/monarch/bin/autosetup
Running those two particular commands as shown without additional command-line arguments will yield a manual page describing all the available options. Additional command-line arguments can be added to those commands as needed to access those options.

Installing Instructions and Trigger Files

All of the instructions files that you create must be kept in your own private master stash, located in some place that you define. See Practical Advice for Auto Setup for more information about this.

To run discovery, a GDMA client must find an instructions file and a corresponding trigger file in a known location on the GDMA server. The names of those two files in the known locations must be myhostname_instructions and myhostname_trigger, respectively, so each GDMA host can tell which file is intended for its own use. The myhostname part may be specified in either fully-qualified or unqualified form; the GDMA client will look first for the one form and then for the other if the first is not found.

You should adopt some consistent conventions up front as to how you will install your instructions and trigger files. In particular, you don't want both fully-qualified and unqualified files for the same host to be present in the special locations known to the GDMA client, or the situation will become muddled as to which of them you intend to be in play.

In order for a discovery instructions file to be downloaded by a GDMA client and used to run a pass of discovery there, the GDMA client must find a corresponding trigger file with its last-modified timestamp later than that on the instructions file. This provides safety against race conditions when these files are copied separately into their respective known locations for pickup by the GDMA clients. If both instructions and trigger files are mentioned on the same autosetup command line, this constraint will be automatically forced. If you use separate autosetup commands to install instructions and trigger files, the autosetup command for the instructions file must be run first, and running the autosetup command that installs the trigger file afterward will ensure that its last-modified timestamp has the proper relationship to the already-existing instructions file.

Generally, you don't concern yourself any of with those details, except for understanding the proper ordering if separate autosetup commands are used. What you care about is specifying the master files you wish to be interpreted by particular GDMA clients, and the autosetup commands used to install your master files in the special known locations, as shown just below. (To keep the column width small in this text, we use a bash alias so we don't need to type the full path to the autosetup command. You can also do the equivalent by other means, such as using the full path in each invocation, or (in GW7) by adding /usr/local/groundwork/gdma/bin to your command-search PATH environment variable.)

Here are typical commands you might use in GroundWork 7:

alias autosetup=/usr/local/groundwork/gdma/bin/autosetup
cd /path/to/my_master_stash
autosetup install -p my_special_class_instructions my-gdma-hostname
autosetup install -p live_action_trigger my-gdma-hostname
If you are dealing with many hosts at the same time, you can also name multiple hosts on such commands:
alias autosetup=/usr/local/groundwork/gdma/bin/autosetup
cd /path/to/my_master_stash
autosetup install -p my_class_instructions myhost1 myhost2 myhost3
autosetup install -p dry_run_trigger myhost1 myhost2 myhost3
Here are typical commands you might use in GroundWork 8. In that context, you must be aware that running docker-compose can only be done in the .../gw8 directory used for your GroundWork 8 installation. Thus all pathnames to your master instructions and trigger files must be written to assume that they are being accessed while the gw8 directory is the current working directory, even though that will typically not be where you store your master copies of those files.
alias autosetup="cd /path/to/gw8; docker-compose exec monarch /usr/local/groundwork/monarch/bin/autosetup"
set stash="/path/to/my_master_stash"
autosetup install -p $stash/my_special_class_instructions my-gdma-hostname
autosetup install -p $stash/live_action_trigger my-gdma-hostname
If you are dealing with many hosts at the same time, you can also name multiple hosts on such commands:
alias autosetup="cd /path/to/gw8; docker-compose exec monarch /usr/local/groundwork/monarch/bin/autosetup"
set stash="/path/to/my_master_stash"
autosetup install -p $stash/my_class_instructions myhost1 myost2 myhost3
autosetup install -p $stash/dry_run_trigger myhost1 myhost2 myhost3

Dry Run Testing

If you have any concerns about how the discovery instructions will be interpreted on a given GDMA client, you should perform a dry-run discovery. The key part is the setting of last_step in the trigger file:

last_step = "test_configuration"
You will typically have it set that way in a previously-prepared trigger file named dry_run_trigger, as we referenced above.

When discovery results are sent from the GDMA client to the server, the registration script on the server saves them in a file. On the GroundWork server, you can probe for what happened during discovery using the following command, using the same autosetup alias as shown above for either GW7 or GW8, respectively:

autosetup print results ‌my-gdma-hostname
You can use either a fully-qualified hostname or an unqualified hostname in that command. If only one form is available in the saved files, it will find the relevant file and display its content. If multiple forms are present and there is any ambiguity as to which one you intended, you will be informed of that at the top of the output.

To go one step further and see what the server did during its dry-run configuration using the discovery results, use the following command, again using the autosetup alias:

autosetup print analysis my-gdma-hostname

Live Action Configuration

To perform live-action discovery, intending for the results to affect the production configuration, the trigger file must specify that the results are to be processed all the way to the end:

last_step = "do_configuration"
You will typically have it set that way in a previously-prepared trigger file named live_action_trigger, as we referenced above.

You can verify the discovery results and the configuration changes in summary form using the same autosetup commands as we just described for a dry run.

Related Resources