On AIX, we the equivalent of:

`uname -v`.`uname -r`

Typical sensor values on this platform are strings like:

5.3

On HP-UX, the proper derivation on this platform is unknown at this time. For the moment, we use the equivalent of:

`uname -r`

Solaris

On Solaris, we use the equivalent of:

`uname -r` `uname -v`

Typical sensor values on this platform are strings like:

5.10 Generic
5.11 11.3

On Windows, programmatically obtaining the OS version on this platform is somewhat difficult. There is a standard Perl package for that purpose, but it is subject to confusion within Windows itself. Using that mechanism, we get a string typically beginning with "Win2003", "Win2008", or "Win2012", such as "Win2003 Enterprise Edition Service Pack 2". But note that the initial "Win2012" string is also generated for Windows 2016, so it is not completely definitive on this platform. See the article Finding your Operating System version programmatically for more information on the mess that is Windows versions in this regard.

We have instead implemented this sensor for Windows by using a WMI call. Together with a little editing of our own inside the sensor, it results in a string similar to these.

Microsoft Windows Server 2003, Enterprise Edition Service Pack 2 Version 5.2.3790
Microsoft Windows Server 2008 Standard Service Pack 2 Version 6.0.6002
Microsoft Windows Server 2012 R2 Standard Version 6.3.9600
Microsoft Windows Server 2016 Datacenter Version 10.0.14393

If you need to make distinctions based on the OS version for this platform, you should try a run of discovery on each of the OS versions that matter to you, and see what OS version string is determined during discovery. Then you can develop sensor patterns to match against whatever strings you obtained.

This value is determined by internal logic within Perl code. Possible sensor values to be matched against using the pattern directive regular expression are:

arm
intel
powerpc
sparc

The command used to probe will probably need to vary depending on the OS distribution and version, to accommodate both initd- and systemd-based operating systems. Look at these commands as possibilities:

chkconfig --list 2>/dev/null | fgrep :on | awk '{print $1}'
systemctl list-unit-files --full --state static,enabled,indirect \
    --no-pager --no-legend | awk '{{print $1}}'

Also look at the service command. See the external articles Check running services in Fedora / RHEL / CentOS Linux server and Ubuntu full list of available services for more possibilities.

We use the equivalent of "sc query" to list the running services, though the actual implementation relies on system calls rather than running that command. We filter to only match if the service STATE is RUNNING. The value to be matched with the pattern directive regular expression will be the SERVICE_NAME as listed in the output from "sc query". A simpler way to view this data is to look at the output from:

wmic service get name,state | findstr Running

This sensor matches the regular expression given as the pattern directive against the full command line of each process. You may restrict the set of processes to be examined by providing a comma-separated or space-separated list of user names as the sensor's resource directive. Extra leading and trailing separators are allowed within the overall userlist. Using the resource directive is recommended where practical and sensible, but optional; if it is not provided, the command lines for all processes running on the system are examined.

On UNIX-like platforms, the command "ps -u 'userlist' -o args=" is used to find the command lines to match against. You can consider that while developing your pattern-directive regex. If the resource directive is not provided, "ps -e -o args=" is used instead.

On Solaris, the command string retrieved by "ps" is truncated to 79 characters, so trying to match something farther out in the command line will be fruitless. This is a limitation of Solaris, not of GDMA. If you need to identify individual processes, make sure the arguments are arranged so the unique parts fit within this constraint. Other platforms (Linux, AIX) do not appear to have this problem.

On Solaris, there is currently no support for limiting the processes under consideration to only those running in a particular zone. If such a capability is needed, talk to GroundWork.

On Windows, a username may be specified as just the user account or prepended with a domain name in the form domain\user as you would expect. That said, there is a level of backslash interpretation that occurs when the instructions file is read, that will tend to swallow a single backslash used in that fashion. Therefore, to specify both a domain and user, double the backslash between them to specify adomain\\user instead. If a simple user account is named, without a prefixed domain name, that is the only component matched against process owners. If both a domain name and a user account are provided in a given user name, then both components are matched against process owners.

On Windows, it is possible for domain names and user account names to contain spaces. That conflicts with the possible use of spaces to separate consecutive user names within the resource definition. To get around that, on this platform we support double-quoting a full username. If either the domain name or the user account name contains a space character, such double-quoting is mandatory for that username. The double quotes used in this fashion nest within the overall double quotes which contain the full resource list of user names without needing to be themselves escaped by backslashes, so you may have a specification like this:

resource = " "NT AUTHORITY\\LOCAL SERVICE" Administrator "

On Windows, you may use "." as an abbreviation for the unqualified hostname used as a domain name. So for instance, on a machine named MyMachine, this resource:

This sensor looks to see whether a particular port or set of ports are open (generally in a LISTEN state, not just in general use on an ESTABLISHED connection) on the local machine. For this sensor type, the resource specifies a list of space-separated IP address blocks, in CIDR-block notation, used to qualify the port matching (e.g., to specify on what network interface the port is open). The pattern specifies one or more port numbers to probe. Broadly speaking, a matched port must be open on some network address within at least one address range which is positively specified by the resource, and within no address range which is negatively specified by the resource. In that regard, IPv4 and IPv6 addresses are handled separately, though their respective CIDR blocks can be mixed arbitrarily in the specified sensor resource. Each individual port which is found to be open will ultimately generate a separate service instance for each service in the sensor definition's service_profile or service; this sensor type used alone is not intended to test that multiple ports are all open before instantiating a single copy of each service in the service_profile or the single named service. If we want that, we can either AND together the results of multiple <condition> objects in the sensor definition, or implement a separate open_local_port_collection sensor type.

There is no default value for the sensor resource. To match ports, you must supply a resource value. CIDR blocks are positively specified in the sensor resource by simply being present there. CIDR blocks are negatively specified in the sensor resource by being prefixed with a "!" character. This can be used, for instance, to suppress matching of ports open on the loopback interface (!127.0.0.0/8). Only IPv4 CIDR blocks affect matching against ports open on IPv4 interfaces, and only IPv6 CIDR blocks affect matching against ports open on IPv6 interfaces.

A positive 0.0.0.0/0 or ::/0 wildcard CIDR block will match a port on any IPv4 or IPv6 interface, respectively. But these wildcards are not allowed as negative CIDR blocks, because they would match all respective addresses and thus completely block any sensor matching of that address type. There is no point in that. If you want to not match any IPv4 addresses, just don't mention any IPv4 CIDR blocks, positive or negative. The same thing applies with IPv6.

Wildcards as a general idea don't just apply with respect to the sensor resources. Ports can be opened by a program either using a specific IPv4 or IPv6 address, or by using an IPv4 or IPv6 wildcarded address (effectively, all zeroes). If a port is found to be open on a wildcarded IP address, it automatically matches every non-negated CIDR block of the same address type (IPv4 or IPv6). If you wish to exclude such wildcarded IP addresses from consideration, you must provide an explicit negated CIDR block of that address type that lists the wildcard address with a full-length netmask (i.e., either !0.0.0.0/32 or !::/128).

IPv4-in-IPv6 mapped addresses, namely those in the ::ffff:0:0/96,64:ff9b::/96, and ::/96 address ranges, are not handled in any special way by the sensor matching. They are treated as ordinary IPv6 addresses. No attempt is made to recognize and handle any other special addresses (network, broadcast, multicast, etc.) in distinct ways.

Each resource component can be one of several objects:

• a specific IPv4 CIDR block, representing some restricted range of IPv4 addresses, but also matching any wildcard IP address

• a specific IPv6 CIDR block, representing some restricted range of IPv6 addresses, but also matching any wildcard IP address

• a negated IPv4 CIDR block, preventing recognition of all IPv4 addresses included in the CIDR block

• a negated IPv6 CIDR block, preventing recognition of all IPv6 addresses included in the CIDR block

• a specific IPv4 wildcard address designator ("0.0.0.0/0"), corresponding to the network programming constant INADDR_ANY

• a specific IPv6 wildcard address designator ("::/0"), corresponding to the network programming constant IN6ADDR_ANY_INIT

The address components of discovered listening ports may have any of the following forms:

• a specific IPv4 address

• a specific IPv6 address

• an IPv4 wildcard address, which may vary between platforms and be seen in diagnostic output as either * or 0.0.0.0, but which is normalized (to 0.0.0.0) before resource matching

• an IPv6 wildcard address, which may vary between platforms and be seen in diagnostic output as either * or[::] or ::, but which is normalized (to ::)  before resource matching

The address matching across all those possibilities can seem a bit complicated. So to see how this plays out in practice, and to guide the construction of your own sensor resource values, consider the following situation. Assume the following IPv4 ports are seen to be open on the machine:

0.0.0.0:123 (a wildcarded IP address, visible in all domains)
127.0.0.1:123 (an address in the loopback domain)
172.28.113.60:123 (an address in a particular private domain)

Then assume we have the following partial sensor definition (missing cardinality and perhaps some other directives) to play with, to match one or more of those open ports as sensor instances:

Some processes listen on a port not by opening an anonymous socket, but by opening a socket which is named by some path which is visible in the filesystem. For instance, /usr/local/groundwork/postgresql/.s.PGSQL.5432is used by GroundWork PostgreSQL to listen for incoming local connections. This sensor type is different from the open_local_port sensor type both because the pattern directive regular expression is matched against a filesystem path rather than a port number, and because it draws the list of candidate items (in this case, filepaths) from a source (netstat output) which does not need qualification partitioning as we do with CIDR blocks for anonymous ports. (If any such partitioning is desired, it would probably be a qualification as to whether the named socket is on a locally-mounted or remote-mounted filesystem. If there turns out to be a practical use case for this or some other resource-level qualification, talk to GroundWork.)

The sensor pattern is matched against all the open named socket paths on the machine, without pre-qualification by a sensor resource definition. This sensor type might typically be used to sense an open MySQL or PostgreSQL database port.

As noted earlier, the list of candidate socket paths is drawn from netstat output instead of being generated from fileglobs. That can produce some items that don't completely look like absolute filepaths. Here are some examples, drawn from a Linux machine:

/run/lvm/lvmpolld.socket
/run/systemd/private
/tmp/.esd-1059/socket
@/tmp/.ICE-unix/1896
@/tmp/.X11-unix/X0
@ISCSID_UIP_ABSTRACT_NAMESPACE
private/rewrite
public/cleanup

Not having a sensor resource to nail down particular filepaths also means that your pattern directive regular expression will probably need to include much of the filepath(s) you intend to match to determine whether the named socket of interest is open. If you have difficulty constructing your pattern directive regex to match the named socket you expect to see, you can look at "netstat -a" output to see what it sees, and what your pattern will be up against.

Sockets named in the filesystem (that is, the AF_UNIX socket family) is only becoming available in Windows internal builds in early 2018; it is not yet in public release. Until that happens and GroundWork has a chance to test this feature, the open_named_socket sensor is not supported under Windows.

When this sensor type becomes available on Windows, named socket paths will ordinarily include backslashes, making it complicated to construct corresponding sensor patterns. So to avoid the ugliness of needing to double each backslash in a pattern match and then double it again to escape backslash interpretation as the instructions file is read, we support using forward slashes in sensor patterns for such paths even on the Windows platform:

pattern = "C:/foo/bar"

As with path matching for patterns in similar sensors, on the Windows platform the forward slashes in your pattern will match backslashes in the actual filepath, and any matched backslashes will be returned as such in strings captured as part of your sensor pattern.

Specify how the discovery results are to be handled if they simply duplicate what happened in the last round of similar-type (dry-run or live-action) discovery. (Dry-run means any value of last_step before do_configuration.) This is an optional directive; it defaults to optimize. The idea is to block wasted effort on both the client and server that would not change any outcome except for recording the fact that another pass of auto-discovery was run. The possible values are:

ignore: If the GDMA client finds that the discovery results are an exact match of the last time this type of auto-discovery was run, it will log that fact locally and go no further. That is to say, even if last_step is set to send_results or some later stage, and even if the GDMA client has no externals in hand and last_step is do_configuration, no results will be sent to the server if the results of this pass are identical to those of the previous pass of such auto-discovery.

optimize: Behave exactly as ignore would have, except in the special case where last_step is do_configuration and the GDMA client has no externals in hand that correspond to those exact discovery results (i.e., that are timestamped later than the earlier run that produced those results). This allows a live-action pass of auto-discovery to punch through any previous server-side problems that caused configuraton to fail, and re-run the steps there so new externals can be generated and then picked up by the GDMA client, while not bothering the server if nothing would change.

force: If the GDMA client finds that the discovery results are an exact match of the last time this type of auto-discovery was run, it will log that fact locally but continue on. That is to say, if last_step is set to send_results or some later stage, the duplicate results will be sent on to the server in the usual manner, and the server should process them all the way through the specified last_step. This setting is provided to heal all prior wounds, ensuring that all components are fully synchronized, at the cost of possibly doing some work that won't change anything.

The most common setting for if_duplicate is:

if_duplicate = "optimize"

Specify how errors will be handled if last_step is either do_analysis or test_configuration and the auto-setup processing gets that far. The possible values are:

ignore: Errors encountered while either analyzing discovery results or running a dry-run application of those results to the Monarch configuration will be ignored, with respect to reporting them to any outside agency. This is the default behavior. It is assumed that an administrator is manually supervising some testing, and will be taking the trouble to look for any problems that might appear. The autosetup tool can help with this.

post: Errors encountered when either analyzing discovery results or running a dry-run application of those results to the Monarch configuration will be posted to the production monitoring, as though they had occurred when last_step was do_configuration. The nature of the reporting will be softened in that it will result in a WARNING-level event instead of a CRITICAL-level event, since this was just a dry-run problem. This behavior might be useful for running periodic selective automated tests to verify that there would be no problems if a pass of live-action auto-discovery were to become necessary.

The most common setting for soft_error_reporting  is:

soft_error_reporting = "ignore"

Optionally, specify a change policy specific to just the pass of discovery run by this trigger, temporarily overriding the default_change_policy set in the GroundWork server's config/register_agent_by_discovery.conf file. This directive is not required to be present in the trigger file, and there is not yet any guarantee that any value of this directive other than non_destructive will be supported on the GroundWork server.

The most common setting for change_policy  is:

change_policy = "non_destructive"

Specify how far to proceed with processing the auto-discovery instructions.  Possible values are:

ignore_instructions: This is a no-op. The GDMA client should not try to download the instructions file. This would be used only for testing the ability for the GDMA client to find and download the trigger file itself.

fetch_instructions: The GDMA client should attempt to download the instructions file, but take things no further.

do_discovery: The GDMA client should run a pass of auto-discovery, but do nothing with the results except to capture them locally in a file. It should only log errors locally if they should arise, or log the successful completion of the discovery actions.

send_results: The GDMA client should send auto-discovery results, failed or not, to the GroundWork server. Said results will include the values of all directives in the trigger file, so the server will know how far to proceed on its side. The server will do a minimal amount of inspecting the results, mainly to validate their construction as a proper auto-discovery packet and then to check the last_step value to see if further work is needed on the server. If the received data constitutes a valid packet, it will be stored in the auto-discovery results directory on the server, so the results can be inspected manually and analyzed via the autosetup  tool.

do_analysis: The GroundWork server should fully analyze the returned auto-discovery results, checking them for internal errors, inconsistencies, and client-side failures, but go no further. Problems will be logged on the server side, and the analysis results will be stored in the auto-discovery results directory for debugging and troubleshooting purposes.

test_configuration: Using the outcome of the analysis, the GroundWork server should do a dry run of configuring the GDMA client in Monarch, taking account of the current content of host and service profiles, service definitions, and so forth. Problems will be logged on the server side, but any changes made to the database for test purposes will be rolled back and not committed.

do_configuration: The GroundWork server should attempt to put all auto-discovery results into production, by both changing the Monarch database as needed and building externals for this GDMA client.

The ignore_instructions, fetch_instructions, and do_discovery choices for last_step are mostly for early development testing. The server has no notion of when the trigger file is downloaded or when the GDMA client takes any subsequent action, so it has no idea that anything should be done with the trigger file. If the trigger file stays around, the GDMA client will find it again on its next polling cycle and interpret it all over again. So it is up to the administrator to manage deleting the trigger file on the server once testing is done. That can be done either manually, by customer scripting, or with the autosetup tool.

For send_results or any of the later choices for the last_step, the last action on the server will be to remove the trigger file, after the other actions described above for that last_step are complete. This means that any given trigger file effectively runs a one-shot action. If the administrator wants to run another cycle of auto-discovery, another trigger file will need to be created and moved into place. That can be easily done with the autosetup tool.

As a practical matter, if you're going to do a dry run for most diagnostic purposes, you may as well set last_step to do_analysis or test_configuration. This both makes the trigger deletion automatic, and puts some data back on the server where it will be easier to access.

The most common settings for last_step  are:

# Use this for a full dry run.
last_step = "test_configuration"

# Use this for a full production run.
last_step = "do_configuration"