About GroundWork TCG

The connection between GroundWork servers and monitoring data sources such as Nagios is facilitated by a new component called TCG, or the Transit Connection Generator (see https://github.com/gwos/tcg). Based on the NATS project, TCG allows the creation of resilient connections that automatically re-establish when broken, and buffers data to ensure a lossless stream of metrics. TCG is easy to configure, and flexible. 

The GroundWork Transit Connection Generator (TCG) is open source and available on GitHub.

Standalone GroundWork servers (8.1.0 >) and TCG

The typical default standalone GroundWork server uses TCG to forward data from Nagios to GroundWork Foundation, our normalization and aggregation layer. You will see a connection called “Local Nagios” under the Connectors menu, which is the local internal connection that does this.

Parent-child servers and TCG

Child servers use TCG to forward Nagios state data and metrics to parent servers. 

Types of TCG connections

Four connection types are supported in TCG as of GroundWork Monitor 8.1.0:

Nagios

This is a connector to a local Nagios with the Data Geyser enhancement running. 

You can also connect this to a parent server to create an independently managed GroundWork child.

Nagios parent-managed child 

This is a special connection type for sending data from a child to a parent. It is special because it is provisioned and managed on the parent, not the child. 

Elastic

This is similar to a Cloud Hub connector, in that it connects to a source of monitoring data, Elastic Stack in this case. It uses TCG to send its state and metric data to GroundWork.

Server

This connector allows a TCG instance running on a Linux server to forward basic resource monitoring metrics to GroundWork. Since TCG is a go program, it runs as a single executable, and is easily daemonized, configured with a single config file, and connected to. 

APM

The Application Performance Monitoring connector with Jaeger also uses TCG, along with a Prometheus client to forward custom metrics from your applications to GroundWork for monitoring. 

Configuration Details

TCG uses NATS for guaranteed delivery of monitoring messages. The NATS component uses on-disk queues to save messages that can't be immediately delivered. These queues can be limited in scope by size and by age, separately. Here's how these settings can be adjusted for your TCG implementations. 

Defaults

The maximum age a message can be in TCG is 10 days (240 hours), and the maximum size a queue can be is 50 GB. This means that by default you need to have 50 GB of disk set aside per connector. In a standalone GroundWork installation, this 50 GB requirement is covered easily by the 200GB minimum needed for running GroundWork 8.x. However, when one or more of the above optional configurations is used, more space has to be provisioned. 

Changing maximum age and queue size for Nagios connections

The Nagios instances on GroundWork standalone or child servers will queue and retransmit monitoring results using TCG. If you have child servers, you may want to adjust the settings for how much data to queue and for how long. 

To adjust the Nagios connector size on a standalone or child GroundWork server: 

1. Access the command line of your GroundWork server and change to the gw8 directory:

   $ cd gw8

2. Edit the datageyser_tcg_config.yaml in the nagios container: 

   docker-compose exec nagios vi /usr/local/groundwork/config/datageyser_tcg_config.yaml

3. Adjust the following lines as needed: 

       natsStoreMaxAge: 240h0m0s
       natsStoreMaxBytes: 53687091200

   e.g. to make the timeout 3 days and the size 1GB: 

       natsStoreMaxAge: 72h0m0s
       natsStoreMaxBytes: 1073741824

   and save the file.

   
   

4. Restart GroundWork to make the changes take effect: 

   $ docker-compose down
   $ docker-compose up -d

Changing maximum age and queue size for containerized connections

If you run connectors on your GroundWork server as connectors, these connections will queue and retransmit monitoring results using TCG. You may want to adjust the settings for how much data to queue and for how long, since the local connectors will potentially consume a lot of disk space if they are unable to communicate with the GroundWork system for a while. 

To adjust the containerized connector size on a GroundWork server: 

1. Access the command line of your GroundWork server and change to the gw8 directory:

   $ cd gw8

2. Edit the tcg_config.yaml in the connector container, for example if you are running the Elastic connector as a container, the container will be called tcg-elastic, and the file to edit will be called /tcg/elastic-connector/tcg_config.yaml: 

   docker-compose exec tcg-elastic vi /tcg/elastic-connector/tcg_config.yaml

3. Adjust the following lines as needed: 

       natsStoreMaxAge: 240h0m0s
       natsStoreMaxBytes: 53687091200

   e.g. to make the timeout 3 days and the size 1GB: 

       natsStoreMaxAge: 72h0m0s
       natsStoreMaxBytes: 1073741824

   and save the file.

   
   

4. Restart GroundWork to make the changes take effect: 

   $ docker-compose down
   $ docker-compose up -d

   Changes to values in this file will be preserved across restarts of GroundWork, but other changes such as comments will not.