Overview

In order to create advanced functionality for simultaneous monitoring of multiple and heterogeneous virtual server environments subject to fairly rapid changes in configuration, GroundWork created Cloud Hub.

A GroundWork hub is a self-contained monitoring subsystem that performs auto discovery, monitoring configuration by an operator, synchronization between monitoring configuration and target system configurations, and monitoring of multiple and different target environments using only the GroundWork API as its data interface to the main GroundWork Monitor system. Hubs rely upon GroundWork central monitoring services for role based access control, event management, notifications, performance graphing and analysis, reporting, dashboards, status maps, and run time user interfaces. For configuration details, see How to configure Cloud Hub connectors.

Monitoring architecture and the Cloud Hub

Cloud Hub is a complete Java application that is pre-installed on the GroundWork server, and can also be installed on free standing VMs and bare metal machines provided that they have a Java environment installed. Cloud Hub is deployed in either a centralized, distributed, or mixed architecture.

Centralized mode

In the centralized mode, the single pre-installed Cloud Hub can be interfaced to multiple server virtualization managers as in the case where a customer has more than one Virtualization Management system and also in the case of virtualization managers of different vendors such as VCenter® Server from VMware, or the OpenStack© framework.

In addition to API based monitoring of vendor and open source virtualization managers, Cloud Hub can also monitor hypervisors directly if virtualization managers are not available.

Decentralized mode

In decentralized Cloud Hub deployment multiple Cloud Hubs are used to monitor a single virtualization API data source. Mixed deployments that are a combination of centralized and decentralized architectures can also be created.

Functional description of Cloud Hub

Cloud Hub performs the following functions:

• Based on operator input, Cloud Hub connects to the port containing the Virtualization Manager API and auto discovers the inventory and configuration relationships of hypervisors, virtual machines (VMs), network domains, storage domains, and resource pools etc.
• Cloud Hub automatically adds the discovered hosts, and configuration items to the GroundWork online database using the following GroundWork virtualization data model:
  • Each hypervisor shall be created as both a host and a host group
  • The Hypervisor profile will be applied to the Host representing the monitoring metrics for the hypervisor
  • All hypervisors shall be automatically added to a host group containing all hypervisors
  • All VMs currently running on each hypervisor will be added to the host group for that hypervisor
  • All VMs sharing a network domain, resource pool, or storage domain shall be collected into host groups associated with each of these entities
  • The VM profile will be applied to all virtual hosts discovered by the API

GroundWork virtualization model

The GroundWork virtualization data model is made visible and actionable in the Status pages of the GroundWork Monitor user interface.

• Automatically commences monitoring of all of the discovered VMs and hypervisors that have been discovered. Monitoring includes collection of each monitoring check result and its defined threshold and for those so configured storing each value for presentation in graphical format.
• For each metric a performance graph can be created depending on the setting in the profiles.
• Automatically updates the configuration relationships when the virtualization manager moves VMs between hypervisors or changes resource assignments without requiring operator action.
• Cloud Hub provides a web based user interface that permits the system administrator to customize the monitoring profile that is used. Changes that are supported in this way include:
  • Adding or subtracting items to be monitored from the full list of several thousand parameters that are exposed by the APIs
  • Assigning short indicative alias names to replace long form strings
  • Setting threshold values
  • Selecting synthetic measurements so that thresholds can be set in percentage of 100% to make threshold setting
  • Choosing whether the parameter is to be graphed or not

Technical description of Cloud Hub

The internal modules and interfaces between the Cloud Hub and the virtualization managers and hypervisors that it monitors are illustrated in the image below.

The functional description of each internal module and interface of Cloud Hub is as follows:

• Virtual Environment Monitoring API will receive and normalize data collected from different connectors before it is sent to the GroundWork Connector.
• Connectors are used to connect the Cloud Hub to the virtual systems, (e.g., VMware connector is used to connect the Cloud Hub to vSphere API interfaces on VMware hypervisors virtualization managers).
• Configuration U/I is a web application accessed from the GroundWork server that performs the configuration functions described previously.
• Synchronization and Monitoring Module listens, based on the contents of the Monitoring Profile and Hub preferences file, to the data stream coming from the Virtual Environment Monitoring API (VEMA) and sends both inventory updates (i.e., synchronization messages) and monitoring metrics to the GroundWork connector to be transmitted to the GroundWork central server.
• GroundWork Connector normalizes the inventory and monitoring data before sending it to the GroundWork server using the secure REST API. This module generates alerts when it detects that the value of a metric exceeds its threshold and generates performance data calls. Performance Data and Alerts messages are sent over the secure REST API to the GroundWork server.

Agent configuration properties files contain the following:

• Configuration for GroundWork API endpoint
• Configuration for virtualization API's
• Check interval
• Enabling monitoring of resource pools, network, and storage devices

Monitoring profiles contain the following:

• Metrics and threshold to be monitored for hypervisors andVM's
• Enabling / disabling automatic performance graph creation

The results of auto discovery, manual configuration, synchronization, and monitoring with Cloud Hub flows into the GroundWork Monitoring system where it is written directly to the central database through the REST API. The data is used as follows:

• Status information (Up/Down, OK, Warning, Critical) and the hierarchical relationships between all of the levels of the virtual server stack are displayed in Status

• Collection of all of the services for all of the layers of virtual server stacks depends upon the use of single consistent host names throughout the environment
• Events coming from Cloud Hub are assigned an application type (e.g., VEMA for VMware, OS for Open Stack) for viewing in the dashboard Events
• Alerts are created within the Cloud Hub feeder and made available to the Notification and Escalation subsystem by the RESTful API that also writes the associated events to the GroundWork database
• The notification and escalation subsystem will communicate alerts to designated contacts in accordance with its configuration
• Performance data is written to the production data base where it is summarized for statistical analysis

The Cloud hub applications all share the application type "VEMA", and the hosts they add to the system have this application type as the owner. For a discussion of ownership see Ownership options.

Description of operation

Cloud Hub is pre-installed in GroundWork Monitor Enterprise that is, in turn, installed by the GroundWork Installer. The Cloud Hub is available to Administrators under the Configuration menu option. Cloud Hub can be distributed as a Docker container and is portable to any Linux system running Docker CE or Docker Enterprise.

The initial configuration of cloud connectors is describe in the How-to article How to configure Cloud Hub connectors. The documentation describes the configuration of the different connector and will be extended when new connector become available.

During normal operations, the Cloud Hub keeps itself in synchronization with the virtualized server inventory and delivers monitoring data for hypervisors, VM containers, and network, storage, and other resource pools. Alerts are generated within the Cloud Hub when it is detected that the value of a metric exceeds its threshold. Such alerts are passed to the Status application for further analysis and processing by NOC operators and are also passed to the Notification and Escalation subsystem that applies its rules to notify those contacts that are scheduled to receive the alert at a particular point in time.

Some connectors have features that others lack: Google Cloud connector has a discovery feature. AWS connector can be made to support tags in EC2 for various functions. Review the documents for each connector before use. 

Related articles

• Page:
  How to determine Cloud Hub metrics to be monitored (Knowledge Base)
• Page:
  Cloud Hub troubleshooting (Knowledge Base)
• Page:
  How to configure Cloud Hub connectors (Knowledge Base)