About Black List

The Black List feature allows you to stop Cloud Hub discovered hosts from being included in the system monitoring. Hosts that are blacklisted will not be displayed, monitored, distribute notifications, be counted against the system license limit, and overall be reported on within the system. This can be useful for hosts that are not currently being used but may be in the future. Blacklisted hosts are not seen by the system: once blacklisted the entry for that host is DELETED from devices and host table, and the CloudHub connector logic will NOT try to add any host that matches a blacklist regular expression.

Cloud Hub consults this blacklist for references to hosts that will not be discovered and added to the Foundation database and used against the license limit. This is useful when Cloud Hub discovers devices like templates and test servers in the same location where production devices are found. To start monitoring blacklisted hosts again, you would remove the host from the list.

Timing of the changes note: As a rule it can take 0-5 min to refresh the regex cache in Cloud Hub where the Black Lists are kept in real time (whatever set of specifications you added to the Black List page will be read in by Cloud Hub and cached for faster access). Then, it can take 0-X minutes to refresh the connector on top of that where X is the refresh interval of the individual connector. Total maximum wait time is X+5 minutes.

Adding Black List Records

  1. As an administrator, go to Configuration > Devices > Black List.
  2. Select the + icon and enter the exact host name or a regular expression (see below).
  3. Click Add which places the record on the list of blacklisted hosts. The added host(s) will be removed from dashboards.
    new black list item

Using Regular Expressions

Regular expressions are patterns used to match character combinations in strings. In JavaScript, regular expressions are also objects. You can select several hosts by using a regex in the Host Name field for the Black List feature. Below we show an example of how to use the Black List feature with a regex. It is recommended to use java/javascript regex patterns instead of using file path wildcard syntax. For additional information regarding regular expressions you may want to visit the following link which is specific to the regex use case in Java 7 we are deploying, we've also listed a link to test regular expressions:

Other reference sites exist but the subject is complicated. Perl regular expressions will not work here.

Example: In this first image we show the Status dashboard for a VMware Cloud Hub connection. You can see the host group NET:VM Network which lists hosts that include the string "6-7" and "southern".

adding regular expressions

Next we use the regular expressions (dev-centos-6-7-)\w+ and (southern-)\w+ where (captures the group character codes), \w matches any word character (alphanumeric and underscore), and + matches one or more of the preceding token.

regex example

This captures all of the hosts in our example except dev-centos-6-7-fuel-plugin. You can see the remaining hosts by searching for southern.

regex results

Additional examples:

Name of black list recordHosts removed from Status dashboard

gw-logstash-0[12]

gw-logstash-01 and gw-logstash-02

ag*r0

aggr0

(ag)\w+

aggr0_gwos_netapp_colo_02_0

(wingdma-)\w*

wingdma-dev

sol10_64(.*)

sol10_64 qa (barbie)

NET-VM Network

NET-VM\WNetwork

Negative Matching

You may find that you have a production infrastructure with wonderfully regular names that are to be monitored. Yet there are many templates, unused instances, test instances, ad hoc clones and other detritus with names made up at random that are not following a pattern. The challenge is how to blacklist the unwanted hosts without slavishly typing in each and every unwanted name, a fool's errand. Here is the answer.

You need a regular expression that identifies the hosts to be blacklisted as a negative, that is blacklist those that are "NOT this pattern". You will identify in ONE regular expression the patterns for ALL the hosts you wish to KEEP; any that do not match at least one of the patterns will be blacklisted. The important point to remember is that this is an inclusive expression that says, if the host Cloud Hub has discovered matches ANY part of the expression it is safe, but if it matches NONE of the patterns it will be blacklisted.

There is a further twist to this approach, that is the hosts seen by a connector are associated with a grouping factor we refer to as the "hypervisor". In VMware the hypervisor is the ESX server where the host instance is located. In Amazon the hypervisor is the Region, for example "us-west-2a". So in this negative blacklist regex you must first identify the regions and the hypervisors to be kept, followed by the regular expressions identifying the host name patterns to be kept. The expression is interpreted from left to right. Leaving out a region results in ALL the hosts in that region being blacklisted.

All the patterns, not just this negative regex but all the others, are applied to every discovered host, by every connector, in every cycle.

Example:

Let's imagine all your production servers are in one region, "us-west-1a", and they are all named with a starting string in the name "PROD-denver" or "PROD-dallas" or "PROD-washington" (with numbers or letters following the prefix). And let's imagine that all the other uninteresting hosts, also in the region "us-west-1a" are not named with any of those prefixes. Let's say they are all prefixed as "TEST" or "QA" or "Template" along with a few outliers "abc123", "Joe-Test", "20180704-large-system", "JD-ubuntu-1604".

Name of blacklist recordHosts removed from system
^(?!us-west-1a|PROD-denver|PROD-dallas|PROD-washington).*$TEST* QA* Template* abc123 Joe-Test 20180704-large-system JD-ubuntu-1604

For clarity and copying the exact string here it is unformatted. You can have one or more of the strings like "allowedpattern1":

^(?!allowedpattern1|allowedpattern2|allowedpattern3).*$
CODE

The example is for matching of first part of each name. A more complex pattern could be used. See reference link above.

It should be clear that you could use this negative match approach to isolate monitoring discovery to a selected set of regions or hypervisors.

It should be clear that the process of identifying a host as one to be blacklisted is a sequential search against all the blacklist rules. The host must not be identified in any of the rules, to survive into the set that is added to and updated in Foundation.

Removing Black List Records

To start monitoring the blacklisted hosts again, you would first remove the host (or pattern) from the list.

  1. As an administrator, select Configuration > Devices > Black List. 
  2. Then, check all or individual host names to be removed from the blacklist, (you may also use the Filter Hosts box to search for specific hosts to manage).
  3. Click the Trashcan icon and then Delete.
  4. Next, Cloud Hub connections don't pull inventory and require a Stop, Start to get the unlisted host to be visible again.

Related Resources