About NoMa Notifications

GroundWork has integrated NoMa, a free standing notification and escalation subsystem which no longer requires the use of Nagios notifications. The NoMa subsystem also permits changes to notification and escalation schedules to be made by users with roles having lower privileges than the system administration role, which makes the system more flexible to operate. 

Notifications and escalations using Nagios remain available for configuration and maintenance as shown by the direct line from Nagios to Notifications in the image below. This permits customers that have made investments in time or scripting for Nagios alerts to continue to use these methods.

There is also a way to send Nagios notifications to NoMa, as shown by the dotted line and further described in the next section. 

As shown, the alert flow coming from monitoring sources such as Nagios and Cloud Hub (and any other source) is sent to the RESTful API for the Foundation database. In turn, NoMa subscribes to alert messages via the REST API and sends any alerts it finds there that pass its filters. This figure shows the relationship between Nagios, NoMa, and GroundWork Foundation.

By using the NoMa front-end, hosts and services can be assigned to notification definitions. When NoMa receives notifications, it searches for matching host and service definitions in its database. If a matching definition is found, contacts and methods are determined and notifications will be sent. The following diagram shows how the product is employed within GroundWork Monitor.

In rare cases, you may need to consider using Nagios notifications to place alerts directly in the NoMa input queue. This is useful if you have specific requirements for escalating alerts from Nagios using contacts and methods already defined in NoMa, or if you want to continuously notify a contact or a group of contacts until an issue is acknowledged or resolved.

Generally, using the continuous notification features of Nagios is not  recommended, since it contributes to alarm fatigue. A single notification on state change should be sufficient. However, many organizations have become accustomed to this mode of operation, so GroundWork provides a Nagios notification script and command definition called alert_via_noma for this purpose. 

If you find you need this functionality and want to implement, see Nagios Notifications.

Configuring NoMa Notifications

Known Issue

The NoMa notification system includes the capability of limiting notifications to a certain time frame. The most common, and default time frame is 24/7, which has a default expiration date of 12/31/2021. This configuration is adjusted in Configuration > Notifications > Configuration > Time Frames. The Validity Information section for the Valid To field needs to be adjust to a later date e.g., 12/31/2099. Note this configuration change may result in queued notifications being sent, please see GroundWork Messenger Timeframes Remediation.

GroundWork alert notifications can be configured using Nagios or NoMa. The Nagios method will only notify on Nagios configured elements, and NoMa will notify on both Nagios and other configured monitors (e.g., Cloud Hub, GDMA).

NoMa notification rules essentially define the who, what, where, when, why, and how of alert notifications.

Rules contain various directives including contacts and contact groups defining who is to receive notifications, what monitored hosts and services should be included or excluded in the notifying, where and when notifications should be sent, which event states to notify upon, and how the notification will be distributed. Notification rules can be enabled (activated) and disabled, and rules can be set to escalate after a specified number of notifications is reached. The image below illustrates the notification rule (first tab), and points out where the components are defined (other tabs) and where they are incorporated within a rule.

To create a notification rule:

1. Go to Configuration > Notifications.
2. Within the Notifications tab, click Create.
   
3. Enter the notification rule directives, starting with the top section Notification Information:
   • Name: Enter a name to identify the notification.
   • Description: (optional) This field describes the notification.
   • Activated: If this directive is checked, the notification rule is enabled (activated).
   • Owner: This directive identifies the owner of a rule for editing, an administrator can edit all rules.
     
4. In the next section Hosts and Services, you'll need to indicate which hosts, host groups, services, service groups are to be included or excluded in the notifications.
   
   • Includes: An applied include constraint will only allow the notification rule to pass if the host or service belongs to at least one of the listed hostgroups or servicegroups. 
   • Excludes: Conversely, an applied exclude constraint will only allow the notification rule to pass if the host or service does not belong to any of the listed hostgroups or servicegroups.
   • For additional information see the below Appendix A: NoMa Includes and Excludes.
     

     

     

     Tips

     • The " * " wildcard character indicates "all" possibilities, (e.g., all hosts). Any include fields that are empty are assigned an implicit " * " and will automatically match all possibilities.
     • In specifying comma-separated lists of hosts, hostgroups, services, or servicegroups for inclusion or exclusion filtering, '*' can be used as a multi-character wildcard and '?' can be used as a single-character wildcard, within any particular item in such a list.
     • For the Hosts and Services fields, typing a host name and pressing Tab will display a list of valid hosts and services.
     • The Recipients field is unimplemented and should not be used.
5.  Continuing with the notification rule, the Contacts and methods to notify section is used to define who is to receive notifications (Contacts, Contact Groups), how the notification will be distributed (Methods), and the when the notifications should be sent (Timezone and Timeframe).

   

   

   Tips

   • When selecting elements in each of the Contacts, Groups, and Methods, it is only associated with the present notification rule if its background is grayed indicating it has been selected. To select a single element click on the element, to select more than one element use the Control, Command, or Shift keys while making a selection.
   • Contacts and Groups are defined in their respective tabs and applied in the Notifications tab. Holidays are applied in the Contacts tab, and indicate a period of time a contact should not receive notifications. The Holiday option is only displayed for previously defined contacts. 
   • Method options are defined in the Methods tab and applied in the Notifications tab:
     • E-Mail: Emails are sent using the standard system mailer, you may need to configure your mail relay to accept mail from the system.
       (Command: sendemail, Contact: email, Sender: root@localhost, Fallback method: none, Acknowledgeable: No)
       For information on sending email notifications to modern email services such as Gmail or Office 365 using sendEmail as a TLS-compliant client, see in this document Appendix B: Enabling sendEmail with TLS with NoMa.
     • Growl: Growl notifications are sent using an included script, it requires the Perl module Net::Growl and it uses UDP 9887 by default. After install, allow notifications from network, optionally require password from LAN, the server only requires Perl module Net::Growl as stated earlier. Client, depending on OS, requires an extra client: Mac: http://growl.info Linux: http://mattn.github.com/growl-for-linux/ Windows: http://www.growlforwindows.com/
       (Command: growl, Contact: growladdress, Sender: none, Fallback method: none, Acknowledgeable: No)
     • SMS: SMS alerts are sent via an attached SMS-capable modem using smstool3 or an iSMS/SMSFinder hardware device.
       (Command: sendsms, Contact: mobile, Sender: none, Fallback method: none, Acknowledgeable: No)
     • Voice: Voice alerts are generated with an Asterisk (or Starface) soft PBX.
       (Command: voicecall, Contact: phone, Sender: none, Fallback method: none, Acknowledgeable: Yes)
     • Voice + E-Mail fallback: Voice alert with email method fallback.
       (Command: voicecall, Contact: phone, Sender: none, Fallback method: E-Mail, Acknowledgeable: Yes)
     • Voice + SMS fallback: Voice alert with SMS method fallback.
       (Command: voicecall, Contact: phone, Sender: none, Fallback method: SMS, Acknowledgeable: Yes)
     • For all method options: The Fallback method field is used if the first method returns an error, and is currently only used by the voicecall plugin, to provide fallback to SMS or Email. And, any method that is configured as Acknowledgeable will cause the escalation chain to end, typically only voice should use this method as E-Mail and SMS provide no guarantee that a message has been received and understood.

   • The Timeframe directive indicated the hours allowed for notifications to be sent. Timeframes are defined in the Timeframes tab and applied in the Contacts, Holidays, Contactgroups and Notifications tabs.
     

     

     Known Issue

     The NoMa notification system includes the capability of limiting notifications to a certain time frame. The most common, and default time frame is 24/7, which has a default expiration date of 12/31/2021. This configuration is adjusted in Configuration > Notifications > Configuration > Time Frames. The Validity Information section for the Valid To field needs to be adjust to a later date e.g., 12/31/2099. Note this configuration change may result in queued notifications being sent, please see GroundWork Messenger Timeframes Remediation.

      
     

6. Next is the section Which events to notify. Here you need to select (check or uncheck) the filter options for recovery, problem, state transitions, and other states of change for which to send notifications.

7. For Numbers and more, set the notify after number to indicate when notifications should start being sent, (e.g., 1 sends a first notification, 2-6 sends second through sixth notifications).

   

   • If the box for Rollover if the last rule is reached is checked, NoMa will start notifications again from number 1 after the last notification (or escalation) has been reached.
   • If the box for Let notifier handle escalations is checked, NoMa is enabled to simulate the reception of notifications from Nagios, this is useful for one-off alerts.
8. Select Create.

9. If you wish to continue notifications after the specified notify after directive, you can add escalations.

   

   • The escalation option is only displayed for previously defined notification rules, so the notification rule would need to be saved first.
   • If you need to delete a contact group and it is associated with an escalation, you need to first delete the escalation, then delete the contact group. If you deleted a contact group that is associated with an escalation, this will cause a crash of the NoMa daemon and an error in the debug log.
   • Go to Configuration > Notifications and select the pencil icon corresponding to the rule to update.
   • Then, select add Escalation toward the bottom of the screen, and set the Contacts and or Groups to be notified, and the Methods of communication for the escalation.
   • As in the first notification, in an escalation the Notify after number indicates when this notification should be escalated.
   • Click Create, adding additional escalations or click Save again to save the notification rule.

Appendices

Appendix A: NoMa Includes and Excludes

In the Hosts and Services section of a notification rule, you can specifically indicate which hosts, host groups, services, service groups and recipients are to be included or excluded.

The default " * " wildcard character indicates "all" possibilities, (e.g., all hosts). Any include fields that are empty are assigned an implicit " * " and will automatically match all possibilities.

In specifying comma-separated lists of hosts, hostgroups, services, servicegroups, or recipients for inclusion or exclusion filtering, '*' can be used as a multi-character wildcard and '?' can be used as a single-character wildcard, within any particular item in such a list. For Hosts and Services fields, typing a host name and pressing Tab will display a list of valid hosts and services.

An applied include constraint will only allow the notification rule to pass if the host or service belongs to at least one of the listed hostgroups or servicegroups. Conversely, an applied exclude constraint will only allow the notification rule to pass if the host or service does not belong to any of the listed hostgroups or servicegroups.

This section describes an example notification rule with host configuration for two hosts, applied includes and excludes filters, and the notification results.

Host configuration:

Host and services filters:

Notification results:

For Host 1 the rule matches and the notification is sent. For Host 2 the rule does not match and the notification is not sent to members of this notification rule.

Appendix B: Enabling sendEmail with TLS with NoMa

GroundWork Monitor allows you to send email notifications with modern email services such as Gmail or Office 365, as a TLS compliant client. These instructions will walk you through the steps of setting up this client with the NoMa notification manager.

Prerequisites

• A service user email account to authenticate and send email on the service (e.g., Gmail account)
• An app password for the user account. An app password is a password that is separate from the normal email client password. You usually have to generate these from your email service account settings, e.g., App Password for Google and App Password for Office 365.
• Outbound network access on port 587 or similar port (usually open if you use one of these hosted email services)

To configure TLS compliant client with NoMa

1. Log in to the command line on your GroundWork 8 server, and go to the gw8 directory.

2. Edit the gw8.env file and add the following lines:

   SMTPSERVER_SERVERPORT=smtp.mymail.com:587
   SMTPSERVER_TLS=yes
   SMTPSERVER_AUTHUSER=myserviceaccount@mymail.com
   SMTPSERVER_AUTHPASS=myapppassword

   CODE

   where smtp.mymail.com:587 is your email service endpoint and port, myserviceaccount@mymail.com is the service user email account, and myapppassword is the app password.

3. Restart the NoMa daemon after making this change to have the system pick up the new configuration:
   

   docker-compose kill noma_daemon
   docker-compose up -d noma_daemon

   CODE

4. To test, enter the following from the command line:
   

   docker-compose exec noma_daemon /usr/bin/sendEmail -f myserviceaccount@mymail.com -t myemailaddress@mydomain.com -u 'test from noma' -m '***** NoMa *****' -u 'test from noma'  

   CODE

   where myserviceaccount@mymail.com is the service account, and myemailaddress@mydomain.com is an email account you have access to. You should see the test email arrive within a few minutes. If you have something other than a message on the screen with email sent successfully, please correct the issue before proceeding. 

5. Restart the NoMa daemon:
   

   docker-compose kill noma_daemon
   docker-compose up -d noma_daemon

   CODE

   At this point, you should be able to configure contacts in the NoMa notification screens, and see email coming through when notifications are triggered.

Related Resources

• Page:
  Home Dashboard
• Page:
  Nagios Notifications
• Page:
  Scheduled Downtime