monitors:submqttm

Error loading plugin struct
ParseError: syntax error, unexpected 'fn' (T_STRING), expecting :: (T_PAAMAYIM_NEKUDOTAYIM)
More info is available in the error log.

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
monitors:submqttm [2020/11/07 21:06] – created wnelismonitors:submqttm [2020/11/13 12:48] (current) – [Installation] wnelis
Line 8: Line 8:
  
 ===== Description ===== ===== Description =====
-NOTE: this page is under construction, and thus not yet complete. 
  
 +Script submqttm.py (short for SUBscribe_to_MQTT_Measurements) is a skeleton for one or more Xymon tests, which receive their measurements via MQTT. An abstract class XymonTest is defined, which handles the receipt of the topics, the detection that all required information is received, and the transmission of the Xymon message to the Xymon server. Per Xymon test a derived class needs to be defined, containing (at least) method BuildMessage, which transforms the measurements received from MQTT into a message to be sent to Xymon. The script contains a receiver thread which subscribes to all required topics, and which calls method XymonTest.Update to handle them. List MqttXymon near the start of the script contains the mapping between MQTT topics and Xymon tests.
  
-Script submqttm.py is a skeleton for one or more Xymon tests, which receive their measurements via MQTT. An abstract class XymonTest is defined, which handles the receipt of the topics, the detection that all required information is received, and the transmission of the Xymon message to the Xymon server. Per Xymon test a derived class needs to be defined, containing (at least) method BuildMessage, which transforms the measurements received from MQTT into a message to be sent to Xymon. The script contains a receiver thread which subscribes to all required topics, and which calls method XymonTest.Update to handle them. +As distributed, the script contains a single Xymon test, which shows the incoming (publish) and outgoing (subscribe) message rate as well as the incoming and outgoing octet rate of the MQTT broker.
- +
-As distributed, the script contains a single Xymon test, which shows the incoming and outgoing message rate as well as the incoming and outgoing octet rate of the MQTT broker.+
  
 +This script was developed to handle multiple small, low power sensors based on an ESP8266 micro-controller. As multiple destinations for the collected data are expected, and as MQTT provides for a simple transfer and distribution mechanism, these micro-controllers publish their measurements using MQTT. 
 ===== Installation ===== ===== Installation =====
 === Client side === === Client side ===
  
-Source modules submqttm.py and watchdog.py in /usr/lib/xymon/client/ext +The source modules submqttm.py and watchdog.py shown in the next chapter are to be installed in directory /usr/lib/xymon/client/ext 
-Continuously running, as the topics can arrive at any time. + 
-Tests with mode 'periodic' are run once every 5 minutes.+Script submqttm.py needs to be running continuously, as the topics can arrive at any time. Thus xymonlaunch is //not// used. In stead, a service is  defined which will start script submqttm.py at system start
 + 
 +Service submqttm is defined by creating a file named /etc/systemd/system as super user. An example of the content of this file is shown below. 
 +<code> 
 +[Unit] 
 +Description=Xymon client with MQTT input 
 +  
 +[Service] 
 +Type=simple 
 +User=xymon 
 +Group=xymon 
 +ExecStart=/usr/lib/xymon/client/ext/submqttm.py 
 +WorkingDirectory=/usr/lib/xymon/client/ext 
 +StandardOutput=null 
 +StandardError=null 
 +Restart=on-abort 
 + 
 +[Install] 
 +WantedBy=multi-user.target 
 +</code> 
 + 
 +Once file submqttm.service is created, the following two commands are needed to activate the script: 
 +<code> 
 +sudo systemctl start submqttm 
 +sudo systemctl enable submqttm 
 +</code> 
 + 
 +Note that though the script is running continuously, the tests with mode 'periodic' are run once every 5 minutes. 
  
 === Server side === === Server side ===
  
-xymonserver.cfg to be extended +For the initial client, which monitors the throughput of one MQTT broker, two extensions of the xymon server configuration are needed. First, xymonserver.cfg should be extended. This is achieved by adding file mqtt-thr.cfg in directory /usr/lib/xymon/server/etc/xymonserver.d. It' contents is: 
-graphs.cfg to be extended+ 
 +<code> 
 +TEST2RRD+=",mqtt=devmon" 
 +GRAPHS_mqtt="mqtttm,mqttto" 
 +</code> 
 + 
 +Secondly, graphs.cfg should be extended with the definitions of the graphs. In directory /usr/lib/xymon/server/etc/graphs.d (also) a file named mqtt-thr.cfg is created containing the following definitions: 
 + 
 +<code> 
 +[mqtttm] 
 +        TITLE , MQTT message rate 
 +        YAXIS Throughput [p/s] 
 +        -l 0 
 +        DEF:mr=mqtt.broker.message.rrd:Rcv:AVERAGE 
 +        DEF:mx=mqtt.broker.message.rrd:Xmt:AVERAGE 
 +        LINE1:mr#0000FF:ingress 
 +        GPRINT:mr:MIN:Min \: %5.1lf %sp/s 
 +        GPRINT:mr:MAX:Max \: %5.1lf %sp/s 
 +        GPRINT:mr:AVERAGE:Avg \: %5.1lf %sp/s 
 +        GPRINT:mr:LAST:Cur \: %5.1lf %sp/s\n 
 +        LINE1:mx#00FF00:egress 
 +        GPRINT:mx:MIN: Min \: %5.1lf %sp/s 
 +        GPRINT:mx:MAX:Max \: %5.1lf %sp/s 
 +        GPRINT:mx:AVERAGE:Avg \: %5.1lf %sp/s 
 +        GPRINT:mx:LAST:Cur \: %5.1lf %sp/s\n 
 + 
 +[mqttto] 
 +        TITLE , MQTT bit rate 
 +        YAXIS Throughput [b/s] 
 +        -l 0 
 +        DEF:or=mqtt.broker.octet.rrd:Rcv:AVERAGE 
 +        DEF:ox=mqtt.broker.octet.rrd:Xmt:AVERAGE 
 +        CDEF:br=or,8,
 +        CDEF:bx=ox,8,
 +        LINE1:br#0000FF:ingress 
 +        GPRINT:br:MIN:Min \: %5.1lf %sb/s 
 +        GPRINT:br:MAX:Max \: %5.1lf %sb/s 
 +        GPRINT:br:AVERAGE:Avg \: %5.1lf %sb/s 
 +        GPRINT:br:LAST:Cur \: %5.1lf %sb/s\n 
 +        LINE1:bx#00FF00:egress 
 +        GPRINT:bx:MIN: Min \: %5.1lf %sb/s 
 +        GPRINT:bx:MAX:Max \: %5.1lf %sb/s 
 +        GPRINT:bx:AVERAGE:Avg \: %5.1lf %sb/s 
 +        GPRINT:bx:LAST:Cur \: %5.1lf %sb/s\n 
 +</code> 
 + 
 +A small note: typically I'll end each TITLE-line with two spaces. This results in a bigger separation between the title and the date-range, which improves the readability of the top line in the graph. Those two trailing spaces are missing in the above definition of the graphs.
  
 ===== Source ===== ===== Source =====
Line 95: Line 168:
  # Throughput statistics of the MQTT broker  # Throughput statistics of the MQTT broker
   { 'Mode'     : 'periodic',   { 'Mode'     : 'periodic',
-    'MqttHost' : '127.0.0.1',+    'MqttHost' : '<your.mqtt.broker.address>',
     'Topic'    : ( '$SYS/broker/bytes/received'   ,     'Topic'    : ( '$SYS/broker/bytes/received'   ,
                    '$SYS/broker/bytes/sent'       ,                    '$SYS/broker/bytes/sent'       ,
Line 103: Line 176:
                  ),                  ),
     'XymonClss': 'XymonTestSys',     'XymonClss': 'XymonTestSys',
-    'XymonHost': '127.0.0.1',+    'XymonHost': '<your.xymon.server.address>',
     'XymonPort': 1984,     'XymonPort': 1984,
-    'XymonSrc' : 'rpi30',+    'XymonSrc' : '<mqtt.server.name>',
     'XymonTest': 'mqtt',     'XymonTest': 'mqtt',
     'Handler'  : None     'Handler'  : None
Line 177: Line 250:
     syslog.syslog ( ' '.join( (self.name,Msg) ) )     syslog.syslog ( ' '.join( (self.name,Msg) ) )
     syslog.closelog()     syslog.closelog()
 +
 +  def join( self, TimeOut=None ):
 +    self.stop()
 +    super().join( TimeOut )
  
   def stop( self ):   def stop( self ):
Line 289: Line 366:
  
  #  #
- # Method InformXymon takes a formatted message for the xymon server, prepends+ # Method InformXymon takes a formatted message for the Xymon server, prepends
  # the status message header and sends the resulting message to the Xymon  # the status message header and sends the resulting message to the Xymon
  # server. The method returns True if sending the message succeeded, otherwise  # server. The method returns True if sending the message succeeded, otherwise
Line 635: Line 712:
 </hidden> </hidden>
  
-Module watchdog.py+==== watchdog.py ==== 
 + 
 +Module watchdog.py  and script submqttm.py are to be installed in the same directory.
  
 <hidden onHidden="Show Code ⇲" onVisible="Hide Code ⇱"> <hidden onHidden="Show Code ⇲" onVisible="Hide Code ⇱">
  • monitors/submqttm.1604783172.txt.gz
  • Last modified: 2020/11/07 21:06
  • by wnelis