Server Monitoring with Zabbix

Server monitoring with ZabbixFor a long time, I have stuck on many hosted monitoring softwares to avoid management effort on
a self-hosted monitoring system. However, free stuffs cannot go long so I must move from services to services. This makes me tired! Thus I decided to return to a self-hosted solution so that I do not need to move from time to time. Currently the number of servers / hosts that I need to manage is gradually over 100, so I must come for a decision soon. After reviewing many current solutions, my favorite Zabbix does still stand on a top position of every recommended list. So the decision is quite simple: Go monitoring with Zabbix 🙂

Long story short: This entry is for installing and configuration Zabbix for my self-hosted monitoring purpose.

I. Zabbix Server & Zabbix Frontend for centralized monitoring purpose

I decided to go with Debian 8, a lightweight, enterprise-reader OS. And the most important reason: the server I choose to go is with Debian 8, and why do I need to waste time to reinstall it :lol ?

  • Install packages for Debian 8 (replace jessie – Debian 8 with stretch – Debian 9 or wheezy – Debian 7 ):
    wget http://repo.zabbix.com/zabbix/3.4/debian/pool/main/z/zabbix-release/zabbix-release_3.4-1+jessie_all.deb
    dpkg -i zabbix-release_3.4-1+jessie_all.deb
    apt-get update
    apt-get install -y sudo nmap
  • Install Zabbix Server and Zabbix Front-end (I use MySQL as the database):
    apt-get install -y zabbix-server-mysql zabbix-frontend-php
  • Add zabbix user to /etc/sudoers (you can use visudo to edit):
    zabbix ALL=(ALL) NOPASSWD: ALL
  • Create MySQL database for Zabbix services (replace your password in ):
    shell> mysql -uroot -p<password>
    mysql> create database zabbix character set utf8 collate utf8_bin;
    mysql> grant all privileges on zabbix.* to [email protected] identified by '<password>';
    mysql> quit;
  • Import initial Zabbix schema (you will need to input your password):
    zcat /usr/share/doc/zabbix-server-mysql/create.sql.gz | mysql -uzabbix -p zabbix
  • Update Zabbix configuration at /etc/zabbix/zabbix_server.conf:
    DBHost=localhost
    DBName=zabbix
    DBUser=zabbix
    DBPassword=<password>
  • Change the timezone configuration at /etc/apache2/conf-enabled/zabbix.conf
  • Start and enable Zabbix services:
    service zabbix-server start
    update-rc.d zabbix-server enable
    systemctl restart apache2
  • Login to web interface at http://IP/zabbix/ with default user Admin and password is zabbix

II. Install Zabbix agent on nodes to be monitored

  1. Install Zabbix Agent on agent nodes

    Again, I chose Debian (Ubuntu will be the same), so if you install on other OS, remember to replace according commands (e.g. replace apt-get with yum for CentOS / Fedora)

    • To install Zabbix agent, run
      wget http://repo.zabbix.com/zabbix/3.4/debian/pool/main/z/zabbix-release/zabbix-release_3.4-1+jessie_all.deb
      dpkg -i zabbix-release_3.4-1+jessie_all.deb
      apt-get update
      apt-get install -y zabbix-agent
    • We will use Pre-Shared Keys (PSK) to secure the connection between the server and agent. So let’s generate a key (we will need the key to configure the host later):
      openssl rand -hex 32 > /etc/zabbix/zabbix_agentd.psk
    • We need to edit the Zabbix agent settings (/etc/zabbix/zabbix_agentd.conf) to set up its secure connection to the Zabbix server. We will need to edit the following configuration items (be sure to note TLSPSKIdentity and the value of TLSPSKFile to use later on Zabbix Web Interface):
      Server=ZABBIX_SERVER_IP_ADDRESS
      Hostname=nocix64g2670v2-01
      TLSConnect=psk
      TLSAccept=psk
      TLSPSKIdentity=PSK-nocix64g2670v2-01
      TLSPSKFile=/etc/zabbix/zabbix_agentd.psk
    • Start and enable Zabbix agent, run
      systemctl start zabbix-agent
      systemctl enable zabbix-agent
  2. Add the new host to the Zabbix server

    Installing Zabbix agent is just a half way to get things done. We will also need to add the according host to our Zabbix server.

    • Access to your Zabbix web interface
    • Locate to Configuration >> Host >> Create host, then
      • Input the Host name and IP ADDRESS params to reflect your client machine. You can also create Group to group many servers there
      • Click the Templates sub-menu, Type “Template OS Linux” and select it in “Link new templates“. Remember to click on the Add button to add it to the template list.
      • Next, navigate to the Encryption tab, select PSK for “Connections to host” and “Connections from host“, then
        • set PSK identity to the value that we set in TLSPSKIdentity in the previous step (PSK-nocix64g2670v2-01 in the above case)
        • set PSK value to the key you generated for the Zabbix agent in the previous set(at /etc/zabbix/zabbix_agentd.psk)
    • Finally, click Add button and wait for several minutes 🙂

III. Secure Zabbix server with Let’s Encrypt

  • Install packages:
    echo 'deb http://ftp.debian.org/debian jessie-backports main' | tee /etc/apt/sources.list.d/backports.list
    apt-get update
    apt-get install -y python-certbot-apache -t jessie-backports
  • Set Up the Apache ServerName and ServerAlias: edit /etc/apache2/sites-available/000-default.conf and put ServerName and ServerAlias information. For example:
        ServerName tienle.com
        ServerAlias www.tienle.com
  • Restart apache:
    service apache2 restart
  • Setup the SSL certification process:
    certbot --apache
  • After finishing, remember to setup crontab to auto-renew the certificate for each month / week with the certbot renew command line.

IV. To be considered: Zabbix Proxy

Zabbix Proxy can help off-load Zabbix servers in case we monitor thousands of nodes, so I do not really need it this case. In case you decide to go with Zabbix Proxy, one note worth remembering is to use a separated database from your Zabbix server if you install them on the same server.

V. Zabbix Notes

  • Remember to associate a user to a Media type (e.g. SMS notification, Email notification, etc.
    which can be configured at Administration >> Media Types menu) so that alert notifications can be sent
  • You can change anything of a template (triggers, applications, etc.) via the menu Configuration >> Templates >> select the according template
  • Lazy setup on Guest machines with ZonePing? Run
    wget --no-check-certificate -N https://www.tienle.com/tools/tool-script-install-zabbix-agent-zoneping.sh && bash tool-script-install-zabbix-agent-zoneping.sh
    

Leave a Reply