How to Monitor Server Logs in Real-Time with "Log.io"

Monitor Server Logs in Real-Time with "Log.io"

Log.io is a small application build on Node.js and Socket.io for monitoring real time Linux server logs through web interface.

On RHEL/CentOS 7

Install Epel repositories using the following command.

yum install http://fedora.mirrors.telekom.ro/pub/epel/7/x86_64/e/epel-release-7-2.noarch.rpm

On RHEL/CentOS 6

--------------------- On RHEL/CentOS 6.x - 32 Bit ---------------------
yum install http://fedora.mirrors.telekom.ro/pub/epel/6/i386/epel-release-6-8.noarch.rpm

--------------------- On RHEL/CentOS 6.x - 64 Bit ---------------------
yum install http://fedora.mirrors.telekom.ro/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

After added the Epel repositories you can do a system upgrade by using following command line.

yum update

Now Install Node.js and NPM

Node.js is an open-source, cross-platform JavaScript run-time environment that executes JavaScript code server-side. Historically, JavaScript was used primarily for client-side scripting, in which scripts written in JavaScript are embedded in a webpage's HTML and run client-side by a JavaScript engine in the user's web browser. Node.js lets developers use JavaScript for server-side scripting—running scripts server-side to produce dynamic web page content before the page is sent to the user's web browser. Consequently, Node.js represents a "JavaScript everywhere" paradigm, unifying web application development around a single programming language, rather than different languages for server side and client side scripts.now install the nodejs using following command line.

curl --silent --location https://rpm.nodesource.com/setup_5.x | bash - 
yum install -y nodejs

Install and Configure Log.io Application
Log.io application must be installed through NPM by specifying a valid local system user ( example - root) i recommend install this application through root user. now install the application using following command.

npm install -g log.io --user "root"

after the completed installation the application will create a folder which is hidden the folder name is "Log.io" in centos 7 the folder path is below

cd /root/.log.io
ls

after that you will see three file

harvester.conf, log_server.conf, web_server.conf  

Now it's time to configure Log.io to monitor local log files in real time.

- The harvester file is for watches the changes in specified local log files declared in its configuration and send new log to the server. - harvester.conf

First open the harvester.conf file, by default you will see have some setting the monitor Apache log 

nano harvester.conf

my harvester.conf file below

exports.config = {
    nodeName: "application_server",
    logStreams: {
      apache: [
        "/var/log/httpd/access_log",
        "/var/log/httpd/error_log"
      ],
      SecureLog: [
        "/var/log/secure"
        ],
      MySqlLog: [
        "/var/log/mysqld.log"
        ]
    },
    server: {
      host: '0.0.0.0',
      port: 28777
    }
  }

Note - if you don't want to send the harvester output to a remote Log.io server then open harvester.conf file and replace 0.0.0.0 address with lookback address(127.0.0.1)

log_server.conf - This configuration is for tell the server on which ip address it should listen.By default it listens on all ip interfaces you can modify the ip address by using the following command 

my log_server.conf file below

nano log_server.conf

exports.config = {
  host: '0.0.0.0',
  port: 28777
}

web_server.conf - This configuration file is for web interface. by default the web portal is accessible on port 28778(all interface). Also using this file you can increase the security by using HTTP authentication,securing the web interface with SSL, disallow logs from specific ip address and restricting the web interface access to the specific ip. 

my web_server.conf file below

nano web_server.conf

 exports.config = {
  host: '0.0.0.0',
  port: 28778,

  /*
  // Enable HTTP Basic Authentication
  auth: {
    user: "hackthesec",
    pass: "1234346"
  },
  */

  /*
  // Enable HTTPS/SSL
  ssl: {
    key: '/path/to/privatekey.pem',
    cert: '/path/to/certificate.pem'
  },
  */

  /*
  // Restrict access to websocket (socket.io)
  // Uses socket.io 'origins' syntax
  restrictSocket: '*:*',
  */

  /*
  // Restrict access to http server (express)
  restrictHTTP: [
    "192.168.29.39",
    "10.0.*"
  ]
  */

}

Allow web interface and socket.io in the firewall for receiving the logs from the harvester.

firewall-cmd --add-port=28778/tcp --permanent
firewall-cmd --add-port=28777/tcp --permanent
firewall-cmd --reload

Start Log.io Application using following command line 

------ First start server in background ----
log.io-server &

--- Start log harvester in background -----
log.io-harvester & 

After the server has been started open your browser and hit enter with your ip and the declared  port

example - http://192.168.2.253:28778

To stop Log.io application run the following command.

pkill node

Log.io = https://github.com/NarrativeScience/Log.io
http://logio.org/