Skip to main content

Configure a universal forwarder to monitor a log file on Linux

Configure a universal forwarder to monitor a log file on Linux


In this article we will see how to configure Splunk forwarder to monitor a log file and ingest the log file events to Splunk. In this example we will perform the steps to monitor a file on the same server where Splunk enterprise is installed.

  • Login to the Linux server using PuTTY on which Splunk is installed.
  • If not already set, then set SPLUNK_HOME variable to the directory where Splunk is installed.
    • export SPLUNK_HOME=/opt/splunk
  • Go to the directory $SPLUNK_HOME/etc/system/local
    • cd $SPLUNK_HOME/etc/system/local
  • Modify the file inputs.conf and append the below lines at the end of the file.
            [monitor:///var/log/messages_copy]
            index=main
            sourcetype=server_log


    • Value of monitor tells the file(s) that you want Splunk to monitor for the events. You can specify a single or multiple files using wild cards. In this example I am using a copy of system file /var/log/messages. I will later add some events in this finally and we will see how Splunk reads these events.
    • Value of index parameter tells the name of the index in which the information of the events will be stored.
    • Sourcetype is a classification of data input, i.e, type of data input being provided.
  • Save the file inputs.conf
  • Restart Splunk
    • #cd $SPLUNK_HOME/bin
    • #./splunk stop
    • #./splunk start

Verify that Splunk is Reading events from the file and the events are visible on Splunk console using below query:

  • First I did a test for searching the events, but no events were found as the file /var/log/messages_copy was not found. See the screenshot below:


  • In the next step, I will create the file /var/log/messages_copy

  • Now check the same query to again and you should be able to see the events for the log file /var/log/messages_copy. 


Comments

Popular posts from this blog

Configure Oracle ASM Disks on AIX

Configure Oracle ASM Disks on AIX You can use below steps to configure the new disks for ASM after the raw disks are added to your AIX server by your System/Infrastructure team experts: # /usr/sbin/lsdev -Cc disk The output from this command is similar to the following: hdisk9 Available 02-T1-01 PURE MPIO Drive (Fibre) hdisk10 Available 02-T1-01 PURE MPIO Drive (Fibre) If the new disks are not listed as available, then use the below command to configure the new disks. # /usr/sbin/cfgmgr Enter the following command to identify the device names for the physical disks that you want to use: # /usr/sbin/lspv | grep -i none This command displays information similar to the following for each disk that is not configured in a volume group: hdisk9     0000014652369872   None In the above example hdisk9 is the device name and  0000014652369872  is the physical volume ID (PVID). The disks that you want to use may have a PVID, but they must not belong to a volu...

Gitlab installation steps on Redhat Linux

In this blog we will see the steps to install Gitlab on Redhat Enterprise Linux 6. I will be using the virtual machine "gitserver" that I have created on Google Cloud. You can use any server or VM running RHEL 6 and follow these steps. Follow the below steps to install gitlab. Run these steps as root user. # yum install -y curl policycoreutils-python openssh-server cronie # lokkit -s http -s ssh  # yum install postfix  # service postfix start  # chkconfig postfix on  # curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.rpm.sh | sudo bash  # EXTERNAL_URL="http://34.69.44.142" yum -y install gitlab-ee  You will see a screen similar to below, once your gitlab installation is successful. You can now access the gitlab console using the http or https url that you provided during the installation, i.e., http://<ip/server_name> http://gitserver.localdomain.com or  http://34.69.44.142 When you open the c...

Load records from csv file in S3 file to RDS MySQL database using AWS Data Pipeline

 In this post we will see how to create a data pipeline in AWS which picks data from S3 csv file and inserts records in RDS MySQL table.  I am using below csv file which contains a list of passengers. CSV Data stored in the file Passenger.csv Upload Passenger.csv file to S3 bucket using AWS ClI In below screenshot I am connecting the RDS MySQL instance I have created in AWS and the definition of the table that I have created in the database testdb. Once we have uploaded the csv file we will create the data pipeline. There are 2 ways to create the pipeline.  Using "Import Definition" option under AWS console.                    We can use import definition option while creating the new pipeline. This would need a json file which contains the definition of the pipeline in the json format. You can use my Github link below to download the JSON definition: JSON Definition to create the Data Pipeline Using "Edit Architect" ...