Akshay Narang

In this article we will see how to join multiple datasets in Splunk using Lookup.  Consider below two datasets: Vendor Sales Data : Provides the detail of which vendors have sold what products. Prices Data : Master data of the products like product name, price, sale price and code for each product. Code is the common field between 2 data sets Requirement is  to display the Products sold by the vendors along with the vendor details and product details To get this information we would need to join the two datasets on a common field which is "Code" Splunk provides lookup command in order to do that. I already have uploaded the vendor sales data to the index "index2". You can refer to my blog  "Upload a dataset to Splunk" for the steps to upload the data. Next step is to upload the prices data, which will be used as a lookup table to get the information on additional fields. Goto Settings -> Lookups -> Lookup table files -> New Lookup Table File Leave...

Extract New Fields in Splunk The process by which Splunk extracts fields  from event data  and the results of that process, are referred to as extracted fields.  Splunk extracts a set of default fields  for each event it indexes. You can also create custom fields by defining additional index-time and search-time field extractions, using search commands, the field extractor , or configuration files. In this article we will see how to extract custom fields from the event data: In my blog " Configure a universal forwarder to monitor a log file on Linux ", we have seen how to configure universal forwarder to monitor the linux server /var/log/messages file. We will use the event data from the same file to extract the message field. As you can see in the left pane of below screenshot that default fields extracted by Splunk does not contain the message field. Steps to Extract New Field: Click on the "Extract New Fields" link on the bottom left corner of the event data sear...

Configure a universal forwarder to monitor a log file on Linux In this article we will see how to configure Splunk forwarder to monitor a log file and ingest the log file events to Splunk. In this example we will perform the steps to monitor a file on the same server where Splunk enterprise is installed. Login to the Linux server using PuTTY on which Splunk is installed. If not already set, then set SPLUNK_HOME variable to the directory where Splunk is installed. export SPLUNK_HOME=/opt/splunk Go to the directory $SPLUNK_HOME/etc/system/local cd $SPLUNK_HOME/etc/system/local Modify the file inputs.conf and append the below lines at the end of the file.                [monitor:///var/log/messages_copy]                index=main                sourcetype=server_log Value of monitor  tells the file(s) that you want Splunk to monitor for the events...

Installing Splunk on AWS EC2 - Red Hat Linux In this article we will see how to install Splunk on Red Hat Linux using Amazon AWS EC2 instance. We would first need to create a EC2 instance on Amazon AWS. Steps to Create EC2 Instance Login to your AWS console.  Under Services -> Click on EC2 Click on Launch Instance Choose an Amazon Machine Image (AMI). In my case case, I am using "Red Hat Enterprise Linux 8 (HVM)" that is available as free tier. Choose an Instance Type - General purpose - t2.micro, as this is eligible as free tier. Click on Review and Launch In the next step, it will show your EC2 instance configuration. Click on "Launch". Another window will open saying to "Select an existing key pair or create a new key pair". You will need to create a new key pair or choose an existing if you already have one. Key-pair file is a .pem file which is used to connect to your AWS EC2 instance using password less authentication. Download key pair file and C...

Searching, Analysing and Visualizing in Splunk In this article we will see how to Analyse and Visualize a dataset using Splunk "Search & Reporting App" We have already uploaded a dataset (.csv file) that contains information on confirmed cases of Covid-19 across different countries. Refer to my blog "Adding dataset to Splunk" to learn about uploading a CSV dataset file to Splunk. The dataset is free to download from https://data.humdata.org Once you have finished loading your dataset, Click on Search & Reporting.  Check that your dataset is uploaded successfully. In my case, I have uploaded the dataset in an index with the name "index2" and the name of my csv file is "time_series_covid19_confirmed_global.csv".  As my index "index2" contains the information of other dataset as well, so I will put another filter to include the source name. Based on when the dataset was uploaded, you can adjust the drop down which I have left to de...

How to upload a Dataset to Splunk In this blog I will show how to upload a dataset to Splunk. The data once uploaded can be later analysed using Splunk Search Processing Language (SPL) and meaningful reports/dashboards can be generated.  Steps to Upload a Dataset 1. Login to Splunk Console 2. Under Settings -> Click  Add Data 3. You can use different type of datasets. In our case we are using a comma separated values file (.csv)  4. Click on Upload from My Computer 5. Click on "Select File" to select the csv file to upload. 6. Once selected it will upload the csv file to Splunk. The time it takes to upload is based upon the size of the file. 7. Set the Source Type of the File. In our case it is csv file, so I will leave it to the default. 8. Next Step is to select an index to store your data. An Index in Splunk is used as the repository of your data. You can create a new index or leave it to default. It is better to create separate indexes based on amount and type of d...