Difference between revisions of "Splunk"
| Line 1: | Line 1: | ||
| − | + | Splunk is a flexible data aggregation system. In laymens' words, Splunk is a system that combs through log files (and anything else that contains structured information that you want to throw at it) and presents the results in a summarized format. It is really a pretty neat thing. See the [http://www.splunk.com splunk website]. | |
| − | |||
| − | Splunk is a flexible data aggregation system | ||
| − | |||
== Splunk at UNH == | == Splunk at UNH == | ||
| − | |||
We are running the free 3.0beta3 on our system [[Jalapeno]]. Splunk is resource hungry. It requires at least 600MB of memory and quite a bit of CPU. Although it is possible to run a splunkd server deamon on each node and have these pass the information to the master node, this is '''not''' how I chose to set it up. Our splunbk setup is as follows: | We are running the free 3.0beta3 on our system [[Jalapeno]]. Splunk is resource hungry. It requires at least 600MB of memory and quite a bit of CPU. Although it is possible to run a splunkd server deamon on each node and have these pass the information to the master node, this is '''not''' how I chose to set it up. Our splunbk setup is as follows: | ||
* Splunk runs on [[Jalapeno]]. It is installed in /data/splunk, with a link to /opt/splunk. | * Splunk runs on [[Jalapeno]]. It is installed in /data/splunk, with a link to /opt/splunk. | ||
| Line 10: | Line 6: | ||
* The free version of splunk does not allow for login. We should restrict access to jalapeno to sysadmins. | * The free version of splunk does not allow for login. We should restrict access to jalapeno to sysadmins. | ||
* This can be extended to do many different tasks! | * This can be extended to do many different tasks! | ||
| − | + | == Connecting to Splunk == | |
| − | + | jalapeno isn't an HTTP server, so it's not possible to access the interface to Splunk by simply opening a web browser and going to the appropriate port. There is a fairly simple workaround, though: | |
| + | # <code>ssh -L8001:localhost:8000 jalapeno</code>. It doesn't necessarily have to be 8001, but some available port on your machine. | ||
| + | # Open a web browser with good Javascript support (and optionally Flash, for some fancy graphing features), and go to ''localhost:8001'' (or whatever port you chose) | ||
Revision as of 17:33, 6 July 2007
Splunk is a flexible data aggregation system. In laymens' words, Splunk is a system that combs through log files (and anything else that contains structured information that you want to throw at it) and presents the results in a summarized format. It is really a pretty neat thing. See the splunk website.
Splunk at UNH
We are running the free 3.0beta3 on our system Jalapeno. Splunk is resource hungry. It requires at least 600MB of memory and quite a bit of CPU. Although it is possible to run a splunkd server deamon on each node and have these pass the information to the master node, this is not how I chose to set it up. Our splunbk setup is as follows:
- Splunk runs on Jalapeno. It is installed in /data/splunk, with a link to /opt/splunk.
- Jalapeno mounts the /var/log directories from einstein and roentgen so that it can be accessed by splunk for aggregation.
- The free version of splunk does not allow for login. We should restrict access to jalapeno to sysadmins.
- This can be extended to do many different tasks!
Connecting to Splunk
jalapeno isn't an HTTP server, so it's not possible to access the interface to Splunk by simply opening a web browser and going to the appropriate port. There is a fairly simple workaround, though:
ssh -L8001:localhost:8000 jalapeno. It doesn't necessarily have to be 8001, but some available port on your machine.- Open a web browser with good Javascript support (and optionally Flash, for some fancy graphing features), and go to localhost:8001 (or whatever port you chose)
