Setting up Ganglia on ubuntu Hadoop cluster:
Dependent on apache2 and php client and php module installation. Therefore install apache
sudo apt-get install apache2 php5 php5_cli libapache2-mod-php5
Master ganglia- chose a low utilization , low consequence node i.e. slave6
sudo apt-get install ganglia-monitor rrdtool gmetad ganglia-webfrontend
get monitor , rrdtools and web ui.
Modify webcontext file and copy to apache
sudo cp /etc/ganglia-webfrontend/apache.conf /etc/apache2/sites-enabled/ganglia.conf
Modify /etc/ganglia/gmond.conf to your specs (we are using unicast configuration). I commented out multicast attributes and replaced default ips with my ganglia master ip:
globals {
daemonize = yes
setuid = yes
user = ganglia
debug_level = 0
max_udp_msg_len = 1472
mute = no
deaf = no
host_dmax = 0 /*secs */
cleanup_threshold = 300 /*secs */
gexec = no
send_metadata_interval = 30
}
/* If a cluster attribute is specified, then all gmond hosts are wrapped inside
* of a tag. If you do not specify a cluster tag, then all will
* NOT be wrapped inside of a tag. */
cluster {
name = "Hadoop Ganglia Monitor"
owner = "hduser"
latlong = "unspecified"
url = "unspecified"
}
/* The host section describes attributes of the host, like the location */
host {
location = "unspecified"
}
/* Feel free to specify as many udp_send_channels as you like. Gmond
used to only support having a single channel */
udp_send_channel {
host = 10.77.201.104
port = 8649
}
/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
port = 8649
}
/* You can specify as many tcp_accept_channels as you like to share
an xml description of the state of the cluster */
tcp_accept_channel {
port = 8649
}
Modify /etc/ganglia/gmetd.conf to state the data collecting node for ganglia
data_source "Hadoop Cluster" 10.77.201.104Starting and stopping:
Restart gmetad
sudo service gmetad restart
Restart ganglia monitor in master node
sudo service ganglia-monitor restart
Ganglia Clusters
Install ganglia monitor on nodes
sudo apt-get install ganglia-monitor
Modify /etc/ganglia/gmond.conf to send data to receiver = datasource
/* If a cluster attribute is specified, then all gmond hosts are wrapped inside
* of a tag. If you do not specify a cluster tag, then all will
* NOT be wrapped inside of a tag. */
cluster {
name = "Hadoop Ganglia Monitor"
owner = "hduser"
latlong = "unspecified"
url = "unspecified"
}
/* The host section describes attributes of the host, like the location */
host {
location = "unspecified"
}
/* Feel free to specify as many udp_send_channels as you like. Gmond
used to only support having a single channel */
udp_send_channel {
host = 10.77.201.104
port = 8649
}
/* You can specify as many udp_recv_channels as you like as well. */
udp_recv_channel {
}
/* You can specify as many tcp_accept_channels as you like to share
an xml description of the state of the cluster */
tcp_accept_channel {
}
Starting and stopping:
sudo service ganglia-monitor start
Hadoop ganglia
Issues:
tried accessing ganglia url
got on html:
There was an error collecting ganglia data (127.0.0.1:8652): fsockopen error: Connection refused.
Problem is a file permissions issue.
Solution:
chown -R nobody:root /var/lib/ganglia/rrds
then restart daemons
MONITORING
send datagram
master node:
ps -ef | grep -v grep | grep gm
Results
ganglia 21116 1 0 12:58 ? 00:00:00 /usr/sbin/gmond --pid-file=/var/run/ganglia-monitor.pid
nobody 21127 1 0 12:58 ? 00:00:01 /usr/sbin/gmetad --pid-file=/var/run/gmetad.pid
network
sudo netstat -plane | egrep 'gmon|gme'
results:
tcp 0 0 0.0.0.0:8649 0.0.0.0:* LISTEN 999 87732663 21116/gmond
tcp 0 0 0.0.0.0:8651 0.0.0.0:* LISTEN 65534 87729072 21127/gmetad
tcp 0 0 0.0.0.0:8652 0.0.0.0:* LISTEN 65534 87729073 21127/gmetad
udp 0 0 0.0.0.0:8649 0.0.0.0:* 999 87732662 21116/gmond
udp 0 0 192.168.179.103:60243 192.168.179.103:8649 ESTABLISHED 999 87732666 21116/gmond
unix 3 [ ] STREAM CONNECTED 87785728 21116/gmond
unix 3 [ ] STREAM CONNECTED 83863311 21127/gmetad
No comments:
Post a Comment