HPCC Components - Service Types & Processes

[amillar]

Good Afternoon,

We have been using a monitoring service named pulse way on windows machines for a few years now and are very impressed with it.

This week I have tried implementing the monitoring agent on one of our Ubuntu 12.04 LTS boxes which are running hpccsystems-platform_community-5.0.2-1trusty_amd64

I have pretty much configured it so it will tell me if drive space is low, CPU usage is high, free memory is low, whether the machine is off or on etc which is great

but I am having problems getting it to tell me of these HPCC components are running :

mydafilesrv
mydali
mydfuserver
myeclagent
myeclccserver
myeclscheduler
myesp
myroxie
mysasha
mythor

I have been trying to establish initially if these are actually services rather than processes, can anyone give me anymore information on this?

It seems Ubuntu has three service types :

SYSVINIT, UPSTART or SYSTEMD

if they are services, does anyone know which type?

The agent is configured via XML, and to give you an example I can get the service SSH monitored like this :

<Service Name="ssh" DisplayName="SSH Daemon" IsDaemon="true" DaemonType="UPSTART" Path="" StartParameters="" CanBeStopped="true" Enabled="true" />

and the NTP process using this :

<Service Name="ntpd" DisplayName="NTPD Process" IsDaemon="false" DaemonType="NONE" Path="/usr/sbin/ntpd" StartParameters="-p /var/run/ntp/ntpd.pid -g -u ntp:ntp -i /var/lib/ntp -c /etc/ntp.conf" CanBeStopped="true" Enabled="false" />

but any HPCC services, show up on the app as stopped, even when they are started so its probably a configuration error on my part.

Here are some service variations I have tried without success :

<Service Name="myesp" DisplayName="myesp" IsDaemon="true" DaemonType="UPSTART" Path="" StartParameters="" CanBeStopped="true" Enabled="true" />
<Service Name="myroxie" DisplayName="myroxie" IsDaemon="true" DaemonType="NONE" Path="" StartParameters="" CanBeStopped="true" Enabled="true" />
<Service Name="mysasha" DisplayName="mysasha" IsDaemon="true" DaemonType="SYSVINIT" Path="" StartParameters="" CanBeStopped="true" Enabled="true" />
<Service Name="mythor" DisplayName="mythor" IsDaemon="true" DaemonType="SYSTEMD" Path="" StartParameters="" CanBeStopped="true" Enabled="true" />

I have done the same with processes e.g.

<Service Name="mythor" DisplayName="mythor" IsDaemon="false" DaemonType="NONE" Path="" StartParameters="" CanBeStopped="true" Enabled="true" />

The only catch here is I am not passing any start-up parameters like in the NTP example I found.

If these HPCC components are processes, does anyone know what parameters I need to pass?

I have had a good look around the internet but cannot find much information, and my Ubuntu knowledge is limited

Any help would be greatly appreciated.

Thanks in advance

[ming]

HPCCSystem Platform uses SYSVINT for service management.
The registered service are hpcc-init and dafilesrv. Majority components are managed through service hpcc-init or /opt/HPCCSystems/etc/init.d/hpcc-init hpcc-init doesn't stay as daemon on the system, instead it start a group of init-<component> daemon (defined under /opt/HPCCSystems/bin). These init-<component> process start/monitor/restart each component.

[Gleb Aronsky]

You may want to examine our offerings for monitoring and alerting with Ganglia and Nagios.

Some of the monitoring utilities released for Nagios maybe useful for Pulse monitoring. The utilities provided do some basic health checks on HPCC components. Other tools provided in the package generate configuration files from the HPCC environment.xml configuration file directly for Nagios.

The utility configgen, released as part of the platform can also list ports and configured nodes. Just check the command line help.

You can find monitoring/alerting docs here: https://hpccsystems.com/download/docume ... -technical

And the source code for here:
https://github.com/hpcc-systems/nagios-monitoring
https://github.com/hpcc-systems/ganglia-monitoring

[amillar]

Hi Ming and Gleb,

Thanks for getting back to me, I now understand the process a little bit better, and can confirm I have HPCC-INIT and DAFILESERV are being monitoring now via pulse way which great.

I do want to implement some more sophisticated monitoring in the future and have been looking at Ganglia and Nagios.

Now I have at least something in place it’s going to give me some breathing space to look into these solutions properly.

Thanks again

Antony