Systemd – Easier management of your HPCC Systems components

Systemd is a system start-up and process manager for Linux. In recent years, it has become the boot manager of choice on newer Linux distributions. It is one of many new features you can try out in our HPCC Systems 7.0.0 Beta version which we are preparing for release soon.

While Systemd is partially compatible with the older SysV init scripts which the HPCC Systems platform uses, there are several reasons to start using the newer Systemd target and service scripts we are providing.

Clearly see all your system components and dependencies with one command

Our platform now generates a hpccsystems-platform.target file which includes the details of all components comprising your HPCC Systems environment.  A service list is produced which includes an entry for each component. To see the service list simply enter this command

 systemctl list-dependencies hpccsystems-platform.target

Here is an example of a service list:

Starting and stopping your HPCC Systems environment will be much easier and faster

Each component starts in parallel on its own thread, which means less startup time. DALI must always be started and running first, which is done automatically as part of the process. 

These are the commands to use to start/stop all components:

systemctl start hpccsystems-platform.target
systemctl stop hpccsystems-platform.target

​​​Since the hpccsystems-platform.target file knows exactly which components are included in your system, it can start all of them with a single command.

To allow the hpccsystems-platform.target to start after a machine boots up, simply use this command to enable/disable the target:

systemctl enable hpccsystems-platform.target systemctl disable hpccsystems-platform.target

​​​That doesn’t mean you can’t stop or (re)start individual components. If you wanted to stop or start ESP only, for example, you would make this request:

sudo systemctl stop esp@myesp.service
sudo systemctl start esp@myesp.service

It’s easier to investigate a failed component

This replaces the corresponding HPCC Systems init system log file. Using a simple command, you can see every field which shows problems such as coring. For example, to investigate why your ESP component failed, you would make this request:

journalctl -eu esp@myesp.service

Environment.xml file changes are automatically propagated to component service files

Every time you change your environment.xml, all the component files need to be regenerated. Using Systemd, any changes are propagated through and the service files are changed automatically. This is very useful and time saving particularly for large clusters.

In order to set up automatic regeneration of the service files, use these commands:

systemctl enable hpcc-environment-monitor.path
systemctl start hpcc-environment-monitor.path

This will allow the hpcc-environment-monitor.path script to run after booting the system. The script checks for modifications to the /etc/HPCCSystems/environment.xml file and launches the corresponding service file, which then regenerates the hpccsystems-platform.target and the necessary service files.

But remember that you must stop the system before modifying the “live” environment.xml file.

Resource limits will be respected

One of the changes in Systemd is how resource limits are checked for and set.  This is one way in which Systemd isn’t entirely compatible with the older SysV Init system.  

Users who try to use the older SysV scrips on newer systems that use Systemd, will often run into issues of resource limits which have been set in the scripts, but are not being appropriately respected by the host system.  Now with Systemd scripts explicitly written for the HPCCSystems Platform, you will not have to deal with this incompatibility issue.

At the moment, we still maintain the SysV init scripts for the HPCC-Platform and they will still be available in HPCC Systems 7.0.0, alongside the ability to use the new Systemd scripts. 

But we expect that somewhere between HPCC Systems 7.0.0 and our next major release, Systemd will become the standard approach to use. So why not make the switch and take advantage of the benefits now? And let us know what you think.

If you want to see a list of the main features you can expect to see in HPCC Systems 7.0.0, read our Fly on the Wall – Developer Offsite Meeting blog. More feature specific blogs are coming soon, so keep checking back for more details.