Strange error in ECL IDE

[michael-mason]

Hello,

I'm going through the tutorial, and when I try to submit the sample code:

IMPORT TutorialMM;
COUNT(TutorialMM.File_OriginalPerson);

I get the following error in the ECL Error window at the bottom:

WARNING: ESP Exception - CSoapResponseBinding: 2012-06-29 10:56:45 GMT: SDS: Dirty client cache members used
SDS Reply Error : CFile::open /var/lib/HPCCSystems/hpcc-data/dali/delta.progress, Disk full (D:\hpccsystems\src\eclide\comms\Dali.cpp, 1281) SoapUtil.h(419)

We have HPCC running on a VM (single node) and I checked and we seem to have several gigabytes of free space.

Can anyone help? Thanks,

[jsmith]

That error certainly suggests that disk was full at the point of failure.
Perhaps space cleared by the time the space was inspected?

The error is originating from Dali creating a very small file (8 bytes).
Is Dali also on the same machine/vm ? Is /var/lib/HPCCSystems/hpcc-data/dali/ on the same partition as the rest?

Has the VM host machine/partition got enough space? I think it's possible to get a guest 'disk full' error if the VDI is dynamic and the host is short of space. i.e. at the point the VM tries to expand the virtual disk.

[michael-mason]

Hi,

It looks like the VM isn't allowing new files to be created on the system even though 'df' reports there is available space. I'm not a VM guy, but I'm guessing the host system isn't allowing the VM to expand its disk space usage or something and this is causing my issues.

When I try to do something as simple as 'touch test.txt', it gives me the 'No space left on device' error. I have an email out to our sysadmin/VM guy, so hopefully he can resolve the issue. Thank you very much for your help! HPCC is a really cool system and I'm really enjoying working with it.

[michael-mason]

Hello,

The problem turned out to be that our system that runs HPCC ran out of inodes. Even though we had disk space left (as reported by df), we didn't have any remaining inodes left as we reached our system limit.

The daemon process daliadmin created over 1,000,000 files in /var/log/daliadmin, adn I believe that was the culprit. I deleted all of the old log files, and everything is working great now.

Thanks,
-Mike

[jsmith]

Glad your found the cause.
It crossed my mind, whether # files per directory (another hard limit I think), or something similar.

daliadmin is a utility command, not a daemon, so I'm curious why there were 1000's of daliadmin log files still though...

[jsmith]

Thor runs daliadmin per startup, so it could well be that, perhaps other things do.
Daliadmin really shouldn't be creating a log per run ( see github issue https://github.com/hpcc-systems/HPCC-Platform/issues/2779 )