Dfuplus fails to spray files
I have been trying to spray data using dfuplus command.
Here are the details of my HPCC setup.
Master IP Address :172.31.45.14
Slave 1 IP Address : 172.31.33.152
Slave 2 IP Address : 172.31.42.187
On my master node, I have 30 GB of kmeans related dataset in csv format spread across 30 files.
The directory is : /mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/
Using the dfuplus command, I am trying to spray the same data across my slave nodes.
Here is the command I am running for an individual file.
$ sudo dfuplus action=spray srcip=172.31.45.14 srcfile=/mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1 dstname=kmeans::dataset::file1 dstcluster=mythor server=http://172.31.45.14:8010 format=csv
Checking for local Dali File Server on port 7100
Variable spraying from /mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1 on 172.31.45.14:7100 to kmeans::dataset::file1
Submitted WUID D20180423-152513
D20180423-152513 status: queued
Failed: No Drop Zone on '172.31.45.14' configured at '/mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1'.
The job fails stating that
No Drop Zone on '172.31.45.14' configured at '/mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1'.
My ECL Watch console shows a single mydropzone entry of 172.31.33.152.
Here are the details of my HPCC setup.
Master IP Address :172.31.45.14
Slave 1 IP Address : 172.31.33.152
Slave 2 IP Address : 172.31.42.187
On my master node, I have 30 GB of kmeans related dataset in csv format spread across 30 files.
The directory is : /mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/
Using the dfuplus command, I am trying to spray the same data across my slave nodes.
Here is the command I am running for an individual file.
$ sudo dfuplus action=spray srcip=172.31.45.14 srcfile=/mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1 dstname=kmeans::dataset::file1 dstcluster=mythor server=http://172.31.45.14:8010 format=csv
Checking for local Dali File Server on port 7100
Variable spraying from /mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1 on 172.31.45.14:7100 to kmeans::dataset::file1
Submitted WUID D20180423-152513
D20180423-152513 status: queued
Failed: No Drop Zone on '172.31.45.14' configured at '/mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1'.
The job fails stating that
No Drop Zone on '172.31.45.14' configured at '/mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/file1'.
My ECL Watch console shows a single mydropzone entry of 172.31.33.152.
- rsghatpa
- Posts: 13
- Joined: Wed Mar 28, 2018 5:13 pm
rsghatpa,
You can only spray from locations that are "registered drop zones" in your environment.
You can either move the file(s) to your drop zone or reconfigure your environment so that
mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/ on 172.31.45.14 is a registered drop zone.
HTH,
Jim
You can only spray from locations that are "registered drop zones" in your environment.
You can either move the file(s) to your drop zone or reconfigure your environment so that
mnt/var/lib/HPCCSystems/dataset/kmeans_30GB/ on 172.31.45.14 is a registered drop zone.
HTH,
Jim
- JimD
- Posts: 160
- Joined: Wed May 18, 2011 1:35 pm
First, take the port number off the server. Second, use the IP addresses found on your ecl watch under the Operations icon then click on System Servers.
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
From a linux (actually it was my master instance) I placed a file in my home directory, /home/ubuntu, and was able to spray it with the following:
- Code: Select all
sudo dfuplus action=spray srcip=10.106.73.212 \
srcfile=/home/ubuntu/myfile_head4.csv \
dstname=tlh::myfile_head4 dstcluster=mythor \
server=http://10.106.73.212 format=csv
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
rsghatpa,
Let us know if you got your spray to work and how, please.
Tim
Let us know if you got your spray to work and how, please.
Tim
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
Hi Tim, it worked. I changed my dropzone location. Added my files to it. And using ECL Watch did the necessary spray jobs.
- rsghatpa
- Posts: 13
- Joined: Wed Mar 28, 2018 5:13 pm
rsghatpa,
Glad to hear it worked. Did you basically do the dfuplus command as I show it above?
Tim
Glad to hear it worked. Did you basically do the dfuplus command as I show it above?
Tim
- tlhumphrey2
- Posts: 260
- Joined: Mon May 07, 2012 6:23 pm
7 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest