Wed Sep 30, 2020 12:12 am
Login Register Lost Password? Contact Us


Publishing queries with remote dali

Comments or questions specific to the features of ECL Watch

Mon Nov 09, 2015 1:57 pm Change Time Zone

I can't find any instructions in the documentation on how to do this. Can someone explain how Roxie queries that reference a remote dali work? I assumed I would write the ECL as normal:

ds := DATASET('~file_on_other_cluster', layout, flat)

and then put the other cluster's dali IP address in the "Remote Dali" field of the ECL Watch "Publish" pop-up. When I do this my query suspends because it can't resolve the file, though.

Credentials on both clusters are the same. Am I doing something else wrong?
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 2:35 pm Change Time Zone

Hi Alex,

I think the design flow is to build all of your data on the remote dali server, and then create your query on the other server where you are publishing. Remember, when you create a ROXIE query you only need to compile, and then when you publish the remote dali should correctly point to your indexes and files that you need.

I have a lesson in the Advanced ROXIE Online course that walks you through this process.

Regards,

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1002
Joined: Wed Jun 29, 2011 7:13 pm

Mon Nov 09, 2015 2:42 pm Change Time Zone

Yes, that is what I'm doing.

Cluster A has all our thors and contains the dataset I care about.

Cluster B has the roxies.

I compile the ECL on cluster B, publish it to the Roxies (either through ECL Watch or the command line; behavior is no different) specifying that the Remote Dali is the dali server IP for cluster A. The query immediately suspends because it can't resolve the file that lives on Cluster A.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 3:21 pm Change Time Zone

Hi Alex,

What are you putting in for the Remote Dali address? It should only be the base address of the cluster (no ports needed). Also, could it be a security issue (not having remote access or a firewall issue). If you run out of options I would suggest that you open a report in our issue tracker and get more community members to look at this.

Regards,

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1002
Joined: Wed Jun 29, 2011 7:13 pm

Mon Nov 09, 2015 3:24 pm Change Time Zone

Yes, it's just the IP with no port. And it's the Dali IP, not ESP or anything, right?

I didn't want to open a JIRA until I ruled out user error, but it's sounding more like a cluster configuration problem than something I'm doing wrong.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 3:29 pm Change Time Zone

Right on both.

Is the ROXIE cluster LDAP protected? If so, you may need to add the user and password on your command line publish.

But it sounds like you are doing things correctly (not user error).

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1002
Joined: Wed Jun 29, 2011 7:13 pm

Mon Nov 09, 2015 3:34 pm Change Time Zone

Yes, both clusters are using LDAP. My credentials are the same on both (they use the same LDAP server, even). Using the command line tool I do have to provide user name and password. It publishes the query but suspends it, so presumably authentication to the Roxie cluster is ok.

Thanks; I'll open a ticket.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 3:38 pm Change Time Zone

If you open up the ECL Watch, there is a section named Published Queries, and the specific error as to why it is suspended should be available from there. I wonder if it is possible that another query might have that file locked.

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1002
Joined: Wed Jun 29, 2011 7:13 pm

Mon Nov 09, 2015 3:53 pm Change Time Zone

Yes, the error is "Can't resolve file". I created this file just for this test, so I do not believe it's being used by any other queries.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 6:50 pm Change Time Zone

Alex,
Yes, the error is "Can't resolve file".
Then the most probable issue will be the way you named the logical file in the DATASET declaration for your query. Since the file is on a remote cluster you should probably use the "foreign file" naming syntax, discussed here: https://hpccsystems.com/download/documentation/ecl-language-reference/html/Foreign_Files.html

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1556
Joined: Wed Oct 26, 2011 7:40 pm

Next

Return to ECL Watch

Who is online

Users browsing this forum: No registered users and 1 guest

cron