Publishing queries with remote dali
I can't find any instructions in the documentation on how to do this. Can someone explain how Roxie queries that reference a remote dali work? I assumed I would write the ECL as normal:
ds := DATASET('~file_on_other_cluster', layout, flat)
and then put the other cluster's dali IP address in the "Remote Dali" field of the ECL Watch "Publish" pop-up. When I do this my query suspends because it can't resolve the file, though.
Credentials on both clusters are the same. Am I doing something else wrong?
ds := DATASET('~file_on_other_cluster', layout, flat)
and then put the other cluster's dali IP address in the "Remote Dali" field of the ECL Watch "Publish" pop-up. When I do this my query suspends because it can't resolve the file, though.
Credentials on both clusters are the same. Am I doing something else wrong?
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Hi Alex,
I think the design flow is to build all of your data on the remote dali server, and then create your query on the other server where you are publishing. Remember, when you create a ROXIE query you only need to compile, and then when you publish the remote dali should correctly point to your indexes and files that you need.
I have a lesson in the Advanced ROXIE Online course that walks you through this process.
Regards,
Bob
I think the design flow is to build all of your data on the remote dali server, and then create your query on the other server where you are publishing. Remember, when you create a ROXIE query you only need to compile, and then when you publish the remote dali should correctly point to your indexes and files that you need.
I have a lesson in the Advanced ROXIE Online course that walks you through this process.
Regards,
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Yes, that is what I'm doing.
Cluster A has all our thors and contains the dataset I care about.
Cluster B has the roxies.
I compile the ECL on cluster B, publish it to the Roxies (either through ECL Watch or the command line; behavior is no different) specifying that the Remote Dali is the dali server IP for cluster A. The query immediately suspends because it can't resolve the file that lives on Cluster A.
Cluster A has all our thors and contains the dataset I care about.
Cluster B has the roxies.
I compile the ECL on cluster B, publish it to the Roxies (either through ECL Watch or the command line; behavior is no different) specifying that the Remote Dali is the dali server IP for cluster A. The query immediately suspends because it can't resolve the file that lives on Cluster A.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Hi Alex,
What are you putting in for the Remote Dali address? It should only be the base address of the cluster (no ports needed). Also, could it be a security issue (not having remote access or a firewall issue). If you run out of options I would suggest that you open a report in our issue tracker and get more community members to look at this.
Regards,
Bob
What are you putting in for the Remote Dali address? It should only be the base address of the cluster (no ports needed). Also, could it be a security issue (not having remote access or a firewall issue). If you run out of options I would suggest that you open a report in our issue tracker and get more community members to look at this.
Regards,
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Yes, it's just the IP with no port. And it's the Dali IP, not ESP or anything, right?
I didn't want to open a JIRA until I ruled out user error, but it's sounding more like a cluster configuration problem than something I'm doing wrong.
I didn't want to open a JIRA until I ruled out user error, but it's sounding more like a cluster configuration problem than something I'm doing wrong.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Right on both.
Is the ROXIE cluster LDAP protected? If so, you may need to add the user and password on your command line publish.
But it sounds like you are doing things correctly (not user error).
Bob
Is the ROXIE cluster LDAP protected? If so, you may need to add the user and password on your command line publish.
But it sounds like you are doing things correctly (not user error).
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Yes, both clusters are using LDAP. My credentials are the same on both (they use the same LDAP server, even). Using the command line tool I do have to provide user name and password. It publishes the query but suspends it, so presumably authentication to the Roxie cluster is ok.
Thanks; I'll open a ticket.
Thanks; I'll open a ticket.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
If you open up the ECL Watch, there is a section named Published Queries, and the specific error as to why it is suspended should be available from there. I wonder if it is possible that another query might have that file locked.
Bob
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Yes, the error is "Can't resolve file". I created this file just for this test, so I do not believe it's being used by any other queries.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Alex,
HTH,
Richard
Then the most probable issue will be the way you named the logical file in the DATASET declaration for your query. Since the file is on a remote cluster you should probably use the "foreign file" naming syntax, discussed here: https://hpccsystems.com/download/documentation/ecl-language-reference/html/Foreign_Files.htmlYes, the error is "Can't resolve file".
HTH,
Richard
- rtaylor
- Community Advisory Board Member
- Posts: 1619
- Joined: Wed Oct 26, 2011 7:40 pm
24 posts
• Page 1 of 3 • 1, 2, 3
Who is online
Users browsing this forum: No registered users and 1 guest