Publishing queries with remote dali

[alex]

I can't find any instructions in the documentation on how to do this. Can someone explain how Roxie queries that reference a remote dali work? I assumed I would write the ECL as normal:

ds := DATASET('~file_on_other_cluster', layout, flat)

and then put the other cluster's dali IP address in the "Remote Dali" field of the ECL Watch "Publish" pop-up. When I do this my query suspends because it can't resolve the file, though.

Credentials on both clusters are the same. Am I doing something else wrong?

[bforeman]

Hi Alex,

I think the design flow is to build all of your data on the remote dali server, and then create your query on the other server where you are publishing. Remember, when you create a ROXIE query you only need to compile, and then when you publish the remote dali should correctly point to your indexes and files that you need.

I have a lesson in the Advanced ROXIE Online course that walks you through this process.

Regards,

Bob

[alex]

Yes, that is what I'm doing.

Cluster A has all our thors and contains the dataset I care about.

Cluster B has the roxies.

I compile the ECL on cluster B, publish it to the Roxies (either through ECL Watch or the command line; behavior is no different) specifying that the Remote Dali is the dali server IP for cluster A. The query immediately suspends because it can't resolve the file that lives on Cluster A.

[bforeman]

Hi Alex,

What are you putting in for the Remote Dali address? It should only be the base address of the cluster (no ports needed). Also, could it be a security issue (not having remote access or a firewall issue). If you run out of options I would suggest that you open a report in our issue tracker and get more community members to look at this.

Regards,

Bob

[alex]

Yes, it's just the IP with no port. And it's the Dali IP, not ESP or anything, right?

I didn't want to open a JIRA until I ruled out user error, but it's sounding more like a cluster configuration problem than something I'm doing wrong.

[bforeman]

Right on both.

Is the ROXIE cluster LDAP protected? If so, you may need to add the user and password on your command line publish.

But it sounds like you are doing things correctly (not user error).

Bob

[alex]

Yes, both clusters are using LDAP. My credentials are the same on both (they use the same LDAP server, even). Using the command line tool I do have to provide user name and password. It publishes the query but suspends it, so presumably authentication to the Roxie cluster is ok.

Thanks; I'll open a ticket.

[bforeman]

If you open up the ECL Watch, there is a section named Published Queries, and the specific error as to why it is suspended should be available from there. I wonder if it is possible that another query might have that file locked.

Bob

[alex]

Yes, the error is "Can't resolve file". I created this file just for this test, so I do not believe it's being used by any other queries.

[rtaylor]

Alex,

Yes, the error is "Can't resolve file".

Then the most probable issue will be the way you named the logical file in the DATASET declaration for your query. Since the file is on a remote cluster you should probably use the "foreign file" naming syntax, discussed here: https://hpccsystems.com/download/documentation/ecl-language-reference/html/Foreign_Files.html

HTH,

Richard