Tue Sep 29, 2020 7:49 pm
Login Register Lost Password? Contact Us


Publishing queries with remote dali

Comments or questions specific to the features of ECL Watch

Mon Nov 09, 2015 7:06 pm Change Time Zone

I was hoping I would be able to keep the code agnostic about where the file lives and just use the "remote dali" feature of published queries.

I know using foreign::<remote dali ip>::path::to::file works, as I've tested it (so the file is definitely where I think it is, and authorization doesn't seem to be an issue). I'm specifically trying avoid having to do that, though.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 7:43 pm Change Time Zone

Alex,

OK, I just tested this. Here are the steps I took:
  • I built an INDEX on my Training1 cluster with a local name for the file (so it only exists on my Training1 Thor).
  • Then I wrote a Roxie query for that INDEX on my Training2 cluster (no, they do not share the same Dali).
  • On Training2, I compiled the query and Published it to my Training2 Roxie. I specified the Training1 Dali IP as the Remote Dali.
  • It published with no errors and ran with no errors.
Therefore, it is not necessary to use "Foreign file" naming (I didn't) to accomplish this.

So, next you'll want to duplicate my experiment with your code and data, step-by-step, and see if you still get an error.

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1556
Joined: Wed Oct 26, 2011 7:40 pm

Mon Nov 09, 2015 8:00 pm Change Time Zone

The only difference I see between what you're doing and what I'm doing is that in my simple experiment, my file is thor file and not an index. Other than that my steps are exactly the same as yours.

I presume that shouldn't make a difference, though.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 8:11 pm Change Time Zone

Alex,

Can you upload all your test code so we can look at it, please?

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1556
Joined: Wed Oct 26, 2011 7:40 pm

Mon Nov 09, 2015 8:22 pm Change Time Zone

This is hand-typed so there might be typos; My HPCCs are not on an Internet-connected network.

Code: Select all
string3 codein1 := '' : stored('ISO3');
string40 namein1 := '' : stored('Name');

countries := DATASET('~120way::Country_To_ISO3', Layout_Country_to_ISO3, thor);
d1 := countries(
if( codein1 != '', (trim(codein1) = iso3_code), true),
if( namein1 != '', stringlib.stringfind(trim(country), trim(namein1), 1) >0, true));

output(d1, named('Matching_Countries'));

alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 8:26 pm Change Time Zone

Also, what we would really like to see is the exact content of the error. It would usually have a logical file name in it.

The reason its failing could be for so many reasons. Like incorrect use of ~ or other naming issues. Or perhaps it can’t get to the dali, but the error could help us diagnose.

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1002
Joined: Wed Jun 29, 2011 7:13 pm

Mon Nov 09, 2015 8:30 pm Change Time Zone

Easy enough:

State: suspended
Error: Could not resolve filename 120way::Country_To_ISO3

I've fiddled with tildes; the error is the same if I leave it out of the code, but filename is obviously wrong since its appends the cluster name to the front of the path.

ETA: As I mentioned, if I use "foreign::<ip address>" in front of the path in the DATASET, it resolves the file ok. So it's not a path issue in that respect.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 8:38 pm Change Time Zone

Alex,

I see a couple of possible things:
  • "120way" appears to be the scope name of your cluster, and you have no directory specified (since directories must follow definition naming rules, which do not allow for name starting with a number), so I would suggest trying with a file that is in a directory under your scope
  • Roxie queries are most often contained within FUNCTION structures (really old ones may still use MACRO) and you code does not, so you may want to change that
Here's my query code:
Code: Select all
IMPORT $;
EXPORT TestPublishRemoteDaliService() := FUNCTION

  STRING30 fname_value := '' : STORED('FirstName');
  STRING30 lname_value := '' : STORED('LastName');

  Fetched := IF(fname_value <> '',
                $.File_People.IDX_LFname(LastName=  lname_value,
                                         FirstName=  fname_value),
                $.File_People.IDX_LFname(LastName=  lname_value));

  RETURN Fetched;
END;
And this is the supporting code:
Code: Select all
EXPORT File_People := MODULE
  Layout := RECORD
    UNSIGNED8 ID;
    STRING15  FirstName;
    STRING25  LastName;
    STRING15  MiddleName;
    STRING2   NameSuffix;
    STRING8   FileDate;
    UNSIGNED2 BureauCode;
    STRING1   MaritalStatus;
    STRING1   Gender;
    UNSIGNED1 DependentCount;
    STRING8   BirthDate;
    STRING42  StreetAddress;
    STRING20  City;
    STRING2   State;
    STRING5   ZipCode;
  END;
  EXPORT File := DATASET('~CLASS::RT::Intro::Persons',Layout,FLAT);
  EXPORT IDX_LFname  := INDEX(File,{Lastname,Firstname},{ID}, 
                              '~RTTEST::RemoteTest::KEY::People::LFname');
  EXPORT BLD_IDX_LFname  := BUILD(IDX_LFname,OVERWRITE);
END;

HTH,

Richard
rtaylor
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1556
Joined: Wed Oct 26, 2011 7:40 pm

Mon Nov 09, 2015 8:52 pm Change Time Zone

In response to your 2 points:

1) Names have been sanitized for public viewing; I typed in "120way" but that's not the actual name of the cluster. The actual path in my code is correct; publishing the query to the cluster that actually contains the file works ok. It's only when publishing it on the remote cluster that the file fails to resolve.

2) Wrapping the code in a function or macro has no effect on the ability to resolve the file.
alex
 
Posts: 38
Joined: Wed Feb 25, 2015 4:06 pm

Mon Nov 09, 2015 8:56 pm Change Time Zone

OK, one last request please, can you post the Roxie log that contains the error? Since the error is reflected on every node, any log from any node will do, as long as it contains the error.

Thanks,

Bob
bforeman
Community Advisory Board Member
Community Advisory Board Member
 
Posts: 1002
Joined: Wed Jun 29, 2011 7:13 pm

PreviousNext

Return to ECL Watch

Who is online

Users browsing this forum: No registered users and 1 guest

cron