Publishing queries with remote dali
I was hoping I would be able to keep the code agnostic about where the file lives and just use the "remote dali" feature of published queries.
I know using foreign::<remote dali ip>::path::to::file works, as I've tested it (so the file is definitely where I think it is, and authorization doesn't seem to be an issue). I'm specifically trying avoid having to do that, though.
I know using foreign::<remote dali ip>::path::to::file works, as I've tested it (so the file is definitely where I think it is, and authorization doesn't seem to be an issue). I'm specifically trying avoid having to do that, though.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Alex,
OK, I just tested this. Here are the steps I took:
So, next you'll want to duplicate my experiment with your code and data, step-by-step, and see if you still get an error.
HTH,
Richard
OK, I just tested this. Here are the steps I took:
- I built an INDEX on my Training1 cluster with a local name for the file (so it only exists on my Training1 Thor).
- Then I wrote a Roxie query for that INDEX on my Training2 cluster (no, they do not share the same Dali).
- On Training2, I compiled the query and Published it to my Training2 Roxie. I specified the Training1 Dali IP as the Remote Dali.
- It published with no errors and ran with no errors.
So, next you'll want to duplicate my experiment with your code and data, step-by-step, and see if you still get an error.
HTH,
Richard
- rtaylor
- Community Advisory Board Member
- Posts: 1619
- Joined: Wed Oct 26, 2011 7:40 pm
The only difference I see between what you're doing and what I'm doing is that in my simple experiment, my file is thor file and not an index. Other than that my steps are exactly the same as yours.
I presume that shouldn't make a difference, though.
I presume that shouldn't make a difference, though.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Alex,
Can you upload all your test code so we can look at it, please?
Richard
Can you upload all your test code so we can look at it, please?
Richard
- rtaylor
- Community Advisory Board Member
- Posts: 1619
- Joined: Wed Oct 26, 2011 7:40 pm
This is hand-typed so there might be typos; My HPCCs are not on an Internet-connected network.
- Code: Select all
string3 codein1 := '' : stored('ISO3');
string40 namein1 := '' : stored('Name');
countries := DATASET('~120way::Country_To_ISO3', Layout_Country_to_ISO3, thor);
d1 := countries(
if( codein1 != '', (trim(codein1) = iso3_code), true),
if( namein1 != '', stringlib.stringfind(trim(country), trim(namein1), 1) >0, true));
output(d1, named('Matching_Countries'));
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Also, what we would really like to see is the exact content of the error. It would usually have a logical file name in it.
The reason its failing could be for so many reasons. Like incorrect use of ~ or other naming issues. Or perhaps it can’t get to the dali, but the error could help us diagnose.
Bob
The reason its failing could be for so many reasons. Like incorrect use of ~ or other naming issues. Or perhaps it can’t get to the dali, but the error could help us diagnose.
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
Easy enough:
State: suspended
Error: Could not resolve filename 120way::Country_To_ISO3
I've fiddled with tildes; the error is the same if I leave it out of the code, but filename is obviously wrong since its appends the cluster name to the front of the path.
ETA: As I mentioned, if I use "foreign::<ip address>" in front of the path in the DATASET, it resolves the file ok. So it's not a path issue in that respect.
State: suspended
Error: Could not resolve filename 120way::Country_To_ISO3
I've fiddled with tildes; the error is the same if I leave it out of the code, but filename is obviously wrong since its appends the cluster name to the front of the path.
ETA: As I mentioned, if I use "foreign::<ip address>" in front of the path in the DATASET, it resolves the file ok. So it's not a path issue in that respect.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
Alex,
I see a couple of possible things:
HTH,
Richard
I see a couple of possible things:
- "120way" appears to be the scope name of your cluster, and you have no directory specified (since directories must follow definition naming rules, which do not allow for name starting with a number), so I would suggest trying with a file that is in a directory under your scope
- Roxie queries are most often contained within FUNCTION structures (really old ones may still use MACRO) and you code does not, so you may want to change that
- Code: Select all
IMPORT $;
EXPORT TestPublishRemoteDaliService() := FUNCTION
STRING30 fname_value := '' : STORED('FirstName');
STRING30 lname_value := '' : STORED('LastName');
Fetched := IF(fname_value <> '',
$.File_People.IDX_LFname(LastName= lname_value,
FirstName= fname_value),
$.File_People.IDX_LFname(LastName= lname_value));
RETURN Fetched;
END;
- Code: Select all
EXPORT File_People := MODULE
Layout := RECORD
UNSIGNED8 ID;
STRING15 FirstName;
STRING25 LastName;
STRING15 MiddleName;
STRING2 NameSuffix;
STRING8 FileDate;
UNSIGNED2 BureauCode;
STRING1 MaritalStatus;
STRING1 Gender;
UNSIGNED1 DependentCount;
STRING8 BirthDate;
STRING42 StreetAddress;
STRING20 City;
STRING2 State;
STRING5 ZipCode;
END;
EXPORT File := DATASET('~CLASS::RT::Intro::Persons',Layout,FLAT);
EXPORT IDX_LFname := INDEX(File,{Lastname,Firstname},{ID},
'~RTTEST::RemoteTest::KEY::People::LFname');
EXPORT BLD_IDX_LFname := BUILD(IDX_LFname,OVERWRITE);
END;
HTH,
Richard
- rtaylor
- Community Advisory Board Member
- Posts: 1619
- Joined: Wed Oct 26, 2011 7:40 pm
In response to your 2 points:
1) Names have been sanitized for public viewing; I typed in "120way" but that's not the actual name of the cluster. The actual path in my code is correct; publishing the query to the cluster that actually contains the file works ok. It's only when publishing it on the remote cluster that the file fails to resolve.
2) Wrapping the code in a function or macro has no effect on the ability to resolve the file.
1) Names have been sanitized for public viewing; I typed in "120way" but that's not the actual name of the cluster. The actual path in my code is correct; publishing the query to the cluster that actually contains the file works ok. It's only when publishing it on the remote cluster that the file fails to resolve.
2) Wrapping the code in a function or macro has no effect on the ability to resolve the file.
- alex
- Posts: 38
- Joined: Wed Feb 25, 2015 4:06 pm
OK, one last request please, can you post the Roxie log that contains the error? Since the error is reflected on every node, any log from any node will do, as long as it contains the error.
Thanks,
Bob
Thanks,
Bob
- bforeman
- Community Advisory Board Member
- Posts: 1006
- Joined: Wed Jun 29, 2011 7:13 pm
24 posts
• Page 2 of 3 • 1, 2, 3
Who is online
Users browsing this forum: No registered users and 1 guest