Query publishing using remote files

[dsanchez]

Hi all,

I am trying to publish a query into a Roxie cluster but avoiding the copy of any file from Thor and instead using them remotely.

I've been reading this thread but I would like to discard other options before I dive into the configuration files:
viewtopic.php?f=17&t=1992

As him, I have an index created in my Thor cluster A, I have some ECL I want to deploy in my Roxie Cluster B.
The line I use to publish is as follows:

Code: Select all

ecl publish -v --name="$folderName.$file" -s $server --target=$target -u $user -pw $password --allow-foreign --main="$folderName.$file" --memoryLimit=2GB --timeLimit=30000 --warnTimeLimit=5000 --daliip=$remoteDali --no-files --allow-foreign

A and B don't share a dali, and this works if I publish normally (Without the last 2 options). The problem is that this is a small development Roxie and it doesn't have a lot of disk space so we can't afford having Roxie to copy the files it needs.

Any Idea on how can we achieve this?

Thanks for the help!
Daniel.

[JimD]

This is achieved in the way your Roxie os configured.

There are two settings in the Roxie configuration that control where Roxie looks for data and index files:

copyResources Copies necessary data and key files from the current location when the query is published.

useRemoteResources Instructs Roxie to look for data and key files in the current location after the query is published.

For the scenarion you descriobe, you should set copyResources to FALSE and useRemoteResources to TRUE.

This directs the Roxie cluster to load the data from a remote location.
The query can be executed immediately, but performance is limited by network bandwidth.
This allows queries to run without using any Roxie disk space but reduces its throughput capabilities.

You can find more details about this and other Roxie configuration settings in this manual:
http://cdn.hpccsystems.com/releases/CE- ... df#page=10

HTH,

Jim

[dsanchez]

This was really helpful. Thanks Jim.

I will test this and thank you one more time if it works!

[dsanchez]

I couldn't get it to work...

This is the RoxieTopology file in the cluster, I changed those two parameters, restarted the cluster but still the files were copied over. The parameter in roxie shows now the right values but when I publish the query it copies the files again or if I specify in the publishing script the options "--no-files --allow-foreign --daliip=$remoteDali" the query gets suspended because it cannot find the file.

Any Idea what I could be missing? Platform is on version 5.6.8-1 by the way.

Thanks!

EDIT*