Wed Dec 01, 2021 3:52 pm
Login Register Lost Password? Contact Us


build web service inside of hpcc source code

Questions around writing code and queries

Sat Feb 02, 2013 11:12 pm Change Time Zone

Hello there,

Currently, I am trying to build some web services and relevant http forms which are based on existing WSs in HPCC WsECL. From the internal source code perspective, I can add a new subfolder in ESP/services/ws_*** for my own need, and it could work if I can continue to develop C++ code on it.

But my intention is that finally we want to build a web service HTTP form, which is separate with the HPCC system, so in this way, could somebody tell me how could I do it more efficiently using ECL instead of using pure C++ code to create such service?

With Thanks and Regards
Bo
buptkang
 
Posts: 36
Joined: Tue Jan 22, 2013 7:19 am

Mon Feb 04, 2013 9:29 pm Change Time Zone

It really depends on what you are trying to accomplish how much I could reccomened your service 1. Calling an ECL based service. 2. Calling into ESP (EclWatch) services via SOAP or REST calls. 3. Being implemented inside of ESP as part of a C++ service.

You can create ECL and publish it. You can then call it from your service. And that ECL can even use SOAPCALL or HTTPCALL to call other services in ESP or elsewhere.

Or you can have your service call ESP services to manipulate the HPPC and / or run ECL.

A basic workflow might help me understand what you are trying to do and where it might fit.
anthony.fishbeck
 
Posts: 61
Joined: Wed Jan 30, 2013 10:18 pm

Wed Feb 20, 2013 11:04 pm Change Time Zone

Hi Tony,

There is a long time interval since the last message on this question. In general, I want to build a machine learning web portal based on ECL-ML toolbox.

I will mostly create and export ECL code as the web service to be called later. Then my question is still here:

My goal is to do the web-development as less as possible, I mainly focus on data manipulation using ECL. So I assume that I need to create some HTML forms such as load file, manipulate file and so forth in my portal.

If so, which way do you recommend me to do now? Developing such portal inside of HPCC system or build my own client app to call? Like what I am saying, I want to minimize any effort on web development.

Under such circumstance, could you give me some ideas on the work flow I should continue to try?

Thanks
Bo
buptkang
 
Posts: 36
Joined: Tue Jan 22, 2013 7:19 am

Thu Feb 21, 2013 10:34 pm Change Time Zone

This is a really difficult question because you have so many options available and much of it is based on your personal preferences and style.

It is possible to write ECL that uses the ECL "SOAPCALL" method to call HPCC system functions. That way you could publish ECL queries that did things like copying files onto the system, and then publish another ECL query that used the files. All of those queries could be accessed using forms from the WsEcl page.

You can btw call published queries using an HTTP GET url. So for example if you
have a query with one parameter called "lastname" you could run the query using the
following url:
Code: Select all
http://ip:port/WsEcl/submit/query/hthor/myquery?lastname=smith


(just replace hthor with the target cluster you are hitting, and myquery with your published query name).

You could also drop html files in /opt/HPCCSystems/componentfiles/files and they will be accessible through esp. For example, if you add myfile.html you can see it from ESP as
Code: Select all
http://ip:port/esp/files/myfile.html
That page could have convenient links to your published queries, or could directly contain Ajax calls out to the system functions.

As soon as you want to start doing really sophisticated web development though I would probably recommend using another front end and making calls from there to the HPCC.

Regards,
Tony
anthony.fishbeck
 
Posts: 61
Joined: Wed Jan 30, 2013 10:18 pm

Fri Feb 22, 2013 5:50 pm Change Time Zone

Hi Tony,

Thanks again for your comment.

In general, I will create most ECL script to manipulate data, then publish the query into the server in order to wait for end-user's data request through web portal.

From client web portal side, I should create HTML+Javascript(JQuery) to pull the data through WS using JSON data format or XML format.

But here, I have a question:you mentions that I can put my client files(html+js) into HPPC deployed source folder. If I want to use HPCC's internal ecm WSDL to represent data in order to reduce lots of web dev work, how could I use JSON to store the data in client side? Does it mean that I have to stick onto XML soap method?

There is another big design issue: Do you think that building another web portal inside of HPCC is a "good" option other than building outside HPCC context? Here my "good" definition is to try to reduce web development work as less as possible, I want to focus on data manipulation side.

With Thanks and Regards
Bo
buptkang
 
Posts: 36
Joined: Tue Jan 22, 2013 7:19 am

Mon Feb 25, 2013 5:52 pm Change Time Zone

Hi Bo,

The generated WSDL/XMLSchema describe the SOAP/XML format for the query. The JSON format is structurally very similar, and you can use the WSDL as a form of documentation that is very close. But we don't currently have a schema specifically for JSON so you would have to hand code the JSON message processing.

If you have tools that generate code for handling SOAP/XML from a WSDL, it makes the question of whether you should use SOAP or JSON trickier. I would suggest playing with both and deciding which better meets your needs.

As for running your html+js from the HPCC ESP component or outside the HPCC, first of all if you have a web server set up I would probably use that rather than embedding in the ESP just to keep things cleen. If not I would base it on the scale of the system, a large or active system should probably have a stand alone web server. A small prototype system may run ok from ESP.

Regards,
Tony
anthony.fishbeck
 
Posts: 61
Joined: Wed Jan 30, 2013 10:18 pm

Wed Feb 27, 2013 11:05 pm Change Time Zone

Hi Tony,

Thanks, I am more clear toward my target now.

Bo
buptkang
 
Posts: 36
Joined: Tue Jan 22, 2013 7:19 am

Mon Mar 04, 2013 8:53 pm Change Time Zone

Hi Tony,

Currently I am trying to develop my ECL function and access the data through SOAP, but I faced up issues on retrieving data.

First, I submitted and published the below function into THOR server(sample code):

Code: Select all
//
//  Example code - use without restriction. 
//
ParseWords(STRING LineIn) := FUNCTION
   PATTERN Ltrs := PATTERN('[A-Za-z]');
   PATTERN Char := Ltrs | '-' | '\'';
   TOKEN   Word := Char+;
               ds     := DATASET([{LineIn}],{STRING line});
   RETURN PARSE(ds,line,Word,{STRING Pword := MATCHTEXT(Word)});
END;   

EXPORT Soapcall3() := FUNCTION
  STRING UID     := '' : STORED('UIDstr');
  STRING LeftIn  := '' : STORED('LeftInStr');
  STRING RightIn := '' : STORED('RightInStr');
  BOOLEAN TokenMatch := FUNCTION
    P1 := ParseWords(LeftIn);
    P2 := ParseWords(RightIn);
    SetSrch := SET(P1,Pword);
    ProjRes := PROJECT(P2,
                         TRANSFORM({BOOLEAN Fnd},
                                      SELF.Fnd := LEFT.Pword IN SetSrch));
    AllRes  := DEDUP(SORT(ProjRes,Fnd));
    RETURN COUNT(AllRes) = 1 AND AllRes[1].Fnd = TRUE;
  END;
  RETURN OUTPUT(DATASET([{UID,TokenMatch}],{STRING UID,BOOLEAN res}));
END;



Then I used another sample code to retrieve the data:

Code: Select all
InRec := RECORD
   STRING UIDstr{XPATH('UIDstr')};
   STRING LeftInStr{XPATH('LeftInStr')};
   STRING RightInStr{XPATH('RightInStr')};
END;
InDS := DATASET([
         {'1','the quick brown fox jumped over the lazy red dog','quick fox red dog'},
         {'2','the quick brown fox jumped over the lazy red dog','quick fox black dog'},
         {'3','george of the jungle lives here','fox black dog'},
         {'4','fred and wilma flintstone','fred flintstone'},
         {'5','yomama comeonah','brake chill'}
                        ],InRec);

RS := SOAPCALL(InDS,
                      RoxieIP,
               svc,
                      InRec,
                      TRANSFORM(LEFT),
                      DATASET({STRING UIDval{XPATH('uid')},
                              BOOLEAN CompareResult{XPATH('res')}}));

OUTPUT(RS);



In the above code, I do not write the variable RoxieIP and svc variable.

RoxieIP = 'http://myIP:8002/WsEcl/soap/query/myroxie/soapcall3.1';
svc = 'soapcall3.1';

But when I was running the above code, it always run without finishing signal.

Do you have any clue on this?

Thanks
Bo
buptkang
 
Posts: 36
Joined: Tue Jan 22, 2013 7:19 am

Tue Mar 05, 2013 5:10 pm Change Time Zone

I think the main issue is that your URL references what we call the process cluster (physical roxie) rather than target cluster. Assuming you are using the default configuration, change 'myroxie' to 'roxie':

RoxieIP = 'http://myIP:8002/WsEcl/soap/query/roxie/soapcall3.1';

And try again.

Regards,
Tony
anthony.fishbeck
 
Posts: 61
Joined: Wed Jan 30, 2013 10:18 pm

Wed Mar 06, 2013 2:29 pm Change Time Zone

Hi Tony,

I have successfully to run that service call which is built on Roxie server, but interesting, if I publish the same query in the THOR server, then I cannot retrieve the result. Do you have any clue?

Regards
Bo
buptkang
 
Posts: 36
Joined: Tue Jan 22, 2013 7:19 am

Next

Return to Programming

Who is online

Users browsing this forum: No registered users and 1 guest