Tue Oct 26, 2021 11:15 am
Login Register Lost Password? Contact Us

question on improvisational complex queries

Topics related to the Hadoop Connector or migrating data from Hadoop

Tue Aug 09, 2011 4:33 pm Change Time Zone


I am supporting a team where hadoop/mapreduce is being used. From what I understand, the current implementation does not allow for the end-user (non-developer) to automatically send complex queries to the system. Can you please help me understand why this is and how HPCC addresses this?

Posts: 1
Joined: Tue Aug 09, 2011 4:25 pm

Tue Aug 09, 2011 8:02 pm Change Time Zone

Well I cannot really comment on : "Why Hadoop doesn't let them ..."

I suspect the issue they are hitting is that Hadoop is generally coded in Java - and it generally requires a lot of Java - so it more lends itself to 'projects' that 'dynamic complex queries'. There are (of course) multiple things in the Hadoop community designed to attack that (eg Pig, Cascalog) - but again - I cannot really tell you what will or won't work for your client.

The ECL answer is rather easier: ECL is a dictionary based language; it is designed to that the heavy hitters go in first and create functions and words that the end users can they use dynamically. The concept is that it is 'simple in the end' and the complexity is buried in the lower layers.

In the case that they queries are FAIRLY dynamic but not entirely free form then the ECL Template language can be used to actually generate the ECL for you.

Even more specialized it is possible to use the ECL graph statement and the PARSE statement in Tomita mode to construct a 'mini-language' which is then parsed and executed by roxie at run-time.

HTH - but as is usually the case - the more specific the question the more helpful the answer ....

Community Advisory Board Member
Community Advisory Board Member
Posts: 109
Joined: Fri Apr 29, 2011 1:35 pm

Return to From Hadoop

Who is online

Users browsing this forum: No registered users and 1 guest