Thu Aug 16, 2018 6:50 am
Login Register Lost Password? Contact Us


HTTPCALL HTTPHEADER

Comments and questions related to the Enterprise Control Language

Wed Jan 31, 2018 8:52 pm Change Time Zone

I'm trying to call Amazon Web Services from an HTTPCALL and I'm running into some issues. I have got functions that construct the request and sign it using the correct hashing methods. The resulting url works from a web browser or Fiddler and returns the requested response. However, when I send the request from ECL using HTTPCALL I get errors. One of the problems is that HTTPCALL seems to automatically add headers to my request even when they are not specified with the HTTPHEADER option. Using tcpdump on my HPCC system I can see the GET request go out and it includes "Authorization: Basic ##somehash##", "Accept: text/xml", and "Host: ec2.amazonaws.com". The host is fine, but the other two cause issues. The problem is that the request, including the headers, needs to be hashed into a signature and they have to be added in a very specific way in a specific order. The best solution would be if I could control the headers that are added and the order using HTTPHEADER. I even tried adding the Accept and Authorization headers manually thinking maybe it would overwrite the "built in" ones. Instead I got two Authorization lines in the request.
jbanner
 
Posts: 2
Joined: Wed Jan 31, 2018 8:31 pm

Thu Feb 01, 2018 5:58 pm Change Time Zone

I've created JIRAs for two different solutions:

1. Allow full control of HTTPHEADERs:
https://track.hpccsystems.com/browse/HPCC-19045

2. Native support for AWS Signatures:
https://track.hpccsystems.com/browse/HPCC-19046

In the meantime if you are not able to use HTTPCALL to make these requests, you could try making the request through one of the embedded languages supported by the HPCC.

Embedding some Javascript, Python, or Java code to make the request and return the data to your ECL code may be a good workaround.

You can find examples of using embedded languages as part of the normal installation on any HPCC server in the following directory:

Code: Select all
/opt/HPCCSystems/examples/embed

or in our source code repository at:

https://github.com/hpcc-systems/HPCC-Platform/tree/master/initfiles/examples/embed
anthony.fishbeck
 
Posts: 47
Joined: Wed Jan 30, 2013 10:18 pm

Thu Feb 01, 2018 7:06 pm Change Time Zone

Thank you for you suggestions. I finally got it working. I had been trying to include the authentication parameters in the query string as described in https://docs.aws.amazon.com/general/lat ... mples.html in the section titled "Using GET with Authentication Information in the Query String". I couldn't get the Basic authentication to go away from the header. I've now switched to the method titled "Using GET with an Authorization Header" where the authorization strings are in the header instead. Doing it that way the default authentication header value was replaced appropriately. This is working well for the simple test requests I am currently making. Hopefully it will continue to function for more complex operations. Thanks again.
jbanner
 
Posts: 2
Joined: Wed Jan 31, 2018 8:31 pm


Return to ECL

Who is online

Users browsing this forum: No registered users and 1 guest