External Service Implementation

ECL external system services are implemented as exported functions in a .SO (Shared Object). An ECL system service .SO can contain one or more services and (possibly) a single .SO initialization routine.

All exported functions in the .SO (hereafter referred to as "entry points") must adhere to certain calling and naming conventions. First, entry points must use the "C" naming convention. That is, function name decoration (like that used by C++) is not allowed.

Second, the storage class of __declspec(dllexport) and declaration type _cdecl needs to be declared for Windows/Microsoft C++ applications. Typically, SERVICE_CALL is defined as _declspec(dllexport) and SERVICE_API is defined as _cdecl for Windows, and left as nulls for Linux. For example:

Extern "C" _declspec(dllexport) unsigned _cdecl Countchars(const unsigned len, const char *string)

.SO Initialization

The following is an example prototype for an ECL (.SO) system service initialization routine:

extern "C" void stdcall <functionName> (IEclWorkUnit *w);

The IEclWorkUnit is transparent to the application, and can be declared as Struct IEclWorkUnit; or simply referred to as a void *.

In addition, an initialization routine should retain a reference to its "Work Unit." Typically, a global variable is used to retain this value. For example:

IEclWorkUnit *workUnit;
     // global variable to hold the Work Unit reference
  
  extern "C" void SERVICE_API myInitFunction (IEclWorkUnit *w)
  {
       workUnit = w; // retain reference to "Work Unit"
  }

Entry Points

Entry points have the same definition requirements as initialization routines. However, unlike initialization routines, entry points can return a value. Valid return types are listed below. The following is an example of an entry point:

extern "C" __int64 SERVICE_API PrnLog(unsigned long len, const char *val)
  {
  }

SERVICE Structure - external

For each system service defined, a corresponding ECL function prototype must be declared (see SERVICE Structure).

  servicename := SERVICE
    functionname(parameter list) [: keyword = value];
    END;
  
  For example:
  email := SERVICE
    simpleSend(STRING address, STRING template, STRING subject)
       : LIBRARY='ecl2cw', INITFUNCTION='initEcl2Cw';
     END;

Keywords

This is the list of valid keywords for use in service function prototypes:

LIBRARY Indicates the name of the .SO module an entry point is defined in.
ENTRYPOINT Specifies a name for the entry point. By default, the name of the entry point is the function name.
INITFUNCTION Specifies the name of the initialization routine defined in the module containing the entry point. Currently, the initialization function is called once.
INCLUDE Indicates the function prototype is in the specified include file, so the generated CPP must #include that file. If INCLUDE is not specified, the C++ prototype is generated from the ECL function definition.
C Indicates the generated C++ prototype is enclosed within an extern "C" rather than just extern.
PURE Indicates the function returns the same result every time you call it with the same parameters and has no side effects. This allows the optimizer to make more efficient calls to the function in some cases.
ONCE Indicates the function has no side effects and is evaluated at query execution time, even if the parameters are constant. This allows the optimizer to make more efficient calls to the function in some cases.
ACTION Indicates the function has side effects and requires the optimizer to not remove calls to the function.
CONTEXT Internal use, only. Indicates an extra internal context parameter is passed to the function.
GLOBALCONTEXT Internal use, only. Same as CONTEXT, but there are restrictions on where the function can be used (for example, not in a TRANSFORM).
CTXMETHOD Internal use, only. Indicates the function is actually a method of the internal code context.

Data Types

Please see the BEGINC++ documentation for data type mapping.

Passing Set Parameters to a Service

Three types of set parameters are supported: INTEGER, REAL, and STRINGn.

INTEGER

If you want to sum up all the elements in a set of integers with an external function, to declare the function in the SERVICE structure:

  SetFuncLib := SERVICE
    INTEGER SumInt(SET OF INTEGER ss) :
       holertl,library='dab',entrypoint='rtlSumInt';
  END;
  x:= 3+4.5;
  SetFuncLib.SumInt([x, 11.79]); //passed two REAL numbers - it works

To define the external function, in the header (.h) file:

__int64 rtlSumInt(unsigned len, __int64 * a);

In the source code (.cpp) file:

  __int64 rtlSumInt(unsigned len, __int64 * a) {
       __int64 sum = 0;
       for(unsigned i = 0; i < len; i++) {
            sum += a[i];
       }
       return sum;
    }

The first parameter contains the length of the set, and the second parameter is an int array that holds the elements of the set. Note: In declaring the function in ECL, you can also have sets of INTEGER4, INTEGER2 and INTEGER1, but you need to change the type of the C function parameter, too. The relationship is:

  INTEGER8 -- __int64
  INTEGER4 -- int
  INTEGER2 -- short
  INTEGER1 -- char

REAL

If you want to sum up all the elements in a set of real numbers:

To declare the function in the SERVICE structure:

  SetFuncLib := SERVICE
       REAL8 SumReal(SET OF REAL8 ss) :
            holertl,library='dab',entrypoint='rtlSumReal';
  END;
  
  INTEGER r1 := 10;
  r2 := 20.345;
  SetFuncLib.SumReal([r1, r2]);
  // intentionally passed an integer to the real set, it works too.

To define the external function, in the header (.h) file:

double rtlSumReal(unsigned len, double * a);

In the source code (.cpp) file:

  double rtlSumReal(unsigned len, double * a) {
    double sum = 0;
    for(unsigned i = 0; i < len; i++) {
       sum += a[i];
    }
    return sum;
  }

The first parameter contains the length of the set, and the second parameter is an array that holds the elements of the set.

Note: You can also declare the function in ECL as set of REAL4, but you need to change the parameter of the C function to float.

STRINGn

If you want to calculate the sum of the lengths of all the strings in a set, with the trailing blanks trimmed off:

To declare the function in the SERVICE structure:

  SetFuncLib := SERVICE
    INTEGER SumCharLen(SET OF STRING20 ss) :
       holertl,library='dab',entrypoint='rtlSumCharLen';
  END;
  str1 := '1234567890'+'xxxx ';
  str2 := 'abc';
  SetFuncLib.SumCharLen([str1, str2]);

To define the external function, in the header (.h) file:

__int64 rtlSumCharLen(unsigned len, char a[ ][20]);

In the source code (.cpp) file:

__int64 rtlSumCharLen(unsigned len, char a[][20]) {
    __int64 sumtrimedlen = 0;
       for(unsigned i = 0; i < len; i++) {
          for(int j = 20-1; j >= 0; j—) {
            if(a[i][j] != ' ') {
              break;
            }
            a[i][j] = 0;
       }
       sumtrimedlen += j + 1;
    }
    return sumtrimedlen;
  } 

Note: In declaring the C function, we have two parameters for the set. The first parameter is the length of the set, the second parameter is char[][n] where n is the SAME as that in stringn. Eg., if the service is declared as "integer SumCharLen(set of string20)", then in the C function the parameter type must be char a[][20].

ECL Plug-Ins

In addition to external services, an ECL code module may be built as an ECL plug-in. These need to be deployed to the ESP and ECL servers through the ConfigEnv utility (see the Systems Operation Manual). A plug-in is accessable to all users of the environment, and appears in the ECL IDE in the same fashion as a service library.

Plug-In Requirements

Plug-ins require an exported function with the following signature under Windows:

Extern "C" _declspec(dllexport) bool getECLPluginDefinition(ECLPluginDefinitionBlock *pb)

The function must fill the passed structure with correct information for the features of the plug-in. The structure is defined as follows:

Struct ECLPluginDefinitionBlock
  {
    Size_t size;
       //size of passed structure - filled in by the calling function
    Unsigned magicVersion ;
       // Filled in by .SO - must be PLUGIN_VERSION (1) 
    Const char *moduleName;
       // Name of the module - e.g. lib_stringlib
    Const char *ECL;
       // ECL Service definition for non-HOLE applications
    Unsigned flags;
       // Type of plug-in - for user plugin use 1
    Const char *version ;
       // Text describing version of plugin - used in debugging
    Const char *description;
       // Text describing plugin
  } 

To initialize information in a plug-in, use a global variable or class and it will be appropriately constructed/destructed when the plugin is loaded and unloaded.

Deployment

External .SOs must be deployed to each node of the target environment as well as the ECL Server and ESP server used for the cluster. PlugIns may be deployed via the ConfigEnv command (see Systems Operation manual). If external data files are required, they should be either manually deployed to each node, or referenced from a network node (the latter requires hard-coding the address in the code for the .SO). Note that manually deployed files are not backed up with the standard SDS backup utilities. [I think].

Constraints

The full set of data types is supported on the Data Refinery and Data Delivery Engines (Thor/Roxie/Doxie).

An Example Service

The following code example depicts an ECL system service (.SO) called examplelib that contains one entry point (stringfind). This is a slightly modified version of the StringFind function found in the StringLib plugin. This version is designed to work in both the Data Refinery and Complex Analysis Engine supercomputers.

ECL definitions

  EXPORT ExampleLib := SERVICE
    UNSIGNED4 StringFind(CONST STRING src,
          CONST STRING tofind,
          UNSIGNED4 instance )
       : c, pure,entrypoint='elStringFind';
  END; 

.SO code module:

  //******************************************************
  // hqlplugins.hpp : Defines standard values included
              in
  // the plugin header file.
  //******************************************************
  #ifndef __HQLPLUGIN_INCL
  #define __HQLPLUGIN_INCL
  
  #define PLUGIN_VERSION 1
  
  #define PLUGIN_IMPLICIT_MODULE 1
  #define PLUGIN_MODEL_MODULE 2
  #define PLUGIN_.SO_MODULE 4
  
  struct ECLPluginDefinitionBlock
  {
    size_t size;
    unsigned magicVersion;
    const char *moduleName;
    const char *ECL;
    const char *Hole;
    unsigned flags;
    const char *version;
    const char *description;
  };
  
  typedef bool (*EclPluginDefinition) (ECLPluginDefinitionBlock *);
  
  #endif //__HQLPLUGIN_INCL
  
  //******************************************************
  // examplelib.hpp : Defines standard values included in
  // the plugin code file.
  //******************************************************
  #ifndef EXAMPLELIB_INCL
  #define EXAMPLELIB_INCL
  
  #ifdef _WIN32
    #define EXAMPLELIB_CALL __cdecl
    #ifdef EXAMPLELIB_EXPORTS
       #define EXAMPLELIB_API __declspec(dllexport)
    #else
       #define EXAMPLELIB_API __declspec(dllimport)
    #endif
  #else
    #define EXAMPLELIB_CALL
    #define EXAMPLELIB_API
  #endif
  
  #include "hqlplugins.hpp"
  
  extern "C" {
  EXAMPLELIB_API bool getECLPluginDefinition(ECLPluginDefinitionBlock *pb);
  EXAMPLELIB_API unsigned EXAMPLELIB_CALL elStringFind(unsigned srcLen,
       const char * src, unsigned hitLen, const char * hit,
       unsigned instance);
  }
  
  #endif //EXAMPLELIB_INCL
  
  //******************************************************
  // examplelib.cpp : Defines the plugin code.
  //******************************************************
  #include <memory.h>
  #include "examplelib.hpp"
  
  static char buildVersion[] = "$Name$ $Id$";
  
  #define EXAMPLELIB_VERSION "EXAMPLELIB 1.0.00"
  
  const char * const HoleDefinition =
    "SYSTEM
"
    "MODULE (SYSTEM)
"
    " FUNCTION StringFind(string src, string search,
        unsigned4 instance),unsigned4,c,name('elStringFind')
"
    "END
";
  
  const char * const EclDefinition =
    "export ExampleLib := SERVICE
"
    " unsigned integer4 StringFind(const string src,
        const string tofind, unsigned4 instance )
            : c, pure,entrypoint='elStringFind'; 
"
    "END;";
  
  EXAMPLELIB_API bool getECLPluginDefinition(ECLPluginDefinitionBlock *pb)
  {
    if (pb->size != sizeof(ECLPluginDefinitionBlock))
       return false;
    pb->magicVersion = PLUGIN_VERSION;
    pb->version = EXAMPLELIB_VERSION " $Name$ $Id$";
    pb->moduleName = "lib_examplelib";
    pb->ECL = EclDefinition;
    pb->Hole = HoleDefinition;
    pb->flags = PLUGIN_IMPLICIT_MODULE;
    pb->description = "ExampleLib example services library";
    return true;
  }
  
  //----------------------------------------------------------------
  EXAMPLELIB_API unsigned EXAMPLELIB_CALL elStringFind(unsigned srcLen,
    const char * src, unsigned hitLen, const char * hit,
    unsigned instance)
  {
    if ( srcLen < hitLen )
       return 0;
    unsigned steps = srcLen-hitLen+1;
    for ( unsigned i = 0; i < steps; i++ )
       if ( !memcmp((char *)src+i,hit,hitLen) )
            if ( !--instance )
                 return i+1;
    return 0;
  }