Cryptographic Standard Library ECL Module
As of HPCC Systems version 7.6.0, a new cryptographic module has been added to the ECL Standard Library. Within this module, there is an assortment of cryptographic features available to ECL developers to utilize in order to safeguard their sensitive data, using industry standard cryptographic algorithms. These features include digital hash algorithms, symmetric and asymmetric encryption and decryption, and digital signatures, all of which can be applied to individual columns within an ECL dataset.
Supported Cryptographic Algorithms
All of the cryptographic functions available in the Crypto ECL Standard Library support one or more security algorithms, developed and verified by the United States National Security Agency (NSA). ECL Developers should begin their cryptographic session by calling one or more of the following ECL interfaces to determine which algorithms are available to them, and choose the one that best meets their security and performance requirements.
SET OF STRING Std.Crypto.SupportedHashAlgorithms(); SET OF STRING Std.Crypto.SupportedSymmetricCipherAlgorithms(); SET OF STRING Std.Crypto.SupportedPublicKeyAlgorithms();
Hashing Functions
A digital hash function is a fast cryptographic method that converts data of any form and size into a unique, fixed size representation. A hash is a one-way function, so that the hash cannot be reversed back into its original form, and is guaranteed to be unique. Additionally, using the same hash algorithm, the same input data will always generate the same hash. As an example, secure email software applications compute a hash of the plain text email, and sends it along with the plain text email body. The email client that receives the email re-computes the hash using the plain text email contents, and if the computed hash matches the one sent, then the contents can be trusted not to have been modified. One use of a hash for an ECL developer could be to create and index a hash of a large data column, and use this as the lookup key, presumably speeding up searches.
The following ECL sample demonstrates the simplicity of creating digital hashes using the Standard Library Cryptographic module.
IMPORT STD;
myHashMod := Std.Crypto.Hashing('SHA256');//Define the hashing module
DATA myHash1 := myHashMod.Hash( (DATA)'HPCC is cool’ );
DATA myHash2 := myHashMod.Hash( (DATA)'ECL is even more cool’ );
Symmetric Encryption/Decryption
A symmetric algorithm is one that uses the same secret key for both encryption and decryption. Sensitive data can be quickly encrypted using a shared secret key, and written to a column in the ECL dataset in its encrypted form. When the dataset is read, it can be decrypted using the same shared key. Care should be taken to protect the shared key, as any compromise could result in sensitive data being harvested or changed.
The following ECL sample demonstrates the simplicity of encrypting and decrypting a simple text string using a symmetric algorithm.
IMPORT STD; //Define the Symmetric Encryption/Decryption module MySymEncMod := Std.Crypto.SymmetricEncryption( 'aes-256-cbc', 'MySecretKey'); DATA encrypted := MySymEncMod.Encrypt( (DATA)'HPCC and ECL are cool'); STRING decrypted := (STRING) MySymEncMod.Decrypt( encrypted ) ;
Asymmetric Encryption/Decryption
Asymmetric encryption/decryption, also known as Public Key Encryption (PKE), uses public and private key files to operate. A public key file can be widely distributed, and is used to encrypt sensitive data. The private key file is tightly held and guarded, and must be used to decrypt the data. Asymmetric algorithms are not as fast as symmetric ones, but they are considered to be more secure. Public and private key files can be self-generated using publicly available tools, or better yet issued by a respected certificate authority.
The Crypto module to perform asymmetric encryption/decryption can be defined by specifying a file system path and filename, or can be loaded from a buffer, as follows.
MyMod1 := Std.Crypto.PublicKeyEncryption(VARSTRING pkAlgorithm, VARSTRING publicKeyFile, VARSTRING privateKeyFile, VARSTRING passphrase); MyMod2 := Std.Crypto.PublicKeyEncryptionFromBuffer(VARSTRING pkAlgorithm, VARSTRING publicKeyBuffer, VARSTRING privateKeyBuffer, VARSTRING passphrase);
The following ECL sample demonstrates the simplicity of encrypting and decrypting using public/private keys that were streamed into an ECL DATA type.
IMPORT STD;
//Define the PKE Encryption module
MyPKEncryptModule := Std.Crypto.PublicKeyEncryptionFromBuffer('RSA', publicKeyBuf, privateKeyBuf, 'passphrase');
DATA encrypted := MyPKEncryptModule.Encrypt( (DATA)'HPCC is fun');
DATA decrypted := MyPKEncryptModule.Decrypt(encrypted);
Typically, your ECL query will only be encrypting, or decrypting data, but not both. If this is the case, you can define your module specifying just the public or private key, as follows.
IMPORT STD;
MyPKEncryptor := Std.Crypto.PublicKeyEncryptionFromBuffer('RSA', publicKeyBuf, NULL, NULL);
MyPKDecryptor := Std.Crypto.PublicKeyEncryptionFromBuffer('RSA', NULL, privateKeyBuf, NULL);
Digital Signatures
A digital signature is similar to a hash, described above, with the main difference being that it is created and verified using public and private keys. A digital signature is computed using the tightly held private key, and can be verified by any entity that has access to the widely distributed public key. As with a hash, a digital signature can be computed and used as a lookup key for larger data objects.
The following ECL sample demonstrates the simplicity of creating and verifying a digital signature using public/private key files stored on the HPCC Systems cluster file system.
IMPORT STD;
//Define the PKI Encryption module
privKeyFile := '/home/hpcc/certificate/key.pem';
pubKeyFile := '/home/hpcc/certificate/public.key.pem';
MyPKIEncryptModule := Std.Crypto.PublicKeyEncryption('RSA', privKeyFile, pubKeyFile, 'passphrase');
DATA sig := MyPKIEncryptModule.Sign( (DATA)'HPCC is fast');
BOOLEAN isValid := MyPKIEncryptModule.VerifySignature( sig, (DATA)'HPCC is fast');
Key and Passphrase Management
Keys and passphrases should never be encoded in ECL, where prying eyes could extract them and maliciously access sensitive data. It is recommended that some external key/value or other secure store be utilized, where the ECL execution engine could query them at run time.