If you are an ECL programmer, there are a lot of things you don’t need to worry about that programmers in low level languages like C or C++ need to think about:

• Memory management
• Out-of-bounds array accesses
• Divide by zero
• Thread-safe programming
• Pointers and all their pitfalls

By freeing you from the need to worry about these, ECL allows you to focus on the problem you really want to solve – analysing your data. But there is one time that you DO need to worry about all these things: If you decide to use the BEGINC++ feature to express a bit of logic in C++ rather than ECL.

Writing error-free C++ is significantly harder than writing error-free ECL. I’ve been writing C++ for 25+ years and I still manage to write code with bugs in it. All the time. Every day. But I generally manage to avoid releasing code with bugs in it. How do I do that? Using every method I know to check the code is correct before I release it. Some of these methods include:

Compiler warnings

I NEVER ignore a compiler warning, and I configure my compiler to warn me about everything I possibly can. Modern compilers can warn about all sorts of things, from uninitialized variables to common typos to mismatched sprintf parameters.

Unfortunately when embedding C++ into ECL, you don’t usually get to see any warnings that might arise from the embedded C++ code. In release 6.0.0, we plan to force all warnings in embedded C++ to errors by default, so they will be much harder to ignore. There are tricks you can use to make sure that you do see them: 

Unit testing

Including boundary cases. I will write a separate post on how best to incorporate unit tests into your code in ECL.

Visual code inspection

Before getting anyone else to review your code, check it yourself. Use a checklist – such as the list of common mistakes below, to make sure that you have not fallen into any of the more common traps and pay particular attention to how errors and exceptions are handled as these are often overlooked.

Assert checks

It is common practice in C/C++ code to use assert to verify that any assumptions your code makes are in fact true. The classical assert macro is designed so that the checks are only made in debug mode and there is no oeverhead in release mode. Unfortunately use of assert inside embedded C++ is not practical, for a couple of reasons:

1. Workunits are generally compiled in release mode.
2. An assert failure normally results in the termination of the program that is running. You don’t want to terminate Roxie or Thor, just the current workunit.

We will consider adding a form of assert that CAN be used in embedded code in a future platform build.

Code review

No code goes into the platform source repository until it has been independently reviewed by an expert coder who is NOT the original author of the code.

If you have embedded C++ that you plan to use in a production job then I hereby offer the services of the HPCC platform team to code review it. I’m not making this offer out of the goodness of my heart, or because the platform team hasn’t got enough to do, or because we like a good laugh. I’m making it because I’d rather spend the time reviewing C++ code up front than tracking down obscure issues (that will be assumed to be platform bugs) that incorrect embedded C++ code can cause. If you are a RELX employee then there’s a good chance your manager will insist that you take me up on this offer but, I’ll make the offer to any ECL programmer, RELX or not.

Valgrind

Valgrind is an automatic memory debugging tool that can detect various bugs in your program as it runs, including accessing uninitialized memory, accessing outside the bounds of an array, and memory leaks.

The simplest way to use valgrind with a piece of embedded C++ code is to create a standalone ECL program from the unit tests, and run that standalone program under valgrind. You may see some false positives from the platform code but, any reports that reference your code should be investigated!

Single stepping

For particularly complex C++ logic where I want to be sure that it is doing what I intended when I coded it, I will use a debugger to step through it line by line (again, use the unit tests) to make sure that not only do I get the right results but, that I take the expected path through the code with the expected values in any intermediate results.

Common embedded C++ errors

STRING parameters are not null terminated

Standard C/C++ library functions work on null terminated strings but, if you declare your parameter to the EMBEDC++ function as a STRING or a STRINGnn it will NOT be null terminated. If you pass it to a function that expects a standard C null-terminated string, then that function is likely to read off the end of the value passed in. Sometimes this will cause no immediately-apparent symptom because there is usually a null byte hanging around soon enough to stop it from reading so far that it gets to unreadable memory. However, eventually, you will encounter a situation where it reads right off the end of the page and crashes.

Sometimes I see embedded C++ that starts off by making new, null-terminated copies of any STRING input parameters so that they can be safely passed to  C library functions but, this can also be error-prone. A much better idea is to use VARSTRING as the parameter type if you want a null-terminated string.

Don’t forget space for the null terminator

If you are using any standard C library functions, or if you are returning a varstring result, remember that there will be an extra null character on the end of the string that you must allow for when allocating memory for it or when declaring local arrays.

STRING can get large

Don’t make any assumptions about the maximum size of a STRING or VARSTRING that might be passed to your function (unless you declared the parameter as a STRINGnn). ECL programmers use big strings. If you are writing a general purpose function that takes a string parameter, then expect that some day someone will pass in a really big string. Don’t assume you can store a copy of the string in a local variable!

Divide by zero

A divide by zero will cause the program to generate a signal, which in effect means that the program will terminate. If the program in question is Roxie or Thor, that’s probably not a good thing. If your code contains a division (or modulus) operation, and there is any chance that the divisor may be zero, then check it first!

Memory leaks

If you allocate any memory from the heap (using malloc, calloc, new, rtlAlloc, strdup, or whatever) then you must make sure that it is released. If the memory you allocated is used to store the function’s return value and is passed back to the caller, then you don’t need to worry about it, but in every other case you do.

Memory leaks are particularly nasty bugs in that the symptom is observed well away from the cause. A bit of embedded C++ that leaks a few bytes each time it is called will sail through unit tests, and quite likely appear to be working fine when used on small jobs, but put it into a Roxie query that is called millions of times and you now have a Roxie that will run out of memory for no apparent reason every few hours. Not good.  

Memory corruptions

Don’t modify your inputs! Be careful about writing outside the memory you allocated. Don’t write to a pointer after it has been freed and don’t free memory more than once.

Like memory leaks, memory corruptions can be really nasty to track down. If your embedded C++ corrupts memory, then it may well pass all its tests, but other queries running on the same roxie at the same time as yours may give incorrect results.

Thread safety

Embedded C++ code may be called from multiple threads at the same time. Even if you think that it’s only called from one place, the underlying platform will do its best to run things in parallel if it can and if used in a Roxie query, there may be multiple queries running at the same time. There isn’t usually any good reason to use static variables in embedded C++ but, if you do then you need to be aware that multiple threads might access the variables at the same time. This is one of those areas where it’s safest to say “Unless you really know what you are doing, don’t…”

Plus all the list of common C++ programming errors – = rather than ==, operator precedence errors, etc

Conclusion

I don’t want to put people off using embedded C++. It’s a powerful tool if used correctly and writing inappropriate ECL to avoid using embedded C++ can also cause major problems. But just remember that embedded C++ is a bit like a chainsaw; when it’s the right tool for the job, nothing can beat it but, if you don’t treat it with the right respect it can kill you.