Bottom up programming and ECL

I was having a conversation with a friend of mine yesterday, and we were discussing how, as a programmer, people are not supposed to fight the language that they are using. If the language is an obstacle rather than a helper, it’s probably time to look for a different programming language for that particular task.

As we were exchanging ideas around this, in the context of how ECL (ECL, as the programming language behind the HPCC Systems platform, not to be confused with this ECL ) works, I remembered having read an excellent essay from Paul Graham, who wrote the book “On Lisp” back in 1993, which is, in my opinion, one of the best books on Lisp out there.

So I looked up the essay and read it, and to my surprise, the entire document is absolutely accurate if you replace every entry of the word “Lisp” by the word “ECL”. One of the paragraphs that caught my attention is the following (I replaced “Lisp” by “ECL” for the reader’s benefit):

“In ECL, you don’t just write your program down toward the language, you also build the language up toward your program. As you’re writing a program you may think “I wish ECL had such-and-such an operator.” So you go and write it. Afterward you realize that using the new operator would simplify the design of another part of the program, and so on. Language and program evolve together. Like the border between two warring states, the boundary between language and program is drawn and redrawn, until eventually it comes to rest along the mountains and rivers, the natural frontiers of your problem. In the end your program will look as if the language had been designed for it. And when language and program fit one another well, you end up with code which is clear, small, and efficient.

It’s worth emphasizing that bottom-up design doesn’t mean just writing the same program in a different order. When you work bottom-up, you usually end up with a different program. Instead of a single, monolithic program, you will get a larger language with more abstract operators, and a smaller program written in it. Instead of a lintel, you’ll get an arch.”

The entire purpose behind the design of ECL, was to create a language that would grow with the programmer, with enough plasticity to become almost a DSL suited to solve the current problem, but general enough to be applied to analogous tasks too, reusing most of the code (and the data attached to it).

The high level data oriented primities, the encouragement of purity across the language (but not to a point that it could become an obstacle to practical implementation) and the ability to encapsulate code and data behind standard interfaces, provides for an excellent collaborative development environment, which not only does create readable and reusable code, but also prevents a vast number of potential programming defects.

The declarative nature of the ECL programming paradigm lets the programmer focus on the task at hand and be less concerned with the specific implementation details. The implicit parallelism abstracts the details around multiprocessing and multinodes, allowing the same exact ECL program to run on a small virtual machine or a cluster composed of hundreds of nodes. The powerful optimizer will ensure that the execution plan is the most optimal, and the fact that ECL compiles to C++, and to machine code, ensures that your process will run at native speed on the hardware, without any extra overhead.

So an extensible declarative data-oriented high level programming language, on top of an extremely high performance data-intensive parallel processing system: What else could you ask for?

Head over here and take a look for yourself (and maybe, while you are at it, take HPCC for a ride).

Flavio Villanustre