Wed Dec 08, 2021 9:58 am
Login Register Lost Password? Contact Us

FROMJSON to DataSet produces empty dataset

Comments and questions related to the Enterprise Control Language

Mon Oct 14, 2019 2:15 am Change Time Zone

I'm seeing some strange behavior when there's an optional member that would become a dataset when reading it in using fromjson(). For the below json lets say we have a member for one service that's optional? For example, what if ServiceA has a member named 'flags' that has members (flaga, flagb, flagc) that's not always present for Service A. 'flags' is optional. When reading this in using the record structure it would produce a dataset when present and an empty dataset when not. How would the record structure look then? If we specify 'flags' in the record structure and it's not in the json it seems to produce an empty dataset named something other than 'flags'. In some instances it gives the empty dataset the name 'name' or whatever the first member of the structure is Is there a way in HPCC/ECL to read this in correctly using fromjson() for both scenarios mentioned?

Code: Select all
jsonstring2   :=
+ '   "text": "Hello World",'
+ '   "services": ['
+ '      {'
+ '         "name": "ServiceA",'
+ '         "body": "default",'
// + '        "flags": {'
// + '         "flaga": "a",'
// + '         "flagb": "b",'
// + '         "flagc": "c"'
// + '        },'
+ '         "count": "1"'
+ '      },'
+ '      {'
+ '         "name": "ServiceB",'
+ '         "body": "custom",'
+ '         "count": "2"'
+ '      },'
+ '      {'
+ '         "name": "ServiceC",'
+ '         "body": "default",'
+ '         "count": "3"'
+ '      }'
+ '   ]'
+ '}';

rFlags := record
  string flaga{xpath('flaga')} := '';
  string flagb{xpath('flagb')} := '';
  string flagc{xpath('flagc')} := '';

rServices := record
  string name{};
  string body{};
  dataset(rFlags) flags{xpath('flags')} := dataset([],rFlags);
  string count{xpath('count')};

rMain := record
  string text{} := '';
  dataset(rServices) services{xpath('services')};

ds := fromjson(rMain, jsonstring2);
Posts: 27
Joined: Mon Jul 11, 2016 12:45 pm

Mon Oct 14, 2019 5:34 pm Change Time Zone

ECL's FROMJSON() and FROMXML() functions don't create a record definition from the data, but rather populate a record definition by examining the data. It's a distinction that implies a couple of things:

  • There is only one record definition used for all records parsed from the data during that function call
  • A field within a record automatically assumes its default value (empty string for strings, zero for numerics, empty dataset, etc) unless parsing finds something for it in the data string. Unless you need a non-standard default value for a field, you do not need to specify one in the record definition

In your example, the only difficulty I saw was with the 'flags' field. In the data, it is defined as a JSON object, but in the record definition it is defined as a dataset (an array of JSON objects). I think that actually parses correctly, but the display in ECL Watch may be a little confusing. Anyway, I would define rServices like this:

Code: Select all
rServices := record
  string name{xpath('name')};
  string body{xpath('body')};
  rFlags flags{xpath('flags')};
  string count{xpath('count')};

On the subject of displaying things in ECL Watch: By default, the output will label fields with the XPATH value rather than the attribute name. That can lead to all kinds of issues, especially with more complex XPATH values (e.g. "foo[1]/@bar"). To always show the ECL attribute names, add a NOXPATH to OUTPUT:

Code: Select all

That wasn't really an issue with your example, but if you continue to play with JSON parsing then you will probably run into it at some point.

Lastly, one thing you might want to check into is the IFBLOCK() option within a RECORD. That option can be a useful way to maintain slightly different views of records within a dataset. It does need to be keyed off something though, so in your example you might need a field like "hasFlags" to indicate whether to show any of the 'flags' data. Also, be aware that IFBLOCK() is not like a C-style union, where the field contents are overlayed; all of the fields will physically exist in the record, end-on-end, but IFBLOCK() will govern which ones you have access to and which ones are shown.

Hope this helps.

Community Advisory Board Member
Community Advisory Board Member
Posts: 568
Joined: Tue Oct 18, 2011 4:45 pm

Return to ECL

Who is online

Users browsing this forum: Google [Bot] and 1 guest