Reading a really deeply and trickly nested json. Currently using the lift json library need some help.Please


Reading a really deeply and trickly nested json. Currently using the lift json library need some help.Please

I have a really nested json very big and very deeply nested.

Here is the link to the json http://ift.tt/2uf4OZM

I was trying to use the lift library to parse it .

I was having difficulties. My main problem was twofold. For example for SchedulesMM

It has a list of nested dates I would to try to extract all those StartDates,Sid,EndDates into a list but I was having difficulty doing this. I dont see how to get this into a list nicely .

I could just reference startDate . Assume I have a jsonObject.

(jsonObject(index) \ "StartDate").children – but this is not as clean Also for some records I actually have a problem with nested records having the same name. so (jsonObject(index) \ "Id").children -returns the ids of both of Destinations and Ch example json here

"Destination": { "Code": "android", "Id": 408, "Ch": { "Id": 56, "Name": "C" }

Back to the question twofold I need some help parsing this type of record with the lift library specifically the edge cases I mentioned. Other than that seems to be working.

Or if there is a better library for this please suggest a coding solution.

My Plan is to parse this out and then flatten it. Hopefully then load it into a spark dataframe.

I was going to try explode with spark but this json is just so huge and it some other cases can be even slightly bigger. Would be easier if I could do it all before spark.

Thank you

Submitted July 12, 2017 at 12:33AM by yanks09champs
via reddit http://ift.tt/2ueSLM7

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s