Ingest JSON
Many systems export information in JSON format and CFS has features to help you convert JSON into IDOL documents.
JSON files are parsed as follows:
- Any document that has the metadata field
AUTN_JSON_PARSING_SECTIONis parsed using the settings specified in the section of the CFS configuration file that matches the field value. - All other JSON documents are parsed using the settings specified in the
[JSONParsing]section of the CFS configuration file.
For information about the JSON parsing configuration parameters, refer to the Connector Framework Server Reference.
The CFS JSON reader accepts a stream of JSON document objects:
{"DREREFERENCE": "one"}{"DREREFERENCE": "two"}{"DREREFERENCE": "three"}
Or an array of JSON document objects:
[{"DREREFERENCE": "one"},{"DREREFERENCE": "two"},{"DREREFERENCE": "three"}]
Example
Consider the following JSON:
{
"ref": "12345abcd",
"someField": [
"value1",
"value2"
],
"otherfield": {
"field_name": "myfield",
"@attribute": true,
"attrs": {
"attr1": "fish",
"attr2": "chips",
"attr3": "peas"
},
"field_value": {
"attr_name": "key",
"attr_value": "value",
"interesting": 7
},
"someField": [
12.9
]
},
"content_stuff": {
"1": "page1",
"2": [
"page2",
"page3"
]
}
}
To ingest this JSON, you might use the following configuration:
[JSONParsing] ReferencePath=ref ContentPaths=content_stuff/^.*$ FieldSections=SomeField,OtherField ExpandArrays=true [SomeField] Path=someField/^.*$ [OtherField] Path=otherfield NamePath=field_name ValuePath=field_value/interesting FieldSections=SomeField AttributeSections=OtherFieldAttr,OtherFieldAttr2,OtherFieldAttr3 [OtherFieldAttr] Path=^@(attribute)$ [OtherFieldAttr2] NamePath=field_value/attr_name ValuePath=field_value/attr_value [OtherFieldAttr3] Path=attrs/^.*$
To ingest the JSON, send the ingest action to CFS:
http://localhost:7000/action=ingest&adds=%3Cadds%3E%3Cadd%3E%3Csource%20
filename%3D%22file.json%22%20
lifetime%3D%22permanent%22%20%2F%3E
%3C%2Fadd%3E%3C%2Fadds%3E
This would produce the following document:
#DREREFERENCE 12345abcd #DREFIELD myfield="7" #DREFIELD myfield/@attribute="true" #DREFIELD myfield/@key="value" #DREFIELD myfield/@attr1="fish" #DREFIELD myfield/@attr2="chips" #DREFIELD myfield/@attr3="peas" #DREFIELD myfield/someField="12.9" #DREFIELD someField="value1" #DREFIELD someField="value2" #DRESECTION 0 #DRECONTENT page1 #DREENDDOC #DREREFERENCE 12345abcd #DRESECTION 1 #DRECONTENT page2 #DREENDDOC #DREREFERENCE 12345abcd #DRESECTION 2 #DRECONTENT page3 #DREENDDOC