Difference between revisions of "JSON Integration Adapter"

From CDOT Wiki
Jump to: navigation, search
(Current Status)
Line 11: Line 11:
 
== Current Status==
 
== Current Status==
 
* '''Completed''' (February 02, 2012)
 
* '''Completed''' (February 02, 2012)
 +
  
 
== Project Repository ==
 
== Project Repository ==

Revision as of 12:20, 4 February 2012

Project Goal

To create an adapter for the NexJ Core similar in methodology and design of existing message adapters (XML, Fixed, CSV etc.) but using the JSON format

Contributors


Current Status

  • Completed (February 02, 2012)


Project Repository

BitBucket : https://bitbucket.org/gbatumbya/nexjexpress-jsonintergrationadapter/
Deprecated: https://bitbucket.org/b_lim/nexj-express-json-integration-adapter/


Definitions

Term Description
Formatter Turns data into JSON
Parser Turns JSON into data
Message The metadata to be used when formatting and parsing a message. The grammar. It is a tree-like structure.
MessageInstance The information to be parsed or formatted

Because the Integration layer allows a Model to interact with external applications asynchronously, a description / grammar of the data to be exchanged between the two systems is defined using a Message in the Model.

  • Formatting is turning internal NexJ representation of a MessageInstance into JSON. The formatter is given a Message (the grammar), MessageInstance (the data), and a pipe. This accomplished by walking the Message tree-like structure while validating the data in the MessageInstance and streaming JSON format to the pipe. If the data in the MessageInstance does not conform to the Message, the formatter throws an exception.
  • Parsing is turning JSON representation of a MessageInstance into an internal NexJ representation of a MessageInstance. The parser is given a Message (the grammar) and an input stream. First the input stream is parsed for JSON using a JSONParser which returns either an Object or an Array. The formatter walks the Message building the structure of the MessageInstance and adding validated values in the object returned by the JSONParser to the MessageInstance. If the data from the input stream does not match the expected input at any time, the parser throws an exception.


Message Fundamentals

Message - Messages can contain values or other messages. The red nodes are messages, the green nodes are values.

NexJ Express Messages

Internally, messages are Transfer Objects. To determine if a node is a message, use instanceof on CompositeMessagePartInstance after retrieving the MessagePart. For example,

public void format(TransferObject tobj, Message message, Output out) throws IntegrationException
{
...
CompositeMessagePart root = message.getRoot(); // Gets the root of the message.
Iterator it = root.getPartIterator();
while (it.hasNext())
{
  part = (MessagePart)it.next());
  if (part instanceof CompositeMessagePartInstance)
  {
    // This part is a message
  }
  else if (part instanceof PrimitiveMessagePart)
  {
    // This part is a value
  }
...
}
...

NOTE: THE MESSAGE PARAMETER DOES NOT CONTAIN THE MESSAGE VALUES. THE MESSAGE PARAMETER CONTAINS THE MESSAGE METADATA. THE TRANSFER OBJECT CONTAINS THE MESSAGE VALUES WHICH MUST BE VALIDATED AGAINST THE MESSAGE TO ENSURE CORRECT FORMAT.

MessagePart
Parts of a message. Messages can contain values or other messages. NexJ Express has two types of message parts, CompositeMessagePart and PrimitiveMessagePart.
CompositeMessagePart
implementation is CompositeMessagePartInstance . The relationship between CompositeMessagePartInstance and PrimitiveMessagePart with the above picture is as follows - CompositeMessagePartInstance are messages (the red nodes) and PrimitiveMessagePart are values (the green nodes). To determine multiplicity of MessageParts, use isCollection() method of MessagePart. Note that multiplicity of the above screenshot are all [0..1], that is zero or one instance. Possible node multiplicities:
[0..1] 
zero or one instance (i.e. an optional entry)
[1..1] 
exactly one instance (i.e. required)
[1..N] 
collection of at least one, at most N instances
[0..0] 
collection of zero to infinite instances (displayed as [*])
[1..0] 
collection of at least one, possibly unlimited instances
XMLJSONMessageMappingLoader
Used by the framework to autoload JSONMessagePartMapping for each of the message parts.
JSONMessagePartMapping
Each node in the above picture may has a corresponding JSONMessagePartMapping. Each node may have its own mapping, with its own values initialized in XMLJSONMessageMappingLoader. In order to get the mapping, first cast MessagePart to a concrete class such as CompositeMessagePartInstance or PrimitiveMessagePart, then use part.getMapping(). The purpose of the mapping is metadata for each node.
JSONMessageFormatter
Used to turn messages into JSON format.
JSONMessageParser
Used to turn JSON into a message.

Algorithm

format(TransferObject tobj, Message message, Output out) Algorithm for format method is as follows

  1. Retrieve the root of the message
  2. Iterate through each of the parts of the root
    1. If the part is a CompositeMessagePartInstance, it is a message node
    2. If the part is a PrimitiveMessagePart, it is a value node
    3. Validate the part against the MessagePartMapping. Different types of validation must be done if the part is a Composite versus if it is a Primitive.
    4. If the part is a CompositeMessagePartInstance, recursively call the format method with TransferObject set to the part. Suggested to overload the format method to format(TransferObject tobj, MessagePart message, Output out) and pass in the part, since retrieving a message root with getRoot() will always get the highest root of the message but what you want is the parent.
    5. If the part is a PrimitiveMessagePart, write the message part to the output.


Project Phases

Phase 1. Research (DONE)

  1. Complete Fundamentals of NexJ Studio tutorial
  2. Complete NexJ Integration tutorial
  3. Install NexJ Studio Express from source

Phase 2. Design Proposal (DONE)

  1. Receive general approval for project
  2. Receive approval for JSON encoding options
  3. Receive approval for JSON formatting options
  4. Final Project Proposal

Phase 3. Create Classes (DONE)

  1. JSONMessagePartMapping
  2. XMLJSONMessageMappingLoader
  3. JSONMessageFormatter
  4. JSONMessageParser
  5. JSONMessageFormatterTest

Phase 4. Internal Code Review (DONE)

  1. Internally review code at CDOT

Phase 5. Code Review 1 (DONE)

  1. August 9, 2011
  2. Code review took place at NexJ with Grace Batumbya, Brian Lim and Andrew Borzenko (NexJ Developer) in attendance.

Phase 6. Apply Changes from Code Review 1

  1. Estimated Duration: 3 Weeks (August 29, 2011)
  2. Changes to make

Summary of changes

  1. Allowing modes for all Composite message parts, not just the root.
  2. Refactoring methods to check for != right condition instead of checking for the wrong condition to throw an exception. (whitelist instead of blacklist)
  3. Formatting all primitives before writing or storing them (e.g. timestamps, decimal numbers.)
  4. Creating a RootJSONMessagePartMapping extending JSONMessagePartMapping

Phase 7. Code Review 2

  • Proposed Date: Week of September 23 October 15

Phase 8. Apply Changes from Code Review 2

Phase 9. Code Review 3

  • Week of November 14

Phase 10. Apply Changes from Code Review 3


Resources

JSON RFC : http://www.ietf.org/rfc/rfc4627.txt
Introduction to NexJ Studio Express : https://www.projects.openhealthtools.org/sf/go/doc1771?nav=1
NexJ Developer's Guide
NexJ Integration Fundamentals
Open Health Tools Integration Platform https://www.projects.openhealthtools.org/sf/projects/oht_aip/