simple is better

JSON-RPC 2.0 Transport: Sockets

Status: proposal/draft
Date: 2013-05-03
Author: Roland Koebler <rk at simple-is-better dot org>

Using sockets (TCP, UDP or Unix Domain Sockets) with JSON-RPC is simple: Simply send and receive the JSON-RPC strings. The only problem is to detect when a Request/Response is complete and where the next Request/Response starts. Here, three ways are presented:

1   shutdown/close after every Request/Response

Use a new connection for every Request/Response.

The client MUST shutdown the writing (SHUT_WR) to signal the end of the Request. The server MUST close the connection to signal the end of the Response.

This is simple, straight-forward and robust. But it may be slow, since every Request needs a new connection.

Client-side:

  1. open a socket, connect
  2. send Request, shutdown socket-writing
  3. receive Response (=receive until close/shutdown)
  4. close socket
  5. goto 1

Server-side:

  1. open socket, bind, listen
  2. accept a connection
  3. receive Request (=receive until shutdown)
  4. send Response
  5. close connection
  6. goto 1

2   Netstrings

See:

Encode JSON-RPC objects in netstrings. Since netstrings are a self-delimiting encoding, a single connection can be used for several Requests/Responses.

But note that the server may close a connection e.g. on error or after a timeout, so use appropriate error-handling and always use a new connection as fallback.

Example:

60:{"jsonrpc": "2.0", "method": "first", "params": 42, "id": 1},66:{"jsonrpc": "2.0", "method": "second", "params": [23, 7], "id": 2},

3   pipelined Requests/Responses / JSON-splitter

It's also possible to use a single connection for several Requests/Responses without an additional encoding. Then, (a) a streaming JSON-decoder or (b) a streaming JSON-splitter is needed, which detects the end of a JSON-object/array. Then, several JSON-RPC Requests can simply be pipelined on a single connection.

Streaming JSON-decoders seem to be rare, but a JSON-splitter which only detects the end of a JSON-object/array can easily be implemented (see below).

But note that the server may close a connection e.g. on error or after a timeout, so use appropriate error-handling and always use a new connection as fallback.

JSON-splitter example in Python 2 (should be easily transferable to other languages):

def json_split(s, start=0):
    r"""Extract 1st JSON-object/array from a string.

    :Returns:   (EXTRACTED_JSON_STRING_or_None, i_remainder)
    :Raises:    ValueError

    :Example::

        >>> stream = '{"first": "object", "data": "x"} {"second": "object", "data": "y"} ["third", "array"]["fourth", "array"]["incomplete", "arr'
        >>> i=0
        >>> while True:
        ...     j, i = json_split(stream, i)
        ...     if j is None:
        ...         break
        ...     print j
        {"first": "object", "data": "x"}
        {"second": "object", "data": "y"}
        ["third", "array"]
        ["fourth", "array"]
        >>> print stream[i:]
        ["incomplete", "arr

        >>> stream = '{"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \\" [ { y"}' * 5
        >>> i=0
        >>> while True:
        ...     j, i = json_split(stream, i)
        ...     if j is None:
        ...         break
        ...     print j
        {"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
        {"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
        {"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
        {"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
        {"a": "b", "1": 2, "c": {"1": [1, 2], "3": [{"d": ["}"]}], "2": {"3": 4}}, "xy": "x ] } \" [ { y"}
    """
    state = 0   # 0=search start, 1=array,  11=string inside array
                #                 2=object, 12=string inside object
    i = start   # current position in string
    b = i       # begin of string-part
    depth = 0   # array/object-depth-counter

    while i < len(s):
        # find start of JSON-array or object
        if 0 == state:
            if s[i] == '[':
                state = 1
                b = i
                depth = 1
            elif s[i] == '{':
                state = 2
                b = i
                depth = 1
            elif s[i].isspace():    # ignore whitespace
                pass
            else:
                raise ValueError("Invalid character '%s' at %d, expected '[' or '{'." % (s[i], i))
        # skip string
        elif state > 10:
            if s[i] == '\\':    #   skip char after \
                i += 1
            elif s[i] == '"':   #   end of string
                state -= 10
        # inside array
        elif 1 == state:
            if s[i] == '"':
                state += 10
            elif s[i] == '[':
                depth += 1
            elif s[i] == ']':
                depth -= 1
                if depth == 0:
                    return s[b:i+1], i+1
        # inside object
        elif 2 == state:
            if s[i] == '"':
                state += 10
            elif s[i] == '{':
                depth += 1
            elif s[i] == '}':
                depth -= 1
                if depth == 0:
                    return s[b:i+1], i+1

        i += 1
    return None, b

View document source.