Idempotent Publish/Subscribe Messaging Environment

Rather than a framework or standardized protocol, IPSME is a set of conventions that forms an architecture where any participant to talk to any other participant, without need for a central authority and without standardization, provided groups of participants speak the same protocol.

Conventions

  • A messaging environment (ME) is defined as:

    • A pubsub system which receives messages and relays those messages to all subscribed participants;

    • Messages must be idempotent or identifiable as duplicates.

  • Each participant:

    • sends and receives messages in a (local) ME;

    • simply ignores messages, if not understood.

  • Translation of messages is done by having a participant listen to messages on a (local) ME and sending out translated messages.

  • Communication across ME boundaries is through a reflector pair: a participant listener with a counterpart in another ME, which resends messages there. Communication between reflectors is left unspecified and completely up to the author(s) of the reflectors, as is the selection of messages to resend.

Messaging Environment (ME)

A local ME employs the usage of a readily available pubsub resource. On the various operating systems, there is usually a platform- specific messaging system for inter-process communication (IPC) where pubsub can be used e.g., NSNotificationCenter on macOS/iOS and MSMQ on Windows. If a platform-specific messaging system is not available, any other available pubsub can be used, as long as all participants that are to be local to that ME, know how to access the ME. If more than one pubsub is utilized on a platform and they are to interact, they must be interconnected by a pair of reflectors.

Expecting all prospective participants to be connected to the same ME is impractical; not all prospective participants would easily route to a single ME and a single ME would certainly be overloaded. Participants can be divided (i.e., into regions) using multiple MEs. IPSME specifies participants should connect to a local ME, but places no limitation on the number of MEs or the topology organization thereof. IPSME specifies reflectors for communication across ME boundaries, and it is this communication that allows MEs to be connected and organized into a general graph topology.

Messages

Because pubsub is a broadcast system, messages in IPSME are required to be idempotent, so as to promote asynchronous communication, by reducing the amount of acknowledges required, and that identifiable duplicates can be eliminated. If messages are passed across multiple MEs a universally unique identifier (e.g., UUID or GUID) can be employed to help achieve idempotence. Idempotent messages can be processed multiple times by a participant, but the processing of each message must give the same result after the application of the initial message. Because messages are broadcast using pubsub, messages are the implicit invocation of potentially multiple participants. This does add more complexity to the sender, since the sender must handle zero or multiple responses e.g., pick the best response or reply to all the responses within a given timeframe.

Rather than trying to obtain interoperability by enforcing a predefined structure or data model in messages, IPSME does not specify a format for message content i.e., it is possible to use strings, binary or any of the topic-, content- or type-based pubsub schemes. By not specifying message content IPSME avoids having a predetermined expressiveness, perhaps having many simultaneously. Participants send messages in their own protocol and only participants that understand those messages will be able to process them i.e., partitioning the semantic and syntactic space into any number of separate spaces. One of the main tenets that enables interoperability for IPSME is the interest management of each participant; if a participant does not understand a message, it simply drops the message and continues processing. This can lead to scenarios where certain participants might want affirmation that another participant has received the message. It is possible to send return messages (e.g., Remote Procedure Calls or Acknowledges) through IPSME, but such a return is not explicitly defined in the IPSME specification.

Translators

To broaden the set of participants that understand a message, specialized participants that translate messages can be inserted in to a ME. These translators listen for messages that adhere to a certain protocol, translate them to a different protocol (i.e., a mapping) and send out the translation; translating participants are considered mediators between other participants.

Between all participants of a local ME and through to other MEs via reflectors, translations have a network effect. Translations are transitively applicable to other participants e.g., if a translation 𝑋 translates from participant 𝐴 to 𝐵, then any participant 𝐶, that can communicate with 𝐴, can also communicate with 𝐵, via 𝑋. The network effect of translations has the potential to reduce the complexity of integrating 𝑛 participant nodes from an exponential 𝑛(𝑛 − 1)/2, to a linear (𝑛 − 1).

Masks

Masks are a sub class of translators that have a slightly different role than just accepting, translate and pushing messages back to a ME. Some APIs are directly compatible with IPSME and so a mask can be used to speak directly to the API. The mask listens for incomming messages from the ME, translates them to the API of system being integrated and calls the API directly. Responses from the API are received by the mask, translated and then pushed out to the ME.

Reflectors

A reflector is a participant in a ME that listens and filters for particular messages that should be routed to another ME. The reflector communicates directly with a reflector (i.e., its counterpart) in the other ME. Upon receiving a message, the counterpart publishes the message to its local ME; participants of that local ME will receive messages from the distant ME transparently i.e., without being aware of any communication complexity thereof. Communication between two reflectors is left as undefined and is completely up to the author(s) of those participants. A reply message is routed back through reflectors in a reverse fashion. Similar to how adding translators adds functionality without changing the existing system, reflectors can also be inserted into a ME without changing existing implementations; to the participants using a reflector the ME is simply expanded.