Automatic Path Creation Service - APC

This document explains the architecture of the APC (Automatic Path Creation Service), which is a component within ICEBERG to establish data flows between heterogeneous communication endpoints by composing appropriate transoding operators. Call agents of the ICEBERG artchitecture make requests to the APC service for data path creation by providing the required endpoint information. APC service provides a clean separation of the data paths from the control paths by encapsulating the data path creation and instantiation process.


Table of Contents


Architecture of APC:

Terminology:

This section describes the terminology used in the following sections.

Operators

An operator is a unit of computation on some data. operators are strongly-typed: they have a clear definition of the input and output types. In addition, operators have various attributes such as communication protocol, computational requirement, static data input (e.g. a database) etc. Only based on these attributes, is automatic composition of services possible.
Operators can have multiple inputs and outputs (i.e. aggregators, broadcasters). Communication using multicast is one way to deal with this.
Operators have only soft-state: this means that if an operator fails and is restarted, there is no need to recover its state information. Application level protocol is needed to provide guarantees such as reliability, ordered, end-to-end data delivery.

Connectors

A connector is an abstraction of the Application Data Unit (ADU) transport mechanism between two operators (e.g, RTP connector, UDP connector, etc.) Each connector is characterized by a specific transport protocol.

Overview:

APC service is responsible for creating, maintaining, and eventually freeing up data paths consisting of operators strung together by connectors. The overall path construction process can be illustrated through the following diagram:

It is a continuous process of optimization.

Implementation:

This section describes some of the interesting implementation details and functionalities provided by APC in the current release.

Simplifications of the design:

For the purpose of prototyping, we made a number of simplifications to the above ideal design:

APC service:

The APC service itself is a Ninja iSpace cluster service. The idea is that multiple nodes will implement the APC service, providing fault-tolerance and load-balancing features. This will require the sharing of data among the nodes. Thus, in this release, we only consider a single-node APC service.

Connection Manger:

This is also an iSpace service that must be implemented by all nodes on which operators will run. Connection Manager has an API that deals with loading, creating, repairing, destroying operators. It is also reponsible for creating and maintaining connections from or to the operators running on the node where the connection manager resides. A connector consists of two objects, each one is part of the two operators in communication. One part is a writer object; the other is a reader object.
For simplicity, the operators created by each connection manager have a process per usage model: operators are not shared among different path instances. Operators are created on demand for each new path instantiated and destroyed after the path is requested to be torn down.
The physical paths created by the APC service always contain the source endpoint operator. In certain cases, it does not make sense to have such an operator; thus, it can be a null operator. This is provisioned by a dummy operator and a dummy connection manager implementing the connection manager interface.

Partial Path Repair (PPR):

At any given time, any operator can fail, or the connection between two operators can be broken. In this release, we support partial path recovery: rather than tearing down and restarting the entire path, failing operators are restarted on new nodes in order to introduce minimal disturbance at the end users.
Failure is discovered by catching the I/O exception at each operator from its neighboring communicating operators. The failing operator is identified by the APC service and restarted at a different node.

Existing implementation of the types of paths:

from MP3 to gsm
from gsm/pcm to pcm/gsm
from text to gsm/pcm
from pcm/gsm to text


API to APC explained:

The API to APC is rather simple and consists only of the following five methods:


How to add operators and connectors to the APC package:


How to run the APC service:

In the iSpace configuration file, which is the file passed as argument to ninja.iSpace.Main, one needs to specify this entry to run the APC service:
iceberg.APCpath.services.APC    host_file_name
The host_file_name contains the name of the hosts available to run operators. Each host name is separated by a new line; an example can be found here.
On each of the host specified in the host_file_name, a connection manager service or a service that implements the connection manager interface needs to be run. For example, one of the following entries needs to be in the iSpace configuration file on each of those hosts.
iceberg.APCpath.services.ConnectorMgr
iceberg.APCpath.services.DummyCM
iceberg.APCpath.services.VatCM
iceberg.APCpath.services.GsmCM


Known caveats/limitations of APC:

This section documents the known caveats/limitations of APC. Due to time limitations, they are not fixed, but do not affect the functionality of the rest of ICEBERG components. In the next release, these bugs will definitely go away.

Javadoc to APC:


Z. Morley Mao, zmao@cs.berkeley.edu
Last modified: Thu Jun 8 17:29:48 PDT 2000