Wide-Area Middleware Links

This is a collection of links related to my research interests

For more DHT related links, click here

 
 Peer-to-Peer & Overlay Networks

Peer-to-Peer Computing. Milojicic, D. S.; Kalogeraki, V.; Lukose, R.; Nagaraja, K.; Pruyne, J.; Richard, B.; Rollins, S.; Xu, Z.; HP Labs Technical Reports HPL-2002-57 20020315. A very interesting paper which describes what we understand for Peer-to-Peer, describing its main goals and different algorithms used. It also includes a survey reviewing the field of P2P systems and applications by summarizing the key concepts and giving an overview of the most important systems. Its intended audience is very heterogeneous, ranging from people unfamiliar with the field to users or developers interested in a comparison of P2P solutions with alternative architectures and models.

Project JXTA. JXTA technology is a set of open protocols that allow any connected device on the network ranging from cell phones and wireless PDAs to PCs and servers to communicate and collaborate in a P2P manner. JXTA peers create a virtual network where any peer can interact with other peers and resources directly even when some of the peers and resources are behind firewalls and NATs or are on different network transports.

Microsoft Windows Peer-to-Peer Networking. Windows Peer-to-Peer Networking is an operating system component that enables the creation of new peer-to-peer (P2P) applications for computers running Windows XP (Service Pack 1 and later). P2P applications can utilize the powerful personal computers that exist at the edge of the Internet to create exciting, distributed experiences. P2P networking can also allow existing applications to work together in new and useful ways.

Pastry. Castro, M.; Druschel, P.; Rowstron, A.; et al. Pastry is a generic, scalable and efficient substrate for P2P applications. Pastry nodes form a decentralized, self-organizing and fault-tolerant overlay network within the Internet. Pastry provides efficient request routing, deterministic object location, and load balancing in an application-independent manner. Furthermore, Pastry provides mechanisms that support and facilitate application-specific object replication, caching, and fault recovery.
(Pastry papers)

Tapestry. Zhao, B. Y.; Kuwiatowicz, J. D.; Joseph, A. D.; et al. Tapestry is an overlay location and routing infrastructure that provides location-independent routing of messages directly to the closest copy of an object or service using only point-to-point links and without centralized resources. The routing and directory information within this infrastructure is purely soft state and easily repaired. Tapestry is self-organizing, fault-resilient, and load-balancing.
(Tapestry papers)

Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications. Stoica, I.; Morris, R.; Karger, D.; Kaashoek, M. F. and Balakrishnan, H. A fundamental problem that confronts peer-to-peer applications is to efficiently locate the node that stores a particular data item. This paper presents Chord, a distributed lookup protocol that addresses this problem. Chord provides support for just one operation: given a key, it maps the key onto a node. Data location can be easily implemented on top of Chord by associating a key with each data item, and storing the key/data item pair at the node to which the key maps. Chord adapts efficiently as nodes join and leave the system, and can answer queries even if the system is continuously changing. Results from theoretical analysis, simulations, and experiments show that Chord is scalable, with communication cost and the state maintained by each node scaling logarithmically with the number of Chord nodes.

The Globe Project. Van Steen, M.; Tanenbaum, A. S. In this research project, a powerful unifying paradigm for the construction of large-scale wide area distributed systems is being looked: distributed shared objects. In their model, the universe consists of a vast number of shared objects, each of which has some associated methods. Authorized users of an object may invoke an object's methods. A method can return the contents of a Web page, but it can also accept an email or news message, look up a name in a worldwide distributed database, access a file or perform an arbitrary other action. Physically, objects are distributed, with active copies on multiple machines at the same time. Objects use peer-to-peer communication: applications load (part of) the object implementation in their address space to participate in the distributed object. Users may contact any copy to have methods performed, but they know nothing about the internal structure and protocols used inside the object. This scheme allows different objects to use different algorithms for data partitioning, replication, consistency, and fault tolerance, in a way transparent to the users.
(Globe papers)

Groove Desktop Collaboration Software. Groove Networks, Inc. Groove is a collaborative P2P system. It is mainly targeted to Internet and intranet users, although it can also be used on mobile devices, such as PDAs, mobile phones and tablets. It is intended to enable communication, content sharing, and tools for joint activities. Groove's main goal, which was P2P by design, was to allow users to communicate directly with other users without relying on a server. Other important goals include security and privacy, and flexibility.

Ben Houston's Peer-to-Peer Idea Page. A very interesting page full of many P2P ideas and links.

P-Grid: The Grid of Peers. P-Grid is a peer-to-peer system based on a virtual distributed search tree: Each peer only holds part of the overall tree, which comes into existence only through the cooperation of individual peers. Every participating peer's position is determined by its path, that is, the binary bit string representing the subset of the tree's overall information that the peer is responsible for.

The Stanford Peers Home Page. The Standford University Peer-to-Peer Research Group.

 Publish / Subscribe and Event Services

The Information Bus - An Architecture for Extensible Distributed Systems. Oki, B.; Pfluegl, M.; Siegel, A.; Skeen, D. Research can rarely be performed on large-scale, distributed systems at the level of thousands of workstations. In this paper, we describe the motivating constraints, design principles, and architecture for an extensible, distributed system operating in such an environment. The constraints include continuous operation, dynamic system evolution, and integration with extant systems. The Information Bus, is a novel synthesis of four design principles: core communication protocols have minimal semantics, objects are self-describing, types can be dynamically defined, and communication is anonymous. A must-read and very interesting paper.

The Many Faces of Publish/Subscribe. Eugster, P. Th.; Felber, P.; Guerraoui, R. and Kermarrec, A.-M. The publish/subscribe communication paradigm has recently received an increasing attention. With systems based on that paradigm, subscribers register their interest in an event, or a pattern of events, and are subsequently asynchronously notified of events fired by publishers. Many variants of the paradigm have recently been promoted, each variant being specifically adapted to some given application or network model. This paper factors out the common denominator underlying these variants: full decoupling of the communication participants in time, space and flow. These decoupling dimensions are used to better identify commonalities and divergences with traditional interaction paradigms. The many existing variations on the theme of publish/subscribe are classified and synthesized. In particular, their respective benefits and shortcomings are discussed both in terms of interfaces and implementations.

CORBA Notification Service v1.0.1. Object Management Group. A CORBA-based Notification Service, a service which extends the existing OMG Event Service, adding to it many new capabilities, including many publish/subscribe features. The defined Notification Service attempts to preserve all of the semantics specified for the OMG Event Service, allowing for interoperability between basic Event Service clients and Notification Service clients.

Elvin Notification Service. Segall, B.; Arnold, D.; Henderson, M.; Phelps, T. and Roy, C. Decoupling the production and consumption of information in software systems facilitates extensibility by removing explicit dependecies between components. So called "publish/subscribe" notification architectures are comprised of undirected production, and subscription to events by their characteristics rather than their source. Elvin is a notification/messaging service with a difference; rather than messages being directed by applications from one component to another, Elvin messages are routed to one or more locations, based entirely on the content of each message.
(Elvin papers)

Hermes: A Distributed Event-Based Middleware Architecture. Pietzuch, P. R.; Bacon, J. M. There is a need for an event-based middleware to build large-scale distributed systems. Existing publish/subscribe systems still have limitations compared to invocation-based middlewares. Hermes, a novel event-based distributed middleware architecture follows a type- and attribute-based publish/subscribe model. It centres around the notion of an event type and supports features commonly known from object-oriented languages like type hierarchies and super-type subscriptions. A scalable routing algorithm using an overlay routing network is presented that avoids global broadcasts by creating rendezvous nodes. Fault-tolerance mechanisms that can cope with different kinds of failures in the middleware are integrated with the routing algorithm resulting in a scalable and robust system.

The NaradaBrokering Project (iNtegrated Asynchronous Real-time Adaptive Distributed Architecture). Fox, G.; Pallickara, S.; et al. This project aims to provide a unified messaging environment that integrates Grid Services, JMS and JXTA. NaradaBrokering is an event brokering system designed to run on a large network of cooperating broker nodes. NaradaBrokering is based on the distributed publish/subscribe paradigm and also supports the flexibility of centralized interactions as well as peer-to-peer (P2P) interactions over the edge. NaradaBrokering is JMS compliant and also supports replacing centralized single server systems with a distributed brokering solution. NaradaBrokering also provides support for JXTA interactions. More recently, NaradaBrokering has been used to provide support for raw RTP clients.
(NaradaBrokering papers)

Scribe. Castro, M.; Druschel, P.; Kermarrec, A.-M. and Rowstron, A. SCRIBE is a generic, scalable and efficient group communication and event notification system. It provides application level multicast and anycast. Scribe is efficient, self-organizing, flexible, highly scalable and supports highly dynamic groups. It is built on top of Pastry, a generic, scalable, self-organizing substrate for peer-to-peer applications.

Siena (Scalable Internet Event Notification Architectures). Carzaniga, A.; Wolf, A. L.; et al. Siena is a research project aimed at designing and constructing a generic scalable publish/subscribe event-notification service. The technical basis of Siena is an innovative type of network service called content-based networking.

xEvents. Slominski, A.; Govindaraju, M.; Gannon, D. This paper presents a SOAP-based event system for Grid events, which in turn aid in wiring together distributed software components. SOAP RPC specifies HTTP as its network protocol and XML as the data format; representation of events using XML allows self-describing formats using XML-Schemas, and language and platform independence. SOAP events are defined as a specification of interfaces and wire-formats compliant with the SOAP 1.1 specification. The advanced features (directory services, firewall friendliness, security considerations and failsafe mechanisms) that event systems should support are described.

 Grid Computing

Active Reliable Multicast Services for Computational Grids. Pham, C.; Maimour, M.; Bouhafs, F. In active networking, routers themselves play an active role by executing application-dependent services on incoming packets. Recently, the use of active network concepts where routers themselves could contribute to enhance the network services by customized functionalities have been proposed in the multicast research community and can be very beneficial to the grid community. Contributing mainly on feedback implosion problems, retransmission scoping and cache of data, these active reliable multicast protocols open new perspectives for achieving high throughput and low latency on wide-area networks. In this project, the benefits a computing grid can draw from an underlying active reliable multicast service are investigated. The Dynamic Replier Active Reliable Multicast protocol for reducing the end-to-end latency is proposed.

Butterfly Plaform for Online Games. They have presented a fully-distributed server technology that uses open grid computing protocols in large-scale immersive game networks that support unlimited numbers of players and require the most demanding levels of service. As of February 2003 it has been announced that IBM and Butterfly.net have signed a a PlayStation®2 tools and middleware agreement with Sony Computer Entertainment, Inc. to provide technology and networking services for the PlayStation®2 computer entertainment system.

The DataGrid Project. DataGrid is a project funded by European Union. The objective is to build the next generation computing infrastructure providing intensive computation and analysis of shared large-scale databases, from hundreds of TeraBytes to PetaBytes, across widely distributed scientific communities.

The Globus Project. The Globus Project is developing fundamental technologies needed to build computational grids. Grids are persistent environments that enable software applications to integrate instruments, displays, computational and informational resources that are managed by diverse organizations in widespread locations.
(Globus papers)

Global Grid Forum. The Global Grid Forum (GGF) is a community-initiated forum of 5000+ individual researchers and practitioners working on distributed computing, or "grid" technologies. GGF's primary objective is to promote and support the development, deployment, and implementation of Grid technologies and applications via the creation and documentation of "best practices" - technical specifications, user experiences, and implementation guidelines. GGF efforts areas are also aimed at the development of a broadly based Integrated Grid Architecture that can serve to guide the research, development, and deployment activities of the emerging Grid communities. Defining such an architecture will advance the Grid agenda through the broad deployment and adoption of fundamental basic services and by sharing code among different applications with common requirements. Wide-area distributed computing, or "grid" technologies, provide the foundation to a number of large-scale efforts utilizing the global Internet to build distributed computing and communications infrastructures. As common Grid services and interoperable components emerge, the difficulty in undertaking these large-scale efforts will be greatly reduced and, as importantly, the resulting systems will better support interoperation.

LEGION: Worldwide Virtual Computer. Legion, an object-based metasystems software project at the University of Virginia, is designed for a system of millions of hosts and trillions of objects tied together with high-speed links. Users working on their home machines see the illusion of a single computer, with access to all kinds of data and physical resources, such as digital libraries, physical simulations, cameras, linear accelerators, and video streams. Groups of users can construct shared virtual work spaces, to collaborate research and exchange information. This abstraction springs from Legion's transparent scheduling, data management, fault tolerance, site autonomy, and a wide range of security options.

 Wide-Area Name Services

Domain Name System (DNS). Domain name servers translate domain names to IP addresses. That sounds like a simple task, and it would be -- except for five things:

  • There are billions of IP addresses currently in use, and most machines have a human-readable name as well

  • There are many billions of DNS requests made every day. A single person can easily make a hundred or more DNS requests a day, and there are hundreds of millions of people and machines using the Internet daily
  • Domain names and IP addresses change daily
  • New domain names get created daily
  • Millions of people do the work to change and add domain names and IP addresses every day

The DNS system is a database, and no other database on the planet gets this many requests. No other database on the planet has millions of people changing it every day, either. That is what makes the DNS system so unique! You can get more information here.

Global Name Service (GNS). Lampson, B. W. et al. The GNS database is composed of a tree of directories. Each directory has a unique directory identifier (DI). Each directory contains a list of names and references. One important thing to note is that leaves in this structures are organised into value trees.

X.500 Directory Service. As for the other name services the data stored in X.500 servers is organized in a tree structure with named nodes. However a wide range of attributes is stored at each node and acess is not just by name but also by attribute combinations. The X.500 name tree is called the Directory Information tree (DIT) and the entire structure is called the Directory Information Base (DIB). Servers are called Directory Service Agents (DSA) and their clients are called Direcory User Agents (DUA). The name of a DIB entry that determines its position in the DIT tree is determined by selecting one or more of its attributes as distingushed attributes.

 Related Technologies

Instant Messaging. Allows you to maintain a list of people that you wish to interact with. You can send messages to any of the people in your list, often called a buddy list or contact list, as long as that person is online. Sending a message opens up a small window where you and your friend can type in messages that both of you can see.

  • Instant Messaging and Presence Protocol (IMPP) Working Group. This working group will eventually define protocols and data formats necessary to build an internet-scale end-user presence awareness, notification and instant messaging system.  Its initial task is to determine specific design goals and requirements for such a service. The design goals document will be submitted for IETF-wide review, and based on that review, the group's charter will be extended.

  • Instant Messaging and Presence Protocol (IMPP) Information Homepage. The official IMPP information page.

  • ICQ. Instant messaging really exploded on the Internet scene in November 1996. That's when Mirablis, a company founded by four Israeli programmers, introduced ICQ, a free instant-messaging utility that anyone could use.

  • Yahoo! Messenger. The Instant Messaging system from Yahoo!

  • MSN Messenger. The Instant Messaging system from Microsoft.

  • Cerulean Studios' Trillian. A must download piece of software which allows you to connect to ICQ, AIM, MSN Messenger, Yahoo! Messenger and IRC in a single, sleek and slim interface. Forget having multiple IM programs running on your computer: with Trillian you can have access to all their networks using only one program!

  • Jabber. Jabber is an open XML protocol for the real-time exchange of messages and presence between any two points on the Internet. The first application of Jabber technology is an asynchronous, extensible instant messaging platform, and an IM network that offers functionality similar to legacy IM systems such as AIM, ICQ, MSN, and Yahoo.

Peer-to-Peer software. Many variations exist; however file sharing and distributed computing applications seem to have gained the battle of popularity.

  • SETI@home. SETI@home is a scientific experiment that uses Internet-connected computers in the Search for Extraterrestrial Intelligence (SETI). You can participate by running a free program that downloads and analyzes radio telescope data.

  • United Devices Grid Computing Projects. The United Devices Cancer Research Project is asking you to volunteer your PC to help process molecular research being conducted by the Department of Chemistry at the University of Oxford in England and the National Foundation for Cancer Research. To participate, you simply download a very small, no cost, non-invasive software program that works like a screensaver: it runs when your computer isn't being used, and processes research until you need your machine. Your computer never leaves your desk, and the project never interrupts your usual PC use. Apart from the Cancer Research Project, there currently also exist the Smallpox Research Grid and the PatriotGrid.

  • eDonkey2000. eDonkey is a File Sharing program. This means that it will give you the opportunity to share files you have on your computer with other users on the network, and you will in turn be able to download files of them.

  • Overnet. Overnet is an application that allows you to share files with millions of other people across the globe. There are a lot of file sharing programs out there, Overnet is a little different from each of them. Unlike several programs (most notably the now defunct Napster) Overnet does not have one central server. There are in fact no servers, only clients. Each client starts up, finds other clients on the network, and communicates with them. Unlike other 'serverless' programs, all clients are equal, each communicating, sharing and assisting each other.

  • eMule. eMule is a new filesharing client which is based on the eDonkey2000 network, but offers more features than the standard eDonkey client, because it's opensource but under the restrictions of the GPL License.

  • Kazaa Lite. Kazaa Lite (sometimes called 'K-Lite') is a Peer-to-Peer File Sharing application that does not contain any 'Spyware'. In actual fact, it is a slightly modified version of the very popular Kazaa Media Desktop (often called the 'Original Kazaa' or 'KMD'), which is made by Sharman Networks. Both programs are very similar, with almost identical features and appearance. The main difference is that Kazaa Lite has been modified so that it does not have any of the so-called 'Spyware' and 'AdWare' that comes with the Original Kazaa.

Multiagent Systems.

Miscellaneous.

  • Network News Transfer Protocol (NNTP). NNTP is a TCP/IP protocol based upon text strings sent bidirectionally over 7 ASCII TCP channels. It is used to transfer articles between servers as well as to read and post articles. Articles are transmitted in the form specified by RFC1036 whereas the NNTP protocol itself is defined in RFC977.

  • JavaSim. JavaSim is a component-based, compositional simulation environment. It has been built upon the notion of the autonomous component programming model. Similar to COM/COM+, JavaBeansTM, or CORBA, the basic entity in JavaSim is components, but unlike the other component-based software packages/standards, components in JavaSim are autonomous and are realization of software ICs For the purpose of network modeling and simulation, we have defined, on top of the autonomous component architecture, a generalized packet switched network model.  The model defines the generic structure of a node (either an end host or a router) and the generic network components, both of which  can then be used as base classes to implement protocols across various layers.  Although the model is derived by featuring out the common attributes of network entities in the current best-effort Internet, it is general enough to accommodate other network architectures, such as the IETF differentated services architecture, the mobile wireless network architecture, and the WDM-based optical network architecture.

 This page was last updated on Tuesday, 18th November, 2003. Carles Pairot - <carles.pairot@urv.net>