Peer to peer networking using Windows Communication Foundation (WCF) Peer Channels and the Peer Mesh (C# .Net)

11/17/2008 01:55:46

What Is P2P? Peer to peer communication became somewhat notorious through its use in the sharing of materials of dubious origins online. File sharing networks like Napster, eDonkey and Kazaa made use of decentralised peer to peer communication, but there's far more to peer to peer networking than file sharing. Essentially, Peer to peer is a method of exchanging data between two points, without the use of a central server. Wikipedia description here (makes for good reading).

Uses For P2P Peer to peer networking is a great way to build decoupled message-based distributed systems. It's a great way of sharing the available amount of bandwidth between a series of nodes effectively, and it's a great way to produce push-based systems.

Some obvious simple uses of p2p communication would be file sharing networks, chat networks, or large file distribution networks (e.g. Valve Software’s' Steam platform).

In business, I've implemented p2p in place of solutions that were traditionally patched together using email (small notification clients that monitor other applications that send peer messages being a specific example) and p2p is ideal for these kinds of usages.

Reliability

P2P differs from traditional sockets programming in the way that it isn't deemed a reliable communication channel. Without your own additional logic the messages you send cannot be guaranteed to arrive, nor should they be, as peer communication is built around a "fire and forget" model. When designing applications that use p2p, it's important that you deal with failure conditions and program defensively. Presume messages will be lost, however unlikely that is to actually happen.

P2P in .NET 3.0/5 - The WCF Peer Channel

In .Net 3.0, Microsoft introduced the Windows Communication Foundation, a new unified communications model for .Net developers. The WCF Peer Channel was provided with the first release of WCF for developers to use. The WCF Peer channel is quite easy to understand as an abstraction. It features two main components; the peer resolver and peer participants.

The Peer Resolver

The peer resolver is the registration server for the mesh. Whilst in principle p2p is decentralised, there needs to be a known address which a new node uses to connect to other peers. The resolver keeps track of all node registrations, when a node connects, the resolver supplies the new node with the peer addresses of its nearest neighbours. The node maintains an internal database of nodes, and (implementation dependant) often requires nodes to phone home at a set interval in order to advertise their membership of the mesh.

Participants

Each and every node on the mesh is a participant. In traditional sockets programming there's a very strong concept of connected and disconnected; not so with p2p connections. A participant is online if it can "see" other nodes. A participant joins the network by registering with a resolver. The resolver then supplies the participant with a number of peer addresses that the participant then attempts to establish connections to. When peers are found, they exchange and relay messages across the mesh.

Working examples

The rest of this piece is concerned with implementing a peer to peer network in C#, along with providing some useful helper libraries, some tests and a production-ready peer resolver.

The Persistent Peer Mesh Resolver

The Microsoft SDK provides a basic peer mesh resolver example. Unfortunately, this resolver is stateless and as such if the resolver is restarted or crashes for any reason it looses its registration database which leads to something akin to an IRC net split if peers register with the resolver before and after a restart. I've extended the default Customer Peer Mesh Resolver so that it backs up its registration database to disk and reloads any active node registrations on restart.

The bulk of the work is done by the PersistentCustomResolver class. It's accessible only as a thread safe singleton and manages the instantiation of a StatefulResolverService (which is an extended instance of CustomPeerResolverService from System.ServiceModel.PeerResolvers) and deals with opening a WCF endpoint for listening for new peer registrations. The StatefulResolverService manages a PersistingRegistrationCache which backs up new peer registrations when they occur or are refreshed.

The Persistent peer mesh resolver can ultimately be used very simply in either a Windows service or a stand alone application.

Usage example:

class Program { public static void Main() { PersistentCustomResolver.Instance.Listen();

Console.WriteLine("Custom resolver service is started"); Console.WriteLine("CleanupInterval: {0}", PersistentCustomResolver.Instance.CleanupInterval); Console.WriteLine("RefreshInterval: {0}", PersistentCustomResolver.Instance.RefreshInterval);

Console.WriteLine("Press <ENTER> to terminate service"); Console.ReadLine();

PersistentCustomResolver.Instance.StopListening(); } }

The resolver has two public properties, one of them (CleanupInterval) defines the length of time between the purging of inactive nodes, and the other (RefreshInterval) defines time period in which the resolver expects that active nodes will refresh their registration information.

In addition to the simple usages example above, a not insignificant amount of WCF configuration has to take place in App.config in order to allow the framework to listen correctly.

<?xml version="1.0" encoding="utf-8" ?> <configuration> <system.serviceModel> <services> <service name="Resolver.StatefulResolverService"> <host> <baseAddresses> <add baseAddress="net.tcp://localhost/peerResolverService" /> </baseAddresses> </host> <endpoint address="net.tcp://localhost/peerResolverService" binding="netTcpBinding" bindingConfiguration="PeerResolverBinding" contract="System.ServiceModel.PeerResolvers.IPeerResolverContract" /> </service> </services> <bindings> <netTcpBinding> <binding name="PeerResolverBinding" transferMode="Buffered" transactionProtocol="OleTransactions" hostNameComparisonMode="StrongWildcard" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:10:00" transactionFlow="false" maxBufferPoolSize="50000000" maxBufferSize="268435455" maxReceivedMessageSize="268435455" maxConnections="10000" > <readerQuotas maxDepth="50000000" maxStringContentLength="50000000" maxArrayLength="50000000" maxBytesPerRead="50000000" maxNameTableCharCount="50000000" /> <security mode="None"/> </binding> </netTcpBinding> </bindings> </system.serviceModel> </configuration>

The above configuration tells the resolver to listen using the endpoint net.tcp://localhost/peerResolverService using the standard peer resolver contract. Participants will need to be aware of this endpoint to register on the mesh. The net.tcp binding configuration has some very large configuration values set that would likely be revised for a production system.

Joining a Peer Mesh

Connecting to a peer mesh requires two things; an active peer mesh connection, and a peer message processor. The peer connection acts as the communication channel and the message processor instance responds to any peer events received on that connection. I've encapsulated this into the class PeerMeshManager. The PeerMeshManager class simplifies joining a mesh, leaving a mesh, and sending a message to the mesh. It's left to the individual implementation to instantiate a Message processor and set the public property of the PeerMeshManager appropriately, allowing the application to respond to events on the mesh. See the following example:

Console.Write("Connecting to Peer Mesh...");

PeerMessageProcessor messageProcessor = new PeerMessageProcessor(); messageProcessor.PeerEventRaised += messageProcessor_PeerEventRaised; PeerMeshManager.Instance.MessageProcessor = messageProcessor; PeerMeshManager.Instance.ConnectedToPeers += Instance_ConnectedToPeers; PeerMeshManager.Instance.DisconnectedFromPeers += Instance_DisconnectedFromPeers; PeerMeshManager.Instance.JoinNetwork();

The PeerMessageProcessor makes a handful of events available to the application:

public event PeerEventHandler PeerEventRaised; public event UnhandledPeerEventHandler UnhandledPeerEventRaised;

public event ConnectedHandler ConnectedPeerEventRaised; public event DisconnectedHandler DisconnectedPeerEventRaised; public event RequestPeerAnnounceHandler RequestPeerAnnouncePeerEventRaised;

The first two events are the more interesting...

For each peer message that is sent, PeerEventRaised will be fired.
If an even of a specific type is fired (say ConnectedPeerEventRaised) and there is no listener wired up to ConnectedPeerEventRaised, then UnhandledPeerEventRaised will fire for that event.
The specific event handlers will fire if a peer event of that type occurs on the mesh.

If you were to expand the provided source code with your own peer events, I'd recommend adding custom events for each peer event, and maintaining the above pattern (always fire a PeerEventRaised event, for every event, regardless, always fire a specific event, and always fall back to firing an UnhandledPeerEventRaised event) for consistency.

I've provided a few helper classes that the PeerMeshConnection uses, however I'll skip explaining them in full. I've provided two working peer applications that talk to each other. One is a listener that listens and outputs peer messages, the other sends them in bulk. Feel free to experiment. The sample listener looks like this:

class Program { static void Main() { Console.Write("Connecting to Peer Mesh...");

var messageProcessor = new PeerMessageProcessor(); messageProcessor.PeerEventRaised += messageProcessor_PeerEventRaised; PeerMeshManager.Instance.MessageProcessor = messageProcessor; PeerMeshManager.Instance.ConnectedToPeers += Instance_ConnectedToPeers; PeerMeshManager.Instance.DisconnectedFromPeers += Instance_DisconnectedFromPeers; PeerMeshManager.Instance.JoinNetwork();

Console.Write("Done.");

Console.ReadLine(); }

private static int _recievedCount = 0; static void messageProcessor_PeerEventRaised(PeerEvent e) { Console.Clear(); Console.WriteLine("Rcvd {0} messages.", _recievedCount); Console.WriteLine("Recieved: {0}", e.UniqueIdentifier); Console.WriteLine("App: {0}", e.Application); Console.WriteLine("From: {0}", e.UserId); Console.WriteLine("At: {0})", e.Timestamp); _recievedCount++; }

private static void Instance_ConnectedToPeers(object sender, EventArgs e) { Console.WriteLine("Connected to peers."); }

private static void Instance_DisconnectedFromPeers(object sender, EventArgs e) { Console.WriteLine("Disconnected from peers."); } }

and requires the configuration...

<?xml version="1.0" encoding="utf-8" ?> <configuration>

<system.serviceModel>

</system.serviceModel>

</configuration>

This configuration should allow the application to register with the persistent resolver provided and join a mesh.

The second application requires exactly the same configuration and looks like this:

class Program { static void Main() { Console.Write("Connecting to Peer Mesh...");

if(string.IsNullOrEmpty(input)) { input = "y"; } }

Console.ReadLine(); } private static void Instance_ConnectedToPeers(object sender, EventArgs e) { Console.WriteLine("Connected to peers."); }

private static void Instance_DisconnectedFromPeers(object sender, EventArgs e) { Console.WriteLine("Disconnected from peers."); }

private static int _recievedCount = 0; static void messageProcessor_PeerEventRaised(PeerEvent e) { //Console.WriteLine("Recieved: {0} from: {1}", e.UniqueIdentifier, e.UserId); _recievedCount++; }

private static void DoMessages() { const int defaultNumberOfMessages = 1000;

Console.Write("Done."); Console.WriteLine(); Console.Write("Number of messages to send (defaults to {0})? ", defaultNumberOfMessages); string upperString = Console.ReadLine();

if (string.IsNullOrEmpty(upperString))

int upper = Int32.Parse(upperString);

Console.WriteLine("Sending {0} messages...", upper);

for (int i = 1; i <= upper; i++) { var peerEvent = new PeerEvent(); PeerMeshManager.Instance.SendMessage(peerEvent); //Console.WriteLine("Sent: {0}, GUID: {1}", i, peerEvent.UniqueIdentifier); }

Console.WriteLine("Sent {0} messages.", upper); Console.WriteLine("Rcvd {0} messages.", _recievedCount); }

In order to run the provided code, you need .Net 3.0+ (I'd recommend 3.5 SP1). First, compile and run the resolver and leave it running. Then run the listener application, followed by the PeerMesh sample application. You should (firewall permitting) be able to send messages in bulk between the two applications over the peer channel. For further tests, you could run the peer clients on different machines (remembering to change the localhost address in the app.config for the resolver) or multiple instances of the client on one/many machines.

I've also provided an additional project containing MbUnit tests for the persistent resolver. They're not required to use the samples, so feel free to remove the project if you're having build issues.

I'll hopefully follow up this post with a larger explanation of the classes that underpin the PeerMeshConnection in the participant, but a working example now is likely more valuable the further exposition!

I apologise for any slightly ambiguous explanations given above, this post is here to serve as a little bit of a brain dump on WCF P2P, especially as the available resources online are alarmingly slim and without useful examples.

DOWNLOAD EXAMPLE SOLUTION HERE