CEP is generally used when we implement real-time systems (of course that is not the only area where CEP is used). Real-time does not mean concurrent or for that matter high-performing system.

Of course there are correlations, but at the same time they are fundamentally different paradigms.

This weekend, I am researching Optimisation Algorithms – both Deterministic and Probabilistic. Specifically interested in Swarm Intelligence (which are a type of Monte Carlo algorithm) and in that in Ant Colony and Particle Swarm routing are the main ones I am researching.

1. Part of the overall solution, some smart client is deployed and installed on the desktop and a web only approach is not possible. The main rational being that a smart client can work in a disconnected mode which of course with a web application is not possible.
2. Underlying infrastructure needs to be in place to support this. Infrastructure is not specifically networks and servers, but also both the operational environment and the user’s environment and machine. The operational environments need to allow things such as: data caching, local storage of user data, user profile details, etc.
3. More robust exception management process – this is not only about handling errors but also understanding the fact that the application is in a disconnected state and needs to do things differently.

When designing an occasionally connected application, there are two design approaches that one can take – data centric or service oriented.

1. Data Centric – Applications had a RDBMS of some sort installed locally and use the built-in capabilities of that RDBMS to propagate and sync data including resolving any conflicts.
   1. Server publishes data, which a client subscribes to and is copied locally. The conflict resolution (as changes can be both on the server or client) needs to be agreed upfront.
   2. Generally the database’s built-in conflict resolution is used – this makes it simpler for the application as one does not need to build this in the application.
   3. As there is only one data repository, the data convergence is guaranteed between the client and the server.
   4. Both the client and the server are tightly coupled.
   5. As a database needs to run locally, machines with small footprints or devices such as mobile phones will not be able to run this.
   6. If deployment is an issue then there is more work required here.
2. Service-Oriented – Applications use the SOA paradigm and store information in messages which are queued (when disconnected) and send to the server when connected for processing.
   1. The client can interact with any service required and focuses on the service requests instead of the local data i.e. are loosely coupled.
   2. No local RDBMS required; of course some state information would still need to be saved.
   3. Better when needs to interact outside of the firewall (e.g. Internet or Intranet)
   4. Deployment is still required, but is simpler.

For Data centric application, from a design perspective the following aspects should be factored in:

1. Application needs to be aware of the merge-replication schemes that are implemented as the application needs to optimise for data updates and conflicts.
2. As a result, ACID properties are not used for transactions; instead a pub-sub model is implemented.

On the other hand, for Service-oriented apps, the application design should address the following:

• Application has to implement asynchronous communication.
• Overall solution needs to keep all the network interactions simple and cannot be complex.
• Application needs to add data caching capabilities
• Application needs to implement robust connection management (e.g. Manual vs. Automatic)
• Implement a store-and-forward mechanism such as using MSMQ.
• Application needs to implement a robust data and business rule conflict manager.
• Interacting with CRUD like Web services.
• The application and the work can be logically broken into “chunks” to allow one using a task-based approach.
• The application should be able to handle both forward and reverse dependencies which in turn could be complex business logic.

As a high level guide, a data centric approach should be used when:

• One can deploy a database instance on the client.
• The application can function in a two-tier environment.
• One can tightly couple the client to the server through data schema definitions and communication protocol.
• There is a need for built-in change tracking and synchronization.
• One wants to rely on the database to handle data reconciliation conflicts and minimize the amount of custom reconciliation code that needs to be written.
• There is no need to interact with multiple disparate services.
• Users are able to connect to a database directly through a LAN/VPN/IPsec.

And, a service oriented approach should be taken when:

• One wants to decouple the client and server to allow independent versioning and deployment.
• There is a need for more control and flexibility over data reconciliation issues.
• The delivery team has expertise to write more advanced application infrastructure code.
• There is a need for a lightweight client footprint.
• The applications can be structured into a service-oriented architecture.
• There is a need for specific business functionality (for example, custom business rules and processing, flexible reconciliation, and so on).
  • Note: One might also need to look at a few good rules engine if this is the case.
• One needs control over the schema of data stored on the client and flexibility that might be different from the server.
• The application needs to interact with multiple services using different communication technologies (Web services, Message Queuing, RPC, etc.).
• There is a need for a custom security scheme.
• The application needs to operate outside of the firewall.

I am already using MOOS, so one option is for me to continue to use that – however this is for another part of the application and it might be easier for me to use something else (still need to think it through a little more).

These are the requirements (these are must haves!). Also if it makes a difference I am using CDT for this project.

• Needs to be able to run on Ubuntu 9.04 (and higher)
• Needs to be Open Source (cannot be commercial)
• Needs to be able to store messages “offline”
• Needs to be able to run on TCP with minimal dependencies. It would be nice not to have a whole bunch of underlying dependencies.
• Preferably be easy to use (as a consumer) – I don’t have much time to read through loads of documentation just to get my head around the underlying object model and how to use it.
• C++ support (if it was not obvious until now)

I did a little research online and came across the following, and wanted to get some feedback:

• ActiveMQ – seems like it has good C++ support via CMS (C++ Messaging Service).
• Amazon SQS –  not sure how good the C++ support is. If there is no library per se that I can use, then writing things around REST APIs might be more painful. Also I suddenly have a dependency to be able to go to the public internet. Also it is not free (though there is a free 100K messages / month).
• MQ4CPP – seems quite amateurish (kudos to the guy writing it though – seems like an interesting project to pick up when once has time).
• RabbitMQ – I know some guys used this at work (though that was using it in .NET); nothing for C++, but there some C experimental code; overall does not inspire confidence (in the context of C++).
• OpenAMQ – seems quite interesting and also has a C++ API based on its WireAPI.
• Anything else??

At face value it seems like this is down to ActiveMQ and OpenAMQ. Just looking at the quick samples between the two ActiveMQ seems like more C++ friendly and easier to use compared to OpenAMQ. Of course this is just the first impression and I could be completely wrong – it is not like I have had a chance to play with this (yet anyways).

Does anyone have any experience and feedback on this matter? Feel free to comment on this post, or tweet me.

Whilst MOOS’s roots are in robotics (MRG) and embedded systems, I wonder if I can extend it to use it some of the grid computing scenarios. Maybe implement a pMapReduce or pHadoop? Or perhaps a .NET implementation. Hmm, just need some time. If you need a robust, cross platform communication layer then check out MOOS.

But as are aware Rational tools are very pricey and I was on the look out for something reasonable. Most people have heard of EA (no, not the game company) which is a decent tool and not too expensive (at least the Desktop edition).

Of course if you are stingy like me and want something completely free then check out StarUML which is an open source UML/MDA platform and is pretty decent. When you work with it enough you will find some annoying things – most of which has workarounds. There are one or two things it just cannot do, but I only have been using it for a few months now and am quite impressed. Is it better than the likes of EA or Rose, nope but it is pretty damn good for the price.

1. If you think you need multi-threading, you're wrong
2. If your specification says "you need threading", see Rule 1
3. (For advanced users only) If you think you need multi-threading, you're probably wrong.

You’ll be able to engage in the technology debate with thought-provoking in-depth sessions from customer and partner architects along with members of the UK Microsoft Architect Council such as Avanade, Capgemini, Conchango and Solidsoft.

The agenda is split into seven tracks covering Enterprise, Real World, Identity, Lifecycle, SaaS, Collaboration and Dynamic Systems; the main content sessions will be 75 or 150 minutes in length to accommodate the different formats and levels of interaction that may be required.

There will be an emphasis on architectural ‘investigation’ through the use of small focus groups, as well as a structured networking clinic where individuals from similar vertical business backgrounds can discuss and work through a particular problem domain.

On a different note, been out and about on a few days of holiday with family visiting, but its good to be back now.

Well for those new to webservices, there are two options that you can choose when encoding your wsdl messages.

So, how do they look like. If I “borrow” and example from Sun, if below was your original class in Java:


package com.examples.xmlstring;
import java.rmi.Remote;

import java.rmi.RemoteException;

public interface IStringService extends Remote {

    public String sayXMLHello(String xml)
        throws RemoteException;
    }

The rpc encoding for that looks like this:

<binding name="IStringServiceBinding" type="tns:IStringService">
    <soap:binding transport="http://schemas.xmlsoap.org/soap/http"
style="rpc"/>
    <operation name="sayXMLHello">
        <soap:operation soapaction=""/>
            <input>
               
<soap:body encodingstyle="http://schemas.xmlsoap.org/soap/encoding/" use="encoded" namespace="http://www.examples.com/wsdl/StringService"/>
            </input>
            <output>
               
<soap:body encodingstyle="http://schemas.xmlsoap.org/soap/encoding/" use="encoded" namespace="http://www.examples.com/wsdl/StringService"/>
            </output>
    </operation>
</binding>

And the document encoding looks like:

<?xml version="1.0" encoding="UTF-8"?>
    <env:envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:enc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:ns0="http://www.examples.com/types">
        <env:body>
            <ns0:sayxmlhello>
                <string_1>Hello World</string_1>

            </ns0:sayxmlhello>

        </env:body>
</env:envelope>

I think in the end the real thing to keep in perspective is who and what will be consuming this? If your clients are primarily MS-based (COM/.NET) then you are better off with literal encoding, on the other hand if its primarily J2EE client then you are better off with the other one. But, more interestingly if you don’t know (or in other words can be both), then which road to take?

The hype surrounding Web services has generated many common misconceptions about the fundamentals of this emerging technology.

Web services are frequently described as the latest incarnation of distributed object technology. This misconception, perpetuated by people from both industry and academia, seriously limits broader acceptance of the true Web services architecture. Although the architects of many distributed and Internet systems have been vocal about the differences between Web services and distributed objects, dispelling the myth that they are closely related appears difficult.

Many believe that Web services is a distributed systems technology that relies on some form of distributed object technology. Unfortunately, this is not the only common misconception about Web services. In this article, I seek to clarify several widely held beliefs about the technology that are partially or completely wrong.

Fundamental Errors: At the International World Wide Web Conference in May 2003, a smart and gifted Internet architect I will call Peter asked me, “Don’t you think Web services will fail like all the other wide-area distributed object technologies that people have tried to build?” I was baffled. How could someone like Peter still view Web services as distributed object technology? Yet, he is not alone in his stubbornness: many developers, architects, managers, and academics still see Web services as the next episode in a saga that includes Corba, DCOM, and remote method invocation (RMI). Web services are distributed systems technologies, but that is where the common ground ends. The only possible relation is that Web services are now sometimes deployed in areas where distributed object applications have failed in the past. Within the distributed technology world, it is probably more appropriate to associate Web services with messaging technologies because they share a common architectural view, although they address different application types.

Given that Web services are based on XML documents and document exchange, we could say their technological underpinning is document-oriented computing. However, exchanging documents is very different from requesting an object’s instantiation, requesting a method’s invocation on the basis of the specific object instance, receiving that invocation’s result in a response, and releasing the object instance after several such exchanges.

I frequently encounter about a dozen other statements that fall into the same basic category. I hear people say, for example, that “Web services are just remote procedure calls for the Internet,” or “You need HTTP to make Web services work.” Before addressing several of the more common misconceptions, we should define a Web service in its purest form in order to begin with a clear model.

The application blocks that comprise the Enterprise Library are the following:

• Caching Application Block. This application block allows developers to incorporate a local cache in their applications.
• Configuration Application Block. This application block allows applications to read and write configuration information.
• Data Access Application Block. This application block allows developers to incorporate standard database functionality in their applications.
• Cryptography Application Block. This application block allows developers to include encryption and hashing functionality in their applications.
• Exception Handling Application Block. This application block allows developers and policy makers to create a consistent strategy for processing exceptions that occur throughout the architectural layers of enterprise applications.
• Logging and Instrumentation Application Block. This application block allows developers to incorporate standard logging and instrumentation functionality in their applications.
• Security Application Block. This application block allows developers to incorporate security functionality in their applications. Applications can use the application block in a variety of situations, such as authenticating and authorising users against a database, retrieving role and profile information, and caching user profile information.

Different applications have different requirements and you will not find that every application block is useful in every application that you build. Before using an application block, you should have a good understanding of your application requirements and of the scenarios that the application block is designed to address.

You can also get some more information on Tom’s blog.

Testing .NET Application Blocks covers many testing areas that were used during testing and verification of the various application blocks provided by Microsoft’s patterns & practices group, such as functional, globalization, performance, integration, and security. The guide uses code examples, sample test cases, and checklists to demonstrate how to plan and implement each type of test; the guide also recommends tools to use to run the tests. It covers test considerations to be taken into account when customizing these application blocks or integrating them with your own applications.

Because this guide focuses on testing NET application blocks, its scope does not include user interface (UI) testing or database testing. Although the guidance is developed within the context of Microsoft patterns & practices application blocks, it still applies to testing .NET code in general.

But, in real life, neither of these issues are really ideal. How best to deal with situations where e.g. a sales person who is on the road could not (or does not) connect to the network to synch up and the person in the office updates the record? How do you deal with those situations. One solution is to update only the changes fields, and the rest are either rejected or the user needs to manually confirm each change and pick which ones take precedence. This along with the fact that in real life data can be segregated to some extent on a user level would suffice? Another thing to dwell upon, for most Mobile applications (especially disconnected), going the SOA way is what makes more sense. Since each time you sync up or submit something (if its on gprs for example), your client running on the device should just consume the service and really cannot be bothered by any more details than that (more on this on another post soon).

But, still seems quite fragile with lots of areas for failure. What are you thoughts? Is there a better way to do this that would make it more robust?

HTTP Handlers: The “handlers” are nothing but assemblies (class libraries) that implement IHttpHandler and IHttpAsyncHandler interfaces. Essentially, ASP.NET maps each http request to a http handler. The handler in turn enables processing individual url’s or groups of url’s with similar extensions such as .aspx, .asmx, .ascx, .disco, etc. The handler can be synchronous or asynchronous, the earlier does not return until it finished processing and the latter typically launches a separate process.

When you inherit from IHttpHandler handler, you need to implement both the ProcessRequest method (which processes the individual http requests) and the IsReusable property (which specifies if pooling is supported or not). If you have a more complex set of logic then you should look at inheriting IHttpHandlerFactory, as that allows finer control and you can create different handler based on your need. E.g. you can create a separate handler for GET and PUT.

When you are finished building the handler (essentially compiling your assembly), you need to register it to use it. To register a handler essentially you copy the assembly in the bin folder of where you web app is running and create a section in the web.config. (I’ll have a sample in a bit in this post). You might also want to ensure that the HTTP handler’s extension is registered with IIS.

General process of Building a Handler:

1. Implement the IHttpHandler interface
2. Handle the ProcessRequest method and IsRuseable property.
3. Deploy the assembly
4. Register the handler

Below is an example of a handler that processes request for the files with the extension “.desigeek”. Note, that you do not need to have a physical file present with that extension.

public class DesigeekHandler : IHttpHandler
{
    bool IHttpHandler.IsReusable
    {
        get
        {
            return false;
        }
    }

void IHttpHandler.ProcessRequest( HttpContext context )
{
    HttpResponse response = context.Response;
    
    response.Write( “< html>” );
    response.Write( “< body>” );
    response.Write( “< h1> Hello from Desigeek custom handler. < /h1>” );
    response.Write( “< /body>” );
    response.Write( “< /html>” );
}

Register the HTTP handler by creating an entry in the web.config:

<httpHandlers>
    <add verb=“*“ path=“*.desigeek“ type=“MyHTTPHandler.DesigeekHandler, MyHTTPHandler“/>
httpHandlers>

The main issue is the “lapsed listener” problem. This occurs when objects subscribe to events and subsequently get out of scope. The problem is that the event subscriber doesn’t get garbage collected because the event is still holding a reference to it inside of the event’s invocation list. The event subscriber is still considered reachable from the GC’s point of view. As such, it doesn’t get collected until the event goes out of scope (which is usually at application shutdown) which means that the event subscriber is effectively “leaked”.

Moral of the story: when you implement an Observer pattern, it’s important to consider the relative lifetime of events and subscribers. If implemented naively, you’ll end up having objects that live a lot longer than you think they should. Unsubscribe() is your friend.

As Fabrice writes, .NET’s delegates and events are implementations of the Observer Design Pattern . But the current problem is one more reminder that Design Patterns should not be applied blindly.

If you write the following code, you’ll see that the object instance gets correctly released and collected:

More Information:

• http://tinyurl.com/56vu4
• http://tinyurl.com/5ss2d
• http://tinyurl.com/5jo5n