Using formal methods to support an offshore global delivery method

I have blogged before on this topic but now I want to share some of the results with you.

I was recently involved in two insurance focused system integration initiatives. The fact that they were in insurance is more suggestive that the same approach can work anywhere as insurance tends to be fairly conservative when it comes to new technology.

The background, in both cases, was an in-flight SOA-road map that was being executed. The road map required integration of legacy applications and portal development to support a user experience. A classic ESB approach was used to manage integration and provide both mapping from and to a legacy message format and mediate between the new functionality and the old legacy system. For the most part this is straight forward, or at least so it seems. Estimates were given for the two initiatives to provide the necessary technical contract that would drive implementation offshore.

In the first instance the allotted time to develop a solution and provide the technical contracts was of the order of 15 days effort. This would involve gathering requirements, functional and non-functional, looking at message formats to support the data model, understanding the business transactions so that the solution could be effectively monitored, delivering service contracts for the mapper and mediator services, providing state diagrams to show the individual service behaviors needed and sequence diagrams to provide a collaboration context. The total number of services was 4 including the client, the mediator, mapper and legacy service.

In the second instance the allotted time to develop a solution was of the order of 60 days of effort. In this case there were many more services (I prefer to call them roles) that needed to be understood and many more interactions between those roles. The same level of technical contracts was required to be delivered over the 8 or so roles.

In both cases we used testable architecture. We gathered the requirements initially framed as use cases in UML like notation and then used the pi4soa collaboration tool suite to create sequence diagrams (in which each link referred to an example message in the scenario being modeled) to support each of the use cases. These were then used to drive the development of a CDL model and that model was tested against the sequence diagrams. This represents the left part of the method shown below:




In the first case it tool less than one day to gather the requirements and deliver the tested model. In the second case it took 4 days to do the same. The technical contracts, the WSDL, the BPEL, the BPMN, the state diagrams, sequence diagrams and much more took a few clicks as they were generated from the CDL model.

It may sound fantastical and mythical that we compressed the effort by some 80%. But the fact is that we did. Not only that we found design flaws early both in our use of BPEL and WSDL. We found competing requirements and corrected them and we produced technical contracts that were far more rigourous and complete and unambiguous than we had managed to achieve by hand.

In the early development of the pi4soa toosuite we developed a full FpML model of derivatives trading from indication of interest through to the end of economic life of a derivative. I don't claim that this model was correct but the process of doing it took 4 weeks to complete and in the process a far greater understanding of derivatives processing was gained by both those doing the model and those providing the expertese of derivative processing. Similarly with TWIST and payments and more recently with HL7.

Some question the value of descriptions. All I can say is that the proof is fairly evident that describing something well has the effect that one better understands what is required and design flaws tend to be highlighted. In the insurance enagagements and the much larger FpML engagement the use of testable architecture, the gathering of requirements as sequence diagrams with messages, the modeling of the entire set of requirements and the formal testing of the model against those requirements and finally the generation of technical contracts is without peer. I know of nothing similar that can do this.

Fortunately I work, these days, for a large systems integrator (Cognizant Technology Solutions). This has given me the chance of trying this stuff out and showing I have always believed to be true which is simply that the impact of description through a formal method will help to change the way in which we can effectively do more for less with higher quality and so move the entire process of creating solutions up one level of abstraction with no loss in our ability to express a solution.