DynaCache Cache Replication with WebSphere Process Server and ESB

As I blogged about previously, there is a useful technique which involves inserting Java components into SCA modules to cache the results of services using the DynaCache mechanism of WebSphere Application Server. See Gabriel Telerman’s excellent article for more information.

However, if you deploy applications using this technique across a WebSphere cluster, which is fairly typical of a production WebSphere environment, you’ll most likely want to look into WebSphere cache replication (using DRS – the data replication service). This means that rather than having an independent caches on each cluster member (i.e. each server), you’ll have caches that replicate data between each other when it is invalidated or updated in the cache.

This is documented in detail towards the bottom of this InfoCenter page, but broadly speaking you’ll want to modify the properties of the object cache you’re already using. In the article referenced above, the default cache services/cache/distributedmap is used, but to extend this with replication across a cluster, it’s probably appropriate to create your own object cache first if you haven’t already. You then need to specify a “replication domain” for that cache (you’ll need to create one if you don’t already have one), and the replication type. Often “Push only” is suitable for most performance requirements – this pushes new cache entries across the cluster when they are created, modified, or invalidated.

Some other points to be aware of:

  • The Data Replication Service doesn’t always start up straight away on server startup – sometimes it will take a few minutes.
  • It’s primarily intended for internal IBM use, but you may find that the tracing string com.ibm.ws.cache.*=all is useful for figuring out what’s going on inside the cache if it’s not behaving as you expect. It will show cache hits, misses, and replication.

WebSphere Business Process Management v7 Production Topologies Draft Redbook Available

This post is a bit of a personal plug, but I’ve just returned from a Redbook Residency writing the book WebSphere Business Process Management (BPM) V7 Production Topologies Redbook, which illustrates how to set up various BPM products for production, including WebSphere Process Server. It’s a fully updated version of the previous book in the series, which was for V6.2 of the BPM products. The book is now available in draft, so please feel free to take a look – you can send feedback via the feedback e-mail address on that page. I’d also encourage you to rate the book, especially if you think it’s good!

Overriding a Web Service Endpoint in WebSphere ESB

When working with Mediation Flows in WebSphere ESB, and you have a Web Service import defined that connects out to an external Web Service, you might want to override the default endpoint. Here’s a (not exhaustive) list of ways to do that (the numbers in brackets indicate the version that the feature is available from):

  • (6.0.2+) The most flexible and long-term way: integrate with the Web Services Registry and Repository product by using the Endpoint Lookup mediation primitive. This allows you to dynamically look up service endpoints (i.e. URLs), by retrieving them from the registry based on criteria you set. The primitive, in some modes, automatically sets the /headers/SMOHeader/Target/address section of the Service Message Object (SMO), which exploits the dynamic callout capability in WebSphere ESB to override the default endpoint when the callout is performed. In other modes, multiple endpoints might be retrieved, so you would to select the appropriate one and populate that section manually.
  • (7.0+) Use the new UDDI Endpoint Lookup primitive, which works in a very similar way to the regular Endpoint Lookup primitive, but instead looks up endpoints from a UDDI registry.
  • (6.0.2+) Use the dynamic callout facilities of WebSphere ESB directly by populating the /headers/SMOHeader/Target/address section of the SMO directly. You could do this by writing your own logic in a custom mediation, for example, or you could populate it with a hardcoded value using a Message Element Setter.
  • (6.0.1+) Override the endpoint statically in the admin console after deploying the application. To do this, you’d first locate the relevant SCA module deployed in the admin console, then select the import binding you are interested in:
    Then, you can modify the endpoint accordingly:

    This might be useful if you are deploying different copies of your mediation flow to different servers, and wanted to use different endpoints in each case. Note that the methods above, which use dynamic endpoint support, will override whatever you set here, so this is only useful if you aren’t using dynamic endpoint support.

(Note: this tip applies equally to WebSphere Process Server)

Understanding SOAP Action in Mediation Modules

Questions often arise around the SOAPAction header used during Web Service interactions. Despite the name, this is an HTTP header (not a SOAP header) that is sometimes transmitted with SOAP messages to give extra information to the consumer of the service (i.e. the server) about the intention of the Web Service request. The SOAP specification describes the SOAPAction header, but although it gives some examples, leaves the exact usage open to interpretation.

Inside Mediation Modules, the header is used according to these rules:

  • If a SOAPAction header is set in the inbound SOAP request (through a Web Service export), its value will be placed into the /headers/SMOHeader/Action stanza of the Service Message Object (which is used inside the mediation flow to represent the request). Otherwise, the value in /headers/SMOHeader/Action is not set.
  • When an outbound SOAP request is performed (through a dynamic callout, not through a Web Service import), the value placed in /headers/SMOHeader/Action will be used if it is present (so you could set this value, with a Message Element Setter, for example, if you want it to be overridden). Otherwise, it is set to the namespace of your WSDL file, concatenated with the operation name being used – which is a convention frequently used for the contents of SOAPAction: e.g. http://myhost/mynamespace/myoperation.
  • When an outbound SOAP request is performed (through a Web Service import), a soapAction attribute might be specified in the <wsdl:operation>/<soap:operation> section of the Web Service Port WSDL attached to the import. If so, that value will be used as the SOAP Action.

Thanks for Matt Roberts and Jens Engelke for the help with this tip.

Default Directory Change for WebSphere Integration Developer/Process Server/ESB in V7

Updated 2010-01-25: Typo corrected.

I’ve just been experimenting with WebSphere Integration Developer (WID) v7. If you’re working with this new release, you might like to know that the default directories for the WID WTE (WebSphere Test Environment, also called ITE or UTE in some places) has changed.

By default (on Windows), WID is now installed in C:\Program Files\IBM\WID7. However, the WTE runtime (and its associated profiles) can now be found in a separate hierarchy instead – C:\Program Files\IBM\WID7_WTE. If you are using WebSphere Process Server or WebSphere ESB, the runtime that’ll interest you is in C:\Program Files\IBM\WID7_WTE\runtimes\bi_v7. The profiles can be found under there, in subdirectories called qesb and/or qwps. This is different from in previous versions of these products, where the profiles were in a separate directory structure from the runtime. Note that as always, there are separate profiles for WebSphere ESB and WebSphere Process Server, even though the underlying runtime is the same – and technically WebSphere Process Server – in both cases. This is possible because WebSphere Process Server is a functionality superset of WebSphere ESB.

Really Easy Logging in WebSphere ESB v7.0 with Weakly-Typed Subflows and the Trace Primitive

I’ve been experimenting with the new release of WebSphere ESB and WebSphere Integration Developer – v7.0. David has already done a good job of describing the new features, but we can combine two of them – untyped subflows and the Trace primitive – to create a very powerful and easy-to-use logging feature.

First, we create a new Mediation Subflow. The best practice is to create this in a Business Integration Library so that it can be re-used between different Modules or Mediation Modules – I placed mine in a Library called LoggingSubflowLib. As of V7, by default the terminal types of this are untyped, which means they don’t expect any particular type of message to flow through, and it can be re-used in any mediation flow. In the middle of this subflow, we place one of the new Trace primitives. The subflow then looks like this:

We leave the settings for the Trace primitive at the default:

These default settings mean that it will output the entire SMO (Service Message Object) that flows through the flow/subflow to the server log. On Windows, if you accept all the default paths during install, this can now be found at C:\Program Files\IBM\WID7_WTE\runtimes\bi_v7\profiles\qesb\logs\server1\SystemOut.log – note that this is changed slightly from the V6.2 location.

Now we can create a mediation flow to try out this logging subflow with. For my tests, I created a mediation flow called ConvertCustomer, which converts between two slightly different customer interfaces – a frontend and a backend – using an XSLT mediation in the both the request and the response flow to do the mapping. I then inserted the subflow by simply dragging it from the library onto the mediation flow, and placed it in both the request flow:

and the response flow:

I then tested it with the Integration Test Client in WID, by right-clicking on my Mediation Flow Component and selecting “Test Component in Isolation” (I manually emulated the front-end and the back-end). In the Events sequence, you can see where the interaction passes through the subflow and the Trace primitive, in both the request and response directions:

Looking at the SystemOut.log file, you can see the output:

<Se:smo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:Se="http://www.ibm.com/websphere/sibx/smo/v6.0.1" xmlns:in="wsdl.http://ConvertCustomerLib/GetCustomerInterface1" xmlns:in_1="http://ConvertCustomerLib/GetCustomerInterface1">
 <context/>
 <headers>
  <SMOHeader>
   <MessageUUID>9C1C9286-0125-4000-E000-03A0C0A8C886</MessageUUID>
   <Version>
    <Version>7</Version>
    <Release>0</Release>
    <Modification>0</Modification>
   </Version>
   <MessageType>Request</MessageType>
  </SMOHeader>
 </headers>
 <body xsi:type="in:getCustomerRequestMsg">
  <in_1:getCustomer>
   <id>123</id>
  </in_1:getCustomer>
 </body>
</Se:smo>, 7

<Se:smo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:Se="http://www.ibm.com/websphere/sibx/smo/v6.0.1" xmlns:in="wsdl.http://ConvertCustomerLib/GetCustomerInterface2" xmlns:in_1="http://ConvertCustomerLib/GetCustomerInterface2">
 <context/>
 <headers>
  <SMOHeader>
   <MessageUUID>9C1D129C-0125-4000-E000-03A0C0A8C886</MessageUUID>
   <Version>
    <Version>7</Version>
    <Release>0</Release>
    <Modification>0</Modification>
   </Version>
  <MessageType>Response</MessageType>
  </SMOHeader>
 </headers>
 <body xsi:type="in:getCustomer_backendResponseMsg">
  <in_1:getCustomer_backendResponse>
   <customerBackendData>
    <name>myName</name>
    <address>myAddress</address>
   </customerBackendData>
  </in_1:getCustomer_backendResponse>
 </body>
</Se:smo>, 7

What this now means is that we have a powerful and re-usable subflow that we can place in any mediation flow to do logging. Of course, in this case, this is by itself of only limited value – since we didn’t configure the Trace primitive beyond the defaults, it would be almost as easy just to place the Trace primitive in every mediation flow instead of the Subflow. However, had we done any configuration of the Trace primitive to alter the output location, written our own custom logging logic, or made the subflow more complicated in any way, it would then mean that this development effort only needs to be carried out once.

Very broadly speaking, what the weakly-typed subflows now allow you to do is build your own mediation primitive, very simply (this was always possible, but required some knowledge of the WebSphere ESB SPIs and Eclipse programming). This is an extremely powerful feature.

All of the above also applies to WebSphere Process Server, as it is a functionality superset of WebSphere ESB.

Caching and WebSphere Process Server / ESB

I don’t normally blog about new DeveloperWorks articles, but Gabriel Telerman has just published an article on how to do caching with WebSphere Process Server and WebSphere ESB, by packaging caching logic into a separate SCA component. This is something that is a common requirement, particularly when working with JDBC databases via the WebSphere JDBC adapter, and Gabriel’s article provides some best practices on how this can be accomplished by using a ‘facade’ component that black-box-ifies the caching – an excellent SOA pattern. Worth a read.

WebSphere ESB / Integration Developer / Process Server 7.0 Announced

For those already working with WebSphere Integration Developer, WebSphere ESB, or WebSphere Process Server, or those considering using them, you might be interested to know that version 7.0 has just been announced and is scheduled to be available around the end of Q4 2009.

Please look at the official announcement for more information, but some of the new things I think are particularly interesting are:

  • Service Federation Management between WebSphere Services Registry and Repository and WebSphere ESB.
  • More pattern-based development.
  • Exploitation of WebSphere Application Server V7.
  • Improved Service Gateway scenario support.
  • Event sequencing support in WebSphere ESB.

Health Center with WebSphere ESB and Process Server

I attended a presentation last week by one of the developers of Health Center, a new monitoring and diagnostic tool IBM has made available for debugging Java Virtual Machines as they are executing. It’s available for any application that runs on an (IBM-based) JVM, but of particular interest to me is WebSphere ESB and Process Server.

I won’t repeat the installation instructions, which you can find here, but there are one or two gotchas you should be aware of:

  • At the time of writing, Java versions shipped with WebSphere ESB and Process Server (version 6.2.x) are sufficiently old that they don’t yet include the Health Center agent, so you will need to install this manually as described in the “Enabling an application for monitoring” link when you set up a connection in Health Center. Be careful to extract files into exactly the right directory.
  • I also suggest you then follow the instructions labelled “Configuring WAS to enable monitoring” in the Health Center Infocenter. If you are using the embedded Integrated Test Environment servers inside WebSphere Integration Developer, I suggest the topic “Configuring WAS test environment in RAD to enable monitoring”. Although the product names are different, the principles are essentially the same.
  • Be aware that (at the time of writing) Java 5 SR8 is used by WebSphere ESB and Process Server, so you’ll need to use the more complex form of JVM boot parameters as described in the help. This also means that it’s not yet recommended that use Health Center in production, although it will be useful in development, particularly when you’re focusing on performance, stability, or capacity issues. Hopefully this situation should change in later versions of WebSphere ESB and Process Server.

To give you a taster, once you’ve managed to connect, you’ll see some very cool screens, like this one that allows you to do trivial method profiling:

2009-07-21_160518

(sort by ‘Tree’ is particularly useful, as this includes time taken by methods that a given method has called).

Another handy view is this one, which allows you to watch your heap and JVM pause times over time (drag a box over a particular area to restrict all statistics to that period):

2009-07-21_160428

The best part of Health Center is that all data is updated in real time (and can be saved away and reloaded later), so you can (for example) watch your heap grow and shrink, your garbage collection happen, and so on. Overhead is generally minimal and since it uses a client/server architecture, is minimally invasive.

You can find out a lot more information on the Health Center website and the Java troubleshooting team’s blog.

Thanks to Toby Corbin for the original presentation and Dave Nice for assistance with Health Center.

Emulating modelled faults in WebSphere Integration Developer

Since version 6.1, WebSphere Integration Developer has come with the ability to create test projects containing test suites and programmatic emulators, allowing developers to create repeatable automated tests for their modules. Projects can contain any number of suites, test cases and variations to cover all the behaviours of the module.

One scenario often overlooked in user-defined test suites is the one where the partner reference returns a modelled fault.

For example:

Example interface with a modelled fault

Fortunately, it is possible to return a fault using a programmatic emulator. In this example we will use Java, but the same behaviour can be accomplished using a visual snippet.

First create the payload for the fault. If the payload is a String or other Java type, you can simply create it and assign it to a variable. If the object is a business object then locate the BOFactory and create it as you would any other SDO DataObject

BOFactory boFactory =
  (BOFactory) ServiceManager.INSTANCE.locateService("com/ibm/websphere/bo/BOFactory");

DataObject reservationFault =
  boFactory.create("http://LB_Example","ReservationFault");

Once you have the object set any attributes that are required.

reservationFault.setString("faultCode", "100");

Next create a Java ServiceBusinessException. The exception has four constructor methods, and we want the one that accepts a String and an Object. The string will be name of the fault being returned, and the object will be the payload created above.

Finally, you throw (not return) the ServiceBusinessException

ServiceBusinessException sbe =
  new ServiceBusinessException(reservationFault,"unknownEventFault");

throw sbe;

For completeness, here is the same code represented visually.

Fault emulator as visual snippet