DynaCache Cache Replication with WebSphere Process Server and ESB

As I blogged about previously, there is a useful technique which involves inserting Java components into SCA modules to cache the results of services using the DynaCache mechanism of WebSphere Application Server. See Gabriel Telerman’s excellent article for more information.

However, if you deploy applications using this technique across a WebSphere cluster, which is fairly typical of a production WebSphere environment, you’ll most likely want to look into WebSphere cache replication (using DRS – the data replication service). This means that rather than having an independent caches on each cluster member (i.e. each server), you’ll have caches that replicate data between each other when it is invalidated or updated in the cache.

This is documented in detail towards the bottom of this InfoCenter page, but broadly speaking you’ll want to modify the properties of the object cache you’re already using. In the article referenced above, the default cache services/cache/distributedmap is used, but to extend this with replication across a cluster, it’s probably appropriate to create your own object cache first if you haven’t already. You then need to specify a “replication domain” for that cache (you’ll need to create one if you don’t already have one), and the replication type. Often “Push only” is suitable for most performance requirements – this pushes new cache entries across the cluster when they are created, modified, or invalidated.

Some other points to be aware of:

  • The Data Replication Service doesn’t always start up straight away on server startup – sometimes it will take a few minutes.
  • It’s primarily intended for internal IBM use, but you may find that the tracing string com.ibm.ws.cache.*=all is useful for figuring out what’s going on inside the cache if it’s not behaving as you expect. It will show cache hits, misses, and replication.

WebSphere Business Process Management v7 Production Topologies Draft Redbook Available

This post is a bit of a personal plug, but I’ve just returned from a Redbook Residency writing the book WebSphere Business Process Management (BPM) V7 Production Topologies Redbook, which illustrates how to set up various BPM products for production, including WebSphere Process Server. It’s a fully updated version of the previous book in the series, which was for V6.2 of the BPM products. The book is now available in draft, so please feel free to take a look – you can send feedback via the feedback e-mail address on that page. I’d also encourage you to rate the book, especially if you think it’s good!

Overriding a Web Service Endpoint in WebSphere ESB

When working with Mediation Flows in WebSphere ESB, and you have a Web Service import defined that connects out to an external Web Service, you might want to override the default endpoint. Here’s a (not exhaustive) list of ways to do that (the numbers in brackets indicate the version that the feature is available from):

  • (6.0.2+) The most flexible and long-term way: integrate with the Web Services Registry and Repository product by using the Endpoint Lookup mediation primitive. This allows you to dynamically look up service endpoints (i.e. URLs), by retrieving them from the registry based on criteria you set. The primitive, in some modes, automatically sets the /headers/SMOHeader/Target/address section of the Service Message Object (SMO), which exploits the dynamic callout capability in WebSphere ESB to override the default endpoint when the callout is performed. In other modes, multiple endpoints might be retrieved, so you would to select the appropriate one and populate that section manually.
  • (7.0+) Use the new UDDI Endpoint Lookup primitive, which works in a very similar way to the regular Endpoint Lookup primitive, but instead looks up endpoints from a UDDI registry.
  • (6.0.2+) Use the dynamic callout facilities of WebSphere ESB directly by populating the /headers/SMOHeader/Target/address section of the SMO directly. You could do this by writing your own logic in a custom mediation, for example, or you could populate it with a hardcoded value using a Message Element Setter.
  • (6.0.1+) Override the endpoint statically in the admin console after deploying the application. To do this, you’d first locate the relevant SCA module deployed in the admin console, then select the import binding you are interested in:
    Then, you can modify the endpoint accordingly:

    This might be useful if you are deploying different copies of your mediation flow to different servers, and wanted to use different endpoints in each case. Note that the methods above, which use dynamic endpoint support, will override whatever you set here, so this is only useful if you aren’t using dynamic endpoint support.

(Note: this tip applies equally to WebSphere Process Server)

Understanding SOAP Action in Mediation Modules

Questions often arise around the SOAPAction header used during Web Service interactions. Despite the name, this is an HTTP header (not a SOAP header) that is sometimes transmitted with SOAP messages to give extra information to the consumer of the service (i.e. the server) about the intention of the Web Service request. The SOAP specification describes the SOAPAction header, but although it gives some examples, leaves the exact usage open to interpretation.

Inside Mediation Modules, the header is used according to these rules:

  • If a SOAPAction header is set in the inbound SOAP request (through a Web Service export), its value will be placed into the /headers/SMOHeader/Action stanza of the Service Message Object (which is used inside the mediation flow to represent the request). Otherwise, the value in /headers/SMOHeader/Action is not set.
  • When an outbound SOAP request is performed (through a dynamic callout, not through a Web Service import), the value placed in /headers/SMOHeader/Action will be used if it is present (so you could set this value, with a Message Element Setter, for example, if you want it to be overridden). Otherwise, it is set to the namespace of your WSDL file, concatenated with the operation name being used – which is a convention frequently used for the contents of SOAPAction: e.g. http://myhost/mynamespace/myoperation.
  • When an outbound SOAP request is performed (through a Web Service import), a soapAction attribute might be specified in the <wsdl:operation>/<soap:operation> section of the Web Service Port WSDL attached to the import. If so, that value will be used as the SOAP Action.

Thanks for Matt Roberts and Jens Engelke for the help with this tip.

Default Directory Change for WebSphere Integration Developer/Process Server/ESB in V7

Updated 2010-01-25: Typo corrected.

I’ve just been experimenting with WebSphere Integration Developer (WID) v7. If you’re working with this new release, you might like to know that the default directories for the WID WTE (WebSphere Test Environment, also called ITE or UTE in some places) has changed.

By default (on Windows), WID is now installed in C:\Program Files\IBM\WID7. However, the WTE runtime (and its associated profiles) can now be found in a separate hierarchy instead – C:\Program Files\IBM\WID7_WTE. If you are using WebSphere Process Server or WebSphere ESB, the runtime that’ll interest you is in C:\Program Files\IBM\WID7_WTE\runtimes\bi_v7. The profiles can be found under there, in subdirectories called qesb and/or qwps. This is different from in previous versions of these products, where the profiles were in a separate directory structure from the runtime. Note that as always, there are separate profiles for WebSphere ESB and WebSphere Process Server, even though the underlying runtime is the same – and technically WebSphere Process Server – in both cases. This is possible because WebSphere Process Server is a functionality superset of WebSphere ESB.

Really Easy Logging in WebSphere ESB v7.0 with Weakly-Typed Subflows and the Trace Primitive

I’ve been experimenting with the new release of WebSphere ESB and WebSphere Integration Developer – v7.0. David has already done a good job of describing the new features, but we can combine two of them – untyped subflows and the Trace primitive – to create a very powerful and easy-to-use logging feature.

First, we create a new Mediation Subflow. The best practice is to create this in a Business Integration Library so that it can be re-used between different Modules or Mediation Modules – I placed mine in a Library called LoggingSubflowLib. As of V7, by default the terminal types of this are untyped, which means they don’t expect any particular type of message to flow through, and it can be re-used in any mediation flow. In the middle of this subflow, we place one of the new Trace primitives. The subflow then looks like this:

We leave the settings for the Trace primitive at the default:

These default settings mean that it will output the entire SMO (Service Message Object) that flows through the flow/subflow to the server log. On Windows, if you accept all the default paths during install, this can now be found at C:\Program Files\IBM\WID7_WTE\runtimes\bi_v7\profiles\qesb\logs\server1\SystemOut.log – note that this is changed slightly from the V6.2 location.

Now we can create a mediation flow to try out this logging subflow with. For my tests, I created a mediation flow called ConvertCustomer, which converts between two slightly different customer interfaces – a frontend and a backend – using an XSLT mediation in the both the request and the response flow to do the mapping. I then inserted the subflow by simply dragging it from the library onto the mediation flow, and placed it in both the request flow:

and the response flow:

I then tested it with the Integration Test Client in WID, by right-clicking on my Mediation Flow Component and selecting “Test Component in Isolation” (I manually emulated the front-end and the back-end). In the Events sequence, you can see where the interaction passes through the subflow and the Trace primitive, in both the request and response directions:

Looking at the SystemOut.log file, you can see the output:

<Se:smo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:Se="http://www.ibm.com/websphere/sibx/smo/v6.0.1" xmlns:in="wsdl.http://ConvertCustomerLib/GetCustomerInterface1" xmlns:in_1="http://ConvertCustomerLib/GetCustomerInterface1">
 <body xsi:type="in:getCustomerRequestMsg">
</Se:smo>, 7

<Se:smo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:Se="http://www.ibm.com/websphere/sibx/smo/v6.0.1" xmlns:in="wsdl.http://ConvertCustomerLib/GetCustomerInterface2" xmlns:in_1="http://ConvertCustomerLib/GetCustomerInterface2">
 <body xsi:type="in:getCustomer_backendResponseMsg">
</Se:smo>, 7

What this now means is that we have a powerful and re-usable subflow that we can place in any mediation flow to do logging. Of course, in this case, this is by itself of only limited value – since we didn’t configure the Trace primitive beyond the defaults, it would be almost as easy just to place the Trace primitive in every mediation flow instead of the Subflow. However, had we done any configuration of the Trace primitive to alter the output location, written our own custom logging logic, or made the subflow more complicated in any way, it would then mean that this development effort only needs to be carried out once.

Very broadly speaking, what the weakly-typed subflows now allow you to do is build your own mediation primitive, very simply (this was always possible, but required some knowledge of the WebSphere ESB SPIs and Eclipse programming). This is an extremely powerful feature.

All of the above also applies to WebSphere Process Server, as it is a functionality superset of WebSphere ESB.

Caching and WebSphere Process Server / ESB

I don’t normally blog about new DeveloperWorks articles, but Gabriel Telerman has just published an article on how to do caching with WebSphere Process Server and WebSphere ESB, by packaging caching logic into a separate SCA component. This is something that is a common requirement, particularly when working with JDBC databases via the WebSphere JDBC adapter, and Gabriel’s article provides some best practices on how this can be accomplished by using a ‘facade’ component that black-box-ifies the caching – an excellent SOA pattern. Worth a read.