Review: WebSphere Application Server Administration Using Jython

Recently, I was asked to review the book WebSphere Application Server Administration Using Jython.

First impressions are excellent – the book is thorough, first introducing the Jython language itself (which is essentially Python running on a Java Virtual Machine – and very easy to learn), then moving to talk about the wsadmin tool, which is used to execute administrative scripts or command in Jython with WebSphere. It’s worth pointing out that WebSphere currently supports two scripting languages – Jython and JACL, and Jython is the strategic direction forward for WebSphere, so it’s good that the book focuses on this (also, in my humble opinion, Jython is much easier to learn).

The book then moves on to the meat of the subject, discussing each of the four primary WebSphere administrative objects in turn. There are dedicated chapters for various important and frequently-used topics, such as security, databases, and messaging. In general, these are thorough and of high quality.

The only minor point against the book is that it covers only WebSphere Application Server, not any of the stack products that build on top of it (such as WebSphere ESB and Process Server, which is the primary focus of this blog). However, this is expected – it’s in the title – so is hardly a criticism.

Generally speaking, this is a thorough and useful technical book for anyone wanting to do scripting with WebSphere Application Server.

Disclosure: I recieved a free copy of the book in exchange for this review – that’s all.

Overriding a Web Service Endpoint in WebSphere ESB

When working with Mediation Flows in WebSphere ESB, and you have a Web Service import defined that connects out to an external Web Service, you might want to override the default endpoint. Here’s a (not exhaustive) list of ways to do that (the numbers in brackets indicate the version that the feature is available from):

• (6.0.2+) The most flexible and long-term way: integrate with the Web Services Registry and Repository product by using the Endpoint Lookup mediation primitive. This allows you to dynamically look up service endpoints (i.e. URLs), by retrieving them from the registry based on criteria you set. The primitive, in some modes, automatically sets the /headers/SMOHeader/Target/address section of the Service Message Object (SMO), which exploits the dynamic callout capability in WebSphere ESB to override the default endpoint when the callout is performed. In other modes, multiple endpoints might be retrieved, so you would to select the appropriate one and populate that section manually.
• (7.0+) Use the new UDDI Endpoint Lookup primitive, which works in a very similar way to the regular Endpoint Lookup primitive, but instead looks up endpoints from a UDDI registry.
• (6.0.2+) Use the dynamic callout facilities of WebSphere ESB directly by populating the /headers/SMOHeader/Target/address section of the SMO directly. You could do this by writing your own logic in a custom mediation, for example, or you could populate it with a hardcoded value using a Message Element Setter.
• (6.0.1+) Override the endpoint statically in the admin console after deploying the application. To do this, you’d first locate the relevant SCA module deployed in the admin console, then select the import binding you are interested in:
  Then, you can modify the endpoint accordingly:

  

  This might be useful if you are deploying different copies of your mediation flow to different servers, and wanted to use different endpoints in each case. Note that the methods above, which use dynamic endpoint support, will override whatever you set here, so this is only useful if you aren’t using dynamic endpoint support.

(Note: this tip applies equally to WebSphere Process Server)

Understanding SOAP Action in Mediation Modules

Questions often arise around the SOAPAction header used during Web Service interactions. Despite the name, this is an HTTP header (not a SOAP header) that is sometimes transmitted with SOAP messages to give extra information to the consumer of the service (i.e. the server) about the intention of the Web Service request. The SOAP specification describes the SOAPAction header, but although it gives some examples, leaves the exact usage open to interpretation.

Inside Mediation Modules, the header is used according to these rules:

• If a SOAPAction header is set in the inbound SOAP request (through a Web Service export), its value will be placed into the /headers/SMOHeader/Action stanza of the Service Message Object (which is used inside the mediation flow to represent the request). Otherwise, the value in /headers/SMOHeader/Action is not set.
• When an outbound SOAP request is performed (through a dynamic callout, not through a Web Service import), the value placed in /headers/SMOHeader/Action will be used if it is present (so you could set this value, with a Message Element Setter, for example, if you want it to be overridden). Otherwise, it is set to the namespace of your WSDL file, concatenated with the operation name being used – which is a convention frequently used for the contents of SOAPAction: e.g. http://myhost/mynamespace/myoperation.
• When an outbound SOAP request is performed (through a Web Service import), a soapAction attribute might be specified in the <wsdl:operation>/<soap:operation> section of the Web Service Port WSDL attached to the import. If so, that value will be used as the SOAP Action.

Thanks for Matt Roberts and Jens Engelke for the help with this tip.

Default Directory Change for WebSphere Integration Developer/Process Server/ESB in V7

Updated 2010-01-25: Typo corrected.

I’ve just been experimenting with WebSphere Integration Developer (WID) v7. If you’re working with this new release, you might like to know that the default directories for the WID WTE (WebSphere Test Environment, also called ITE or UTE in some places) has changed.

By default (on Windows), WID is now installed in C:\Program Files\IBM\WID7. However, the WTE runtime (and its associated profiles) can now be found in a separate hierarchy instead – C:\Program Files\IBM\WID7_WTE. If you are using WebSphere Process Server or WebSphere ESB, the runtime that’ll interest you is in C:\Program Files\IBM\WID7_WTE\runtimes\bi_v7. The profiles can be found under there, in subdirectories called qesb and/or qwps. This is different from in previous versions of these products, where the profiles were in a separate directory structure from the runtime. Note that as always, there are separate profiles for WebSphere ESB and WebSphere Process Server, even though the underlying runtime is the same – and technically WebSphere Process Server – in both cases. This is possible because WebSphere Process Server is a functionality superset of WebSphere ESB.

Really Easy Logging in WebSphere ESB v7.0 with Weakly-Typed Subflows and the Trace Primitive

I’ve been experimenting with the new release of WebSphere ESB and WebSphere Integration Developer – v7.0. David has already done a good job of describing the new features, but we can combine two of them – untyped subflows and the Trace primitive – to create a very powerful and easy-to-use logging feature.

First, we create a new Mediation Subflow. The best practice is to create this in a Business Integration Library so that it can be re-used between different Modules or Mediation Modules – I placed mine in a Library called LoggingSubflowLib. As of V7, by default the terminal types of this are untyped, which means they don’t expect any particular type of message to flow through, and it can be re-used in any mediation flow. In the middle of this subflow, we place one of the new Trace primitives. The subflow then looks like this:

We leave the settings for the Trace primitive at the default:

These default settings mean that it will output the entire SMO (Service Message Object) that flows through the flow/subflow to the server log. On Windows, if you accept all the default paths during install, this can now be found at C:\Program Files\IBM\WID7_WTE\runtimes\bi_v7\profiles\qesb\logs\server1\SystemOut.log – note that this is changed slightly from the V6.2 location.

Now we can create a mediation flow to try out this logging subflow with. For my tests, I created a mediation flow called ConvertCustomer, which converts between two slightly different customer interfaces – a frontend and a backend – using an XSLT mediation in the both the request and the response flow to do the mapping. I then inserted the subflow by simply dragging it from the library onto the mediation flow, and placed it in both the request flow:

and the response flow:

I then tested it with the Integration Test Client in WID, by right-clicking on my Mediation Flow Component and selecting “Test Component in Isolation” (I manually emulated the front-end and the back-end). In the Events sequence, you can see where the interaction passes through the subflow and the Trace primitive, in both the request and response directions:

Looking at the SystemOut.log file, you can see the output:

<Se:smo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:Se="http://www.ibm.com/websphere/sibx/smo/v6.0.1" xmlns:in="wsdl.http://ConvertCustomerLib/GetCustomerInterface1" xmlns:in_1="http://ConvertCustomerLib/GetCustomerInterface1">
 <context/>
 <headers>
  <SMOHeader>
   <MessageUUID>9C1C9286-0125-4000-E000-03A0C0A8C886</MessageUUID>
   <Version>
    <Version>7</Version>
    <Release>0</Release>
    <Modification>0</Modification>
   </Version>
   <MessageType>Request</MessageType>
  </SMOHeader>
 </headers>
 <body xsi:type="in:getCustomerRequestMsg">
  <in_1:getCustomer>
   <id>123</id>
  </in_1:getCustomer>
 </body>
</Se:smo>, 7

<Se:smo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:Se="http://www.ibm.com/websphere/sibx/smo/v6.0.1" xmlns:in="wsdl.http://ConvertCustomerLib/GetCustomerInterface2" xmlns:in_1="http://ConvertCustomerLib/GetCustomerInterface2">
 <context/>
 <headers>
  <SMOHeader>
   <MessageUUID>9C1D129C-0125-4000-E000-03A0C0A8C886</MessageUUID>
   <Version>
    <Version>7</Version>
    <Release>0</Release>
    <Modification>0</Modification>
   </Version>
  <MessageType>Response</MessageType>
  </SMOHeader>
 </headers>
 <body xsi:type="in:getCustomer_backendResponseMsg">
  <in_1:getCustomer_backendResponse>
   <customerBackendData>
    <name>myName</name>
    <address>myAddress</address>
   </customerBackendData>
  </in_1:getCustomer_backendResponse>
 </body>
</Se:smo>, 7

What this now means is that we have a powerful and re-usable subflow that we can place in any mediation flow to do logging. Of course, in this case, this is by itself of only limited value – since we didn’t configure the Trace primitive beyond the defaults, it would be almost as easy just to place the Trace primitive in every mediation flow instead of the Subflow. However, had we done any configuration of the Trace primitive to alter the output location, written our own custom logging logic, or made the subflow more complicated in any way, it would then mean that this development effort only needs to be carried out once.

Very broadly speaking, what the weakly-typed subflows now allow you to do is build your own mediation primitive, very simply (this was always possible, but required some knowledge of the WebSphere ESB SPIs and Eclipse programming). This is an extremely powerful feature.

All of the above also applies to WebSphere Process Server, as it is a functionality superset of WebSphere ESB.

Caching and WebSphere Process Server / ESB

I don’t normally blog about new DeveloperWorks articles, but Gabriel Telerman has just published an article on how to do caching with WebSphere Process Server and WebSphere ESB, by packaging caching logic into a separate SCA component. This is something that is a common requirement, particularly when working with JDBC databases via the WebSphere JDBC adapter, and Gabriel’s article provides some best practices on how this can be accomplished by using a ‘facade’ component that black-box-ifies the caching – an excellent SOA pattern. Worth a read.

WebSphere ESB / Integration Developer / Process Server 7.0 Announced

For those already working with WebSphere Integration Developer, WebSphere ESB, or WebSphere Process Server, or those considering using them, you might be interested to know that version 7.0 has just been announced and is scheduled to be available around the end of Q4 2009.

Please look at the official announcement for more information, but some of the new things I think are particularly interesting are:

• Service Federation Management between WebSphere Services Registry and Repository and WebSphere ESB.
• More pattern-based development.
• Exploitation of WebSphere Application Server V7.
• Improved Service Gateway scenario support.
• Event sequencing support in WebSphere ESB.

WebSphere Application Server 7.0 Administration Guide: Review

I was recently asked to review the new book WebSphere Application Server 7.0 Administration Guide by Steve Robinson. First impressions are good; the book looks professional and comprehensive. Steve starts from the basics, installing WebSphere Application Server from scratch. He walks through some administration tasks that I might expect an experienced system administrator or developer to be able to do without trouble (details on un-taring a file, for example), and a little more information at various points would be welcome: for example, although Chapter 1 describes creation of a profile, it doesn’t really explain what a profile is or why it would be needed, referring only to an “application server definition”.

Nevertheless, taken as a whole, the book is generally thorough, and certainly covers the basics for someone with little WebSphere Application Server expertise or exposure. Steve looks at administrative security, scripting, messaging via the System Integration Bus, and so on. As the title implies, the book doesn’t talk much about application development, but that’s fine. There’s nothing here you wouldn’t find elsewhere, but Steve does a good job of collecting it all together, and presenting WebSphere administration topics in a coherent and logical manner.

It’s a good book for WebSphere Application Server administrators who are just getting started – and it would also be useful for those learning to administrate products that use it as a base, such as WebSphere Enterprise Service Bus or WebSphere Process Server.

Quick ‘n’ Dirty Dogear Backup

A little bit of a deviation from our usual WebSphere content here, but: if you’re using Lotus Connections in your organisation, you might want a way to backup the bookmarks you save to Dogear locally. Maybe there’s a lot of valuable content in there that you need should it go offline for a few hours, for example. I’m just paranoid, and I always like to have personal backups of anything important to me – which my bookmarks sure are.

So I wrote a quick and dirty Python script that backs up your bookmarks by connecting via the Dogear API and saving them to an XBEL file. It doesn’t support all fields – in particular, in doesn’t currently back up tags, only the title and the URL.

Feel free to take and adapt to your needs (as with all material on this site, this is not supported by IBM in any way and there are no guarantees that this will even work for you). Feedback appreciated as always.

You’ll need to pass in three command-line arguments to the script: hopefully the code is self-explanatory.

#!/usr/bin/python

import urllib2
import feedparser
import base64

from urllib import quote
from optparse import OptionParser

parser = OptionParser()

parser.add_option("-u", "--username", dest="username")
parser.add_option("-p", "--password", dest="password")
parser.add_option("-s", "--servername", dest="servername")

(options, args) = parser.parse_args()

print "<?xml version=\"1.0\"?>"
print "<xbel version=\"1.0\">"

PAGESIZE = 50
url = "https://" + options.servername + "/dogear/atom?email=" + quote(options.username) + "&access=all&ps=" + str(PAGESIZE)

entrieslength = PAGESIZE
currentpage = 1

base64string = base64.encodestring('%s:%s' % (options.username, options.password))[:-1]
authheader = "Basic %s" % base64string

while entrieslength >= PAGESIZE:
 req = urllib2.Request(url + "&page=" + str(currentpage))
 req.add_header("Authorization", authheader)
 data = urllib2.urlopen(req).read()
 feed = feedparser.parse(data)
 entrieslength = len(feed.entries)
 for entry in feed.entries:
  print "\t<bookmark href=\"" + entry.link + "\">"
  print "\t\t<title>" + entry.title.encode('utf-8') + "</title>"
  print "\t</bookmark>"
 currentpage = currentpage + 1

print "</xbel>"

Health Center with WebSphere ESB and Process Server

I attended a presentation last week by one of the developers of Health Center, a new monitoring and diagnostic tool IBM has made available for debugging Java Virtual Machines as they are executing. It’s available for any application that runs on an (IBM-based) JVM, but of particular interest to me is WebSphere ESB and Process Server.

I won’t repeat the installation instructions, which you can find here, but there are one or two gotchas you should be aware of:

• At the time of writing, Java versions shipped with WebSphere ESB and Process Server (version 6.2.x) are sufficiently old that they don’t yet include the Health Center agent, so you will need to install this manually as described in the “Enabling an application for monitoring” link when you set up a connection in Health Center. Be careful to extract files into exactly the right directory.
• I also suggest you then follow the instructions labelled “Configuring WAS to enable monitoring” in the Health Center Infocenter. If you are using the embedded Integrated Test Environment servers inside WebSphere Integration Developer, I suggest the topic “Configuring WAS test environment in RAD to enable monitoring”. Although the product names are different, the principles are essentially the same.
• Be aware that (at the time of writing) Java 5 SR8 is used by WebSphere ESB and Process Server, so you’ll need to use the more complex form of JVM boot parameters as described in the help. This also means that it’s not yet recommended that use Health Center in production, although it will be useful in development, particularly when you’re focusing on performance, stability, or capacity issues. Hopefully this situation should change in later versions of WebSphere ESB and Process Server.

To give you a taster, once you’ve managed to connect, you’ll see some very cool screens, like this one that allows you to do trivial method profiling:

(sort by ‘Tree’ is particularly useful, as this includes time taken by methods that a given method has called).

Another handy view is this one, which allows you to watch your heap and JVM pause times over time (drag a box over a particular area to restrict all statistics to that period):

The best part of Health Center is that all data is updated in real time (and can be saved away and reloaded later), so you can (for example) watch your heap grow and shrink, your garbage collection happen, and so on. Overhead is generally minimal and since it uses a client/server architecture, is minimally invasive.

You can find out a lot more information on the Health Center website and the Java troubleshooting team’s blog.

Thanks to Toby Corbin for the original presentation and Dave Nice for assistance with Health Center.