DBAdapter - Merge operation

If you use merge operation for updating records using DBAdapter, make sure property DetectOmission is set to false as by default it's true. Or else values coming as Null will not get udpated in database records - causing updates buggy.

  property name="DetectOmissions" value="false"/ 


update:dec'03

There was another interesting learning on DBAdapter, when we had to insert a record to 5 related tables. All the tables were related by foreign keys. DBAdapter wizard lets you pick your tables, define their relationship, pick database sequences for primary keys. And all this internally uses Eclipselink JPA - looked pretty good.

OSB - MQ transport - reply to queue manger name / queue name

Recently we had a good (rather hard) experience with consuming a legacy application service in OSB. The legacy application exposes a MQ based interface. It receives the request message in one queue and reads the reply to queue manager and reply to queue names from the message header and then puts the response in that specified queue manager/queue which is on their side.

If you have worked with MQs, setting up local queues pointing them to remote queues with correct queue manager names etc. can be quite tricky. We wasted many weeks to debug an issue where the remote response queue manager host name was not correct, coordinating with another team, doing it across 2 different environments was a big pain.

More painful was the request/response never worked for months, after quite a bit of heartburn, It was found that while we were setting the reply to response queue manager/queue names in the request message header, it was getting overwritten by OSB with the local queue manger/queue names.

It turned out if you are using mq-transport in request-response mode (which is the synchronous mode) - the response queue manager/queue name has to be same as the request queue manager/queue name if not OSB will overwrite it. This however is not a problem if the message pattern is one-way (i.e only request).

please see more details about 'reply to queue manager' and 'reply to queue' transport header properties here

So to solve the problem we introduced an intermediate queue in between our request queue and remote request queue. The first BS (business service) uses the mq transport in request/response mode and pushes the request to an intermediate queue. Another proxy service listening on to this intermediate queue picks up the message and pushes it to the real request queue with reply to queue manager/queue names in the transport headers as required by the legacy application.

There were some apprehension about correlation id, message id etc, however this worked well, and we were able to move forward.

Framework for picking a web-framework

If you have built database-oriented web applications in the past and wondering now which is the best framework to build going ahead, the below classification might help. I tried creating a decision tree for some of the most popular frameworks.

As you can see in one extreme it's the  Java based standard (JEE) frameworks and at the other extreme it's the non Java,  non sharing (no servlet session) completely new frameworks and in the middle is the Spring MVC and Grails.

A very good presentation in favor of Play here and slides  here

A good comparison of JVM based web app frameworks here, winner is Grails, comparison matrix here

Whatever you pick between the HTML/Javascript and your backend/middleware, it’s important to know how these frameworks differ at a fundamental level which can affect scalability, performance, developer productivity, cost etc.

It's important to note that these frameworks are just not presentation frameworks, they extend to use an ORM framework (mostly hibernate) and have their own build tool, ide etc. Each one of them pretty much extends to a platform and will support cloud deployment.


Interesting video here on why twitter is moving from Ruby on Rails to JVM.

Now time for the new debate how much client side MVC and how much Server side MVC? / REST API?

OSB11g

Finally I got my first project on OSB11g into production, We had used OSB11g before, however primarily for service-virtualization, routing, light transformation, security etc. This project is special as OSB11g was the only platform for all service implementation as well as composition. It's kind of conflicting to build services on OSB11g using DBAdapters instead of using JEE components such as JPA or any other OR mapping product. I had a discussion on this here

Unfortunately I didn't get to code, however I tried a bit to do code-reviews, couple of findings on the IDE part here -

• If you want to only read OSB code - still you have to install OSB as the OSB IDE plugin for OEPE is bundled in the OSB install - hopefully 12c will remove this issue
• Proxy message flows are pretty unreadable at first sight, what helped was - use 'Collapse All' to see the highest level stages and then expand
• Overall Proxy message flows are divided with multiple stages (with multiple pipeline pairs) ending with a Route Node
• It's very important to understand Service callout, Publish and Route Node - also the related thread behavior - great article here 
• As we implemented a very complex orchestration in OSB - one proxy service had almost 100s of activities calling around 30 services - with error handling code - It is very important that the code is readable and maintainable - so it's important to use the Notes feature to add documentation 

There were many challenges in the design aspects as we were building all the application logic in OSB services -

• how to design the PS-LPS structure - in order to improve code modularity we tried using as many LPS, however there was a debate on how to expose these LPSs if there is a need to expose them as services (by using Operation branching in one PS or the LPS by having its own PS)
• How to design the CRUD operations - as separate DBAdapter services or having one service with routing to all the business services example here
• Handling transactions and concurrency - we needed to update multiple records in a database table. Though OSB supports transactions, handling concurrency was a big challenge. We wanted to implement  optimistic locking - however the nature of fine-grained services made the interfaces difficult to implement the feature 
• We decided to implement all the stateless business logic in Java, which was primarily to iterate through multiple hash-maps and do matching kind of logic. From OSB these Java methods were called using Java Callout. XML Beans was used for the data binding between OSB and Java. This worked very well, however there was one challenge to make this Java code compiled as part of OSB code - we end up using Jar files as part of OSB code
• As all developers were doing parallel development of the same service (each developer doing some LPS) - there were challenges in building the code without errors initially

Couple of other issues -

• One issue that I will remember always from this implementation was about File transport. OSB needs File/FTP transports to have the staging directory in a shared storage (or network file system) so that all nodes in cluster can access it. As the polling configuration in proxy service is active-passive however the actual processing is always active-active. So if you don't have NFS or SAN in your cluster environment - you cannot use this transport. more here. Similar requirement for using HAFileAdapter or HAFtpAdapter.
• Also when we were trying using SFTP transport - the requirement was to use public-key based authentication, for which we needed to create a Service Key provider - for which unfortunately we didn't get any clear steps to follow. Which was easier for SFTP Adapter where the private key is provided as part of the adapter configurations.
• Had a issue with MQ transport around setting the correct encoding - UTF-8 - multiple properties on mq connector, transport header, MQRFH2 header properties etc. following these 2 steps resolved the issue
• Set the Character Set to 1208 in Transport Header activity.
• Copy All headers property in the Transport header activity, here the outbound context will be populated to queue

Is your SOA project unable to create WOW effect!

Are your customers not so impressed with the results of SOA projects and is the reason the projects take too long.

SOA/BPM came as an alternative for the only reason that it provides higher developer productivity and business agility (because for reusability). Is that promise met?

Even today all SOA projects (any small to medium package or feature) takes somewhere 4 to 6 months. This is where the WOW effect goes out.

The reasons are well known, recently a number of discussions on Industrializing SOA (or Factory model for SOA) cover that, can be found here and here

Like memory leaks, time leaks in the SOA project. In one we see 'Out of memory' and the other 'Out of budget'. Ask your customers how much time gets wasted because of these issues not well managed.

•        Development Approach
•   Agile / Iterative
•        Deployment automation
• Maven/Hudson Vs ANT
•  Continuous Integration
•  Configuration plans
•  Dependency between components
•      Testing automation
•  Regression testing / unit test scripts
•   Test coverage
•   Way to manage test data sets
•  Integration testing
•  Error Handling / monitoring
•        Version control
•    Binary management
•    Use of MDS
•    Virtualization for dev and test environments
•   Automation scripts (devops)
•  stubbing out system dependencies

 It's time to move the pre-industrial SOA customers to industrialization

What’s your favorite App Server?

If you are like me, you have worked all your life around a JVM + App server. From JDK1.2 to now Java7 And from J2EE to now JEE6. While we started as the Sun loyalists, somewhere in 2004/2005 - Spring became more popular, clearly killing Entity Beans also to some extent EJBs? Fast forward 2013 now as it stands, many competing stacks with JEE footprints

 1. JEE (Full profile) - Weblogic12c, Glassfish3.0, JBOSS AS7.1/EA6.0P, WebSpehere8.0
 2. JEE (WebProfile) - ApacheTomEE4.0 more here
 3. Servlet Containers - Jetty, tc server

To see the difference between JEE Web-profile and Full profile please see towards end of this page here (main difference are JMS, Web service technologies, JCA, Full EJB etc not included in Web profile)

 Now based on these Java platforms we have a number of PaaS coming up

1. Cloudfoundry from vmware based on tc server
2. Oracle Java cloud  based on Weblogic
3. OpenShift (JBoss cloud) based on JBoss EAP (or the new name Wildfly)

 (Not to forget Apache, MuleSoft and other open source platforms supported on AWS like IaaS)

Fight for Java PaaS is getting quite hot with the 3 major app server based PaaS providers try to win market share. Who will emerge winner? Your views?

Random Thoughts - 4

I have been thinking about multiple application types, patterns trying to label all the styles of applications we see in our world. Multiple triggers for these thoughts 1) This whitepaper by mulesoft on Evolution of SOA 2) Three Gartner magic quadrants related to application infrastructure here 3) arrival of Java EE 6, more info here
And in general people wondering, Is SOA dead? Is BPM worth the investment? should we go back to Java and not BPEL?
Here are two application maps I put together, please navigate and comment what kind of application you are building and what kind you want to build in future? (Also please mention if I missed any application styles)

That is a Technology view of applications, however it is important to know where the application fits in the overall business process of the Company. Here is another view showing applications based on business processes. There are many more business processes and applications, the picture is showing some of them.

So the questions one could ask for every project are -
1) Which business process does this application will be part of? e.g. customer booking process through customer service agents in a logistics company
2) What business objectives and benefits (competitive advantage?) does the application address/bring? e.g. carrying most profitable cargo, increased revenue
3) What technology platform will the application be built on? e.g. Oracle Fusion Middleware
4) What application pattern or style does this application has? e.g. SOA style application

Product Overlap or Choice

In the Oracle middleware product stack often there are products which seem to overlap in functionality causing confusion for customers/consultants to pick one over the other. Oracle explains it as a matter of choice that customers have to select the right product for their need. Over time things are getting clearer and this confusion is settling down. Below are my views on how these products should be seen and positioned.

BPM Suite 11g Vs SOA Suite 11g

There has been lot of discussions on this, a lot of good blogs are available on google, what makes it more interesting is BPM Suite and SOA Suite share a common SCA runtime platform on top of Weblogic. BPM Suite is based on BPMN notation and SOA Suite on BPEL. Now without getting into the controversy of what is BPM and what is SOA, one thing to remember is BPM Suite license costs are higher. So it’s much more likely that customer has SOA suite or it’s easier to justify the license cost as it has BPEL, human-workflow, business rules, b2b, Adapters and BAM. It has all the tools you would need to solve your integration problems, customizing workflow requirements for existing oracle apps. So SOA Suite along with OSB can be a complete middleware platform (mid-tier) for the enterprise. BPM Suite on the other hand is better fit to build top-down business process applications in cases where a packaged application is not suitable or available.

OSB (Oracle Service Bus) 11g Vs SOA Suite 11g

This is not much of selecting one over other issue as most customers have both the products, the challenge comes as to what to implement where, how much in OSB and how much in SOA Suite. What integration patterns fit one over the other? So the basic difference between OSB and SOA Suite is that OSB is much more light-weight unlike SOA Suite, SOA Suite because has a lot of engines running in it needs a much higher capacity, tuning and better administration. So for synchronous (req/resp) integration patterns with simple compositions OSB would be sufficient. It’s important to build a logging, auditing, monitoring framework with UI around OSB so as to complement what Enterprise Manager provides for BPEL. BPEL should be used where there is a case for orchestration/complex compositions, human-workflow and long-running processes/async. A lot of customers have implemented a lot more in BPEL which seems like an anti-pattern however with better capacity and administration it should not be a problem.

OSR (Oracle Service Registry) and OER (Oracle Enterprise Repository)

These two products under SOA Governance Suite cause lot of confusion, most customer end up not using either. As Governance is a political area, most customers remain in a policy-paralysis state without knowing how to answer the questions such as 1) how to identify new services? 2) what policies and process should be in place to build/deploy the services 3) ownership of services and how to reduce TCO (Total cost of ownership)


Now OSR and OER are both metadata repositories, they keep service metadata to improve visibility and control. The main difference is what kind of metadata they keep and what kind of analytics you can run on them. OSR is simply a UDDI compliant Registry (oem agreement with HP Systinet). It’s used for programmatic, dynamic discovery of services and provides control over service provisioning. OER on the other hand is a complete Asset management product used during the plan, design, and build phase of the service lifecycle (automate approval process) as a single source of truth for service and composite application development. It provides analytics with 20+ reports such as Investment ROI, Proactive Impact Analysis, value of service & asset portfolio.



There is another product OWSM (Oracle WebService Management) which causes some confusion. These confusions are primarily from a roadmap perspective as to how they will be positioned in future. OWSM is the policy-based security product for SOA Suite. There is another product called OEG (Oracle Enterprise Gateway -oem agreement with Vordel) which is similar to IBM Datapower product. OWSM is part of SOA Suite and OSB both and is the strategic product for all last-mile security implementations.



One more area of confusion is around ODI (Oracle Data Integrator) product, as to when to use ODI and when to use SOA Suite. This is more of an ETL vs SOA debate. In fact ODI uses a methodology called ELT and not ETL. This product is interesting with the cloud integrations on rise. Currently ODI is positioned to do primarily bulk Data Load kind of use cases however it supports web services based integrations.


I will cover the Weblogic suite products in another blog post.

SOA 11g

I finally got my first SOA11g project few months back, unfortunately I am not getting to do development as much I would like to, and that’s why my blogging frequency has also gone down. :-(

Anycase, here are some my experiences with SOA11g.

1) Customer had SOA11.1.1.2 (PS1), and now the latest version is 11.1.1.6(PS5). It's not too back that 11gPS1 was released but now there is no support for it. So first thing which bewilders me is the new versions that keep coming and customers not able to upgrade to them as quickly as we developers would like. Not to forget the exact version of JDev and the plugins.

2) We continued our work on PS1, and we hit a wall with one bug.

We were passing a custom variable in soap header along with wsse security header. Apparently there was a bug #13070260 - which was throwing java.lang.ArrayIndexOutOfBoundsException

As we could not get a back-port patch, we had to look for work-around and luckily there was one, as found here, to supress the wsa headers we used this flag oracle.soa.addressing.response.enabled=false in the composite.

3) We had used a Mediator-BPEL-Mediator pattern, which looked elegant but the repetitive transformations in Mediator and BPEL were bit of a pain, particularly when we could not see the payload information for mediator in enterprise manager (may be another bug). Also the interesting property passThroughHeader that has to be set in composite for the mediator to pass the soap headers forward.

4) We were using OWSM to secure the composites and that was having a few hick ups in setting up the jks keys, understanding the polices etc.

5)We used Business Rules in our composites, and that was quite a challenge to start using properly, as we had lot of bucketsets and everytime we re-imported the schemas, had to redo the decision tables. One interesting finding was, while rules execution in one of the rules it was always going to to default flow - so by using the conflict resoultion as Override instead of RunAfter or RunBefore solved the issue.

update25/Apr: If the schemas for rules has changed, instead of re-importing everything, re-importing only the changed element helps.


I also had my first OSB11g experience in this project - and that is for another blog.

more on AIA2.4/10g

Last week while deploying some integration flows on AIA2.4/SOA10g/Weblogic, had two interesting errors -

1) While deploying the ESB (EBS in AIA) got the error -

A Service or ServiceGroup of name "XXXEBSV1" already exists under parent AIASystem

Found a good blog on this here, which talks about an ESB bug - and the solution to that is
Either you delete the esb before redeploying, or if you are doing it from a script - we can deploy a esb project with no routing and having the same GUIDs for all the services. Deploying the ESB with No routing firth and they deploying the ESB with routing will resolve the issue.


2) The second issue was with correlation sets - we were running a huge loop across EBF -> ESBV1 -> Provider BPEL -> ResponseESBV1 -> EBF

And it was important that the correlation sets work properly for the loop to continue. So we had 3 points in the EBF where callbacks were to be received. We created 3 correlation sets in each scope invoke with yes and receive with no, associated with property with type normalized string, for property alias we picked the EBM_ID, It needs 2 entries in the wsdl one for requestEBM and one for Response EBM. On the Provider BPEL part also we created correlation set on the first received activity (with yes) and then at the invoke ResponseEBS (with no) one.

AIA2.4FP installation on SOA10.1.3.4/Weblogic9.2

After a long hiatus, I am back with a useful blog. I had to install AIA2.4 (AIA2.3 downloads were not available on edelivery) on SOA10.1.3.4 on the WebLogic9.2. I tried to avoid doing it as much as possible, given there were no documentation on this, other than a few OTN forum entries and few other useful links here and here , but finally after 3 days, it was successful.

this blog already pointed to issues with this install and the metalink note#822831.1

To start with - had to first set up SOA10.1.3.4 on Weblogic9.2. This is a pretty well established process, however I faced some challenges because of Windows7

First issue was the 10g installer would not even run complaining SWAP file size issues, I commented the SWAP size checks following this link

The second issue was a big issue, the installer kept failing at the OWSM step (Unable to locate a valid OC4J URI), the error was opmn would not stop properly, due to IP6 support in Windows7. Solution found from here and here

Basically I had to edit the opmn.xml file under opmn/config to change the stop-paramters as suggested in metalinknote#1206667.1. The other important step is to uncheck the IPv6 in the network properties.

After that, things were smooth barring a few "Error occurred during initialization of VM", "OPATCH_JAVA_ERROR=OPatch Exception while trying" errors. For VM issue, closing some applications helped. For opatch issues, running the patch inside opatch directory once resolved the issues.

So after installing
Oracle XE (increase process count)
irca.bat
SOA 10.1.3.1.0 with Advance option
Patch set 7272722 for 10.1.3.4
Weblogic9.2
Patch 7337034
Patch 7490612 For weblogic SOA Domain creation
MLR #9 p8589928_101340_GENERIC
No AIA patches required
AIA installer only Software Copy mode

(for all these ensure your directory names doesn't have spaces or else you might get "Unzip: No matching wildcard found for.." )

Now was the time for AIAFP2.4 AIA 2.4 Installer V17393-01

AIAFP install needs lot of prep work, I did the below
1)increased JTA time out to 300
2)setup AQJMS Startup class (getting the aqjms_v2.0.zip was a challenge, its no longer available on OTN, luckily I had it from a previous attempt)
here only create the startup class, no need to add as a weblogic startup class
3) setting up ADF libararies, downloaded adfinstaller10.1.3.4 and followed the instruction to copy the libararies to weblogic instance
run setupadf.cmd, copy the classpath and add to the weblogic startup classpath
4)copy the esbbootstrap.jar under the JVM_HOME/jre/lib/ext

After this was ready to run AIAFP, first thing was to setup the property files
1)set up the aiaenv.bat under AIA_HOME/bin
2)deploy.properties under AIA_HOME/config
3)FPwlInstall.proprties and server.properties under AIA_HOME/Infrastructure/install/wlscripts

Templates for these property files are available under AIA_HOME/install/templates

next was to run the install scripts under AIA_HOME/install/wlscripts

If you are lucky the following line should do it all...
ant -buildfile FPWLInstall.xml

But that doesn't happen often, so I had following error
Servlet: "AIAValidationSystemAPIServiceSoapHttpPort" failed to preload on startup in Web application

It turned out, this is a known bug AIA 2.4 INSTALLATION ERROR ON WEBLOGIC [ID 1059780.1], this patch didn't work easily, so I just modified the AIAApplicationWL.EAR to change the ValidationSystemAPIService.xml under ValidationSystemAPIService.war

So if your build fails in between, It's better to run the other sections separately, The ant scripts are well organized, you can run the DeployCAVS, DeployServices targets.

That's it, then you can see the beautiful screen of http://localhost:9700/AIA :-)

update26/July
update the weblogic start arguments with -Daia.home={your AIA Home}, this is required otherwise the aia core code will not work and you will see FileNotFound for AIAConfigurationProperties.xml

Random Thoughts -3

Last few months working on few pre-sales proposals, seeing couple of trends/patterns in many of them. Many of the projects customers are planning to start have a cloud piece. Many of the customers are using SFDC (Salesforce.com) or planning to move particularly in media, hospitality industries. Subscription based cost is driving a lot of technology choices. Customers are becoming very cost sensitive and want a very flexible costing model and execution models, where they can pay as they use in buy, build or cloud choices.

From solution perspective, It becomes new challenges around how to integrate with cloud, how to handle security, how to handle RASP (Performance in particular).

Other than the cloud trend, the other trend is the regular debate of orchestration vs choreography. So for every industry or cross-industry problem there are many packaged applications, now these applications also provide regular supporting peripheral/plumbing functionality. For example, an Order-Management App will have its own inbuilt workflow, will have OOB UI to manage the process flow or administration. Now the debate is whether to use the App's such in-built features or to use an esb/bpel layer to unify these functionalities centrally across applications. That's the choreography vs orchestration debate, ESB for stateless choreography and BPEL/BPM for stateful orchestration.

Either way positioning an ESB is easy, with all the security and visibility story around services. Positioning the BPM story is a challenge, customers in hurry resort to point-to-point integrations and some even don't go for commercial esbs. That's another trend I saw open-source ESBs the entire story around JBI, Mule, Fuse, Spring, OSGi etc.

Another trend I see is customers are more interested in data/dw, bi/reporting, mdm etc. And different compliance requirements are driving lot of strategic projects in this area.

My next interest is to find architecture patterns, how soa/bpm/e2.0 can be used both strategically and tactically, particularly oracle soa.

Random Thoughts - 2

My random thoughts series continues in the new year as well, good time to look at Middleware Tech in general and SOA11g in particular. One specific step in this regard was my SOA Certification 1Z0-451, I took this in a hurry to avoid the pain of reading endless pdfs. Luckily I passed the test with an average score of 73%. The test did throw many new things about SOA11g which I discovered after reviewing the questions in the test. Few of them are below, otherwise the test is a 'developer-certification' and very broadly spread across all the products (BPEL, Mediator, HWF, Rules, OWSM etc)

new features
1. Mediator now supports a new validation called schematron validation
2. Rules have the new feature of decision tables (bucketset is a new term)
3. Rules have lot of new features functions, links etc.
4. Fault-management framework has new features (at-least location of the files)
5. OWSM is complete new territory
6. SDO / Entity variable - brand new features
7. Mediator externalizing routing rules to Business rules
8. SpringContext integration support
9. Lot of new features in Mediator
10. EDN as configurable publish-subscribe infrastructure

My next focus area is to deep-dive SOA11g and find best-practices/integration patterns. Stay tuned.

Random Thoughts

Recently I joined a Vendor-neutral Indian Services Company, where Oracle SOA is just another Middleware among TIBCO, WebMethods, IBM, Microsoft and all the BPM vendors like Pega, Lombardi(now IBM), MetaStorm etc.

I had Couple of very interesting findings while interacting with people who have traditionally worked on EAI products like TIBCO, Webmethods and JCAPs etc.

One thing I noticed people tend to relate Service with WebService and hence synchronous or real-time. So they were more comfortable with terms like Messaging, JMS, near real-time etc. I was getting a feel that EAI is a much better term than SOA. To justify the whole SOA story, I researched more and found some discussions like wrapping asynchronous service by synchronous services pattern and so ESB, which can do that. So that reminded of an Article I had read in 2007 SOA = ESB + EAI, how true, I thought.

Moving away from SOA, I found strong discussions around BPM, I started goggling to understand what the BPM story is, and found that since 2009 BPM has gained more traction than SOA. Instead to put in other words BPM is front-ending SOA. Instead BPM and SOA are used in one breath like BPM/SOA. Also I found there are two types of BPM, pure-play BPM and the other BPM which is more like BPM/SOA. My understanding of BPM was, it’s an alternative to packaged applications and custom development. It can provide the rapid development benefits + it's centered on the human centric workflow tasks and SOA complementing with the system centric integration tasks.

Reading the book on Oracle BPM11g, It sounds perfect, unified platform, BPMN2.0 runtime in same server, my next quest is to find out what killer feature BPMN will do so that business folks will use it instead of Visio.

Migration to SOA11g

I had a sneak peek at SOA11g early this year, which I had blogged here and here. This time I wanted a migration-oriented approach to it, so I wanted to take a 10g bpel and migrate to 11g and check out. And what best 10g app to try other than SOA Order booking, and guess what, there is a tutorial on migrating SOA Orderbooking too. here it is. All this on 11gR1 PS1. Following the steps, I could migrate/deploy all the composites except the Customer Service which kept throwing

10:50:52 AM] [Deployer:149034]An exception occurred for task [Deployer:149026]deploy application CustomerService on soa_server1.: [J2EE:160149]Error while processing library references. Unresolved application library references, defined in weblogic-application.xml: [Extension-Name: oracle.soa.workflow.wc, exact-match: false]..

However I could run the complete flow in 11g by keeping the webservices running in 10g.


So What I found was the migration efforts would depend on the components you have like bpel, esb, rules, tasks, java-webservices. Things are pretty straight-forward for bpel, esb, rules and tasks. However for webservices it’s somewhat tricky - based on the three kind of webservice implementations in Orderbooking the ways to migrate differed.

Other than the new files composite.xml, xxx.ComponentType files for each component, I noticed the wsdls for adapter services are now split between abstract wsdls and xx.jca files and xx.mplan for esbs. Good Intro here.
One problem with such migrations would be that bpel and esbs from 10g will migrate as new composites, while if one would redesign them - they might be designed as one composite with both components. So migrations would defeat a major design aspect of composites. That is pretty evident if you look at the 11g design of SOA Orderbooking here.

One complete wow factor is the rules editor - all integrated in JDev, and all migration of rules and tasks - worked just fine. However there was no task forms migrated as part of this demo. I could understand the services, components and references in the composite.xml, however didn't understand what wires are doing. One important point while migration was to keep all the partnerlinks up and running, so while migrating to 11g, the bpels should actually be working on a running 10g environment.

High-level steps for migration, would be -

1. Document all your external resource configurations like dbadapter, jmsadpater etc. And create them in 11g.
2. Keep 10g running and migrate the bpels/esbs using JDev
3. Make necessary changes for adapter configurations, partnerlink host:port information, esb to bpel routing endpoints.
4. Compile/Deploy one by one and Test
Additional effort will be required for webservices, rules, tasks and any other special case configurations.

WebService Oracle AS Vs Weblogic

I had this issue - "java.lang.IllegalStateException: could not find schema type named" before(here) faced it again while trying to deploy a webservice that came as part of a Hyperion product called DRM. Basically the webservices are not portable across OracleAS and Weblogic. Even I guess webservices are not portable between 9.2 and 10.3, so much about standards but it is not that straight forward.

Anycase I could think of two ways to resolve the issue 1) to re build the web service using the WSDL or Java 2) using a different ANT script for weblogic build which can use wsdlc for the specific weblogic

I didnt had much success either way, finally bumped into something called smartupgrade, these upgrade tools are provided as part of JDev to upgrade OracleAS J2EE apps to weblogic10.3.

The smartupgrade tutorial here, can be downloaded here

Using this I could at least deploy my webservice to Weblogic10.3 and solved the problem temporarily.

more Weblogic

I realized I have not blogged anything for over 60days now, also suggests not much quality findings, however I have been working on mostly weblogic setup. We did a portal cluster setup, also did a webcenter install - I see more and more weblogic in all these.

Firstly on weblogic portal sever cluster setup, there were couple of issues in setting up the remote server and starting up the managed servers - the errors we encountered are generally

1)weblogic.ldap.EmbeddedLDAPException: Empty initial replica

This issue, I am still not sure why it happens, but with proper restarts it gets away.

2)com.bea.common.engine.SecurityServiceRuntimeException: [Security:097533]SecurityProvider service class name for WSRPIdentityAsserter is not specified..

This issue was a big issue while starting the managed servers from nodemanager, what I saw was the managed servers could be started from prompt using startManagedServer scripts. While strting using script it starts in Development mode, uses slightly different Java arguments, and while starting from nodemanager it uses production mode. One of the arguments ALTERNATE_TYPES_DIRECTORY could cause this - see this

Also in nodemanager.properties - there is a property called StartScriptenabled, setting that to true also might help.

I am not sure how the domain/ servers and the nodemanager work together however all these issues are due to improper path settings.

3)java.io.IOException: Executable /app/bea/wlp1032/user_projects/domains/f1domain_prod/common/nodemanager/bin/startWebLogic.sh does not exist

Again this one another nodemanger issue, while enrolling a remote server in node manager, it creates the server under domain/common/nodemanger instead of domain/servers - and the start scripts are not there. I am yet to find what’s the correct way to avoid these.

4)Caused By: weblogic.common.ResourceException: Failed to bind remote object (ClusterableRemoteRef(8770601822249760121S:amrndhl206.pfizer.com:f1domain_prod:AdminServer_prod null)/294 [weblogic.jdbc.common.internal.RemoteDataSource]) to replica aware stub at

This was a issues with setting up datasources for RAC db, it needs all JNDI names to be unique, see this

5) Connection reset Exception encountered while communicating with this node manager:
Nodemanger unreachable

This was a issue with setting up remote weblogic managed servers, there are wlst commands to enroll remote servers using nmEnroll(), this resolves this issue, however it may cause issues while startup if not done properly, steps here


Also I installed webcenter for adf and spaces development, It is another product on weblogic, some interesting stuff, more on this later.

Java Clients for Workflow services

You can build clients for workflow services using the APIs exposed by the workflow service. The APIs enable clients to communicate with the workflow service using local and remote EJBs, SOAP, and HTTP. Worklistapp is one such Client which uses the local EJBs to access the services. When we access worklfow services from bpel it uses the WSIF bindings which is nothing but native Java wrapped as WSDL. From any Java client code we can use either Remote or SOAP client types.

Good example from Oracle is here, however this is for OC4J based, for weblgic there will be slight changes in the wf_client_config.xml and the JAR files. In wf_client_config.xml we can use t3:// instead of ormi and weblogic.jndi.WLInitialContextFactory. For JAR files better to put all the ones mentioned also in specific order, a good discussion here

Another useful link here

API Java DOC rerference here

Very useful discussion here if you are getting Nullpointer while trying to update task outcomes obtained using ITaskQueryService here

update-2/june/10
While using the java client code from portal, for a logged in user, we were getting
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[dasha
02, BusinessUnitController, SoaGroup]
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef
.java:348)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef
.java:259)
It turned out, the portal server and soa server weblogic domains needed to be enabled for global trust. And it worked by setting same credential at the Domain/Securty/Advanced page.

Weblogic Custom-Authenticator

While creating the SSO solution between weblogic portal server and soa server, we used the OAM/Webgate approach, very well explained here and here.

However we realized that integrating with webgate was not enough, the user has to be authenticate in the weblogic container also, in order that different deployed web applications can use the same container authentication/authorization services.

Interesting posts on this here and here

The options for us was to use SSPI connector to OAM, however we couldn't use that as the enterprise-wide OAM deployment was not supporting it yet.

So the other option was to use weblogic custom identity assertion and authentication. To understand identity assertion good post here

so we got the sample code from here

After making necessary code changes to read our header variable (Please check the readme.pdf to build/deploy the jar), we were ready to go. We didn't use token or cookie decryption and for authentication we used a LDAP authenticator. This is a bit of duplication on authentication but we need the roles/groups that the user belongs. And for creating a LDAP authenticator very good entry here

Next is to build the portal and see how it performs with these users/groups.


update:may17
I had to SSO ruleauthor applictaion which is on SOA10.1.3.4/Weblogic9.2. So easiest way to SSO weblogic applications was to use the Identity assertion. The Default weblogic identity assertion would work fine, only requirement was to use as CLIENT-CERT in the web.xml for the ruleauthor. However I had to do SSO with the OAM/WebGate infrastructure. So as per the OAM policy we configured the ruleauthor URL, and when its accessed through the Apache OAM sends the header variables and obSSOCookie, Using a custom identity asserter built using as described above, we could SSO ruleauthor.

Worklistapp, Weblogic, Webgate, LDAP, SSO

All the items in my subject were a good challenge for me over last month, finally some success and so the blog-post. Generally all worklistapp customization documents which is here is on OC4J, only weblogic documentaion I could find is this. I had to get the worklistapp do authentication against LDAP and SSO with a weblogic portal application.

Now first some basics, worklistapp works out-of-box with JAZN (even with the SOA10.1.3.4 on Weblogic9.2). JAZN is a XML-file based user/group repository, this cannot work with any enterprise-wide deployment. All enterprises have centralized LDAP/OAM (Oracle Access Manager) for managing all their user identities. Now to change worklistapp to use LDAP was actually easy following this, only catch was metalink ID[729163.1], you need to make sure is_config.xml have all the correct attributes.


The next challenge was installing webgate on our 64bit linux, I tried with 64-bit webgate, correct libraries but the following error persisted -
httpd: Syntax error on line 425 of /app/soatst/apache2.2.6/conf/httpd.conf: API module structure 'obWebgateModule' in file /app/soatst/netpoint05/webgate/access/oblix/apps/webgate/bin/webgate.so is garbled - expected signature 41503232 but saw 41503230 - perhaps this is not an Apache module DSO, or was compiled for a different Apache version?

Finally I got the solution in metlaink ID [982922.1], which basically suggested compiling the apache in a 32 bit linux and moving the binaries, and then use 32bit webgate, and that just worked perfect. Not to forget the target 64bit linux cannot be a lower Linux version than the source.

Doing SSO was the last and final challenge, it took a good amount of discussion as how SSO works with webgate/OAM. We had a portal app running on a portal server, and worklistapp running on a different soa server both weblogic. We fronted both servers with webgate agent/apache, so when any protected URL is accessed webgate pushed the Login page in the portal, provided with userid/password, webgate did authentication, and set an obSSOCoockie and some header variables in the request. Same with SSO, when the SSOed application URL is accessed, webgate provided the same header variables and Cookie.

For worklistapp SSO, we had to deploy the customworklistapp, the code comes with the samples, it has to be built and deployed. Again to build and deploy in weblogic, one major error we faced was the local ejb references, which got resolved by using hot deployment of the customworklist.war directly under /soaApps/BPELPM and making an entry in META-INF/application.xml. Looking at the already deployed worklistapp.war helped package the customworklist.war with the weblogic.xml and weblogic-application.xml.

Anyway, to make SSO work, Login.Java required to be changed. The change I did is to introduce a usercn that I read from the header and use authenticate on behalf of admin user which required only the userid, which I could get from the header variables set by Webgate.

Using SSPI or SAML was not possible as the authentication was directly with LDAP and not through the weblogic realm. So for Identity assertions probably a custom authenticator could be required, need to explore that. good article here.