Random Thoughts -3

Last few months working on few pre-sales proposals, seeing couple of trends/patterns in many of them. Many of the projects customers are planning to start have a cloud piece. Many of the customers are using SFDC (Salesforce.com) or planning to move particularly in media, hospitality industries. Subscription based cost is driving a lot of technology choices. Customers are becoming very cost sensitive and want a very flexible costing model and execution models, where they can pay as they use in buy, build or cloud choices.

From solution perspective, It becomes new challenges around how to integrate with cloud, how to handle security, how to handle RASP (Performance in particular).

Other than the cloud trend, the other trend is the regular debate of orchestration vs choreography. So for every industry or cross-industry problem there are many packaged applications, now these applications also provide regular supporting peripheral/plumbing functionality. For example, an Order-Management App will have its own inbuilt workflow, will have OOB UI to manage the process flow or administration. Now the debate is whether to use the App's such in-built features or to use an esb/bpel layer to unify these functionalities centrally across applications. That's the choreography vs orchestration debate, ESB for stateless choreography and BPEL/BPM for stateful orchestration.

Either way positioning an ESB is easy, with all the security and visibility story around services. Positioning the BPM story is a challenge, customers in hurry resort to point-to-point integrations and some even don't go for commercial esbs. That's another trend I saw open-source ESBs the entire story around JBI, Mule, Fuse, Spring, OSGi etc.

Another trend I see is customers are more interested in data/dw, bi/reporting, mdm etc. And different compliance requirements are driving lot of strategic projects in this area.

My next interest is to find architecture patterns, how soa/bpm/e2.0 can be used both strategically and tactically, particularly oracle soa.

Random Thoughts - 2

My random thoughts series continues in the new year as well, good time to look at Middleware Tech in general and SOA11g in particular. One specific step in this regard was my SOA Certification 1Z0-451, I took this in a hurry to avoid the pain of reading endless pdfs. Luckily I passed the test with an average score of 73%. The test did throw many new things about SOA11g which I discovered after reviewing the questions in the test. Few of them are below, otherwise the test is a 'developer-certification' and very broadly spread across all the products (BPEL, Mediator, HWF, Rules, OWSM etc)

new features
1. Mediator now supports a new validation called schematron validation
2. Rules have the new feature of decision tables (bucketset is a new term)
3. Rules have lot of new features functions, links etc.
4. Fault-management framework has new features (at-least location of the files)
5. OWSM is complete new territory
6. SDO / Entity variable - brand new features
7. Mediator externalizing routing rules to Business rules
8. SpringContext integration support
9. Lot of new features in Mediator
10. EDN as configurable publish-subscribe infrastructure

My next focus area is to deep-dive SOA11g and find best-practices/integration patterns. Stay tuned.

Random Thoughts

Recently I joined a Vendor-neutral Indian Services Company, where Oracle SOA is just another Middleware among TIBCO, WebMethods, IBM, Microsoft and all the BPM vendors like Pega, Lombardi(now IBM), MetaStorm etc.

I had Couple of very interesting findings while interacting with people who have traditionally worked on EAI products like TIBCO, Webmethods and JCAPs etc.

One thing I noticed people tend to relate Service with WebService and hence synchronous or real-time. So they were more comfortable with terms like Messaging, JMS, near real-time etc. I was getting a feel that EAI is a much better term than SOA. To justify the whole SOA story, I researched more and found some discussions like wrapping asynchronous service by synchronous services pattern and so ESB, which can do that. So that reminded of an Article I had read in 2007 SOA = ESB + EAI, how true, I thought.

Moving away from SOA, I found strong discussions around BPM, I started goggling to understand what the BPM story is, and found that since 2009 BPM has gained more traction than SOA. Instead to put in other words BPM is front-ending SOA. Instead BPM and SOA are used in one breath like BPM/SOA. Also I found there are two types of BPM, pure-play BPM and the other BPM which is more like BPM/SOA. My understanding of BPM was, it’s an alternative to packaged applications and custom development. It can provide the rapid development benefits + it's centered on the human centric workflow tasks and SOA complementing with the system centric integration tasks.

Reading the book on Oracle BPM11g, It sounds perfect, unified platform, BPMN2.0 runtime in same server, my next quest is to find out what killer feature BPMN will do so that business folks will use it instead of Visio.

Migration to SOA11g

I had a sneak peek at SOA11g early this year, which I had blogged here and here. This time I wanted a migration-oriented approach to it, so I wanted to take a 10g bpel and migrate to 11g and check out. And what best 10g app to try other than SOA Order booking, and guess what, there is a tutorial on migrating SOA Orderbooking too. here it is. All this on 11gR1 PS1. Following the steps, I could migrate/deploy all the composites except the Customer Service which kept throwing

10:50:52 AM] [Deployer:149034]An exception occurred for task [Deployer:149026]deploy application CustomerService on soa_server1.: [J2EE:160149]Error while processing library references. Unresolved application library references, defined in weblogic-application.xml: [Extension-Name: oracle.soa.workflow.wc, exact-match: false]..

However I could run the complete flow in 11g by keeping the webservices running in 10g.


So What I found was the migration efforts would depend on the components you have like bpel, esb, rules, tasks, java-webservices. Things are pretty straight-forward for bpel, esb, rules and tasks. However for webservices it’s somewhat tricky - based on the three kind of webservice implementations in Orderbooking the ways to migrate differed.

Other than the new files composite.xml, xxx.ComponentType files for each component, I noticed the wsdls for adapter services are now split between abstract wsdls and xx.jca files and xx.mplan for esbs. Good Intro here.
One problem with such migrations would be that bpel and esbs from 10g will migrate as new composites, while if one would redesign them - they might be designed as one composite with both components. So migrations would defeat a major design aspect of composites. That is pretty evident if you look at the 11g design of SOA Orderbooking here.

One complete wow factor is the rules editor - all integrated in JDev, and all migration of rules and tasks - worked just fine. However there was no task forms migrated as part of this demo. I could understand the services, components and references in the composite.xml, however didn't understand what wires are doing. One important point while migration was to keep all the partnerlinks up and running, so while migrating to 11g, the bpels should actually be working on a running 10g environment.

High-level steps for migration, would be -

1. Document all your external resource configurations like dbadapter, jmsadpater etc. And create them in 11g.
2. Keep 10g running and migrate the bpels/esbs using JDev
3. Make necessary changes for adapter configurations, partnerlink host:port information, esb to bpel routing endpoints.
4. Compile/Deploy one by one and Test
Additional effort will be required for webservices, rules, tasks and any other special case configurations.

WebService Oracle AS Vs Weblogic

I had this issue - "java.lang.IllegalStateException: could not find schema type named" before(here) faced it again while trying to deploy a webservice that came as part of a Hyperion product called DRM. Basically the webservices are not portable across OracleAS and Weblogic. Even I guess webservices are not portable between 9.2 and 10.3, so much about standards but it is not that straight forward.

Anycase I could think of two ways to resolve the issue 1) to re build the web service using the WSDL or Java 2) using a different ANT script for weblogic build which can use wsdlc for the specific weblogic

I didnt had much success either way, finally bumped into something called smartupgrade, these upgrade tools are provided as part of JDev to upgrade OracleAS J2EE apps to weblogic10.3.

The smartupgrade tutorial here, can be downloaded here

Using this I could at least deploy my webservice to Weblogic10.3 and solved the problem temporarily.

more Weblogic

I realized I have not blogged anything for over 60days now, also suggests not much quality findings, however I have been working on mostly weblogic setup. We did a portal cluster setup, also did a webcenter install - I see more and more weblogic in all these.

Firstly on weblogic portal sever cluster setup, there were couple of issues in setting up the remote server and starting up the managed servers - the errors we encountered are generally

1)weblogic.ldap.EmbeddedLDAPException: Empty initial replica

This issue, I am still not sure why it happens, but with proper restarts it gets away.

2)com.bea.common.engine.SecurityServiceRuntimeException: [Security:097533]SecurityProvider service class name for WSRPIdentityAsserter is not specified..

This issue was a big issue while starting the managed servers from nodemanager, what I saw was the managed servers could be started from prompt using startManagedServer scripts. While strting using script it starts in Development mode, uses slightly different Java arguments, and while starting from nodemanager it uses production mode. One of the arguments ALTERNATE_TYPES_DIRECTORY could cause this - see this

Also in nodemanager.properties - there is a property called StartScriptenabled, setting that to true also might help.

I am not sure how the domain/ servers and the nodemanager work together however all these issues are due to improper path settings.

3)java.io.IOException: Executable /app/bea/wlp1032/user_projects/domains/f1domain_prod/common/nodemanager/bin/startWebLogic.sh does not exist

Again this one another nodemanger issue, while enrolling a remote server in node manager, it creates the server under domain/common/nodemanger instead of domain/servers - and the start scripts are not there. I am yet to find what’s the correct way to avoid these.

4)Caused By: weblogic.common.ResourceException: Failed to bind remote object (ClusterableRemoteRef(8770601822249760121S:amrndhl206.pfizer.com:f1domain_prod:AdminServer_prod null)/294 [weblogic.jdbc.common.internal.RemoteDataSource]) to replica aware stub at

This was a issues with setting up datasources for RAC db, it needs all JNDI names to be unique, see this

5) Connection reset Exception encountered while communicating with this node manager:
Nodemanger unreachable

This was a issue with setting up remote weblogic managed servers, there are wlst commands to enroll remote servers using nmEnroll(), this resolves this issue, however it may cause issues while startup if not done properly, steps here


Also I installed webcenter for adf and spaces development, It is another product on weblogic, some interesting stuff, more on this later.

Java Clients for Workflow services

You can build clients for workflow services using the APIs exposed by the workflow service. The APIs enable clients to communicate with the workflow service using local and remote EJBs, SOAP, and HTTP. Worklistapp is one such Client which uses the local EJBs to access the services. When we access worklfow services from bpel it uses the WSIF bindings which is nothing but native Java wrapped as WSDL. From any Java client code we can use either Remote or SOAP client types.

Good example from Oracle is here, however this is for OC4J based, for weblgic there will be slight changes in the wf_client_config.xml and the JAR files. In wf_client_config.xml we can use t3:// instead of ormi and weblogic.jndi.WLInitialContextFactory. For JAR files better to put all the ones mentioned also in specific order, a good discussion here

Another useful link here

API Java DOC rerference here

Very useful discussion here if you are getting Nullpointer while trying to update task outcomes obtained using ITaskQueryService here

update-2/june/10
While using the java client code from portal, for a logged in user, we were getting
java.lang.SecurityException: [Security:090398]Invalid Subject: principals=[dasha
02, BusinessUnitController, SoaGroup]
at weblogic.rjvm.ResponseImpl.unmarshalReturn(ResponseImpl.java:234)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef
.java:348)
at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(ClusterableRemoteRef
.java:259)
It turned out, the portal server and soa server weblogic domains needed to be enabled for global trust. And it worked by setting same credential at the Domain/Securty/Advanced page.

Weblogic Custom-Authenticator

While creating the SSO solution between weblogic portal server and soa server, we used the OAM/Webgate approach, very well explained here and here.

However we realized that integrating with webgate was not enough, the user has to be authenticate in the weblogic container also, in order that different deployed web applications can use the same container authentication/authorization services.

Interesting posts on this here and here

The options for us was to use SSPI connector to OAM, however we couldn't use that as the enterprise-wide OAM deployment was not supporting it yet.

So the other option was to use weblogic custom identity assertion and authentication. To understand identity assertion good post here

so we got the sample code from here

After making necessary code changes to read our header variable (Please check the readme.pdf to build/deploy the jar), we were ready to go. We didn't use token or cookie decryption and for authentication we used a LDAP authenticator. This is a bit of duplication on authentication but we need the roles/groups that the user belongs. And for creating a LDAP authenticator very good entry here

Next is to build the portal and see how it performs with these users/groups.


update:may17
I had to SSO ruleauthor applictaion which is on SOA10.1.3.4/Weblogic9.2. So easiest way to SSO weblogic applications was to use the Identity assertion. The Default weblogic identity assertion would work fine, only requirement was to use as CLIENT-CERT in the web.xml for the ruleauthor. However I had to do SSO with the OAM/WebGate infrastructure. So as per the OAM policy we configured the ruleauthor URL, and when its accessed through the Apache OAM sends the header variables and obSSOCookie, Using a custom identity asserter built using as described above, we could SSO ruleauthor.

Worklistapp, Weblogic, Webgate, LDAP, SSO

All the items in my subject were a good challenge for me over last month, finally some success and so the blog-post. Generally all worklistapp customization documents which is here is on OC4J, only weblogic documentaion I could find is this. I had to get the worklistapp do authentication against LDAP and SSO with a weblogic portal application.

Now first some basics, worklistapp works out-of-box with JAZN (even with the SOA10.1.3.4 on Weblogic9.2). JAZN is a XML-file based user/group repository, this cannot work with any enterprise-wide deployment. All enterprises have centralized LDAP/OAM (Oracle Access Manager) for managing all their user identities. Now to change worklistapp to use LDAP was actually easy following this, only catch was metalink ID[729163.1], you need to make sure is_config.xml have all the correct attributes.


The next challenge was installing webgate on our 64bit linux, I tried with 64-bit webgate, correct libraries but the following error persisted -
httpd: Syntax error on line 425 of /app/soatst/apache2.2.6/conf/httpd.conf: API module structure 'obWebgateModule' in file /app/soatst/netpoint05/webgate/access/oblix/apps/webgate/bin/webgate.so is garbled - expected signature 41503232 but saw 41503230 - perhaps this is not an Apache module DSO, or was compiled for a different Apache version?

Finally I got the solution in metlaink ID [982922.1], which basically suggested compiling the apache in a 32 bit linux and moving the binaries, and then use 32bit webgate, and that just worked perfect. Not to forget the target 64bit linux cannot be a lower Linux version than the source.

Doing SSO was the last and final challenge, it took a good amount of discussion as how SSO works with webgate/OAM. We had a portal app running on a portal server, and worklistapp running on a different soa server both weblogic. We fronted both servers with webgate agent/apache, so when any protected URL is accessed webgate pushed the Login page in the portal, provided with userid/password, webgate did authentication, and set an obSSOCoockie and some header variables in the request. Same with SSO, when the SSOed application URL is accessed, webgate provided the same header variables and Cookie.

For worklistapp SSO, we had to deploy the customworklistapp, the code comes with the samples, it has to be built and deployed. Again to build and deploy in weblogic, one major error we faced was the local ejb references, which got resolved by using hot deployment of the customworklist.war directly under /soaApps/BPELPM and making an entry in META-INF/application.xml. Looking at the already deployed worklistapp.war helped package the customworklist.war with the weblogic.xml and weblogic-application.xml.

Anyway, to make SSO work, Login.Java required to be changed. The change I did is to introduce a usercn that I read from the header and use authenticate on behalf of admin user which required only the userid, which I could get from the header variables set by Webgate.

Using SSPI or SAML was not possible as the authentication was directly with LDAP and not through the weblogic realm. So for Identity assertions probably a custom authenticator could be required, need to explore that. good article here.

Business Rules, WebDAV

I am working on a weblogic portal integration with workflow/rules (SOA10.1.3.4 on weblogic9.2) project, so good opportunity to deep-dive into workflow/rules in bpel engine. The first candidate was rules. Oracle business rules has two important parts 1)rule author 2)decisionservice. More details about rules here in the Userguide.

I faced 2 distinct challenges

1) to use WebDAV repository, as using files repository was not the solution for production/clusters. To set up WebDAV on Apache2.2.15, here is a link to how to install. Take special care to enable all the six modules webDAV needs (./configure --prefix /app/soadev/apache --enable-dav=shared --enable-dav-fs=shared --enable-setenvif=shared --enable-alias=shared --enable-auth-digest=shared --enable-authn-file=shared). here is a good link to test WebDAV. Make sure you assign the user.group properly for /var and /uploads folders. In my case it was 'daemon'. I used auth type digest, and no SSL.

Once WebDav was set up, you can test it from browser (open with web folder option) to be able to write or not, From ruleauthor as well it connects, and you can create/import dictionaries.

2)The other challenge is to deploy the decision services after proper changes, for weblogic9.2 there are special instructions here. Also need to make sure to correct the context-root in application.xml, as I had multiple rules in multiple rule-sets, one decisionservice for rule-set was deployed and the context-root has to be modified to be specific for that ear.


After making sure some of these challenges, it worked just perfect. Next would be to look at workflow integration with rules.

update:Apr27
good article on migarting between file based and Webdav based repositories

SOA Projects

I could have named this post as SOA entry points, basically I wanted to list down the different flavors of SOA projects, which also indicate the different SOA Entry points.

SOA Entry points and the product line -

1.Process Automation - BPA / BPM / BPEL / ESB / Rules / Workflow
2.EAI - BPEL / ESB / AIA
3.B2B - B2B / BPEL / ESB
4.Process Portal - Portal / BPEL / ESB / Rules / Workflow
5.MDM - BPEL / ESB / MDM
6.BAM - BAM / BPEL
7.GOV - OER / OSR / BPEL / AIA
8.Data Services - ODI / OSB / BPEL
9.Appgrid/XTP - Weblogic / Coherence / BPEL


Looks like BPEL is the common thread, however with the other product mix the Solution Offering gets very unique and different. The other product OSB can be used in many of these projects. So while doing SOA projects, one should keep cognizance of the kind-of SOA project it is.

SOA 10.1.3.4 on Weblogic 9.2

I had installed SOA10.1.3.4 on Weblogic9.2 on Windows platform in 2008, when it was out first time. I had to install it on linux (single node) few days back, and it was simply perfect. I got it up and running in less than 2days. This blog entry is of good help. Also since I had a 11g DB, this entry is crucial.

I was stuck at one point while applying a patch to SOA10.1.3.4, and was getting opatch error - ERROR: "The Oracle Home does not meet OUI version requirement. OPatch failed with error code 73" [ID 729218.1]. This got resolved by using the 10.1 version of opatch patch#6880880.

It will be interesting to upgrade customer platforms running on 10g (10.1.3.4) to 10g (10.1.3.5) / weblogic10.3 or to 11g. The installation for sure is much simplified.

SOA 11g FOD (Fusion Order Demo)

FOD or Fusion Order Demo is the platform to learn any new Oracle product, I had used it while learning 10gR3, also AIA2.0, and now for 11g. I downloaded the FOD from here, the packaged deployment scripts were great and it just went through besides few hick-ups, tips here

However I couldn't execute the Order-booking demo end to end (some or other errors crop up oracle.jbo.SQLStmtException: JBO-27122: SQL error during statement preparation, or Caused By: javax.xml.soap.SOAPException: Message send failed: Connection refused), I could run the composite and played around em to learn the essential new concepts.

So from 10gR3(weblogic-based), 11g is much different and many important changes, let me try summarize some

1. Earlier we were creating 3 schema (orabpel, oraesb, oraowsm), now it’s more - mds, soainfra, bam and user messaging. MDS is the new thing, which is the metadata storage for all composites.

2. The terminology has changed, In one soa_server there are 4 service engines bpel, mediator (earlier esb), workflow and rules. The same server also has b2b, owsm. The bam server is separate and that’s new Java version.

3. The em is completely new, its shows all the composites, there instances, traces, component traces etc. I couldn't find test screens for components, its there at composite level. Admin stuff by right clicking on soa_infra.

4. Of course the whole idea of composite is new, and a composite has 3 parts service, components and References. So a lot of new config files, deployment fils etc. Complete new composite editor in JDev11g.

5. New frameworks around EDN (even driven networks), SDO (Service Data Objects), Spring etc. to make coding a different experience.

More details as and when I work on them, for now 11g looks just great!

SOA 11g install

finally I got some time to look at 11g and the first set of challenges were to 1) set Jdev 11g and 2) to install SOA 11g on Linux

1) For Jdev we need 11.1.1.2.0 above to be able to see SOA projects, After installing JDev the SOA composite editor can be installed using -> help -> check for updates

2) Installing 11g on Linux is covered in couple of blogs one here, official site here, broadly 5 steps
a) installing DB - 11.1.0.7 (linux_11gR1_database_1013.zip and patch 6890831)
b) using RCU for SOA schemas (ofm_rcu_linux_11.1.1.2.0_disk1_1of1.zip)
c) installing weblogic 10.3.2.0 (ope111130_wls1032_linux32.bin) using bundled Sun JDK
d) installing SOA 11g (ofm_soa_generic_11.1.1.2.0_disk1_1of1.zip)
e) configuring SOA for Weblogic

Good thing is this install is much easier that 10gR3 on weblogic9.2 as all configurations are wizard based.

For DB install all the steps are covered here - make sure all steps r followed also as in patch 6890831. For additional linux packages yum can be used. While dbconsole stop/start there will be an OC4J Configuration errror, the solutions is here

For weblogic/soa set up create a Middleware directory, chowner to the oracle user created as part of db install, as OUI will not work with root user.

While starting weblogic admin server if you see following error - your weblogic version is not proper, use 10.3.2.0

Failed to initialize the application 'em' due to error java.lang.NoSuchMethodError: weblogic.application.ApplicationLifecycleEvent.getDeploymentOperation()Lweblogic/application/DeploymentOperationType;.java.lang.NoSuchMethodError: weblogic.application.ApplicationLifecycleEvent.getDeploymentOperation()Lweblogic/application/DeploymentOperationType

While starting managed serevrs if you see errors like -
BEA-000386 Server subsystem failed. Reason: java.lang.NullPointerExceptionjava.lang.NullPointerException at weblogic.management.bootstrap.BootStrap.find(BootStrap.java:473)
or
BEA-000286 Failed to invoke startup class "JPS Startup Class", java.lang.ClassNotFoundException: oracle.security.jps.wls.JpsWlsStartupClassjava.lang.ClassNotFoundException: oracle.security.jps.wls.JpsWlsStartupClass at java.lang.Class.forName0(Native Method)

start the servers as root user and best way to start the managed servers is from the admin console, and for that start nodemanager, also make sure to modify nodemanger.properties under wlserver_10.3/common/nodemanager/ with StartScriptEnabled to true.

Next is to understand the big-bang changes in 11g and appreciate the value it has to offer.

bpel tuning, best practices and troubleshooting

Last week we went to production on SOA10.1.3.4 on Weblogic9.2, 3 node cluster, with lot of interfaces, and It was a very good exercise to prepare the servers, here are some references, if you read through all of them then you can avoid most of the issues ;)

1. tuning
check the apache-weblogic plugin parameters here
Oracle 10g tuning guide here (however lot of info here doesn't apply to weblogic)
tuning bpel quartz scheduler mentioned here
weblogic tuning guide here
linux parameters here (some good benchmark data here)
JVM parameters here (JVM Proxy settings here)
dehydation db tuning already covered as part of bpel tuning, however make sure DBAs tune processes, sessions, sga space, tablespace properly

2. best practices
if you have heavy batch jobs make sure you understand optSoapShortcut in cluster scenario, pls check this
metalink IDs for esb-bpel communication optimizations (780822.1 (esb-bpel) 726490.1 (bpel-esb))
bpel performance best practices here
esb performance best practices here , one more ppt, one more
a must read on bpel-adapter Integration here and optimizations metalink id [565944.1 and 730580.1]

3. troubleshooting
gcviewer to monitor gc memory issues
threaddump can be created on weblogic console, also can be created by kill -QUIT to monitor socket, thread issues
ORA-600 Errors Metalink ID [754336.1,18485.1, 333338.1 and 460244.1]
Cannot deserialize DOM element" Error When Running a BPEL Process [ID 559261.1]
FOTY0001: TYPE Error Using ORA:PARSEESCAPEDXML() and "&" in Input [ID 861637.1]
ORABPEL-11802 Error Calling Stored Procedure Using a Database Adapter [ID 887736.1]
BPEL waits well after expiry time set by wait or retry by fault policy[ID: 561172.1]

Careers in SOA

As the erstwhile EAI jobs become SOA jobs, it’s interesting to see how many dimensions are there to this job and how to enhance one's skill to be on top of it.

Technical (Type 1-4) - It is also possible that one person might have to be involved in all these types (also to be occupied through-out the project life-cycle), so one has to prepare for it. Also some of the work is product specific roles. By the way, learning the technology might not guarantee success, as they say attitude is the key :)

Work-Type1- This goes to the folks good in linux/unix, mostly the unix admin or dba - they can help in installing all the software (linux, database, app servers, jdk, etc.), RAC install, DR setup, backup activities, code migration, system upgrades, applying patches. These people can grow to be Infrastructure/Data Architects or Cloud Architects :)

Work-Type2- This goes to folks good in JVM, App Server, SOA Server, product architecture, call them 'Specialists' - they can configure the SOA/App servers, clusters, tune them as required, monitor system health, read log file, create SRs, finding patches, system sizing, create solutions for a product stack etc. These people can grow to be Technical/Solution Architects.

Work-Type3- This goes to the folks good in Java, XSLT, JMS, EJBs, transactions, SQL, PL/SQL, BPEL API, Adapters, they are the Developers - (Here I am not including the skills which are more primary in a E2.0 space including JSF, Spring, Hibernate, Struts etc., however a Developer may have to work on some of these if the project demands) . This group can grow to be Application/Solution Architects.

Work-Type4- Lets say this is the group of Architects, who take care governance aspects of the organization, including documentations as per Enterprise standards, coming up with the physical/logical architectures, enterprise security, hardware provisioning, roadmaps, big picture etc. This group is the Enterprise Architects.

Work-Type5- This is the group of project managers, sponsors who take care of planning, budgeting/financials, status tracking, business users reporting etc. This group is the Managers/Leaders of organizations.

I have not included the sales, testing, business analyst, training roles in these categories. Also assumed all role types know SOA and its principles.

Email CSV File As Attachment

I had to create a bpel, which gets data from couple of db tables, and create a csv file and send it as email attachment. I bumped into couple of interesting challenges, luckily google is always there to help :), let me capture some of the lessons learnt.

1. creating a csv file in append mode
2. setting/getting filename in the fileadapter
3. handling multiple records, nillable fields in nxsd
4. interesting xslt, xpath functions
5. sending email with attachment


Creating a csv file using file adapter was quite straightforward, you have to get a sample csv file that you want to crate and give that to the fileadapter wizard to create an nxsd (native xsd). more on it creating csv file here reading csv file here

As the file was created in append mode (how-to here), the file name has to be unique, it had to be set by creating a fileadapter header variable and passing it in invoke. If the file name is created by fileadapter, its name can be obtained as prescribed in technotes 10.1.3.3 by adding an output of header type to the write operation.

I had to write a header to the file, and fileadpater supports multiple record types, however all these record types need a condition Value for each record type to follow, which was not possible to define for data w/o starting with any fixed value, so I had to abandon the idea of using records, I printed the header separately as all my fields were string type. So the nxsd can be tweaked to change datatypes or making fields as optional (nillable=true). more here


While working thru transformations came across very effective xslt functions like translate, which replaces any specific character in the string with any other value. function create-delimited-string creates a delimited string out of a particular node in a repeating XML structure.

Finally sending the file as email attachment, bpel sample sendEMailWithAttchment clearly shows how to do it, however I faced two challenges. One was if the ora:readFile cannot find the file it will throw XSLT error (here ), which is difficult to debug. And some reason my text/html data was overwriting the csv data, for which I changed the order or data setting, setting csv first and then text/html.

update-Nov10

I could not use the translate method to replace newline characters, the sql replace(col,chr(10),null) helped there.

Also if you get javax.xml.xpath.XPathExpressionException: FOTY0001: type error for nothing wrong in XSLT, its because you edited the XSLT even before bpel loaded the parts in the Xform activity, so always wait till the parts load before clicking edit for the XSLT.

Update Dec-7

In order to make the directory to which we write the file as dynamic, we can edit the Outboundheader xsd to add 'directory' there after fileName. And send te directory name as a preference to the bpel at runtime.

transaction=participate

The first thing to note is global transactions will only work among sync processes, so an async bpel/esb cannot participate in a global transaction, so the way to make an async process sync is set delievryPersistPolicy to off.immediate.

Now partnerlinks in sync processes participate in global tx in 2 ways
1. set transaction=participate at the partnerlink (child BPEL or esb call) level
2. set transaction=participate at global level under in bpel.xml (It seems default in 10.1.3.4) - This is for all adapter calls (DBAdapter, AQ, JMS etc.) to participate in global tx

Now for the global tx to work, you have to use datasource with global tx enabled and using an XA driver(may not be true XA).

So when there is error (bindingfault) from dbadapter calls, it will mark the tx for rollback, but it will actually not rollback.

What we have seen is you have to set handleTopLevelFault=false in the configurations/bpel.xml so that rollback works. The other approach is to throw rollback exception in catch-all block.

The following seems to be best-way for global tx handling (10.1.3.4 on weblogic 9.2)
1. XA driver with Global tx enabled
2. in catch-all throwing rollback exception

This is rolling back the db inserts, as well dehydrating the bpel.

As bpel dehydration is happening, it seems it’s happening in a separate thread/tx (need to check if its a supported feature in 10.1.3.4). However we also saw the instance getting into recover(invoke) while using handleTopLevelFault=false.

a good discussion in OTN here and here
some good bpel tx details here and here

update on 23/10
10.1.3.4 does support audit in a seprate thread in async mode as per release notes.

ORABPEL-11825Attempt to use an unsupported database platform.

If you get this error -

WSIF JCA Execute of operation 'T' failed due to: Attempt to use an unsupported database platform.Database platform is not supported: oracle.toplink.platform.database.DatabasePlatform; nested exception is: ORABPEL-11825Attempt to use an unsupported database platform.

Then DBAdpater configuration needs set the PlatformClassName, sample values

• oracle.toplink.platform.database.Oracle9Platform
• oracle.toplink.platform.database.DB2Platform
• oracle.toplink.platform.database.SQLServerPlatform

you can also take this value from the MCF property settings.

Dynamic partnerlink

While crating EAI flows involving many bpels/esbs one common requirement is to be able to call some dynamic logic at run-time. For example say we have a set of requester and provider bpels, which transform the source-format to canonical and canonical to destination-format respectively. However we also have a requirement to do certain enrichment to the data if the source is A and some other enrichment if the source is B. since our requester bpel is common to both the sources we need to extend it in a manner that both the specific enrichment requirements for A and B are taken care while creating the canonical. So one-way to do this is to create source specific or "whatever" (which needs extension) specific bpels and call them from the common requester or provder bpels. And which bpel to call gets decided at run-time. This is a typical requirement for creating dynamic partnerlinks. This is covered in detail in this article

So lets say your master bpel is M and you need to call either A or B based on the source system. In order to create a dynamic link to A or B, roughly below are the steps
1. A and B should be based on same wsdl/xsd
2. In M we need to include this wsdl/xsd with an service endpoint url (this url doesn't matter as at runtime it will be replaced with actual url)
3. Creating a variable of type EndpointReference - and set the following XML fragment
[wsa:EndpointReference xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing"] [wsa:Address/][/wsa:EndpointReference]
4. The actual address has to be obtained from somewhere, either from database or from a config file
5. Once the actual address is obtained, set it to the EndpointReference address
6. Assign the EndpointReefrence to the partnerlink directly

Please refer to the article for sceenshots.