Monday, January 19, 2015

Oracle SOA Platform Overview

Introduction to Oracle SOA Administration 

Latest version from Oracle we can see Oracle SOA 12c that is in the year 2014 or 2015. As every new release there are many challenges expected for the existing projects who already running with Oracle SOA 11g. For our learning purpose better exercise on 11g first then migrate to 12c would give more clearity on every component why we are using this way!

Download Links: downloads.oracle.com
You can download 3GB size .zip file for 12c which is more focused on cloud specific features. I like the one step to go - one file to download in 12c. Well most of the projects initiated with 11g and the stable version of Oracle SOA Suite is available as 11g. So, ready for any thing at the moment :) You can select previous version that is 11g from the download bottom link contains here
select 11.1.1.7.0

Installation Products in Oracle SOA suite framework
SOA Practical environment for Development and Learning Labs

  • ofm_rcu_linux_11.1.1.7.0_64_disk1_1of1.zip
  • ofm_soa_generic_11.1.1.7.0_disk1_1of2.zip
  • ofm_soa_generic_11.1.1.7.0_disk1_2of2.zip
Click on the include features

Prerequsites SOA Installation

SOA Suite 2 parts
OSB [optional]

1. DB XE
Windows XE 64bit,32bit available now.
DB xe is for personal learnings, Complete oracle db is required for realtime implementation.

2. Oracle SOA Suite
adapter Seibel, Peoplesoft, iflex applications can connect with SOA Layer.

Middleware
Content service,

Oracle focused on Infrastructure
TIBCO is competitor - Middleware
Websphere Process Server - separate adapters

Why we have to use SOA?
1. Agility
2. Visibility
3. Efficiencies

Composites are created and sold to others. We are going to restarunt. You will be occupied a table. If the server taking you inputs from you and using a touch pad and send to kitchen services. Two SOA service, once this done the automatically bill will be generated. That is the 3rd service. This is called internal integration. When you use the credit card. that is external service that is payment gateway.
ShoppingCart example.
We can prepare the project that will prepare for online purchase.

Hairball integration architecture it is tightly coupled. Every composite is running from years.  before the SOA it was a mess.

Many people working on application is such a long the complete architecture is too difficulty.

Process gaps - this is integrations architecture is complex.

SOA suite you can track it with EM console. It is easy to find what happend in where in the application side.

Example: stock market what is happening in the site.
Last of month you did transaction it is not completed. how to trace these? Solution is provided by SOA. SOA will give you the date ranges, transactions that happened on that date. Which flow failed, rejected or issue. A transactions failed in 6 months back the customer report is valid then we can reprocessed.  If your application is Bank they maintain 6 months data in the active state. In some other businesses it will be every month we need to purge.

In the migrations you might get RCU upgrade.
Workflow automation - BPM after SOA in use it is separate composite. Loan service if you in need kitchen something then we can directly can go there and pick, instead of
navigation of

SOA Platform components in short description -
BAM - Monitoring and Reporting tool
EM - EM is with Fusion Middleware Infrastructure monitoring
Mediator and OSB looks like same. Both have similar functionality. Mediator work in between with in the organization components. OSB works when we have external services communications.

OSB - Integration

For example if you consider two banks -- HDFC, ICICI common Loan processing system. I need a loan that is best rates provided. If user is set a limitation <10% for this condition satisfied by HDFC work flow forwards it.

Service virtualization input is common the output would be in differs. OSB will understand the requirements and hand over/routing services to right destinations. OSB also having business logic will be there.



SOA Domain Creation


Directory structure after the SOA 12c installation got the following:
SOA 12c directory structure
Any fusion products you will get the Oracle_common, which contains common tools like repository creation utility. OPatch - Upgrading SOA the every quarter, we get a new patch from the Oracle.
Normally in WebLogic bsu command utility was inside the weblogic installation common, from WebLogic 12.1.3 onwards OPatch. Once you go for patching for SOA or FMW products 100% of patches will be effected only when the domain restarted.

Note: You must have the downtime for patching in FMW/SOA12c.

rda - work with Oracle Support for issues, It will collect the configurations of FMW products.
In SOA 11g it is separate directory Oracle_SOA1/OPatch
in SOA12c it is only soa folder

when we have the patches for SOA it will be given

To create SOA domain we have several paths where configuration wizard for WebLogic domain creation available. But, here I will create from SOA installation that is Oracle_SOA1 is for 11g the following ...

 cd $MW_HOME/Oracle_SOA1/
./config.sh

We can also can create a domain with WLST. See the references given below.
Oracle recommends to run the config.sh from Oracle_common/bin path

SOA 12c domain creation takes path and doamin name together

Domain Template with Composites select


By default Basic WebLogic server domain will be part of selected composites without that nothing will progress. And for your SOA administration requires following templates in 12c much clearer with groups.
1. Oracle SOA Suite
2. Oracle SOA Developer -> selected only for Developers
3. JRF, WSF Policy
4. Enterprise Manager

If any composite you missed while configuring the domain then we can use extend the domain.



Application directory for SOA_domain

Database details for SOA Domain 
Here if you get timezone error use export TZ="IST" with your preferred timezone for my system used IST. Provide the Database details such as service name, hostname, port, schema owner and password. here in this screen also we can see few improvements 12c compare to early version.

Create Admin, Managed servers in SOA_domain
After configuring Machines, and assigning the managed servers to the machine then you will reach to the following step..




In SOA11g we had the choice of select JRockit for production and Sund JDK for development environment where as in SOA 12c we have single choice Oracle JDK.

Schema already created then we get 4

Test Connection issue
IO Error - in this case the Database is not up.

Database Service Running
The default service ORCL for full version of Oracle Database. If it is XE make sure that XE service must be in running state.



Go back the domain creation wizard and test one of the schema test connection if it is successful then select all.


For our learning purpose SOA_domain, we can work with ORACLE XE. For OFMW products for WebCenter or ADF environments we must use ORCL service with Complete Oracle database other wise it throws errors saying not supported.


Summary of SOA Domain configuration


  1. Generally the SOA Installation process uses the following sequence.


  • Java
  • SOA, WebLogic
  • Oracle Database
  1. RCU Metadata repository - Data about Data. Each deployment also we need to purge the data.
  • SOA infrastructure
  • MDS
  • BAM schema
  • USER messaging schema
  1. Configure the AdminServer, managed servers
  • SOA_Server1
  • BAM_Server1
  • Machine creation.
  • add above servers
  • default datasources, JMS resources will be configured on top of the SOA, BAM servers
  • Thats the SOA domian got created.


References:
  1. Time Zone setting for Java Options

Install Oracle XE and Execution of RCU

Objective:
The RCU schemas can be executed on Oracle XE or Oracle RAC Database. For experimenting purpose we can work on XE.
  • Perform Install Oracle 10g/11g XE
  • Create the SOA related database schemas RCU

Install Oracle XE

Do the following pre-requisite things:

 
root@ubuntu:~# apt-get install libaio-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
  libaio1
The following NEW packages will be installed:
  libaio-dev libaio1
0 upgraded, 2 newly installed, 0 to remove and 48 not upgraded.
Need to get 20.2 kB of archives.
After this operation, 141 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://in.archive.ubuntu.com/ubuntu/ trusty/main libaio1 amd64 0.3.109-4 [6,364 B]
Get:2 http://in.archive.ubuntu.com/ubuntu/ trusty/main libaio-dev amd64 0.3.109-4 [13.8 kB]
Fetched 20.2 kB in 1s (19.9 kB/s)
Selecting previously unselected package libaio1:amd64.
(Reading database ... 181312 files and directories currently installed.)
Preparing to unpack .../libaio1_0.3.109-4_amd64.deb ...
Unpacking libaio1:amd64 (0.3.109-4) ...
Selecting previously unselected package libaio-dev.
Preparing to unpack .../libaio-dev_0.3.109-4_amd64.deb ...
Unpacking libaio-dev (0.3.109-4) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up libaio1:amd64 (0.3.109-4) ...
Setting up libaio-dev (0.3.109-4) ...
Processing triggers for libc-bin (2.19-0ubuntu6.5) ...
Now the latest libraries which required for Oracle XE can be installed with simple option -f in the following way
 
root@itpc-ubuntu:/home/itpc/Downloads# apt-get -f install
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following extra packages will be installed:
  gcc-4.9-base:i386 libaio1:i386 libc-dev-bin libc6 libc6:i386 libc6-dbg
  libc6-dev libgcc1:i386
Suggested packages:
  glibc-doc glibc-doc:i386 locales:i386
The following NEW packages will be installed:
  gcc-4.9-base:i386 libaio1:i386 libc6:i386 libgcc1:i386
The following packages will be upgraded:
  libc-dev-bin libc6 libc6-dbg libc6-dev
4 upgraded, 4 newly installed, 0 to remove and 44 not upgraded.
1 not fully installed or removed.
Need to get 4,064 kB/14.2 MB of archives.
After this operation, 9,879 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://in.archive.ubuntu.com/ubuntu/ trusty-updates/main gcc-4.9-base i386 4.9.1-0ubuntu1 [14.9 kB]
Get:2 http://in.archive.ubuntu.com/ubuntu/ trusty-updates/main libgcc1 i386 1:4.9.1-0ubuntu1 [47.9 kB]
Get:3 http://in.archive.ubuntu.com/ubuntu/ trusty-updates/main libc6 i386 2.19-0ubuntu6.6 [3,994 kB]
Get:4 http://in.archive.ubuntu.com/ubuntu/ trusty/main libaio1 i386 0.3.109-4 [6,578 B]
Fetched 4,064 kB in 5s (702 kB/s)
Preconfiguring packages ...
(Reading database ... 183715 files and directories currently installed.)
Preparing to unpack .../libc6-dev_2.19-0ubuntu6.6_amd64.deb ...
Unpacking libc6-dev:amd64 (2.19-0ubuntu6.6) over (2.19-0ubuntu6.5) ...
Preparing to unpack .../libc-dev-bin_2.19-0ubuntu6.6_amd64.deb ...
Unpacking libc-dev-bin (2.19-0ubuntu6.6) over (2.19-0ubuntu6.5) ...
Preparing to unpack .../libc6-dbg_2.19-0ubuntu6.6_amd64.deb ...
Unpacking libc6-dbg:amd64 (2.19-0ubuntu6.6) over (2.19-0ubuntu6.5) ...
Preparing to unpack .../libc6_2.19-0ubuntu6.6_amd64.deb ...
Unpacking libc6:amd64 (2.19-0ubuntu6.6) over (2.19-0ubuntu6.5) ...
Selecting previously unselected package libc6:i386.
Preparing to unpack .../libc6_2.19-0ubuntu6.6_i386.deb ...
Unpacking libc6:i386 (2.19-0ubuntu6.6) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up libc6:amd64 (2.19-0ubuntu6.6) ...
Processing triggers for libc-bin (2.19-0ubuntu6.5) ...
Selecting previously unselected package libgcc1:i386.
(Reading database ... 184020 files and directories currently installed.)
Preparing to unpack .../libgcc1_1%3a4.9.1-0ubuntu1_i386.deb ...
Unpacking libgcc1:i386 (1:4.9.1-0ubuntu1) ...
Selecting previously unselected package gcc-4.9-base:i386.
Preparing to unpack .../gcc-4.9-base_4.9.1-0ubuntu1_i386.deb ...
Unpacking gcc-4.9-base:i386 (4.9.1-0ubuntu1) ...
Setting up gcc-4.9-base:i386 (4.9.1-0ubuntu1) ...
Setting up libc6:i386 (2.19-0ubuntu6.6) ...
Setting up libgcc1:i386 (1:4.9.1-0ubuntu1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.5) ...
Selecting previously unselected package libaio1:i386.
(Reading database ... 184025 files and directories currently installed.)
Preparing to unpack .../libaio1_0.3.109-4_i386.deb ...
Unpacking libaio1:i386 (0.3.109-4) ...
Setting up libc-dev-bin (2.19-0ubuntu6.6) ...
Setting up libc6-dev:amd64 (2.19-0ubuntu6.6) ...
Setting up libc6-dbg:amd64 (2.19-0ubuntu6.6) ...
Setting up libaio1:i386 (0.3.109-4) ...
Setting up oracle-xe (10.2.0.1-1.0) ...
update-rc.d: warning: /etc/init.d/oracle-xe missing LSB information
update-rc.d: see 
Executing Post-install steps...
-e You must run '/etc/init.d/oracle-xe configure' as the root user to configure the database.

Processing triggers for libc-bin (2.19-0ubuntu6.5) ...
Processing triggers for ureadahead (0.100.0-16) ...
a) Create a special chkconfig script : The RedHat Linux installer of Oracle XE relies on
 /sbin/chkconfig
 which is not applicable in Ubuntu Linux. The chkconfig package available for the current version of Ubuntu but that produces errors and it may not be safe to use. So, below given shell script will do our need of install Oracle XE successfully way:
sudo vi /sbin/chkconfig

#!/bin/bash
# Oracle XE installer chkconfig hack for Ubuntu

F=/etc/init.d/oracle-xe
if [[ ! `tail -n1 $F | grep INIT` ]]; then
 echo >> $F
 echo '### BEGIN INIT INFO' >> $F
 echo '# Provides: OracleXE' >> $F
 echo '# Required-Start: $remote_fs $syslog' >> $F
 echo '# Required-Stop: $remote_fs $syslog' >> $F
 echo '# Default-Start: 2 3 4 5' >> $F
 echo '# Default-Stop: 0 1 6' >> $F
 echo '# Short-Description: Oracle Express Edition' >> $F
 echo '### END INIT INFO' >> $F
fi
update-rc.d oracle-xe defaults 80 01

(copy and paste the following into your vi editor) Save this file using :x and provide appropriate execute privilege shown below:
chmod 755 /sbin/chkconfig
b) Now lets set the Ubuntu Kernel parameters for Oracle XE requires to set the following:
sudo vim /etc/sysctl.d/60-oracle.conf 
(Enter the following)
# Oracle 11g XE kernel parameters  
fs.file-max=6815744  
net.ipv4.ip_local_port_range=9000 65000  
kernel.sem=250 32000 100 128 
kernel.shmmax=536870912 
(Save the file) 
Note: Here the kernel.shmmax = max possible value, The RAM size or Physical Memory need for Oracle XE installation. Remember that value must be provide in bytes only. (e.g. 512MB RAM == 512*1024*1024 == 536870912 bytes) Now lets verify the change :
sudo cat /etc/sysctl.d/60-oracle.conf 
Load new kernel parameters:
sudo service procps start  
Verify: sudo sysctl -q fs.file-max 
       -> fs.file-max = 6815744 
c) Increase the system swap space : Analyse your current swap space by following Linux command :
           free -m 

Check your swap space

Minimum swap space requirement of Oracle XE is 2 GB . In case, yours system fall under lower than that expected, you can increase it by following steps:

Create empty file:

This file will contain virtual memory contents so make file big enough for your needs. This one will create 1Gb file which means +1Gb swap space for your system:

  
dd if=/dev/zero of=/mnt/swp/swapfile.img bs=1024 count=6M
mkswap /mnt/swp/swapfile.img
vi /etc/fstab
swapon /mnt/swp/swapfile.img
free -m
cat /fdisk /dev/sdb
partprobe /dev/sdb
fdisk -l
mkfs /dev/sdb1 -t ext4
cd /mnt
mkdir swp
chmod 0777 swp
ls -l
mount /dev/sdb1 /mnt/swp -t ext4
chmod 0777 swp
ls -l
touch /mnt/swp/test.txt
ls swp/test.txtproc/swaps
grep Swap /proc/meminfo
increase swap space on Ubuntu
dpkg --install /home/pavanbsd/Downloads/oracle-xe_10.2.0.1-1.0_i386.deb
Oracle 10g XE installation on Ubuntu

/etc/init.d/oracle-xe configure
Oracle Database running as service on Ubuntu
Open apex url that will confirms us that installation is successful.
Oracle XE apex console

 
==========
How to do?
We can do first unzip the ofm_rcu* if it is SOA 11g. But in SOA 12c it is little different you don't need to download multiple files for installating RCU because it is part and parcel of SOA 12c.
Better store the extracted rcuHome at C:\ if it is on Windows. In Linux/Unix preferrably at home directory of user.

RCU is used to create the schemas in the Oracle XE DB.
When we run the rcu.bat or .sh, First it validate all pre-checks.

When /tmp not enough space
You can increase the /tmp space using the command as follows:

root@ubuntu:~# mount -t tmpfs -o size=4096M,mode=1777  overflow /tmp
root@ubuntu:~# mount
/dev/sda1 on / type ext4 (rw,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
none on /sys/fs/fuse/connections type fusectl (rw)
none on /sys/kernel/debug type debugfs (rw)
none on /sys/kernel/security type securityfs (rw)
udev on /dev type devtmpfs (rw,mode=0755)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755)
none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880)
none on /run/shm type tmpfs (rw,nosuid,nodev)
/dev/sdc1 on /mnt/apps type ext4 (rw)
/dev/sdc5 on /mnt/apps2 type ext4 (rw)
/dev/sdb1 on /mnt/swp type ext4 (rw)
gvfs-fuse-daemon on /home/pavanbsd/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=pavanbsd)
overflow on /tmp type tmpfs (rw,s
root@ubuntu:~# df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
/dev/sda1      ext4      6.8G  5.4G  1.1G  84% /
udev           devtmpfs  1.8G  4.0K  1.8G   1% /dev
tmpfs          tmpfs     351M  780K  350M   1% /run
none           tmpfs     5.0M     0  5.0M   0% /run/lock
none           tmpfs     1.8G  148K  1.8G   1% /run/shm
/dev/sdc1      ext4      7.8G  3.0G  4.4G  41% /mnt/apps
/dev/sdc5      ext4      7.8G   18M  7.4G   1% /mnt/apps2
/dev/sdb1      ext4      7.8G  6.1G  1.4G  82% /mnt/swp
overflow       tmpfs     4.0G     0  4.0G   0% /tmp
You may get some warnings that you can Ignore that is related to version warning.

Create new Schema 
We need to provide the Prefix : DEV
If you are using Single DB for Development, Test environments then for further use better to use prefix. This is the convenience to differentiate the environments.

cd $ORACLE_HOME/oracle_common/bin
./rcu

Running RCU

RCU welcome screen

Repository selection in RCU wizard

Entering Database details in RCU

Username - sys
Role we must use  SYSDBA, We have another option as Normal role this will be used to create non -SOA Schemas.

RCU Schema selection
 Component
SOA and BPM
1. SOA infra DEV_SOAINFRA
2. Business Activity  Monitoring -- DEV_BAM
3. User Messaging Service DEV_ORASDPM










One of the above component you selected then, MDS schemas are automatically selected.
Audit Service - trace who logged to the Database what query they are performed such information will be collected, This can be enabled on demand, most of the Production environment we don't prefer to use this. But Banking project may require this service schema must be enabled.

You need to provide the schema credentials, Best practice is to use different credentials and realtime environments Oracle DBA will assign this. It will create schema with same user credentials - here I prefer to use: welcome1

Next step will perform that, RCU will validated and create default tablespace, Temp tablespaces.
You can see rcu log can be created in the rcuHome/logs

Once everything completed click on "Finish" button.

Now if you have SQL Developer or SQL Plus command tool use it to confirm what we have done it is correct or anything missing.

When we create BAM schema need to select Existing prefix.

 connect sys as sysdba;

welcome1

You can confirm by connecting to each schema that you have created.

connect dev_mds


welcome1

To show list of the table:

select * from tab;


Remember this, there are many database tables created for SOAINFRA 10 different groups.


execute the following  query on sys user.

select distinct username from dba_users;


References:

  1. How to increase swap space?
  2. How to attach new hard drive to your Ubuntu OS? 
  3. How to install Oracle XE on Ubuntu?