Oracle EBS R12.2 Weblogic Admin Server Fails To Start Issue

 -- Oracle EBS R12.2 with Weblogic 11g

------------------------Error Description------------------------------

# While Starting Application With up_app.sh, Following Server Runs

1. OHS 

2. NODE Mgr

3. Concurrent 

=========================================================================================

But Admin Server Fails with Following Error in =====> adadminsrvctl.txt 

=========================================================================================


Checking if the Node Manager is already up..

Connecting to Node Manager ...

Successfully Connected to Node Manager.


The Node Manager is already up.


Starting server AdminServer ...

Error Starting server AdminServer: weblogic.nodemanager.NMException: Exception while starting server 'AdminServer'




ERROR: Unable to connect the AdminServer.



StackTrace: 

java.io.IOException

at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:196)

at weblogic.management.remote.common.ClientProviderBase.newJMXConnector(ClientProviderBase.java:84)

at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:369)

at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)

at oracle.apps.ad.util.WLUtil.initMBeanServerConnection(WLUtil.java:131)

at oracle.apps.ad.tools.configuration.EBSProvisioner.ebs_nmstart_adminsrv(EBSProvisioner.java:4141)

at oracle.apps.ad.tools.configuration.EBSProvisioner.ebs_nmstart_adminsrv(EBSProvisioner.java:4404)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at oracle.apps.ad.tools.configuration.EBSProvisioner.main(EBSProvisioner.java:9761)

Caused by: javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3://prodapps.nift.com.pk:7002: Destination unreachable; nested exception is: 

java.net.ConnectException: Connection refused; No available router to destination]

at weblogic.jndi.internal.ExceptionTranslator.toNamingException(ExceptionTranslator.java:40)

at weblogic.jndi.WLInitialContextFactoryDelegate.toNamingException(WLInitialContextFactoryDelegate.java:792)

at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:366)

at weblogic.jndi.Environment.getContext(Environment.java:315)

at weblogic.jndi.Environment.getContext(Environment.java:285)

at weblogic.jndi.WLInitialContextFactory.getInitialContext(WLInitialContextFactory.java:117)

at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:684)

at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:307)

at javax.naming.InitialContext.init(InitialContext.java:242)

at javax.naming.InitialContext.<init>(InitialContext.java:216)

at weblogic.management.remote.common.ClientProviderBase.makeConnection(ClientProviderBase.java:178)

... 11 more

Caused by: java.net.ConnectException: t3://prodapps.nift.com.pk:7002: Destination unreachable; nested exception is: 

java.net.ConnectException: Connection refused; No available router to destination

at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:216)

at weblogic.rjvm.RJVMFinder.findOrCreate(RJVMFinder.java:170)

at weblogic.rjvm.ServerURL.findOrCreateRJVM(ServerURL.java:165)

at weblogic.jndi.WLInitialContextFactoryDelegate$1.run(WLInitialContextFactoryDelegate.java:345)

at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)

at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:146)

at weblogic.jndi.WLInitialContextFactoryDelegate.getInitialContext(WLInitialContextFactoryDelegate.java:340)

... 19 more

Caused by: java.rmi.ConnectException: Destination unreachable; nested exception is: 

java.net.ConnectException: Connection refused; No available router to destination

at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:470)

at weblogic.rjvm.ConnectionManager.bootstrap(ConnectionManager.java:321)

at weblogic.rjvm.RJVMManager.findOrCreateRemoteInternal(RJVMManager.java:262)

at weblogic.rjvm.RJVMManager.findOrCreate(RJVMManager.java:199)

at weblogic.rjvm.RJVMFinder.findOrCreateRemoteServer(RJVMFinder.java:238)

at weblogic.rjvm.RJVMFinder.findOrCreateInternal(RJVMFinder.java:200)

... 25 more



AdminServer logs are located at /u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/logs


07/21/23-12:01:46 :: adadminsrvctl.sh: exiting with status 1 


================================================================================

 


Validated the passed arguments for the option ebs-get-serverstatus

AdminServer is currently not running.



 

-----------------------------END---------------------------------------


=========================================================================================

While Admin Server Log File Named => AdminServer.log Cited Following Error

=========================================================================================


####<Jul 21, 2023 12:01:44 PM PKT> <Info> <Server> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904590> <BEA-002609> <Channel Service initialized.> 

####<Jul 21, 2023 12:01:44 PM PKT> <Info> <Socket> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904603> <BEA-000415> <System has file descriptor limits of - soft: 65,536, hard: 65,536> 

####<Jul 21, 2023 12:01:44 PM PKT> <Info> <Socket> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904603> <BEA-000416> <Using effective file descriptor limit of: 65,536 open sockets/files.> 

####<Jul 21, 2023 12:01:44 PM PKT> <Info> <Socket> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904603> <BEA-000406> <PosixSocketMuxer was built on Apr 24 2007 16:05:00> 

####<Jul 21, 2023 12:01:44 PM PKT> <Info> <Socket> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904626> <BEA-000436> <Allocating 4 reader threads.> 

####<Jul 21, 2023 12:01:44 PM PKT> <Info> <Socket> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904627> <BEA-000446> <Native IO Enabled.> 

####<Jul 21, 2023 12:01:44 PM PKT> <Info> <IIOP> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922904764> <BEA-002014> <IIOP subsystem enabled.> 

####<Jul 21, 2023 12:01:45 PM PKT> <Error> <EmbeddedLDAP> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922905041> <BEA-000000> <Error opening the Transaction Log: /u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/data/ldap/ldapfiles/EmbeddedLDAP.tran (Permission denied)> 

####<Jul 21, 2023 12:01:45 PM PKT> <Error> <EmbeddedLDAP> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922905041> <BEA-000000> <Error Instantiating 'dc=EBS_domain': null> 

####<Jul 21, 2023 12:01:45 PM PKT> <Critical> <EmbeddedLDAP> <prodapps.nift.com.pk> <AdminServer> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1689922905046> <BEA-171522> <An error occurred while initializing the Embedded LDAP Server. The exception thrown is java.lang.ClassCastException: com.octetstring.vde.backend.BackendRoot cannot be cast to com.octetstring.vde.backend.standard.BackendStandard. This may indicate a problem with the data files for the Embedded LDAP Server. If the problem is with the data files and it can not be corrected, backups of previous versions of the data files exist in /u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/data/ldap/backup.> 

####<Jul 21, 2023 12:01:45 PM PKT> <Critical> <WebLogicServer> <prodapps.nift.com.pk> <AdminServer> <main> <<WLS Kernel>> <> <> <1689922905047> <BEA-000362> <Server failed. Reason: 


There are 1 nested errors:


java.lang.ClassCastException: com.octetstring.vde.backend.BackendRoot cannot be cast to com.octetstring.vde.backend.standard.BackendStandard

at weblogic.ldap.EmbeddedLDAP.start(EmbeddedLDAP.java:303)

at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)

at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)

at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)


> 

####<Jul 21, 2023 12:01:45 PM PKT> <Notice> <WebLogicServer> <prodapps.nift.com.pk> <AdminServer> <main> <<WLS Kernel>> <> <> <1689922905068> <BEA-000365> <Server state changed to FAILED> 

####<Jul 21, 2023 12:01:45 PM PKT> <Error> <WebLogicServer> <prodapps.nift.com.pk> <AdminServer> <main> <<WLS Kernel>> <> <> <1689922905069> <BEA-000383> <A critical service failed. The server will shut itself down> 

####<Jul 21, 2023 12:01:45 PM PKT> <Notice> <WebLogicServer> <prodapps.nift.com.pk> <AdminServer> <main> <<WLS Kernel>> <> <> <1689922905072> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN> 

####<Jul 21, 2023 12:01:45 PM PKT> <Info> <WebLogicServer> <prodapps.nift.com.pk> <AdminServer> <main> <<WLS Kernel>> <> <> <1689922905080> <BEA-000236> <Stopping execute threads.> 


====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Reason >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Appliation Run Script was executed through Root. it had changed permission of LDAP Data Files in Admin Server , Form Server , CoreServer

File Locations 

/u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/forms_server1/data/ldap/ldapfiles/EmbeddedLDAP.tran

/u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/oacore_server1/data/ldap/ldapfiles/EmbeddedLDAP.tran

/u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/oafm_server1/data/ldap/ldapfiles/EmbeddedLDAP.tran

/u01/oracle/R1228/fs2/FMW_Home/user_projects/domains/EBS_domain/servers/AdminServer/data/ldap/ldapfiles/EmbeddedLDAP.tran

=========================================================================================================================================

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Solution >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

1. Downapps running server

2. Change revoke permissions from Root and Grant to Oracle on above provided DataFiles.

3. Start again Apps Server

Comments

Post a Comment

Popular posts from this blog

Oracle Database Listener Configuration

Image
Since Listener is s Program to establish connection with the Database from other applications. Hence to establish a connection it must have a Database Service to Bind If no Service is binded then it listener will be working but showing no service in the LOG There are two type of  Service Binding in TCP/IP 1. Dynamic Service Registration Through Database . 2. Static Service Registration Through Listenr.ora .   1.  Dynamic Service Registration In this type of Registration,  service name is mapped in database in a way that, we either map listener description in the LOCAL_LISTENER parameter of the database or just add alisas in this parameter. Here I have just added the alias LOCAL_ORCLP and its detail string in mentioned in TNSNAME.ORA . You could also directly mention the string into the LOCAL_LISTENER PARAMETER Anyway, once this all is setup then we have two options to initiate the service,     1. execute the command Alter System Register     2....
Read more

Oracle Weblogic Repository DB Users Password Reset

Image
When Repository DB Schema Password  expires, then WEblogic Admin Server Could not be started, hence it will prompt following error in the Weblogic Admin  Console. ------------------------------------------------------------------------------------------------------------------------ Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.6.4.v20160829-44060b6): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: java.sql.SQLException: ORA-28000: The account is locked. Error Code: 28000         at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:331)         at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:326)         at org.eclipse.persistence.sessions.DefaultConnector.connect(DefaultConnector.java:138)         at org.eclipse.persistence.sessions.DatasourceLogin.connectToDataso...
Read more

Emailing With Large Text

 While using conventional PLSQL procedure, the Oracle UTL_SMTP.Write_Data have input parameter as varchar2 -- which has a limit of size, so while emailing large content through single Write_data, is not possible. so the solution is to pass the data to email through chunks. Below is the old Procedure. (Having trouble with size) CREATE OR REPLACE PROCEDURE SUPPORT_PORTAL.send_mail_html_new (    p_to            IN   VARCHAR2,    p_from          IN   VARCHAR2,    p_cc            IN   VARCHAR2,    p_bcc           IN   VARCHAR2,    p_subject       IN   VARCHAR2,    p_text_msg      IN   VARCHAR2 DEFAULT NULL,    p_attach_name   IN   VARCHAR2 DEFAULT NULL,    p_attach_mime   ...
Read more