Today i got this error ORA-00600: internal error code, arguments: [ksnpost:ksnigb], [], [], [], [], [], [], [] in one of our shakedown instance (pre go live instance).
In alert log there were many errors like
Errors in file /fsd1/oracle/OROD/oroddb/10.2.0/admin/OROD_admin/udump/orod_ora_6911.trc: ORA-00600: internal error code, arguments: [ksnpost:ksnigb], [], [], [], [], [], [], []
ORA-00609: could not attach to incoming connection
ORA-06403: Unable to allocate memory.
Fri Jul 31 13:55:37 2009
Incremental checkpoint up to RBA [0x1a.21bf9.0], current log tail at RBA [0x1a.234ae.0]
Fri Jul 31 14:00:23 2009
Process startup failed, error stack:
Fri Jul 31 14:00:23 2009
Errors in file /fsd1/oracle/OROD/oroddb/10.2.0/admin/OROD_admin/udump/orod_psp0_18212.trc: ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn5
ORA-27303: additional information: skgpspawn5
Fri Jul 31 14:00:24 2009
trace file orod_psp0_18212.trc had error messages
Process startup failed, error stack:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn5
ORA-27303: additional information: skgpspawn5
*** 2009-07-31 14:01:24.304
In orod.log (listener log file) there were many error messages like
TNS-12518: TNS:listener could not hand off client connection
TNS-12549: TNS:operating system resource quota exceeded
TNS-12560: TNS:protocol adapter error
TNS-00519: Operating system resource quota exceeded
HPUX Error: 12: Not enough space
as you can see above messages clearly states that system is running out of resources. OS was totally out of physical memory as per swapinfo -tam command
problem was we had only 3GB memory allocated to SGA but processes parameter was set to 1000 and this cause an issue in the instance and to solve this we had to reduce Processes parameter to lower value around 400 to make it work.
No comments:
Post a Comment