How to Resolve ORA-00600:[kwqmfidliot:ins] Internal Error

   In our production database we got this error ORA-00600: internal error code, arguments: [kwqmfidliot:ins], [], [], [], [], [], [], [] during OATM implementation.


In one of the trace file we found that our Workflow background queue "WF_DEFERRED_TABLE_M" got corrupted.


Trace file Contents

Dump event group for SYSTEM

kwqicaclcur: Error 600

             Cursor Session Number : 7921

             Cursor Session Serial : 2

             Cursor Pin     Number : 9

Error 600 in Queue Table "APPLSYS"."WF_DEFERRED_TABLE_M"

error 600 encountered during serving 37461

*** 2010-08-15 13:22:30.111

ORA-00600: internal error code, arguments: [kwqitnmptme:wait/done/exp], [0], [], [], [], [], [], []

  In oracle metalink any bugs were reported for this specially during OATM implementation which can cause workflow background queue to corrupt due to intransit data in the queue tables.

 In this case we had to rebuild our workflow background queue based on the following steps.

1. Drop the corrupted workflow queue. In this case make sure we need to use uppercase for the APPLSYS APPS:

sqlplus apps/apps @$FND_TOP/patch/115/sql/wfqued.sql APPLSYS

2. Now it's time to Recreate the corrupted workflow queue

sqlplus apps/apps @$FND_TOP/patch/115/sql/wfquec2.sql APPS APPLSYS 

3. Recreate WF_DEFERRED_TABLE_M_N1 index on the workflow queue table.

sqlplus apps/apps @FND_TOP/patch/115/sql/wfqidxc.sql APPLSYS APPS 

4. Re-populate Workflow Queue table WF_DEFERRED_TABLE_M with data. We need to make sure all the components of workflow are down like background engine, listeners etc.

sqlplus apps/apps @$FND_TOP/sql/wfbkgbld.sql APPLSYS

5. Recreate following indexes if they are mssing after queue creation.

WF_INBOUND_TABLE_PK
WF_OUTBOUND_TABLE_PK

6. Gather stats for Workflow tables to recreate missing histograms if any.

sqlplus apps/apps @$FND_TOP/patch/115/sql/wfhistc.sql APPLSYS

7. Recreate the Subscribers for queues 

sqlplus apps/apps @$FND_TOP/patch/115/sql/wfmqsubc2.sql applsys apps ( 11.5.10.2)

sqlplus apps/apps @$FND_TOP/patch/115/sql/wfmqsubc.sql applsys apps ( 11.5.10)

wfmqsubc.sql is changed to wfmqsubc2.sql that creates subscribers for
WF_NOTIFICATION_OUT and WF_NOTIFICATION_IN. (after 11.5.10.2)

8. Recreate the Grant & Synonyms

sqlplus apps/apps @FND_TOP/patch/115/sql/afwfqgnt.sql apps apps applsys apps

After implementing all steps mentioned above restart all the workflow engine related components.