Memory leak when selecting records with CLOB fields

Memory leak when selecting records with CLOB fields

Postby hpalacio » Wed 27 Jun 2007 21:05

I was hoping you could help us verify a Memory Leak problem we're having. We have developed a couple of applications which, over time, select hundreds of thousands to possibly millions of records.

We've notice a memory leak over time - an issue magnified when we started selecting CLOB fields. Initially, we tracked the leak to OraDirect .Net version 3.20.7.0 in non-direct mode. We recently upgraded and found that the problem still exists in the latest version 3.55.22.0.

To verify the issue, we have created a sample application which compares the MICROSOFT, ORACLE and CORELAB data providers. The Visual Studio 2005 Project files can be downloaded from Here: http://www.savefile.com/files/846348

This will make more sense when you run the application, but the sample application performs the following:

    1) Defines a SELECT statement that selects a CLOB field based on an ID field. (SELECT SOME_CLOB_FIELD FROM SOME_TABLE WHERE ID = :ID)

    2) Executes the SELECT statement once for each ID in a range beginning with a Starting ID to an Ending ID. For example:

    for ID = 1 to 100, select each row individually,
    where :ID=1, :ID=2, :ID=3... ID=100.

    3) Repeats the selection over the given range for the number of iterations defined in the UI. (For example, repeat :ID = 1-100 ten times.)

    4) Displays the STARTing memory, ENDing memory, DIFFerence in memory, and execution TIME.
The main purpose is to select hundreds of rows, individually, one at a time.

If I PREPARE and RE-USE the same command over and over again, the statistics produced look similar to the following:

Code: Select all
-------------------------------------------------
RUNNING CORELAB DATA READER...
-------------------------------------------------
   START      END      DIFF      TIME
-------------------------------------------------
0:   32,912   73,340   40,428   2,774.2051
1:   73,352   113,072   39,720   2,595.4647
2:   113,072   152,788   39,716   2,576.3235
3:   152,788   192,500   39,712   2,573.8167
4:   192,500   232,212   39,712   2,580.2148
5:   232,212   271,924   39,712   2,577.8351
6:   271,924   311,632   39,708   2,581.8845
7:   311,632   351,348   39,716   2,567.0974
8:   351,348   391,052   39,704   2,581.0777
9:   391,052   430,760   39,708   2,585.1092
-------------------------------------------------
DONE RUNNING CORELAB DATA READER
-------------------------------------------------


Notice the memory STARTS at 32,912K and ends at 430,760K ! It continually increases at about 39.7K per iteration.

However, if I RE-CREATE a NEW command for each row, the memory stays flat:

Code: Select all
-------------------------------------------------
RUNNING CORELAB DATA READER...
-------------------------------------------------
   START      END      DIFF   TIME
-------------------------------------------------
0:   431,096   431,132   36   3,914.8048
1:   431,132   431,132   0   3,804.5315
2:   431,132   431,132   0   3,810.4809
3:   431,132   431,132   0   3,802.6648
4:   431,132   431,132   0   3,804.7899
5:   431,132   431,132   0   3,790.4367
6:   431,132   431,132   0   3,810.8424
7:   431,132   431,132   0   3,813.1150
8:   431,132   431,132   0   3,802.0840
9:   431,132   431,132   0   3,800.7679
-------------------------------------------------
DONE RUNNING CORELAB DATA READER
-------------------------------------------------

Although continuously RE-CREATING the command solves the memory problem, we take a significant performance hit in the amount of time per execution.

Also note that we don't see the same issues with the other providers (Microsoft and Oracle).

Could you verify what we're seeing here?

Of course, you'll need to change the following on your side:

    - The COMMAND_TEXT in the code to a SELECT statement relevant
    to a schema you can test on.

    - The ID_PARAMETER_NAME in the code to match the ID of the
    WHERE clause in your SELECT statement.

    - The connection string.

    - Project references to:
    CoreLab.Oracle (3.55.22.0)
    Oracle.DataAccess (10.2.0.100)
    System.Data.OracleClient (2.0.0.0)

We would greatly appreciate your help as we may need to re-evaluate our ability to integrate Corelab into our products. Please let me know if I need to clarify anything or provide more information.
hpalacio
 
Posts: 6
Joined: Fri 17 Mar 2006 15:29

Postby Alexey » Mon 02 Jul 2007 07:30

Thanks for your test project.
We will investigate this problem.
Look forward to hearing from me again.
Alexey
 
Posts: 2756
Joined: Mon 13 Mar 2006 07:43

Postby hpalacio » Tue 04 Sep 2007 14:30

I just received a notification about the latest 4.20 release: http://www.crlab.com/news/2007/directs420.html

Has the memory leak problem been addressed in this release?
hpalacio
 
Posts: 6
Joined: Fri 17 Mar 2006 15:29

Postby Alexey » Wed 05 Sep 2007 09:16

Yes, of course.
Alexey
 
Posts: 2756
Joined: Mon 13 Mar 2006 07:43


Return to dotConnect for Oracle