Memory leak when selecting records with CLOB fields

Discussion of open issues, suggestions and bugs regarding ADO.NET provider for Oracle
Post Reply
hpalacio
Posts: 6
Joined: Fri 17 Mar 2006 15:29

Memory leak when selecting records with CLOB fields

Post by hpalacio » Wed 27 Jun 2007 21:05

I was hoping you could help us verify a Memory Leak problem we're having. We have developed a couple of applications which, over time, select hundreds of thousands to possibly millions of records.

We've notice a memory leak over time - an issue magnified when we started selecting CLOB fields. Initially, we tracked the leak to OraDirect .Net version 3.20.7.0 in non-direct mode. We recently upgraded and found that the problem still exists in the latest version 3.55.22.0.

To verify the issue, we have created a sample application which compares the MICROSOFT, ORACLE and CORELAB data providers. The Visual Studio 2005 Project files can be downloaded from Here: http://www.savefile.com/files/846348

This will make more sense when you run the application, but the sample application performs the following:
  • 1) Defines a SELECT statement that selects a CLOB field based on an ID field. (SELECT SOME_CLOB_FIELD FROM SOME_TABLE WHERE ID = :ID)

    2) Executes the SELECT statement once for each ID in a range beginning with a Starting ID to an Ending ID. For example:

    for ID = 1 to 100, select each row individually,
    where :ID=1, :ID=2, :ID=3... ID=100.

    3) Repeats the selection over the given range for the number of iterations defined in the UI. (For example, repeat :ID = 1-100 ten times.)

    4) Displays the STARTing memory, ENDing memory, DIFFerence in memory, and execution TIME.
The main purpose is to select hundreds of rows, individually, one at a time.

If I PREPARE and RE-USE the same command over and over again, the statistics produced look similar to the following:

Code: Select all

-------------------------------------------------
RUNNING CORELAB DATA READER...
-------------------------------------------------
	START		END		DIFF		TIME
-------------------------------------------------
0:	32,912	73,340	40,428	2,774.2051
1:	73,352	113,072	39,720	2,595.4647
2:	113,072	152,788	39,716	2,576.3235
3:	152,788	192,500	39,712	2,573.8167
4:	192,500	232,212	39,712	2,580.2148
5:	232,212	271,924	39,712	2,577.8351
6:	271,924	311,632	39,708	2,581.8845
7:	311,632	351,348	39,716	2,567.0974
8:	351,348	391,052	39,704	2,581.0777
9:	391,052	430,760	39,708	2,585.1092
-------------------------------------------------
DONE RUNNING CORELAB DATA READER
-------------------------------------------------
Notice the memory STARTS at 32,912K and ends at 430,760K ! It continually increases at about 39.7K per iteration.

However, if I RE-CREATE a NEW command for each row, the memory stays flat:

Code: Select all

-------------------------------------------------
RUNNING CORELAB DATA READER...
-------------------------------------------------
	START		END		DIFF	TIME
-------------------------------------------------
0:	431,096	431,132	36	3,914.8048
1:	431,132	431,132	0	3,804.5315
2:	431,132	431,132	0	3,810.4809
3:	431,132	431,132	0	3,802.6648
4:	431,132	431,132	0	3,804.7899
5:	431,132	431,132	0	3,790.4367
6:	431,132	431,132	0	3,810.8424
7:	431,132	431,132	0	3,813.1150
8:	431,132	431,132	0	3,802.0840
9:	431,132	431,132	0	3,800.7679
-------------------------------------------------
DONE RUNNING CORELAB DATA READER
-------------------------------------------------
Although continuously RE-CREATING the command solves the memory problem, we take a significant performance hit in the amount of time per execution.

Also note that we don't see the same issues with the other providers (Microsoft and Oracle).

Could you verify what we're seeing here?

Of course, you'll need to change the following on your side:
  • - The COMMAND_TEXT in the code to a SELECT statement relevant
    to a schema you can test on.

    - The ID_PARAMETER_NAME in the code to match the ID of the
    WHERE clause in your SELECT statement.

    - The connection string.

    - Project references to:
    CoreLab.Oracle (3.55.22.0)
    Oracle.DataAccess (10.2.0.100)
    System.Data.OracleClient (2.0.0.0)
We would greatly appreciate your help as we may need to re-evaluate our ability to integrate Corelab into our products. Please let me know if I need to clarify anything or provide more information.

Alexey
Posts: 2756
Joined: Mon 13 Mar 2006 07:43

Post by Alexey » Mon 02 Jul 2007 07:30

Thanks for your test project.
We will investigate this problem.
Look forward to hearing from me again.

hpalacio
Posts: 6
Joined: Fri 17 Mar 2006 15:29

Post by hpalacio » Tue 04 Sep 2007 14:30

I just received a notification about the latest 4.20 release: http://www.crlab.com/news/2007/directs420.html

Has the memory leak problem been addressed in this release?

Alexey
Posts: 2756
Joined: Mon 13 Mar 2006 07:43

Post by Alexey » Wed 05 Sep 2007 09:16

Yes, of course.

Post Reply