Increase Memory usage when read blob after upgrade to CrLab drivers [D2006 - D2007]

Discussion of open issues, suggestions and bugs regarding usage of dbExpress drivers for InterBase & Firebird in Delphi and C++Builder
Post Reply
savoweb
Posts: 3
Joined: Thu 14 Jun 2007 14:39

Increase Memory usage when read blob after upgrade to CrLab drivers [D2006 - D2007]

Post by savoweb » Wed 21 Nov 2007 11:10

Hi,
I begin used CrLab for real application.
My application read BLOB to stream (picture).
I see that the memory allocated increase slow at each blob read.
The increase is not the size of the blob but a smaller size.
I restore old borland drivers and memory usage not increase.

I make little mono thread demo that read blob and I check that the problem is in this code.
I make a loop in witch we create a SqlDataSet, select a record from the db, read the blob in a stream and release all objects.
The memory usage still increase (not all stream size).
It seem that the memory increase at each query.

When close delphi not show report leak.

I check with Firebird 1.54, Firebird 2.03 and D2006 or D2007 and we use as dblibrary fbclient.dll of the correct firebird version.

Code:

Code: Select all

procedure TfrmTest.btntestClick(Sender: TObject);
var
  dts: TSQLDataSet;
  Str: TMemoryStream;
  i: integer;
(*const
  coFetchAll = TSQLConnectionOption(301);*)
begin
  for i := 1 to 10  do
  begin
    Str := TMemoryStream.Create;
    dts := TSQLDataSet.Create(nil);
    try
      SQLConnection1.Open;
      (*SQLConnection1.SQLConnection.SetOption(coFetchAll, Integer(False));*)
      dts.SQLConnection := SQLConnection1;
      //  dts.CommandText := 'select FIRST 1000 * from FOTOSEDEHR order by IDFOTO';
      dts.CommandText := 'select FIRST 1000 IDFOTO from FOTOSEDEHR order by IDFOTO';
      dts.Open;
      while not dts.Eof do
      begin
        Str.Clear;
        GetFotoHR_ToStream_FromID(dts.FieldByName('IDFOTO').AsInteger, Str);
      //    GetFotoHR_ToStream_FromID2(dts.FieldByName('IDFOTO').AsInteger, Str);
        dts.Next;
      end;
      Str.Clear;
      dts.Close;
    finally
      Str.Free;
      dts.Free;
      SQLConnection1.Close;
    end;
  end;

  ShowMessage('OK');
end;

function TfrmTest.GetFotoHR_ToStream_FromID(id:integer;Stream:TStream):boolean;
var DS     :TSQLDataSet;
    StreamIn  :TStream;
begin
  result:=false;
    DS:=TSQLDataSet.Create(nil);
    DS.SQLConnection:=SQLConnection1;
    DS.CommandType:=ctQuery;
    DS.CommandText:='select IMG from FOTOSEDEHR where idfoto=:id';
    DS.ParamByName('id').AsInteger:=id;
    DS.Open;
    if (DS.Eof) or (ds.fieldbyname('img').IsNull) then
      stream.Size:=0
    else begin
      try
        StreamIn:=DS.CreateBlobStream(DS.fieldbyname('IMG'),bmRead);
        StreamIn.Position:=0;
        Stream.CopyFrom(StreamIn,StreamIn.Size);
        result:=true;
        StreamIn.Free;
      except
        result:=False;
      end;
    end;
    DS.Close;
    DS.Free;
end;

function TfrmTest.GetFotoHR_ToStream_FromID2(id:integer;Stream:TStream):boolean;
var
  dts     :TSQLDataSet;
begin
  result:=false;
  dts := TSQLDataSet.Create(nil);
  try
    dts.SQLConnection := SQLConnection1;
    dts.CommandType := ctQuery;
   // dts.GetMetadata := false;
    dts.CommandText := 'select IMG from FOTOSEDEHR where idfoto=:id';
    dts.ParamByName('id').AsInteger:=id;
    dts.Open;
    if (dts.Eof) or (dts.fieldbyname('IMG').IsNull) then
        stream.Size:=0
    else
    begin
      TBlobField(dts.FieldByName('IMG')).SaveToStream(Stream);
      Result := true;
    end;
    dts.Close;
  finally
    dts.Free;
  end;
end;

DB: Setting:
DriverName=InterBase by Core Lab
DataBase=127.0.0.1:DBPICUTURE
RoleName=
User_Name=sysdba
Password=masterkey
SQLDialect=3
BlobSize=-1
ErrorResourceFile=
LocaleCode=0000
InterBase by Core Lab TransIsolation=ReadCommited
WaitOnLocks=True
Charset=
CharLength=1
EnableBCD=True
OptimizedNumerics=True
LongStrings=True
UseQuoteChar=False
FetchAll=False
UseUnicode=False

VendorLib = FBCLIENT.DLL
LibraryName = dbexpida30.dll or dbexpida40.dll


CREATE TABLE FOTOSEDEHR
(
IDFOTO INTEGER NOT NULL,
IMG BLOB SUB_TYPE 0 SEGMENT SIZE 1024,
DIMENSIONE INTEGER DEFAULT 0
);

CREATE UNIQUE ASC INDEX IDX_FOTOSEDEHR_IDFOTO ON FOTOSEDEHR (IDFOTO);

Plash
Devart Team
Posts: 2844
Joined: Wed 10 May 2006 07:09

Post by Plash » Fri 23 Nov 2007 12:04

We could not reproduce the problem with memory leak on reading BLOBs. If you open and close TSQLConnection in a loop, memory usage is increased. It is also increased on reopenning connection when using standard Borland driver. But on creating a dataset and reading data, memory usage remains on the same level.

Please specify the exact version of your DbxIda (including release number).

savoweb
Posts: 3
Joined: Thu 14 Jun 2007 14:39

Post by savoweb » Wed 28 Nov 2007 09:57

I make little demo and screen shot.

CoreLab Drivers: 2.20.0.5
Firebird 2.0.3
Delphi 2006 SP2 + All Hot Fix
Delphi 2007/RadStudio SP3

Link demo:
Source + Firebird Backup
http://servicesadsl.proton.it/memorysample.zip
Binary + Firebird Embedded
http://servicesadsl.proton.it/memorysample_embed.zip

(Your library not included)

Screen shot memory usage:
Image

Plash
Devart Team
Posts: 2844
Joined: Wed 10 May 2006 07:09

Post by Plash » Mon 03 Dec 2007 13:55

We have fixed several memory leaks. This fix will be included in the next build of DbxIda.

Post Reply