performance issue of using Mysql driver with TClientDataSet

Discussion of open issues, suggestions and bugs regarding usage of dbExpress drivers for MySQL in Delphi and C++Builder
Post Reply
eric_lmi
Posts: 3
Joined: Tue 16 Sep 2008 11:20

performance issue of using Mysql driver with TClientDataSet

Post by eric_lmi » Fri 31 Oct 2008 18:03

Hi

there are two main problem. example code provided below.

1) memory usage is really hight
the database is about 10Mb size on mysql, but when i dump to TClientDataSet, i take over 500Mb, Any idea why, or did something wrong.
1.1) if i inserted all data to TClientDataSet first and call ApplyUpdates, the memory usage and pagefile(VM) used by my program will increas to almost double 1Gb.

2) slow speed. about 100 seconds to update(insert) only 500 records
each record size about 10Mb / 65000 = 153b, fetch from mysql server also slow, 6Mb size database take 15 mins - half hours to fetch back.

is that normal? my internet is over 4M ADSL. i am in UK and server in US. normal download speed from our US server over 100kbps.

any idea what cause the slow speed.

explain code below
--------------------
I am using the lastest Mysql dbx driver to send and fetch data from mysql server with the following structure

TSQLConnection - TSQLQuery - TDataSetProvider - TClientDataSet

the method to insert record which i used

for 0 to 65000
begin
ClientDataSet^.Insert;

ClientDataSet^.FieldByName('_ID').AsInteger := ID;
// insert all the field data to ClientDataSet
//insert is pretty fast but, take 50 times memory ?????
end

ClientDataSet^.ApplyUpdates(-1); // very slow here, and memory usage continue increasing.

Dimon
Devart Team
Posts: 2885
Joined: Mon 05 Mar 2007 16:32

Post by Dimon » Tue 04 Nov 2008 13:32

I could not reproduce the problem. Please, try to test application work on a local database. Is the problem arised in this case?

eric_lmi
Posts: 3
Joined: Tue 16 Sep 2008 11:20

Post by eric_lmi » Thu 06 Nov 2008 14:19

Dimon wrote:I could not reproduce the problem. Please, try to test application work on a local database. Is the problem arised in this case?
tried local database, very fast. but i did a test with mysql administrator, backup the remote database server. seems over 50 times faster then using dbexprtess driver. is that normal?

Dimon
Devart Team
Posts: 2885
Joined: Mon 05 Mar 2007 16:32

Post by Dimon » Fri 07 Nov 2008 14:33

Try to use standard MySQLConnection driver. Does it solve the problem with performance?

Post Reply