performance issue of using Mysql driver with TClientDataSet

[eric_lmi]

Hi

there are two main problem. example code provided below.

1) memory usage is really hight
the database is about 10Mb size on mysql, but when i dump to TClientDataSet, i take over 500Mb, Any idea why, or did something wrong.
1.1) if i inserted all data to TClientDataSet first and call ApplyUpdates, the memory usage and pagefile(VM) used by my program will increas to almost double 1Gb.

2) slow speed. about 100 seconds to update(insert) only 500 records
each record size about 10Mb / 65000 = 153b, fetch from mysql server also slow, 6Mb size database take 15 mins - half hours to fetch back.

is that normal? my internet is over 4M ADSL. i am in UK and server in US. normal download speed from our US server over 100kbps.

any idea what cause the slow speed.

explain code below
--------------------
I am using the lastest Mysql dbx driver to send and fetch data from mysql server with the following structure

TSQLConnection - TSQLQuery - TDataSetProvider - TClientDataSet

the method to insert record which i used

for 0 to 65000
begin
ClientDataSet^.Insert;

ClientDataSet^.FieldByName('_ID').AsInteger := ID;
// insert all the field data to ClientDataSet
//insert is pretty fast but, take 50 times memory ?????
end

ClientDataSet^.ApplyUpdates(-1); // very slow here, and memory usage continue increasing.

[Dimon]

I could not reproduce the problem. Please, try to test application work on a local database. Is the problem arised in this case?

[eric_lmi]

I could not reproduce the problem. Please, try to test application work on a local database. Is the problem arised in this case?

tried local database, very fast. but i did a test with mysql administrator, backup the remote database server. seems over 50 times faster then using dbexprtess driver. is that normal?

[Dimon]

Try to use standard MySQLConnection driver. Does it solve the problem with performance?