very slow to insert a record

Discussion of open issues, suggestions and bugs regarding MyDAC (Data Access Components for MySQL) for Delphi, C++Builder, Lazarus (and FPC)
Post Reply
maxchenjun
Posts: 8
Joined: Tue 13 Nov 2007 10:26

very slow to insert a record

Post by maxchenjun » Wed 13 Aug 2008 19:50

Hello, I have a table with more than 100,000 records. And I have to sometimes add a few new records. Just insert, no need to read and show.
But I found the speed is very very slow. I think even though I just want to insert the record, but TMyTable load at first all the data records, so it is very very slow. Is this the case? Or how I can solve this problem? Thanks in advance.

And more for my slow solution:

I tried to set fetchAll property to false, and just fetch one row from the table.
And using both table.InsertRecord method and table.Insert and then table.post.
From debuging I know the program waits at the Post point.
Also I tried with MyQuery, but I need unicode value support and I could not insert unicode value directly to the sql statement.

Please help. Thanks!!

kaffeburk
Posts: 214
Joined: Mon 29 Jan 2007 08:03

Re: very slow to insert a record

Post by kaffeburk » Thu 14 Aug 2008 22:54

maxchenjun wrote:Hello, I have a table with more than 100,000 records. And I have to sometimes add a few new records. Just insert, no need to read and show.
I have a similar problem, with 800 000 records... What i did was using TmyDump (maybe i should have used TMyLoader...). Works a lot faster specially if You lock the table during the import. If i remeber right the problem is that TQuery cashes the information regardless if You have set fetch all or not...

A drawback is that You have to spend some time writing code that convert the data to SQL code first. Let me know if u need a object to generate the insert statements.

maxchenjun
Posts: 8
Joined: Tue 13 Nov 2007 10:26

Re: very slow to insert a record

Post by maxchenjun » Fri 15 Aug 2008 01:04

kaffeburk wrote: A drawback is that You have to spend some time writing code that convert the data to SQL code first. Let me know if u need a object to generate the insert statements.

Hi Kaffeburk,

Thanks for your suggestion. I will try as you suggested with TMyDump. Anyway if you could provide an object to generate the insert statements, it will be very useful for me. Thanks for your help. And if possible you can send this to me via email: [email protected]

kaffeburk
Posts: 214
Joined: Mon 29 Jan 2007 08:03

Re: very slow to insert a record

Post by kaffeburk » Sat 16 Aug 2008 11:36

done.

oz8hp
Posts: 151
Joined: Mon 18 Feb 2008 13:28
Location: Denmark
Contact:

Re: very slow to insert a record

Post by oz8hp » Mon 18 Aug 2008 07:21

kaffeburk wrote:done.
Do you still have this example?

I have an application that imports records from >200 Paradox tables and inserts them in MySQL and it is very slow, so any speedup suggestions is welcome.

You may use oz8hp @ hotmail . com for direct contact

maxchenjun
Posts: 8
Joined: Tue 13 Nov 2007 10:26

Post by maxchenjun » Mon 18 Aug 2008 09:23

TMyLoad is the solution.

It was fast enough for my situation. Thanks all!

oz8hp
Posts: 151
Joined: Mon 18 Feb 2008 13:28
Location: Denmark
Contact:

Post by oz8hp » Mon 18 Aug 2008 09:47

maxchenjun wrote:TMyLoad is the solution.

It was fast enough for my situation. Thanks all!
Any sample apps where used??

maxchenjun
Posts: 8
Joined: Tue 13 Nov 2007 10:26

Post by maxchenjun » Tue 19 Aug 2008 13:08

Any sample apps where used??
See help files for TMyLoad. There is an example that shows how it works.

Post Reply