Excessive memory usage populating database

[sandy771]

I am using unidac to populate an sqlite database witha few million rows in a multiple tables using TUniTable->Append()...

I call startTransaction before the first transaction and then regularly throughout the process call commit and then start a new transaction. However the memory usage as seen in TaskManager just climbs until i get an out of memory error.

Am I missing something - should I be doing something else to keep the memory usage down?

[AlexP]

hello,

Regardless of whether you use transactions or not, all data inserted into a table with the help of the Append or Insert methods remain in the DataSet, this is why when inserting a large number of records you get the 'out of memory' exception. To solve this problem, you can use the following INSERT query:

UniQuery1.SQL.Text := 'INSERT INTO TABLE(FIELD1,..,FIELDN) VALUES(.....)'

[sandy771]

Thanks Alex

[sandy771]

I assume your TDALoader does not suffer from the same issue?

adn a further related (probably basic) question, how do I encode a date and time that I have been loading using TUniTable->AsDateTime ?

[AlexP]

Hello,

Yes, to load large amounts of data you can use TUniLoader, and in this case there will be no problems with memory.
To assign the DateTime type to a data field, you can use the standard IDE methods for value conversion to DateTime, e.g. StrToDateTime

TUniTable->AsDateTime = StrToDateTime("01.01.2010 12:12:12");