Virtual Table - How to prevent Out of Memory with huge XML files?

Virtual Table - How to prevent Out of Memory with huge XML files?

Postby Hans Luijten » Tue 26 Jun 2007 20:26

How do i prevent an out of memory error when loading (a rather large) XML file?
Hans Luijten
 
Posts: 12
Joined: Thu 07 Jun 2007 16:39
Location: Osceola, WI, USA

Postby Antaeus » Wed 27 Jun 2007 07:58

Please specify what is the size of the file you try to load, and how much memory do you have.
Antaeus
 
Posts: 2098
Joined: Tue 14 Feb 2006 10:14

Postby Hans Luijten » Wed 27 Jun 2007 12:17

1Gb RAM, 300 Gb disk space free on C (swap drive), 369 Mb XML data file ...
I know it's large, I was wondering if some kind of "paging" would be possible ...
Hans Luijten
 
Posts: 12
Joined: Thu 07 Jun 2007 16:39
Location: Osceola, WI, USA

Postby Antaeus » Wed 27 Jun 2007 16:22

No, the LoadFromFile method does not support paging mode, and we do not plan to implement it in the near future, but we will investigate possibility to optimize memory usage when loading data from xml files.
Antaeus
 
Posts: 2098
Joined: Tue 14 Feb 2006 10:14

Postby Hans Luijten » Thu 28 Jun 2007 01:22

OK sounds good.
I suspect the file to be growing in the future even more.
Would it be an idea to add something like a LIMIT?
For example 0,1000 for the first 1000 etc (like SQL LIMIT)?

Oh wait, I guess that would be close to paging then ;)
Hans Luijten
 
Posts: 12
Joined: Thu 07 Jun 2007 16:39
Location: Osceola, WI, USA

Postby Antaeus » Mon 02 Jul 2007 07:03

Thank you for suggestion. We will consider possibilty to add this feature in one of the nearest builds.
Antaeus
 
Posts: 2098
Joined: Tue 14 Feb 2006 10:14


Return to VirtualDAC