Virtual Table - How to prevent Out of Memory with huge XML files?

Hans Luijten · Post by **Hans Luijten** » Tue 26 Jun 2007 20:26

How do i prevent an out of memory error when loading (a rather large) XML file?

Antaeus · Post by **Antaeus** » Wed 27 Jun 2007 07:58

Please specify what is the size of the file you try to load, and how much memory do you have.

Hans Luijten · Post by **Hans Luijten** » Wed 27 Jun 2007 12:17

1Gb RAM, 300 Gb disk space free on C (swap drive), 369 Mb XML data file ...
I know it's large, I was wondering if some kind of "paging" would be possible ...

Antaeus · Post by **Antaeus** » Wed 27 Jun 2007 16:22

No, the LoadFromFile method does not support paging mode, and we do not plan to implement it in the near future, but we will investigate possibility to optimize memory usage when loading data from xml files.

Hans Luijten · Post by **Hans Luijten** » Thu 28 Jun 2007 01:22

OK sounds good.
I suspect the file to be growing in the future even more.
Would it be an idea to add something like a LIMIT?
For example 0,1000 for the first 1000 etc (like SQL LIMIT)?

Oh wait, I guess that would be close to paging then

Antaeus · Post by **Antaeus** » Mon 02 Jul 2007 07:03

Thank you for suggestion. We will consider possibilty to add this feature in one of the nearest builds.