Page 1 of 1

Out Of Memory Error

Posted: Tue 03 Sep 2013 23:33
by ashleysbuss
Sirs,

I am reading in US postal codes and some other data from a Excel Spreadsheet that contains 16 Columns and 80108 rows. I would like to read in all of the objects first and then do one write to the database using the dotConnect Entities framework latest download. But, this keeps error-ing out, so I tried this:

Code: Select all

 while (!String.IsNullOrEmpty(Sheet.getText(intIndex, 0)))
                    {
                        ZIPSEntities context = new ZIPSEntities();

                        var LocalZIPS = context.CreateObjectSet<ZIPS>();

                        LocalZIPS.AddObject(new ZIPS
                        {
                            strZIPCode = Sheet.getText(intIndex, 0),
                            strZIPType = Sheet.getText(intIndex, 1),
                            strCityName = Sheet.getText(intIndex, 2),
                            strCityType = Sheet.getText(intIndex, 3),
                            strCountyName = Sheet.getText(intIndex, 4),
                            strCountyFIPS = Sheet.getText(intIndex, 5),
                            strStateName = Sheet.getText(intIndex, 6),
                            strStateAbbr = Sheet.getText(intIndex, 7),
                            strStateFIPS = Sheet.getText(intIndex, 8),
                            strMSACode = Sheet.getText(intIndex, 9),
                            strAreaCode = Sheet.getText(intIndex, 10),
                            strTimeZone = Sheet.getText(intIndex, 11),
                            strUTC = Sheet.getText(intIndex, 12),
                            strDST = Sheet.getText(intIndex, 13),
                            longLatitude = Sheet.getNumber(intIndex, 14),
                            longLongitude = Sheet.getNumber(intIndex, 15)
                        });

                            intIndex++;

                            label1.Text = intIndex.ToString();
                            label1.Update();

                            context.SaveChanges();
                    }

                    Sheet.Dispose();

                    label1.Text = "Finished";
                    label1.Update();
                }
            } 


And it keeps error-ing with a Out Of Memory error at object 31,403 now if I comment out the context.SaveChanges() it will load all the objects just fine.

So what am I doing wrong?

Thanks,

Ashley

Re: Out Of Memory Error

Posted: Wed 04 Sep 2013 13:35
by ashleysbuss
Support,

Can I send you my project file and Excel file for you to play with? The Excel file is 7Mb and the project file is about 10Mb but I see you have a 2Mb limit on submissions.

Also, dotConnect Universal does it have the same functions as dotConnect SQLite apart from being able to target several databases with one set of code?

Thanks,

Ashley

Re: Out Of Memory Error

Posted: Thu 05 Sep 2013 16:13
by Shalex
ashleysbuss wrote:Can I send you my project file and Excel file for you to play with? The Excel file is 7Mb and the project file is about 10Mb but I see you have a 2Mb limit on submissions.
Could you please upload your test project to any file exchange server and send us a download link?
ashleysbuss wrote:Also, dotConnect Universal does it have the same functions as dotConnect SQLite apart from being able to target several databases with one set of code?
dotConnect Universal Professional Edition includes a runtime of the Standard Edition of dotConnect for SQLite: http://www.devart.com/dotconnect/sqlite/editions.html.

Re: Out Of Memory Error

Posted: Fri 06 Sep 2013 14:51
by ashleysbuss
Sirs,

Please find the following links to the project and the data file:

http://869fdd87471edbe99e20-b79814d8905 ... SQLite.zip

http://869fdd87471edbe99e20-b79814d8905 ... om/5-digit Commercial.zip

If you would be so kind as to let me know when I can take them down.

Thanks,

Ashley

Re: Out Of Memory Error

Posted: Thu 12 Sep 2013 10:13
by MariiaI
Thank you for the sample project. We have investigated it. In fact, no ORM is intended for insertion large amounts of data. For these purposes using the ADO.NET is more consistent - via the SQLiteConnection and SQLiteCommand with parameters, for example.

Re: Out Of Memory Error

Posted: Tue 17 Sep 2013 15:56
by ashleysbuss
Hi Support,

I finally, added a sleep for one second into my code and SQLite was happy to take in all of the records. It just took awhile!

This helps some of my projects to no-end, thanks for looking in this and for all of your help.

Take care,

Ashley

Re: Out Of Memory Error

Posted: Wed 18 Sep 2013 13:41
by MariiaI
Glad to see that the issue was resolved. If you have any further questions, feel free to contact us.