dump-backup memory errors

Discussion of open issues, suggestions and bugs regarding ADO.NET provider for MySQL
Post Reply
Strider
Posts: 1
Joined: Mon 25 Jul 2005 10:29

dump-backup memory errors

Post by Strider » Tue 30 Aug 2005 18:28

mysqldirect 3.0

I seem to be having a problem with the backup function MySqlDump.Backup. For a small database it seems to work very good and fast, yet for databases over 130 meg it dies during backup with a memory error unless I use

AddHandler MySqlDump.Progress, New MySqlDumpProgressEventHandler(AddressOf WriteProgress)

then it will continue through the backup with tables/databases less than 140 meg yet if I take this addhandler line out it dies with the aforementioned memory error.

I have also found that in trying to save the dumptext with the example of unicode encoding with stream.write it dies yet if I use just streamwriter it works. I am running this on a 2000 machine with a couple gig of ram and plenty of drive space.
My question is what can I do about this? I really need this functionality yesterday I mean what good is a live database if you can't back it up. Is there a way I can use the addhandler function to do something like the .progress yet to get the data and write that out as the backup progresses without going through dumptext? or can I increase the amount of memory uses from the 500 meg that the process appears to need from the task manager screen?

What I would like to be able to do is

function setup write data

function write data

function backup

turn off dump text
addhandler write data
make table strings
backup

since backup has a memory error I would assume restore would has the same memory error and since I have your attention is there someway around that error also

thanks for your time

Serious

Post by Serious » Wed 31 Aug 2005 08:55

At this moment MySqlDump cannot backup and restore data to/from stream. Maybe this feature will be added in the next versions of MySQLDirect .NET.
To avoid OutOfMemory exception you have to backup/restore tables separately.

GWoodcock
Posts: 3
Joined: Thu 16 Jun 2005 17:55

Out of memory exception with MySqlDump

Post by GWoodcock » Sat 15 Oct 2005 22:13

[quote="Serious"]At this moment MySqlDump cannot backup and restore data to/from stream. Maybe this feature will be added in the next versions of MySQLDirect .NET.
To avoid OutOfMemory exception you have to backup/restore tables separately.[/quote]

Just chiming in with my two cents - I also have hit the OutOfMemory exception, and the solution you propose does not solve my problem. The data in the tables I'm backing up routinely exceed 200 MB in size (each), so I can't use MySqlDump unless I partition the table data in some odd way (which is not particularly convenient). This is really a show-stopper bug for me, as far as using MySqlDump.

Being able to specify a stream/streamwriter for MySqlDump to use to periodically dump data to disk would be one way to solve this problem - I'd settle for a method that allowed me to specify a buffer size for MySqlDump to use, something like:

int bufferSize = 131072;
mySqlDump.BeginBackup();
while (mySqlDump.Backup(bufferSize))
{
myStreamWriter.WriteLine(mySQLDump.DumpText);
}
mySqlDump.EndBackup();

The idea here is that MySqlDump would keep track of the position in the table data between the BeginBackup and EndBackup calls (EndBackup would reset the position to the start of the data).

Any idea when this issue might be addressed? If it isn't going to be fixed soon, I'll have to find another solution other than MySQLDirect.NET to handle my needs (and I would like to avoid that).

Serious

Post by Serious » Tue 18 Oct 2005 06:38

Thanks for advice. We'll examine your suggestions.
If MySqlDump with its current design does no suit you, you still can backup database data manually with MySqlDataReader.

Post Reply