OutOfMemoryException and connection pooling

OutOfMemoryException and connection pooling

Postby tomcat2001 » Wed 16 Sep 2009 18:26

I will try and explain the situation as best I can. I think I know the problem, but not sure what to do about this currently. We are getting an outofmemoryexception and the following will cause it. Note in the information I am using below, we are using big binary files. I'm referring to say 20-30 mb.

I have created a test app to simulate the behavior also.

When the app first runs we have some calls that in turn call other things and the long and short is that we are probably adding 4 items to the connection pool by doing this. See SomeOp method below.

After that occurs, I am doing a loop that reads a big binary file and then does an update in the db. Once it hits the 3rd or 4th loop it will generally give me an OutOfMemoryException. As I'm assuming things here since I don't know for sure, I'm expecting the connection pool has these 4 connections out there from the way SomeOp works. Now, when my loop is processing these big files, it starts going down these connection pool objects. The problem is since we're using such big data, the pools in memory start to take up way too much memory. And this in turn gives us our OutOfMemoryException.

I may be totally off base here, but my question is assuming we need to use connection pooling is there any alternative aside from not allowing that many items to get in the connection pool? And/Or what can we do? We are currently on the 4.0.20 version.

internal IDataReader RetrieveSections()
{
string query = string.Format("Select * From {0}",
"soapware_customizations_customstringsections");

PgSqlConnection connection = GetConnectionPgSql();
IDbCommand command = GetCommand(connection, query);
return command.ExecuteReader(CommandBehavior.CloseConnection);
}

private void SomeOp()
{
using (RetrieveSections())
{
using (RetrieveSections())
{
using( RetrieveSections())
{
using( RetrieveSections())
{
}
}
}
}
}
tomcat2001
 
Posts: 21
Joined: Wed 10 Sep 2008 20:29

Postby Shalex » Fri 18 Sep 2009 10:20

1. Try upgrading to the latest 4.55.42 version of dotConnect for PostgreSQL. Does this resolve the problem?
2. Please post here your call stack when the error occurs.
3. Does the problem persist if connection pooling is turned off (Pooling=false; in the connection string)?
4. Please send a test project with the DDL/DML script to us via contact form (http://www.devart.com/company/contact.html). We will try to reproduce the issue.
Shalex
Devart Team
 
Posts: 7659
Joined: Thu 14 Aug 2008 12:44

Postby tomcat2001 » Sat 19 Sep 2009 19:53

Shalex wrote:1. Try upgrading to the latest 4.55.42 version of dotConnect for PostgreSQL. Does this resolve the problem?

No

2. Please post here your call stack when the error occurs.

at System.String.GetStringForStringBuilder(String value, Int32 startIndex, Int32 length, Int32 capacity)\r\n at System.Text.StringBuilder.GetNewString(String currentString, Int32 requiredLength)\r\n at System.Text.StringBuilder.Append(String value)\r\n at Devart.Data.PostgreSql.e.b(Byte[] A_0)\r\n at Devart.Data.PostgreSql.e.a(Encoding A_0, PgSqlType A_1, Object A_2, Boolean A_3)\r\n at Devart.Data.PostgreSql.PgSqlCommand.a(String A_0, Boolean A_1)\r\n at Devart.Data.PostgreSql.PgSqlCommand.a(CommandBehavior A_0, IDisposable A_1, Int32 A_2, Int32 A_3)\r\n at Devart.Common.DbCommandBase.ExecuteDbDataReader(CommandBehavior behavior)\r\n at System.Data.Common.DbCommand.ExecuteReader()\r\n at Devart.Common.DbCommandBase.ExecuteNonQuery()\r\n at PerformanceTester.PerformOperations.UpdatePdf(Guid documentID, String itemType, String itemData, String checksum, String name, Guid itemID, Byte[] binaryData) in C:\Users\Lance Johnson\Documents\Visual Studio 2008\Projects\PerformanceTester\PerformOperations.cs:line 143\r\n at PerformanceTester.PerformOperations.Run() in C:\Users\Lance Johnson\Documents\Visual Studio 2008\Projects\PerformanceTester\PerformOperations.cs:line 181\r\n at PerformanceTester.Program.Main() in C:\Users\Lance Johnson\Documents\Visual Studio 2008\Projects\PerformanceTester\Program.cs:line 29\r\n at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)\r\n at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)\r\n at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()\r\n at System.Threading.ThreadHelper.ThreadStart_Context(Object state)\r\n at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)\r\n at System.Threading.ThreadHelper.ThreadStart()

3. Does the problem persist if connection pooling is turned off (Pooling=false; in the connection string)?

No it doesn't.

4. Please send a test project with the DDL/DML script to us via contact form (http://www.devart.com/company/contact.html). We will try to reproduce the issue.


I will do this.
tomcat2001
 
Posts: 21
Joined: Wed 10 Sep 2008 20:29

Postby Shalex » Mon 21 Sep 2009 14:57

We have answered you by e-mail.
Shalex
Devart Team
 
Posts: 7659
Joined: Thu 14 Aug 2008 12:44


Return to dotConnect for PostgreSQL