connection open failure with SSL

[dsoltesz]

We have an application which uses dotConnect for postgreSQL 7.8. Application has multiple threads with table adapters and they are able to connect to the database parallel.

At site randomly there is a hang to the database communication. It is very hard to reproduce, but we managed remote debug, saved dump file and found the following current call stacks for different threads at the "hanged" status.
Connection pooling is on with 100, SSL is "Required", connection timeout = 0 and therefore in this status the application hangs forever. We understand it is not adviced to be set to 0, and we will change it in the next release, but I assume it will just transform the "hanging forever" situation to an application crash with "Timeout expired", which is still not good enough. Checked with dbmonitor, the ususal connection number is 6-8, so nothing close to 100.

Thread 1:
mscorlib.dll!System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle waitableSafeHandle, long
millisecondsTimeout, bool hasThreadAffinity, bool exitContext) + 0x2b bytes
mscorlib.dll!System.Threading.WaitHandle.WaitOne(int millisecondsTimeout, bool exitContext) + 0x2d bytes
Devart.Data.PostgreSql.dll!Devart.Security.SSL.al.d(System.IAsyncResult A_0) + 0x141 bytes
Devart.Data.PostgreSql.dll!Devart.Security.SSL.s.b(System.IAsyncResult A_0) + 0x2b bytes
Devart.Data.PostgreSql.dll!Devart.Security.SSL.s.a(byte[] A_0, int A_1, int A_2, System.Net.Sockets.SocketFlags A_3) + 0x4a bytes
Devart.Data.PostgreSql.dll!Devart.Security.SSL.af.b(byte[] A_0, int A_1, int A_2) + 0x18 bytes
Devart.Data.PostgreSql.dll!Devart.Common.ah.b(byte[] A_0, int A_1, int A_2) + 0x29 bytes
Devart.Data.PostgreSql.dll!Devart.Common.l.e(byte[] A_0, int A_1, int A_2) + 0x42 bytes
Devart.Data.PostgreSql.dll!Devart.Common.an.b(byte[] A_0, int A_1, int A_2) + 0x16 bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.s.a(byte[] A_0, int A_1, int A_2) + 0x33 bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.s.h() + 0x5a bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.d.b(bool A_0) + 0x1a bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.d.a(bool A_0, bool A_1) + 0x28 bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.h.s() + 0x26 bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.ag.a(Devart.Common.DbConnectionInternal A_0) + 0x29 bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionPool.GetObject(Devart.Common.DbConnectionBase owningConnection) + 0x2de bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionFactory.b(Devart.Common.DbConnectionBase A_0) + 0xda bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionClosed.Open(Devart.Common.DbConnectionBase outerConnection) + 0x6d bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionBase.Open() + 0xb5 bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.PgSqlConnection.Open() + 0x98 bytes
E77.UWA.Database.dll!E77.UWA.Database.uwaDbDataSetTableAdapters.CResultsTableAdapter.Get701Flags(int resultId) + 0xb5 bytes
E77.UWA.Database.dll!E77.UWA.Database.ConcurrentDB.CResultsAdapter.SetSedFlag.AnonymousMethod__4b() + 0x2c bytes
E77.UWA.Shared.dll!UtilThread.Threading.CThreadBase<E77.UWA.Threading.EUWAThreads>.Invoke<long>.AnonymousMethod__0(object i) + 0xd bytes
E77.UWA.Shared.dll!UtilThread.Threading.CWorkItem.SetFunction<object,long>.AnonymousMethod__0(object par) + 0x4c bytes
E77.UWA.Shared.dll!UtilThread.Threading.CProcessCollectionBase<System.__Canon>.ProcessFunction(UtilThread.Threading.CWorkItem workItem) + 0x145 bytes
E77.UWA.Shared.dll!UtilThread.Threading.CProcessCollectionBase<UtilThread.Threading.CPriorityWorkItem>.Process() + 0x1bc bytes
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state) + 0x63 bytes
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool ignoreSyncCtx) + 0xb0 bytes
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) + 0x2c bytes
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() + 0x44 bytes

Thread 2:
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionPool.GetObject(Devart.Common.DbConnectionBase owningConnection) + 0x34 bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionFactory.b(Devart.Common.DbConnectionBase A_0) + 0xda bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionClosed.Open(Devart.Common.DbConnectionBase outerConnection) + 0x6d bytes
Devart.Data.PostgreSql.dll!Devart.Common.DbConnectionBase.Open() + 0xb5 bytes
Devart.Data.PostgreSql.dll!Devart.Data.PostgreSql.PgSqlConnection.Open() + 0x98 bytes
System.Data.dll!System.Data.Common.DbDataAdapter.FillInternal(System.Data.DataSet dataset, System.Data.DataTable[] datatables, int startRecord, int maxRecords, string srcTable, System.Data.IDbCommand command, System.Data.CommandBehavior behavior) + 0x7c bytes
System.Data.dll!System.Data.Common.DbDataAdapter.Fill(System.Data.DataTable[] dataTables, int startRecord, int maxRecords, System.Data.IDbCommand command, System.Data.CommandBehavior behavior) + 0xa7 bytes
System.Data.dll!System.Data.Common.DbDataAdapter.Fill(System.Data.DataTable dataTable) + 0x74 bytes
E77.UWA.Database.dll!E77.UWA.Database.uwaDbDataSetTableAdapters.CRackAssignTableAdapter.GetData() + 0x65 bytes
E77.UWA.FlowControl.dll!E77.UWA.CRackAssignManager.IsQCRack(string rackBC, E77.UWA.Shared.EInstrument instrument) + 0x35 bytes
E77.UWA.FlowControl.dll!E77.UWA.FlowControl.CFlowControl.ProcessQCorWashRack(string rackBC, string tubeBC, byte tubePos, long lowId, E77.UWA.Shared.EInstrument instrument, bool flowError, object header) + 0x23 bytes
E77.UWA.FlowControl.dll!E77.UWA.FlowControl.CFlowControl.ProcessOrders(string rackBarcode, string tubeBarcode, int tubePos, long lowID, E77.UWA.Shared.EInstrument instrument, object originalOrder, bool isDuplicatedDB, bool flowError, bool measuringError, int statusError, int errorError) + 0x72d bytes
E77.UWA.FlowControl.dll!E77.UWA.FlowControl.CFlowControl.Start601OrderThread.AnonymousMethod__28() + 0x142 bytes
E77.UWA.Shared.dll!UtilThread.Threading.CThreadFactory.BeginInvoke.AnonymousMethod__9(object par) + 0xd bytes
E77.UWA.Shared.dll!UtilThread.Threading.CWorkItem.SetFunction<object,UtilThread.Threading.CNullReturnType>.AnonymousMethod__0(object par) + 0x31 bytes
E77.UWA.Shared.dll!UtilThread.Threading.CProcessCollectionBase<System.__Canon>.ProcessFunction(UtilThread.Threading.CWorkItem workItem) + 0x145 bytes
E77.UWA.Shared.dll!UtilThread.Threading.CProcessCollectionBase<UtilThread.Threading.CPriorityWorkItem>.Process() + 0x1bc bytes
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state) + 0x63 bytes
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool ignoreSyncCtx) + 0xb0 bytes
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) + 0x2c bytes
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() + 0x44 bytes

Our theory is the Thread 1 is stuck somewhere with the SSL authentication and therefore the connection pool handler is blocked and cannot continue distribution of other connections to our other threads either. There are multiple threads stopped like Thread 2, in the same function on the top "GetObject".

I am interested in any suggestions to avoid this problem fast. Can SSL turning off be a solution? I am interested in also best practices when we will set connection timeout, how to handle the rizen exception and what to do with the problematic query. Simple retry?

Thank you very much for your fast help in advance.
Daniel

[Shalex]

Our theory is the Thread 1 is stuck somewhere with the SSL authentication and therefore the connection pool handler is blocked and cannot continue distribution of other connections to our other threads either. There are multiple threads stopped like Thread 2, in the same function on the top "GetObject".

I am interested in any suggestions to avoid this problem fast. Can SSL turning off be a solution?

1. We have sent the internal build with the fix to the email specified in your forum profile. If this doesn't help, please upload a small test project with the corresponding DDL/DML script for reproducing to ftp://ftp.devart.com (credentials: anonymous / yourEmail).

I am interested in also best practices when we will set connection timeout, how to handle the rizen exception and what to do with the problematic query. Simple retry?

2. Yes, do a retry.