Application hangs frequently

[cp_Holger]

Hello,

we have a web application ( ASP.Net 4.0, dotconnect 6.10.135, oracle client 10g, IIS7, Windows Server 2008) that hangs frequently in production (~300 concurrent users).

At least once a day, the application stops responding, and we see errors in eventlog or our own trace log like:

Message: Server did not respond within the specified timeout interval
...
StackTrace - at Devart.Data.Oracle.az.a(y A_0, a2 A_1)
...
at System.Data.EntityClient.EntityConnection.OpenStoreConnectionIf(
...

or

Exception message: The underlying provider failed on Open
...
at System.Data.EntityClient.EntityConnection.OpenStoreConnectionIf(
...

We suspected this to be a connection pool problem, but increasing the pool to 300 did not help.
We tried to monitor the connections with dbMonitor. This tool shows that connections onlx go up to ~ 20, but does no longer connect to the application after some time ( dbMonitor restarted or not ) and shows nothing e.g. the next day, so we do not know the value in case of the hang.

The team that runs the oracle db says that the db side shows nothing in the case of our hang, no anomalies, no LOCKS.

And, of course, it does not happen at our test systems.

Setting the Connection Timout to 120 did not change anything. Maybe because its only for direct mode, which we do not use.

This problem onls exists since we moved to .NET 4 with our application, the database and the db client remained the same, only Framework and devart provider changed.

Do you have any idea what to check or what to do, to identify the real reason of this problem?

Any help would be appreciated.

Thanks
Holger

[cp_Holger]

Additional Question:

We would like to monitor something like "Pool has x connection(s)" in our own trace log.

Is there a way to read the number of connections from the dbMonitor object while tracing in the MonitorEvent ?

Thanks.
Holger

[Shalex]

1. Application hangs frequently
a)

~300 concurrent users

How many simultaneous connections to the database can each of concurrent users open? Tell us the value of your "Max Pool Size" connection string parameter;
b) try switching off pooling (Pooling=false;). Does it make any difference?
c) try using Oci Session Pooling instead of dotConnect for Oracle pooling. Does it help?

2. dbMonitor
a)

This tool shows that connections onlx go up to ~ 20, but does no longer connect to the application after some time

Tell us the version (x.x.x) of your dbMonitor. I cannot reproduce the issue with dbMonitor v 3.0.3. Can it be that your application does not establish more connections? Can you reproduce the issue with simple ADO.NET code?
b)

We would like to monitor something like "Pool has x connection(s)" in our own trace log.
Is there a way to read the number of connections from the dbMonitor object while tracing in the MonitorEvent ?

There is no such possibility at the moment. We will notify you when this feature is implemented.

[cp_Holger]

1. Application hangs

a) We have set the Max Pool Size to 300 for each of our 5 different connections. But only one of them is used for the hard work and, as far as we could see, this pool uses max. 20 connections at peak.

b) and c) we are currently testing the normal application behavior with these settings in our test environment before using them in production.

Actually we suspect the DB Server to be the reason, but we would like to measure and monitor that, like with performance counters or something like that.


2. dbMonitor

We run version 3.0.3 on the application server and watch the UI via Terminal Client (mstsc). A Filter is set for Event Type Connection Pool. Maybe it's a redraw / refresh problem. I reduced the settings for # of events and set redraw interval to 2000, which, at least with some patience, allows the tool to redraw and show more than a white screen.
Currently we don't have the ressources to build a test application.

We would like to monitor something like "Pool has x connection(s)" in our own trace log.
Is there a way to read the number of connections from the dbMonitor object while tracing in the MonitorEvent ?

There is no such possibility at the moment. We will notify you when this feature is implemented.

This would be great!

[Shalex]

We have implemented the ReturnToPool member in the MonitorEventType enumeration that allows to track the pool state via OracleMonitor.TraceEvent.:

Code: Select all

       static void OnEvent(object sender, MonitorEventArgs e) {
            if (e.TracePoint == MonitorTracePoint.AfterEvent && e.EventType == MonitorEventType.ReturnToPool) {
                Console.WriteLine("Description: " + e.Description);
            }
        }

We will post here when the corresponding build is available for download.

[Shalex]

New build of dotConnect for Oracle 6.30.202 is available for download!
It can be downloaded from http://www.devart.com/dotconnect/oracle/download.html (trial version) or from Registered Users' Area (for users with valid subscription only): http://secure.devart.com/ .

For more information, please refer to http://www.devart.com/forums/viewtopic.php?t=21724 .

[fredlegrain]

We'd be very interested in more monitoring features on connection pools.
Like knowing the number of connections in use, connections available, min/max connections of the pool. Or even better, get the state of each connection in the pool, etc.

Can you plan that?

[Shalex]

knowing the number of connections in use, connections available

Thank you for your suggestion. We will investigate the possibility of improving the MonitorEventArgs.Description content.

min/max connections of the pool

You can find out this information from the Min Pool Size/Max Pool Size connection string parameters.

get the state of each connection in the pool

Could you please describe the situation when you need to know this specific information?

[fredlegrain]

knowing the number of connections in use, connections available

Thank you for your suggestion. We will investigate the possibility of improving the MonitorEventArgs.Description content.

Thank you.

min/max connections of the pool

You can find out this information from the Min Pool Size/Max Pool Size connection string parameters.

Indeed...

get the state of each connection in the pool

Could you please describe the situation when you need to know this specific information?

We had the experience that some conections can remain in an unstable state in some situations *. Having a list of existing active connections in the pool and knowing their state would be of great interest for diagnostic and monitoring.

* Some situations of the empty array issue made the connection remain open, even impossible to kill unless we restarted the database.
http://forums.devart.com/viewtopic.php? ... ray#p84745

[Shalex]

Some situations of the empty array issue made the connection remain open, even impossible to kill unless we restarted the database.

We recommend you to trace and control sessions state with Session Manager which is available in OraDeveloper Tools, which is included in Developer Edition of dotConnect for Oracle.

As an alternative, you can use dbForge Studio for Oracle (Session Manager is shipped with this product as well).

[fredlegrain]

Some situations of the empty array issue made the connection remain open, even impossible to kill unless we restarted the database.

We recommend you to trace and control sessions state with Session Manager which is available in OraDeveloper Tools, which is included in Developer Edition of dotConnect for Oracle.

As an alternative, you can use dbForge Studio for Oracle (Session Manager is shipped with this product as well).

Thanks for the tips. I'll take a look at it.