Application hangs frequently

Application hangs frequently

Postby cp_Holger » Wed 20 Jul 2011 09:43

Hello,

we have a web application ( ASP.Net 4.0, dotconnect 6.10.135, oracle client 10g, IIS7, Windows Server 2008) that hangs frequently in production (~300 concurrent users).

At least once a day, the application stops responding, and we see errors in eventlog or our own trace log like:

Message: Server did not respond within the specified timeout interval
...
StackTrace - at Devart.Data.Oracle.az.a(y A_0, a2 A_1)
...
at System.Data.EntityClient.EntityConnection.OpenStoreConnectionIf(
...

or
Exception message: The underlying provider failed on Open
...
at System.Data.EntityClient.EntityConnection.OpenStoreConnectionIf(
...


We suspected this to be a connection pool problem, but increasing the pool to 300 did not help.
We tried to monitor the connections with dbMonitor. This tool shows that connections onlx go up to ~ 20, but does no longer connect to the application after some time ( dbMonitor restarted or not ) and shows nothing e.g. the next day, so we do not know the value in case of the hang.

The team that runs the oracle db says that the db side shows nothing in the case of our hang, no anomalies, no LOCKS.

And, of course, it does not happen at our test systems.

Setting the Connection Timout to 120 did not change anything. Maybe because its only for direct mode, which we do not use.

This problem onls exists since we moved to .NET 4 with our application, the database and the db client remained the same, only Framework and devart provider changed.

Do you have any idea what to check or what to do, to identify the real reason of this problem?

Any help would be appreciated.

Thanks
Holger
cp_Holger
 
Posts: 4
Joined: Wed 20 Jul 2011 09:04

Tracing # of connections

Postby cp_Holger » Thu 21 Jul 2011 09:35

Additional Question:

We would like to monitor something like "Pool has x connection(s)" in our own trace log.

Is there a way to read the number of connections from the dbMonitor object while tracing in the MonitorEvent ?

Thanks.
Holger
cp_Holger
 
Posts: 4
Joined: Wed 20 Jul 2011 09:04

Postby Shalex » Thu 21 Jul 2011 13:25

1. Application hangs frequently
a)
cp_Holger wrote:~300 concurrent users

How many simultaneous connections to the database can each of concurrent users open? Tell us the value of your "Max Pool Size" connection string parameter;
b) try switching off pooling (Pooling=false;). Does it make any difference?
c) try using Oci Session Pooling instead of dotConnect for Oracle pooling. Does it help?

2. dbMonitor
a)
cp_Holger wrote:This tool shows that connections onlx go up to ~ 20, but does no longer connect to the application after some time

Tell us the version (x.x.x) of your dbMonitor. I cannot reproduce the issue with dbMonitor v 3.0.3. Can it be that your application does not establish more connections? Can you reproduce the issue with simple ADO.NET code?
b)
cp_Holger wrote:We would like to monitor something like "Pool has x connection(s)" in our own trace log.
Is there a way to read the number of connections from the dbMonitor object while tracing in the MonitorEvent ?

There is no such possibility at the moment. We will notify you when this feature is implemented.
Shalex
Devart Team
 
Posts: 7391
Joined: Thu 14 Aug 2008 12:44

Postby cp_Holger » Fri 22 Jul 2011 09:19

1. Application hangs

a) We have set the Max Pool Size to 300 for each of our 5 different connections. But only one of them is used for the hard work and, as far as we could see, this pool uses max. 20 connections at peak.

b) and c) we are currently testing the normal application behavior with these settings in our test environment before using them in production.

Actually we suspect the DB Server to be the reason, but we would like to measure and monitor that, like with performance counters or something like that.


2. dbMonitor

We run version 3.0.3 on the application server and watch the UI via Terminal Client (mstsc). A Filter is set for Event Type Connection Pool. Maybe it's a redraw / refresh problem. I reduced the settings for # of events and set redraw interval to 2000, which, at least with some patience, allows the tool to redraw and show more than a white screen.
Currently we don't have the ressources to build a test application.


We would like to monitor something like "Pool has x connection(s)" in our own trace log.
Is there a way to read the number of connections from the dbMonitor object while tracing in the MonitorEvent ?


There is no such possibility at the moment. We will notify you when this feature is implemented.


This would be great!
cp_Holger
 
Posts: 4
Joined: Wed 20 Jul 2011 09:04

Postby Shalex » Wed 03 Aug 2011 10:26

We have implemented the ReturnToPool member in the MonitorEventType enumeration that allows to track the pool state via OracleMonitor.TraceEvent.:
Code: Select all
       static void OnEvent(object sender, MonitorEventArgs e) {
            if (e.TracePoint == MonitorTracePoint.AfterEvent && e.EventType == MonitorEventType.ReturnToPool) {
                Console.WriteLine("Description: " + e.Description);
            }
        }

We will post here when the corresponding build is available for download.
Shalex
Devart Team
 
Posts: 7391
Joined: Thu 14 Aug 2008 12:44

Postby Shalex » Fri 12 Aug 2011 14:59

New build of dotConnect for Oracle 6.30.202 is available for download!
It can be downloaded from http://www.devart.com/dotconnect/oracle/download.html (trial version) or from Registered Users' Area (for users with valid subscription only): http://secure.devart.com/ .

For more information, please refer to http://www.devart.com/forums/viewtopic.php?t=21724 .
Shalex
Devart Team
 
Posts: 7391
Joined: Thu 14 Aug 2008 12:44

Re: Application hangs frequently

Postby fredlegrain » Mon 25 Feb 2013 17:11

We'd be very interested in more monitoring features on connection pools.
Like knowing the number of connections in use, connections available, min/max connections of the pool. Or even better, get the state of each connection in the pool, etc.

Can you plan that?
fredlegrain
 
Posts: 8
Joined: Tue 12 Jul 2011 09:47

Re: Application hangs frequently

Postby Shalex » Tue 26 Feb 2013 16:44

fredlegrain wrote:knowing the number of connections in use, connections available

Thank you for your suggestion. We will investigate the possibility of improving the MonitorEventArgs.Description content.

fredlegrain wrote:min/max connections of the pool

You can find out this information from the Min Pool Size/Max Pool Size connection string parameters.

fredlegrain wrote:get the state of each connection in the pool

Could you please describe the situation when you need to know this specific information?
Shalex
Devart Team
 
Posts: 7391
Joined: Thu 14 Aug 2008 12:44

Re: Application hangs frequently

Postby fredlegrain » Tue 26 Feb 2013 22:40

Shalex wrote:
fredlegrain wrote:knowing the number of connections in use, connections available

Thank you for your suggestion. We will investigate the possibility of improving the MonitorEventArgs.Description content.

Thank you.

Shalex wrote:
fredlegrain wrote:min/max connections of the pool

You can find out this information from the Min Pool Size/Max Pool Size connection string parameters.

Indeed...

Shalex wrote:
fredlegrain wrote:get the state of each connection in the pool

Could you please describe the situation when you need to know this specific information?

We had the experience that some conections can remain in an unstable state in some situations *. Having a list of existing active connections in the pool and knowing their state would be of great interest for diagnostic and monitoring.

* Some situations of the empty array issue made the connection remain open, even impossible to kill unless we restarted the database.
http://forums.devart.com/viewtopic.php?f=1&t=24428&p=84745&hilit=date+array#p84745
fredlegrain
 
Posts: 8
Joined: Tue 12 Jul 2011 09:47

Re: Application hangs frequently

Postby Shalex » Wed 27 Feb 2013 12:58

fredlegrain wrote:Some situations of the empty array issue made the connection remain open, even impossible to kill unless we restarted the database.

We recommend you to trace and control sessions state with Session Manager which is available in OraDeveloper Tools, which is included in Developer Edition of dotConnect for Oracle.

As an alternative, you can use dbForge Studio for Oracle (Session Manager is shipped with this product as well).
Shalex
Devart Team
 
Posts: 7391
Joined: Thu 14 Aug 2008 12:44

Re: Application hangs frequently

Postby fredlegrain » Wed 27 Feb 2013 21:16

Shalex wrote:
fredlegrain wrote:Some situations of the empty array issue made the connection remain open, even impossible to kill unless we restarted the database.

We recommend you to trace and control sessions state with Session Manager which is available in OraDeveloper Tools, which is included in Developer Edition of dotConnect for Oracle.

As an alternative, you can use dbForge Studio for Oracle (Session Manager is shipped with this product as well).


Thanks for the tips. I'll take a look at it.
fredlegrain
 
Posts: 8
Joined: Tue 12 Jul 2011 09:47


Return to dotConnect for Oracle