DataContext & Memory Leaks

Discussion of open issues, suggestions and bugs regarding ADO.NET provider for MySQL
luigisaggese
Posts: 17
Joined: Fri 04 Nov 2011 16:43
Location: Italia

DataContext & Memory Leaks

Post by luigisaggese » Mon 07 Nov 2011 08:13

Hi,

I have developed an ASP.Net web application that uses dotConnect for MySQL v.6.50.214 and Framework 3.5.

I have a memory leak problem.

I have used .NET Memory Profiler 4.0 to profile my web application and the profiler tells me that there are +1000 instances of Datacontext object that have been disposed but have still not been Garbage Collected (even when calling GC.collect).

I want to find the memory leak, so i want to make sure that i am using the datacontext correctly. I use:

Code: Select all

using (DataContext dc = GetContext())           
{
   result = dc.GetUtenteCompletoById(pIdUtente).FirstOrDefault();
}

public Devart.Data.Linq.DataContext GetContext()
{
    DataContext fdc = new DataContext(Merqurio.DottNetCore.Core.Singleton.Connection.ConnectionString);               
    fdc.Log = new DebuggerWriter();              
    return fdc;            
}
Can you help me?
Thanks very much.

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Mon 07 Nov 2011 18:07

Could you please specify the following:
- what is the GetUtenteCompletoById method (e.g., a wrapper for a stored procedure returning a result set, or a manually implemented method that executes a LINQ query);
- what exactly is the DebuggerWriter class, and whether the issue can be reproduced if it is not used;
- how exactly are you creating multiple DataContext instances (e.g., can it be a simple loop?).
If possible, please send us a small test project.

We couldn't reproduce the issue in our environment.

luigisaggese
Posts: 17
Joined: Fri 04 Nov 2011 16:43
Location: Italia

Post by luigisaggese » Wed 09 Nov 2011 10:05

Hi StanislavK,
thanks for the answer.
StanislavK wrote:- what is the GetUtenteCompletoById method (e.g., a wrapper for a stored procedure returning a result set, or a manually implemented method that executes a LINQ query);
as you understood, GetUtenteCompletoById method is a wrapper for a stored procedure returning a result set;
StanislavK wrote:- what exactly is the DebuggerWriter class, and whether the issue can be reproduced if it is not used;
to display Sql code in Debugger output window i have used DebuggerWriter class that is a wrapper around Sytem.Diagnostics.Debugger; but this not compromise the problem result (i could also to comment it);
StanislavK wrote:- how exactly are you creating multiple DataContext instances (e.g., can it be a simple loop?).
i am sure that is not a loop because i see with Memory Profiler that some instances of DataContext are Disposed but not Collected.

In attach, there is a screenshot of .NET Memory Profiler' comparison snapshot, in which MerqurioDataContext (which is a DataContext 's wrapper used as showed) has 999 instances that have been disposed but are still not GCed.

Image
StanislavK wrote:If possible, please
send us a small test project.
My project is not small and send a little part of it'll not be relavant.
But i can explain how i access to database data (using Devart DataContext).

In my UI Layer i call my Business Layer that implement database interface using Devart dotConnect for my SQL Entity Model.
The way i call this Layer is using by DDD Model (Presenter-Service-Repository). In the Repository Layer there is a call to a method (example) that return a result set of utente_completo type (utente_completo is a view of my DB mapped by Entity Developer as result set of StoreProcedure GetUtenteCompletoById)
luigisaggese wrote:

Code: Select all

utente_completo IUtenteRepository.GetUtenteCompletoById(Int32 pIdUtente)
{
utente_completo result = new utente_completo();
    using (MerqurioDataContext dc = conn.GetContext())
   {
       result = dc.GetUtenteCompletoById(pIdUtente).FirstOrDefault();
   }
return result;
}
We always use "using statement" in all invoked method of Repository layer.

Please note we have used a single DataContext for all DB knowledge (~200 Stored Procedure, ~20 View, ~100 Table...) and reading some discussions i have noticed
that it is better to use a multiple DataContext(ideally per business object) than single DataContext, as i have seen in:
http://stackoverflow.com/questions/2261 ... atacontext

-What do you suggest with single or multiple Devart DataContext? There are particular optimizations that i could use?

Thanks in advance,
LS

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Fri 11 Nov 2011 18:03

We are analyzing the situation, we will inform you about the results as soon as possible.

luigisaggese
Posts: 17
Joined: Fri 04 Nov 2011 16:43
Location: Italia

Post by luigisaggese » Tue 15 Nov 2011 10:40

StanislavK wrote:We are analyzing the situation, we will inform you about the results as soon as possible.
We are waiting your reply! :-)

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Wed 16 Nov 2011 15:54

Apparently, some object(s) in your application (that belongs to either your application itself or the LinqConnect runtime) refers to DataContext instances, thus preventing them (DataContexts) from being garbage-collected. However, we couldn't reproduce this problem in simple situations in our environment.

Could you please try tracking your application with a profiler tool to check the type of the object(s) referring to non-collected DataContexts?

luigisaggese
Posts: 17
Joined: Fri 04 Nov 2011 16:43
Location: Italia

Post by luigisaggese » Wed 16 Nov 2011 16:54

Instances to which they refer are present
in all objects
that are returned by calls to DataContext
(so embedded in the object).
This is an example screenshot:
Image
Image

If you want to deepen analysis and see all the calls and
undisposed istances made by Devart I have uploaded a work session
conducted on my site that you can analyze with
.NET Memory Profiler 4.0 (http://memprofiler.com/download.aspx).
This is the link to my session with memory profiler:

http://www.dottnet.it/public/16112011_1517.zip

Believe me they are really sure by the way we
use the DataContext (by using clause)
then I suggest you to analyze
this session because I think there is something
that you could evaluate in software code.

Thanks a lot.
-LS

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Fri 18 Nov 2011 15:35

Thank you for the clarification, we've reproduced the issue. The problem is that compiled queries are saved in the cache with a reference to the DataContext instance for queries including a call to a stored procedure.

As a temporary workaround, you can disable query caching. To do so, you can, e.g., comment the following line in the generated code of your DataContext type:

Code: Select all

public static Devart.Data.Linq.CompiledQueryCache compiledQueryCache =
  Devart.Data.Linq.CompiledQueryCache.RegisterDataContext(typeof([Your DataContext type here]));

luigisaggese
Posts: 17
Joined: Fri 04 Nov 2011 16:43
Location: Italia

Post by luigisaggese » Mon 21 Nov 2011 14:01

Ok I'm glad to hear that i have found a bug in Devart and not within my application :-)
I hope this will solve as soon as.
We make intensive use of your instrument having a social network in the medical industry.

If you allow me I'd like to suggest possible changes in the instrument which I think is necessary at this level in order to make this software the more competitive and efficient.

In my opinion, would:

- Implementation of a DataContext thread-safe so you can use it as a singleton.
- Possibility to disable caching in an easy and tracking queries and objects.
- Implementation of an easy interface to switch between test and production database even after the generation of the DataContext.
- Resolution of the bug in the generation of inheritance. The discriminator must all times be awarded every time you open the DataContext.
- Attributes Pipelined in StoreProcedure when is set to true give an error (that is valid for all stored procedure...)

We hope that our community will grow exponentially in order to make these unpleasant memories bugs (we spent more than 2 months of testing and debugging to find a solution to memory leaks) and make this tool more and more powerful and faster.

P.s. We'd love to be listed among your customers being the largest community in Italy in the field. :-)


Thanx a lot for the support!
-LS

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Tue 22 Nov 2011 18:01

Thank you for your suggestions.

- Since it keeps a connection object, which is not thread-safe, a DataContext instance cannot be thread-safe either. Though it is possible to use a single context object for the whole application, a developer should control locks on the context on his own. Besides, DataContext is a light-weight object, and it is recommended to use new DataContext for each unit of work.

- We will consider implementing the possibility of managing object caches. As for the query cache, could you please describe this suggestion in more details?

- Provided that the model is created schema-ignorant, switching between different schemata is as simple as changing the connection strings passed to the context constructor. As for removing the schema name from the storage of existing models, we plan to implement this possibility, though cannot provide any timeframe for this.

- Could you please describe this problem in more details? If you've reported about it somewhere on the forum or by mail, please specify the link to the corresponding post or the mail subject.

- The Pipelined property of methods wrapping stored procedures determines whether the procedure returns a result set that can be accessed like a table. If you encounter any problems when setting this property to true, please describe them more precisely; e.g., please specify the definition of the procedure with which the issue occurs, and the exact error message you are getting.

luigisaggese
Posts: 17
Joined: Fri 04 Nov 2011 16:43
Location: Italia

Post by luigisaggese » Tue 20 Dec 2011 09:31

Hi StanislavK,

i noticed that it was released a new version of Devart dotConnect for MySQL.

Thank you for the clarification, we've reproduced the issue. The problem is that compiled queries are saved
in the cache with a reference to the DataContext instance for queries including a call to a stored procedure.
Now I would like to know if it solve this problem.

Thanks a lot.

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Thu 22 Dec 2011 14:55

The latest version of dotConnect for MySQL does not include the fix for this issue. However, it (the fix) will be available in one of the nearest builds. We will inform you when the corresponding build is released.

calfonso
Posts: 4
Joined: Mon 30 Aug 2010 16:21

Post by calfonso » Wed 15 Feb 2012 16:19

Hi StanislavK,

I'm with the same problems described by luigisaggese but using SQLServer or Oracle DB with LinqConnect version 3.1.25.

The latest release (LinqConnect 3.1.34) solved this?

ps.: let me know when the corresponding build is released. ok

Thanks in advance.

StanislavK
Devart Team
Posts: 1710
Joined: Thu 03 Dec 2009 10:48

Post by StanislavK » Thu 16 Feb 2012 16:33

The problem described in this topic was caused by an error in compiling queries that included calls to stored procedures. More precisely, such compiled queries were holding a reference to the context instance. This issue was fixed in the 3.1.25 version of LinqConnect.

Apparently, the issue you are observing has another origin. Thank you for the sample project, we will investigating it and inform you about the results as soon as possible.

calfonso
Posts: 4
Joined: Mon 30 Aug 2010 16:21

DataContext & Memory Leaks

Post by calfonso » Mon 05 Mar 2012 12:35

Hi StanislavK,

We are with a lot of problems in a production environment, related with this problem. We are stuck waiting for a solution.

You discovered something about. Could you put a high priority in this task and publish the modifications in the next release?
Could you provide me, in private, some phone number to contact the suport team? (This is not published in your web site).
If you don't have a solution in a short time we'll be forced to change our provider (i think it's not a good solution).

thanks in advance

Post Reply