Weird Relationship not found error when doing a Batch Update

Weird Relationship not found error when doing a Batch Update

Postby cjbiggs » Thu 09 Feb 2012 03:52

I have been getting a runtime UpdateExeception using dotConnect for PostgreSQL 5.70.293. When I am doing alot of inserts into my tables with Batch Update enabled and using multi-core/threads (TPL or Parallel.Foreach). I get a relationship not found error message between two tables that dont have a relationship defined in the EDM. It is a valid model, and the relationship doesnt exist, so the message is correct.

1.) Why would it be checking an non-existing relationship on a valid EDM?

2.) Why does it work correctly when I disable Batch Update?

3.) Why does it work when I have Batch Update enable, but use a single core/thread?

I have been having this issue for a couple of releases.

Is there any special logging I can turn on to give you more information to debug into it? I dont know is I can create a sample to reproduce it, but I can easily create it in my application.

Thanks,

Charlie J.
cjbiggs
 
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Postby Shalex » Fri 10 Feb 2012 17:39

Please send us a small test project with the corresponding DDL/DML script to reproduce the issue in our environment.
Shalex
Devart Team
 
Posts: 7659
Joined: Thu 14 Aug 2008 12:44

Postby cjbiggs » Fri 10 Feb 2012 20:49

I dont think I can create a sample that would reproduce the problem, but I will try. I am able to reproduce it within my application very easily.

Thanks,

Charlie J.
cjbiggs
 
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Postby cjbiggs » Mon 13 Feb 2012 15:16

I am having a hard time get a sample together to we recreate this issue. You never answer my questions about it that could have me debug it some more on my side.

1.) Why would it be checking an non-existing relationship on a valid EDM?

2.) Why does it work correctly when I disable Batch Update?

3.) Why does it work when I have Batch Update enable, but use a single core/thread?


4.) Is there any special logging I can turn on to give you more information to debug into it?



Thanks,

Charlie J.
cjbiggs
 
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Postby Shalex » Fri 17 Feb 2012 16:50

Which Batch Updates options have you set? To which values?
cjbiggs wrote:1.) Why would it be checking an non-existing relationship on a valid EDM?
2.) Why does it work correctly when I disable Batch Update?
3.) Why does it work when I have Batch Update enable, but use a single core/thread?

We cannot reproduce the issue in our environment. That's why we cannot answer to these questions.
cjbiggs wrote:4.) Is there any special logging I can turn on to give you more information to debug into it?

Logging is turned off not to decrease performance.
Shalex
Devart Team
 
Posts: 7659
Joined: Thu 14 Aug 2008 12:44

Postby cjbiggs » Fri 17 Feb 2012 18:24

PgSqlEntityProviderConfig config = PgSqlEntityProviderConfig.Instance;
config.DmlOptions.BatchUpdates.Enabled = true;

config.DmlOptions.ReuseParameters = true;
config.DmlOptions.BatchUpdates.BatchSize = 50;
config.DmlOptions.BatchUpdates.AsynchronousBatch = true;
config.DmlOptions.InsertNullBehaviour = InsertNullBehaviour.InsertDefaultOrOmit

Just having Batching enabled and the rest of the lines commented out it breaks. If I set MaxDegreeofPallellism to 1 in my ForEach.Parallel, it works with all the batching code uncommented. It is only when I set MaxDegressof Pallellism to greater than 1 it complaining at runtime about non-existing missing relationships between tables that dont have a relationship.

Thanks,

Charlie J.
cjbiggs
 
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Postby Shalex » Tue 28 Feb 2012 12:39

We have not received a test project to reproduce the issue in our environment. So we can give only general recommendations:

1. ADO.NET (including ObjectContext) is not thread-safe.
http://msdn.microsoft.com/en-us/library/system.data.objects.objectcontext.aspx: "The ObjectContext class is not thread safe. The integrity of data objects in an ObjectContext cannot be ensured in multithreaded scenarios."
That's why the same ObjectContext must not be shared between several threads. I.e. creation of a context instance has to be placed within a cycle:
Code: Select all
    Parallel.ForEach(items, parallelOptions, item =>
    {
       PostgreEntities context = new PostgreEntities();
       ...
    }

2. As a result of the previous point, a program logic should be constructed in the way when dozens of records are added to the context in scope of one iteration. This makes sense to use Batch Updates.

3. Do not autoincrement the values of outer variables in different threads because this is not safe. Use synchronization:
Code: Select all
    System.Threading.Interlocked.Add(ref i, 1);
    item.Id = i;
instead of
Code: Select all
    item.Id = i++;

4. Is usage of Parallel necessary in your scenario? Generally, a solution without Parallel has a better performance: one ObjectContext, one SaveChanges(); Batch Updates is turned on.
Shalex
Devart Team
 
Posts: 7659
Joined: Thu 14 Aug 2008 12:44

Postby cjbiggs » Tue 28 Feb 2012 15:08

Thanks I am aware of Recommendation 1-3. I have that handled in my code. I created a ObjectContext on a different thread and enable BatchUpdate on the ObjectContext. This gives me better performance if the BatchUpdate would work with using Parallel.Foreach.

Thanks,

Charlie J.
cjbiggs
 
Posts: 105
Joined: Fri 15 Jan 2010 19:56


Return to dotConnect for PostgreSQL