Weird Relationship not found error when doing a Batch Update

Discussion of open issues, suggestions and bugs regarding ADO.NET provider for PostgreSQL
Post Reply
cjbiggs
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Weird Relationship not found error when doing a Batch Update

Post by cjbiggs » Thu 09 Feb 2012 03:52

I have been getting a runtime UpdateExeception using dotConnect for PostgreSQL 5.70.293. When I am doing alot of inserts into my tables with Batch Update enabled and using multi-core/threads (TPL or Parallel.Foreach). I get a relationship not found error message between two tables that dont have a relationship defined in the EDM. It is a valid model, and the relationship doesnt exist, so the message is correct.

1.) Why would it be checking an non-existing relationship on a valid EDM?

2.) Why does it work correctly when I disable Batch Update?

3.) Why does it work when I have Batch Update enable, but use a single core/thread?

I have been having this issue for a couple of releases.

Is there any special logging I can turn on to give you more information to debug into it? I dont know is I can create a sample to reproduce it, but I can easily create it in my application.

Thanks,

Charlie J.

Shalex
Site Admin
Posts: 8239
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Fri 10 Feb 2012 17:39

Please send us a small test project with the corresponding DDL/DML script to reproduce the issue in our environment.

cjbiggs
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Post by cjbiggs » Fri 10 Feb 2012 20:49

I dont think I can create a sample that would reproduce the problem, but I will try. I am able to reproduce it within my application very easily.

Thanks,

Charlie J.

cjbiggs
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Post by cjbiggs » Mon 13 Feb 2012 15:16

I am having a hard time get a sample together to we recreate this issue. You never answer my questions about it that could have me debug it some more on my side.

1.) Why would it be checking an non-existing relationship on a valid EDM?

2.) Why does it work correctly when I disable Batch Update?

3.) Why does it work when I have Batch Update enable, but use a single core/thread?


4.) Is there any special logging I can turn on to give you more information to debug into it?



Thanks,

Charlie J.

Shalex
Site Admin
Posts: 8239
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Fri 17 Feb 2012 16:50

Which Batch Updates options have you set? To which values?
cjbiggs wrote:1.) Why would it be checking an non-existing relationship on a valid EDM?
2.) Why does it work correctly when I disable Batch Update?
3.) Why does it work when I have Batch Update enable, but use a single core/thread?
We cannot reproduce the issue in our environment. That's why we cannot answer to these questions.
cjbiggs wrote:4.) Is there any special logging I can turn on to give you more information to debug into it?
Logging is turned off not to decrease performance.

cjbiggs
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Post by cjbiggs » Fri 17 Feb 2012 18:24

PgSqlEntityProviderConfig config = PgSqlEntityProviderConfig.Instance;
config.DmlOptions.BatchUpdates.Enabled = true;

config.DmlOptions.ReuseParameters = true;
config.DmlOptions.BatchUpdates.BatchSize = 50;
config.DmlOptions.BatchUpdates.AsynchronousBatch = true;
config.DmlOptions.InsertNullBehaviour = InsertNullBehaviour.InsertDefaultOrOmit

Just having Batching enabled and the rest of the lines commented out it breaks. If I set MaxDegreeofPallellism to 1 in my ForEach.Parallel, it works with all the batching code uncommented. It is only when I set MaxDegressof Pallellism to greater than 1 it complaining at runtime about non-existing missing relationships between tables that dont have a relationship.

Thanks,

Charlie J.

Shalex
Site Admin
Posts: 8239
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Tue 28 Feb 2012 12:39

We have not received a test project to reproduce the issue in our environment. So we can give only general recommendations:

1. ADO.NET (including ObjectContext) is not thread-safe.
http://msdn.microsoft.com/en-us/library ... ntext.aspx: "The ObjectContext class is not thread safe. The integrity of data objects in an ObjectContext cannot be ensured in multithreaded scenarios."
That's why the same ObjectContext must not be shared between several threads. I.e. creation of a context instance has to be placed within a cycle:

Code: Select all

    Parallel.ForEach(items, parallelOptions, item =>
    {
       PostgreEntities context = new PostgreEntities();
       ...
    }
2. As a result of the previous point, a program logic should be constructed in the way when dozens of records are added to the context in scope of one iteration. This makes sense to use Batch Updates.

3. Do not autoincrement the values of outer variables in different threads because this is not safe. Use synchronization:

Code: Select all

    System.Threading.Interlocked.Add(ref i, 1);
    item.Id = i;
instead of

Code: Select all

    item.Id = i++;
4. Is usage of Parallel necessary in your scenario? Generally, a solution without Parallel has a better performance: one ObjectContext, one SaveChanges(); Batch Updates is turned on.

cjbiggs
Posts: 105
Joined: Fri 15 Jan 2010 19:56

Post by cjbiggs » Tue 28 Feb 2012 15:08

Thanks I am aware of Recommendation 1-3. I have that handled in my code. I created a ObjectContext on a different thread and enable BatchUpdate on the ObjectContext. This gives me better performance if the BatchUpdate would work with using Parallel.Foreach.

Thanks,

Charlie J.

Post Reply