Collection was modified; enumeration operation may not execute

Discussion of open issues, suggestions and bugs regarding ADO.NET provider for Oracle
Post Reply
ricsmania
Posts: 9
Joined: Fri 17 Jun 2016 16:37

Collection was modified; enumeration operation may not execute

Post by ricsmania » Wed 29 Jun 2016 16:11

Hello,

I'm getting this error when trying to read records concurrently and using Include to get the children:

Collection was modified; enumeration operation may not execute

The line that throws the error is this:

Code: Select all

return Ok(_context.Blogs.Include(blog => blog.Posts).FirstOrDefault(blog => blog.Id == id));
There doesn't seem to be anything wrong with the code, so it might be a bug.

dotConnect version 9.1.45.

Full explanation and code samples here: http://stackoverflow.com/questions/3810 ... enumeratio

Thanks.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Re: Collection was modified; enumeration operation may not execute

Post by Shalex » Thu 07 Jul 2016 16:02

Entity Framework Core 1.0 RTM is supported in the newest (9.1.55) build of dotConnect for Oracle: https://www.devart.com/dotconnect/oracl ... story.html. Please try it and upgrade your project to Core 1.0 RTM. Does this help?

ricsmania
Posts: 9
Joined: Fri 17 Jun 2016 16:37

Re: Collection was modified; enumeration operation may not execute

Post by ricsmania » Wed 13 Jul 2016 11:12

Hello Shalex,

I tried the 9.1.55 version with EF 1.0 but got the same error.

It is pretty easy to reproduce:

Create a Nuget from DotConnect 9.1.55 with the name Devart.Data.Oracle and version 9.1.55 and add the source to VS
Clone https://github.com/ricsmania/BlogsOracleEntityFramework
Run the script src/BlogOracle/Database/Blogs.sql
Change the BlogOracle model with the correct connect string
Run BlogOracle
Run SendRequests

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Re: Collection was modified; enumeration operation may not execute

Post by Shalex » Fri 15 Jul 2016 17:35

Thank you for your report. We will investigate the issue and notify you about the result.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Re: Collection was modified; enumeration operation may not execute

Post by Shalex » Fri 09 Sep 2016 17:13

Your project is configured to use .NET Core SDK version 1.0.0-preview1-002702. Please upgrade it to the release version and try the newest (9.1.97) build of dotConnect for Oracle.

Butzei
Posts: 12
Joined: Fri 26 May 2017 07:41

Re: Collection was modified; enumeration operation may not execute

Post by Butzei » Mon 14 Aug 2017 07:21

Hi,

ricsmania was this bug fixed by the update Shalex mentioned?
I have a similar bug that looks like a async problem in dotConnect/EfCore but with a slighly different stacktrace.

Code: Select all

at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.Accept(ExpressionVisitor visitor)
Instead of your 
at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.GetProjectionTypes()
It only occures on higher server load with multiple requests executed in parallel. So i think it would be hard to build an example project.



We are using Devart.Data.Oracle.Entity.EFCore 9.4.326.0 with Microsoft.EntityFrameworkCore 1.1.2.

This is our full stacktrace:

Code: Select all

at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
 at System.Collections.Generic.List`1.Enumerator.MoveNext()
at System.Data.Common.CommandTrees.b.b(SelectExpression A_0)
at System.Data.Common.CommandTrees.b.a(SelectExpression A_0)
at System.Data.Common.CommandTrees.a.e(SelectExpression A_0)
    at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.Accept(ExpressionVisitor visitor)
at System.Data.Common.CommandTrees.a.Visit(Expression node)
at System.Linq.Expressions.ExpressionVisitor.VisitAndConvert[T](T node, String callerName)
	 at Devart.Data.Oracle.Entity.ag.a(SelectExpression A_0)
at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.CreateDefaultQuerySqlGenerator()
at Microsoft.EntityFrameworkCore.Query.Internal.ShaperCommandContext.GetRelationalCommand(IReadOnlyDictionary`2 parameters)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable.Enumerator.BufferlessMoveNext(Boolean buffer)
at Microsoft.EntityFrameworkCore.Storage.Internal.NoopExecutionStrategy.Execute[TState,TResult](Func`2 operation, Func`2 verifySucceeded, TState state)
at Microsoft.EntityFrameworkCore.ExecutionStrategyExtensions.Execute[TState,TResult](IExecutionStrategy strategy, Func`2 operation, TState state)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable.Enumerator.MoveNext()
at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider.<_ShapedQuery>d__3`1.MoveNext()
	at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Qm.Tda.Dal.Services.ReadoutTimeDbService.Get(Guid boxId, Int32 sequenceNo, IReadOnlyCollection`1 lotNames) in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Dal\Services\ReadoutTimeDbService.cs:line 58
at Qm.Tda.Logic.Services.ReadoutTimeService.Get(Guid boxId, Int32 sequenceNo, IReadOnlyCollection`1 lotNames) in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Logic\Services\ReadoutTimeService.cs:line 29
at Qm.Tda.Logic.Services.FailureChartService.GetDataPoints(IEnumerable`1 errors, FailureChartRequestUnit unit, Int32 numberOfDevices) in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Logic\Services\FailureChartService.cs:line 0
at Qm.Tda.Logic.Services.FailureChartService.GetFailureChartSerie(Int32 numberOfDevices, DiagramFailureType failureType, IReadOnlyDictionary`2 convertSeriesTypes, LotMetaDao lotMetaDao, FailureChartRequestUnit unit) in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Logic\Services\FailureChartService.cs:line 146
at Qm.Tda.Logic.Services.FailureChartService.<>c__DisplayClass18_0.<GetFailureChartSeries>b__1(LotMetaDao lotMetaDao) in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Logic\Services\FailureChartService.cs:line 139
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Qm.Tda.Logic.Services.FailureChartService.GetFailureChartSeries(Int32 numberOfDevices, FailureChartRequestUnit unit, Dictionary`2 convertSeriesTypes) in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Logic\Services\FailureChartService.cs:line 139
at Qm.Tda.Logic.Services.FailureChartService.<GetFailureChartAsync>d__12.MoveNext() in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Logic\Services\FailureChartService.cs:line 72
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Qm.Tda.Api.Controllers.FailureChartController.<GetFailureChart>d__4.MoveNext() in C:\BuildAgent\work\4c7ef0c5657430ca\Qm.Tda.Api\Controllers\FailureChartController.cs:line 73
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Threading.Tasks.TaskHelpersExtensions.<CastToObject>d__3`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Re: Collection was modified; enumeration operation may not execute

Post by Shalex » Tue 15 Aug 2017 16:55

Butzei wrote:It only occures on higher server load with multiple requests executed in parallel. So i think it would be hard to build an example project.
If possible, please create a test project and upload it to ftp://ftp.devart.com (credentials: anonymous / yourEmail) so that we can reproduce and investigate the problem.

rmatbg
Posts: 8
Joined: Mon 11 Jun 2018 09:40

Re: Collection was modified; enumeration operation may not execute

Post by rmatbg » Wed 13 Jun 2018 08:04

I'm finding this same problem with dotConnect v9.5.520 and EF Core 2.0.3 running a simple expression of the type:

Code: Select all

_dbContext.Table.Include(t => t.FkRecords).Single(t => t.Pk == key);
Resulting in an exception like:

Code: Select all

System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
   at System.Collections.Generic.List`1.Enumerator.MoveNext()
   at System.Linq.Enumerable.Any[TSource](IEnumerable`1 source)
   at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.VisitSelect(SelectExpression selectExpression)
   at ¿¿.VisitSelect(SelectExpression )
   at Microsoft.EntityFrameworkCore.Query.Expressions.SelectExpression.Accept(ExpressionVisitor visitor)
   at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit(Expression expression)
   at ¿¿.Visit(Expression )
   at Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator.GenerateSql(IReadOnlyDictionary`2 parameterValues)
   at ¿¿.GenerateSql(IReadOnlyDictionary`2 )
   at Microsoft.EntityFrameworkCore.Query.Internal.ShaperCommandContext.GetRelationalCommand(IReadOnlyDictionary`2 parameters)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(Boolean buffer)
   at Microsoft.EntityFrameworkCore.ExecutionStrategyExtensions.<>c__DisplayClass12_0`2.<Execute>b__0(DbContext c, TState s)
   at Microsoft.EntityFrameworkCore.Storage.Internal.NoopExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.ExecutionStrategyExtensions.Execute[TState,TResult](IExecutionStrategy strategy, Func`2 operation, Func`2 verifySucceeded, TState state)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryBuffer.IncludeCollection(Int32 includeId, INavigation navigation, INavigation inverseNavigation, IEntityType targetEntityType, IClrCollectionAccessor clrCollectionAccessor, IClrPropertySetter inverseClrPropertySetter, Boolean tracking, Object entity, Func`1 relatedEntitiesFactory)
   at lambda_method(Closure , QueryContext , Gallery , Object[] )
   at Microsoft.EntityFrameworkCore.Query.Internal.IncludeCompiler._Include[TEntity](QueryContext queryContext, TEntity entity, Object[] included, Action`3 fixup)
   at lambda_method(Closure , Gallery )
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source)
   at lambda_method(Closure , QueryContext )
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass17_1`1.<CompileQueryCore>b__0(QueryContext qc)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at System.Linq.Queryable.Single[TSource](IQueryable`1 source, Expression`1 predicate)
My application runs this same query concurrently (4 - 8 simultaneous threads) but from distinct DbContext instances, each with their own dedicated DbConnection, so there should be no risk of corrupting state unless either EFCore or dotConnect is keeping static data somewhere and this is getting manipulated whilst EFCore is iterating through the records.

To reproduce ensure you have a machine with at least 4 logical processors available (check your Environment.ProcessorCount, but the higher this is the more likely you are of finding the issue) and run something along the lines of:

Code: Select all

for (int i = 0; i < 1000; ++i)
{
    Task.Run(() =>
    {
        using (var dbContext = new YourDbContext(new DbConnection()))
        {
            var result = dbContext.Table.Include(t => t.FkRecords).Single(t => t.Pk == key);
        }
    });
}
You might need to simulate some more work before disposing of the context like that but hopefully you get the idea.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Re: Collection was modified; enumeration operation may not execute

Post by Shalex » Thu 14 Jun 2018 14:38

EF Core 2.1 includes significant enhancements comparing to EF Core 2.0.3. The Microsoft.EntityFrameworkCore.Query.Sql.DefaultQuerySqlGenerator class in Microsoft.EntityFrameworkCore.Relational.dll is improved as well.

We recommend you to upgrade to EF Core 2.1. The internal build of Devart NuGet packages with the Entity Framework Core 2.1 support is available at https://www.devart.com/pub/nuget_oracle_9_5_534.zip.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Re: Collection was modified; enumeration operation may not execute

Post by Shalex » Fri 22 Jun 2018 17:04

Entity Framework Core 2.1 is supported: viewtopic.php?f=1&t=37342.

Post Reply