Devart Forums

Discussion forums for open issues and questions concerning database tools, data access components and developer tools from Devart
https://forums.devart.com/

Permanent failure if First Query attempted while DB server is unavailable

https://forums.devart.com/viewtopic.php?f=30&t=45101

Page 1 of 1

Permanent failure if First Query attempted while DB server is unavailable

Posted: Wed 14 Apr 2021 22:24
by PVHills
Hi There,
We have several microservices built on top of aspnetcore and entity framework and using the MySQL EntityFramework connector from Devart. I've noticed that if the first query that any of our applications attempts fails due to the data store being inaccessible, then the failure seems to be cached, and when the DB is finally alive, the exception is still returned to the user code.

As part of the diagnosis of this problem I tried disabling the database when the application is running and in this situation the application appears to be able to recover once the DB is available again, so I suspect the problem is related to a connection that may be cached during model initialisation.

I've also tried disabling the service provider caching and this also lets the application recover but at the cost of having to initialise the service provider each time, so this can't be used in our application but lends weight to the idea that the error is related to some form of caching.

Has anyone else had/seen this problem before? What was your solution?
Here is a stack trace for context

Code: Select all

   at   .    (String , String , String , String , Int32 , String , Int32 , SshOptions  , SslOptions  , ProxyOptions  , MySqlHttpOptions  , HttpOptions  , Int32  )
   at   .(MySqlConnection , String , String , String , String , Int32 , Int32 , MySqlProtocol  , Boolean  , Boolean  )
   at   ..ctor(   , MySqlConnection )
   at   .   (  , Object , DbConnectionBase )
   at  .(  ,   , DbConnectionBase )
   at  .(  , DbConnectionBase )
   at  .(DbConnectionBase )
   at  .(DbConnectionBase )
   at  .(DbConnectionBase )
   at  .   (DbConnectionBase )
   at Devart.Common.DbConnectionBase.Open()
   at Devart.Data.MySql.MySqlConnection.()
   at Devart.Data.MySql.MySqlConnection.Open()
   at Devart.Common.Entity.c7.a(DbConnection A_0)
   at Devart.Common.Entity.c7.b(DbConnection A_0)
   at Devart.Data.MySql.Entity.ad.a(DbConnection A_0)
   at Devart.Data.MySql.Entity.aa.b(RelationalOptionsExtension A_0)
   at Devart.Common.Entity.c3.a()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Devart.Common.Entity.c3.b()
   at Devart.Data.MySql.Entity.w..ctor(TypeMappingSourceDependencies A_0, RelationalTypeMappingSourceDependencies A_1, c4 A_2)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkRelationalServicesBuilder.<>c.<TryAddCoreServices>b__3_0(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel()
   at Microsoft.EntityFrameworkCore.Internal.DbContextServices.get_Model()
   at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__7_3(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies()
   at Microsoft.EntityFrameworkCore.DbContext.get_InternalServiceProvider()
   at Microsoft.EntityFrameworkCore.DbContext.get_DbContextDependencies()
   at Microsoft.EntityFrameworkCore.DbContext.get_Model()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.get_EntityType()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.CheckState()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.get_EntityQueryable()
   at Microsoft.EntityFrameworkCore.Internal.InternalDbSet`1.System.Linq.IQueryable.get_Provider()
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.Include[TEntity,TProperty](IQueryable`1 source, Expression`1 navigationPropertyPath)
(Our stack from here on)

Re: Permanent failure if First Query attempted while DB server is unavailable

Posted: Thu 15 Apr 2021 17:08
by Shalex
Please try turning off pooling ("Pooling=false;" in the connection string) or clear the pool explicitly with MySqlConnection.ClearAllPools(true).

For more information, refer to:
* https://www.devart.com/dotconnect/mysql ... tring.html
* https://www.devart.com/dotconnect/mysql ... lean).html

Re: Permanent failure if First Query attempted while DB server is unavailable

Posted: Thu 15 Apr 2021 20:52
by PVHills
Hi thanks for your response. I tried changing this config using Pooling=false; in the connection string. This didn't help. My casual model of the breakage is that the model builder is failing during initialisation, and the exception is being captured and passed on to the user code. System.Lazy is well known for this behaviour and I noticed that the failing call stack contains a use of one of these. I suspect that the connection isn't even retried, just that the failed lazy evaluation is played back

Code: Select all

   at Devart.Common.Entity.c3.a()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Devart.Common.Entity.c3.b()
I've managed to work around this by supplying an open DB connection in the configuration options action, here's my workaround:

Code: Select all

            services.AddTransient<DbConnection, MySqlConnection>();
            services.AddDbContext<BusinessContext>((sp, opt) =>
            {
                lock (this)
                {
                    var sql = sp.GetRequiredService<DbConnection>();
                    sql.ConnectionString = Configuration.GetConnectionString("connectionstring");
                    if (!_isConnected)
                    {
                        sql.Open(); // Ensure we can actually get a connection
                        _isConnected = true;
                    }
                    opt.UseMySql(sql);
                }
            });
This throws an exception if the first connection attempt occurs when the DB is unavailable, and prevents the devart component from attempting the model creation until the first connection succeeds. This isn't great, and I'd prefer not to have to do this, and it won't guarantee success

Re: Permanent failure if First Query attempted while DB server is unavailable

Posted: Tue 20 Apr 2021 14:19
by Shalex
The bug with connection recovery after the initial connection attempt failure in EF Core is fixed. We will notify you when a new public build of dotConnect for MySQL is available for download.

Re: Permanent failure if First Query attempted while DB server is unavailable

Posted: Tue 20 Apr 2021 20:50
by PVHills
Great news! Looking forward to installing it

Re: Permanent failure if First Query attempted while DB server is unavailable

Posted: Mon 07 Jun 2021 16:36
by Shalex
New build of dotConnect for MySQL 8.19.1905 is available for download now: viewtopic.php?f=2&t=46768.
All times are UTC
Page 1 of 1
Powered by phpBB® Forum Software © phpBB Limited
https://www.phpbb.com/