Pool Fragmentation
Posted: Tue 31 Jul 2018 09:50
Hi,
I have a Company table in my master database.There are 200 rows(each for one company) in that table.Each company has its own database according to there name in Company table in master database.so i have 200 other databases.I have a windows service running continuously in a specified intervals.Windows service actually import data from different ERP & Accounting systems and also export to those systems.in each run, it compares a field named "LastExecutedTime" in Company table to estimate numbers of companies, for which it needs to import and export in this run.The service create same number(numbers of companies) of parrell task and import & export data.
In the each parallel task, after import/export, it connections to different databases in my server to save imported data.So if the number is 20, it is creating 21 connection pools(20 different databases and one master Database).As the number of company is increasing i have to do the pool fragmentation.
https://docs.microsoft.com/en-us/dotnet ... on-pooling shows the "Pool Fragmentation Due to Many Databases", it suggest to connect with master database and use Transact-SQL USE statement to change to the desired database, it this case it will create one connection pool and i need to increase MaxPoolSize to 200.As number of companies is increasing day by day, i need to set a big value to MaxPoolSize for safety.This is one solution.
https://msdn.microsoft.com/en-us/librar ... L.80).aspx said "In an efficient production system, typically the number of pools is low (1 to 10) and the total number of connections in use is also low (fewer than 12)".I have another solution in my mind.Is it possible to use custom key value pair in connection string? if possible, i will distribute this value to 200 companies.20 companies will fall with each value. i will use this value with connection string of master database to create different pools for different key/values. In this way it will allow me to maximum 10* 100 = 1000 connections with 10 connection pools.i don't need to increase the MaxPoolSize
Is there anything wrong with my second solution?
I have a Company table in my master database.There are 200 rows(each for one company) in that table.Each company has its own database according to there name in Company table in master database.so i have 200 other databases.I have a windows service running continuously in a specified intervals.Windows service actually import data from different ERP & Accounting systems and also export to those systems.in each run, it compares a field named "LastExecutedTime" in Company table to estimate numbers of companies, for which it needs to import and export in this run.The service create same number(numbers of companies) of parrell task and import & export data.
In the each parallel task, after import/export, it connections to different databases in my server to save imported data.So if the number is 20, it is creating 21 connection pools(20 different databases and one master Database).As the number of company is increasing i have to do the pool fragmentation.
https://docs.microsoft.com/en-us/dotnet ... on-pooling shows the "Pool Fragmentation Due to Many Databases", it suggest to connect with master database and use Transact-SQL USE statement to change to the desired database, it this case it will create one connection pool and i need to increase MaxPoolSize to 200.As number of companies is increasing day by day, i need to set a big value to MaxPoolSize for safety.This is one solution.
https://msdn.microsoft.com/en-us/librar ... L.80).aspx said "In an efficient production system, typically the number of pools is low (1 to 10) and the total number of connections in use is also low (fewer than 12)".I have another solution in my mind.Is it possible to use custom key value pair in connection string? if possible, i will distribute this value to 200 companies.20 companies will fall with each value. i will use this value with connection string of master database to create different pools for different key/values. In this way it will allow me to maximum 10* 100 = 1000 connections with 10 connection pools.i don't need to increase the MaxPoolSize
Is there anything wrong with my second solution?