mono linux blob performance issue

Discussion of open issues, suggestions and bugs regarding ADO.NET provider for Oracle
Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

mono linux blob performance issue

Post by Sao » Sat 23 Apr 2011 13:33

Hello,
I got strange performance issue with reading blob from Oracle 11g R1
Platform is SLES 11.1 x64, mono 2.8.2

I got speed around 15 kbytes/sec ! It is locally on server!

Can you help me, or we can jointly investigate tis issue?

My code example is very simple:

Code: Select all

        static void Main(string[] args)
        {
            using (var ora = new Devart.Data.Oracle.OracleConnection())
            {
                ora.ConnectionString = "User Id=...;Password=...;Server=127.0.0.1;Direct=True;Sid=...";
                ora.Open();
                var cmd = ora.CreateCommand();
                DateTime t1 = DateTime.Now;
                Console.WriteLine("Now is {0}", t1);
                cmd.CommandText = "select blob_data from blob_table where id=1";
                byte[] blob = cmd.ExecuteScalar() as byte[];
                DateTime t2 = DateTime.Now;
                Console.WriteLine("{0}: {1}; size:{3} MB; speed: {2} MB/s", "cmd.ExecuteScalar()", t2 - t1, (blob.Length * 1000.0) / (t2 - t1).TotalMilliseconds / 1024 / 1024, blob.Length/1024.0/1024);
                byte[] blob1 = new byte[0];
                using (var r = cmd.ExecuteReader())
                {
                    r.Read();
                    var lob = r.GetOracleLob(0);
                    blob1 = new byte[lob.Length];
                    lob.Read(blob1, 0, (int)lob.Length);
                }
                DateTime t3 = DateTime.Now;
                Console.WriteLine("{0}: {1}; size:{3} MB; speed: {2} MB/s", "cmd.ExecuteReader()", t3 - t2, (blob1.Length * 1000.0) / (t3 - t2).TotalMilliseconds / 1024 / 1024, blob1.Length / 1024.0 / 1024);
            }
        }
Program output is:

Code: Select all

cmd.ExecuteScalar(): 00:01:00.5573980; size:0,948768615722656 MB; speed: 0,0156672619210399 MB/s
cmd.ExecuteReader(): 00:01:02.2995090; size:0,948768615722656 MB; speed: 0,0152291507742486 MB/s

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Wed 27 Apr 2011 09:22

I have tried your code with dotConnect for Oracle v 6.10.141 (Direct=true) in the following environments:
a) .NET Framework 4.0
cmd.ExecuteScalar(): ... speed: ~ 8 MB/s
cmd.ExecuteReader(): ... speed: ~ 8 MB/s
b) mono 2.8.1
cmd.ExecuteScalar(): ... speed: ~ 3.5 MB/s
cmd.ExecuteReader(): ... speed: ~ 3.5 MB/s

1. We think that your results are caused by mono and your platform compatibility. Also maybe some functions of your platform work extremely slowly (e.g., domain sockets, etc). If possible, try different platform for testing purposes.
2. Try setting the DirectUtils.PacketSize property.
3. Upgrade to the latest version of dotConnect for Oracle.

Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

Tested on other available to me platforms

Post by Sao » Wed 27 Apr 2011 12:41

1. Tested on other available to me platforms
a)

Code: Select all

$cat /etc/SuSE-release
openSUSE 11.4 (x86_64)
VERSION = 11.4
CODENAME = Celadon
$mono --version
Mono JIT compiler version 2.10.1 (tarball Wed Feb 23 20:29:11 UTC 2011)
Copyright (C) 2002-2011 Novell, Inc and Contributors. www.mono-project.com
 TLS:           __thread
 SIGSEGV:       altstack
 Notifications: epoll
 Architecture:  amd64
 Disabled:      none
 Misc:          debugger softdebug 
 LLVM:          yes(2.9svn-mono)
 GC:            Included Boehm (with typed GC and Parallel Mark)

$mono testblob.exe 8192
Now is 27.04.2011 15:34:50
cmd.ExecuteScalar(): 00:00:24.3291520; size:0,948768615722656 MB; speed: 0,0389971921636503 MB/s
cmd.ExecuteReader(): 00:00:24.6390750; size:0,948768615722656 MB; speed: 0,0385066653566604 MB/s

$mono testblob.exe 4096
Now is 27.04.2011 15:36:27
cmd.ExecuteScalar(): 00:00:59.9309040; size:0,948768615722656 MB; speed: 0,0158310412891929 MB/s
cmd.ExecuteReader(): 00:01:01.4260280; size:0,948768615722656 MB; speed: 0,0154457100127434 MB/s
b)

Code: Select all

$ uname -a
Linux vms 2.6.25-gentoo-r7 #1 SMP Fri Aug 29 22:47:42 MSD 2008 i686 Intel(R) Xeon(TM) CPU 2.66GHz GenuineIntel GNU/Linux

$ mono --version
Mono JIT compiler version 2.8 (tarball Sat Nov 13 01:35:05 MSK 2010)
Copyright (C) 2002-2010 Novell, Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  x86
        Disabled:      none
        Misc:          debugger softdebug
        LLVM:          supported, not enabled.
        GC:            Included Boehm (with typed GC and Parallel Mark)

$ mono testblob.exe 8192
Now is 04/27/2011 14:19:14
cmd.ExecuteScalar(): 00:00:25.4104050; size:0.948768615722656 MB; speed: 0.0373377998391862 MB/s
cmd.ExecuteReader(): 00:00:25.6608080; size:0.948768615722656 MB; speed: 0.036973450552401 MB/s
2. Tried different PacketSize (as parameter in calls above). Value 4096 seems to be default, because repeats my initial result. Value 8192 gives approximately double speed, but still unacceptable.

3. My version is latest dotConnect for Oracle Standard
Devart.Data.Oracle.dll 6.10.141.0

P.S.
Also I must say that your test result of 3,5 MB/s is unasseptable for me.
I need to clear what is the cause. If it is somewhere in mono, i can make patch, but I must know where is the problem.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Fri 29 Apr 2011 13:51

1. Please try running your test program under .NET Framework (Windows) againt the same Oracle server (Linux) and notify us about the results.
2.
Sao wrote:Also I must say that your test result of 3,5 MB/s is unasseptable for me.
If I set DirectUtils.PacketSize=8192 in my environment (mono 2.8.1), the results are:
cmd.ExecuteScalar(): ... speed: ~ 6.5 MB/s
cmd.ExecuteReader(): ... speed: ~ 7 MB/s

Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

In Windows all Ok

Post by Sao » Sat 30 Apr 2011 06:37

1. In windows I get all my bandwidth of 10Mbit/sec

Code: Select all

>testblob.exe 8192
Now is 30.04.2011 10:27:12
cmd.ExecuteScalar(): 00:00:01.4800021; size:0,948768615722656 MB; speed: 0,641058965877586 MB/s
cmd.ExecuteReader(): 00:00:00.8800012; size:0,948768615722656 MB; speed: 1,07814468403299 MB/s
2. My program should work locally on server or in Tbit/sec connection with it. I expected to recieve speed limitation from Oracle DB storage susbsystem.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Wed 04 May 2011 14:02

1. As I understood, you were running all your tests against the same Oracle 11g R1 installed on the SLES 11.1 x64, mono 2.8.2 platform. Is this correct?
2. Platform SLES 11.1 x64, mono 2.8.2:
a) is it a real computer or virtual machine?
b) tell us its hardware characteristics.
3. You have got 15 kbytes/sec when running your test program on the same SLES 11.1 x64, mono 2.8.2 platform where Oracle 11g R1 is installed, haven't you?
4.
Sao wrote:My program should work locally on server or in Tbit/sec connection with it.
Is Tbit/sec your desirable network speed or speed of transferring data with Oracle? Which speed of transferring data with Oracle is acceptable for you?

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Wed 04 May 2011 14:38

Shalex wrote:I have tried your code with dotConnect for Oracle v 6.10.141 (Direct=true) in the following environments:
a) .NET Framework 4.0
cmd.ExecuteScalar(): ... speed: ~ 8 MB/s
cmd.ExecuteReader(): ... speed: ~ 8 MB/s
These are the results for remote server in local network (limitation of network).

Local server:
cmd.ExecuteScalar(): ... speed: ~ 34 MB/s
cmd.ExecuteReader(): ... speed: ~ 60 MB/s

Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

Replying on post Wed 04 May 2011 14:02

Post by Sao » Thu 05 May 2011 13:49

Thank you!

1. Yes. You understood right.
2. This is a real machine
Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz
RAM 4GB
Raid0 on 3ware controller.
3.I got 15 kbytes/sec from all my available linux platforms.
a) the same server locally, connecting to localhost
b) others I posted Wed 27 Apr 2011 12:41
4. Tbit/sec is desirable network speed. I understand, that Oracle transfers data slowly.
The result you got of ~ 60 MB/sec I would treat as ucceptable.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Tue 10 May 2011 13:52

We think that a low speed in your Linux environment is caused by some hardware problem (HDD write/read). For example: http://www.linuxquestions.org/questions ... id-310035/. Better performance with Windows on the client side in your tests can be explained that Oracle data were cached in RAM.

Please try using another .NET provider. Does the problem persist in this case as well?

Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

I disagree

Post by Sao » Tue 10 May 2011 21:16

I have no performance issue with this server and native mono/.NET oracle provider (System.Data.OracleClient)
I have no performance issue with this server and any Microsoft Windows client, even with dotConnect provider.
And I already wrote this on Sat 30 Apr 2011.

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

please confirm

Post by Shalex » Wed 11 May 2011 14:41

Sao wrote:I have no performance issue with this server and native mono/.NET oracle provider (System.Data.OracleClient)
I have no performance issue with this server and any Microsoft Windows client, even with dotConnect provider.
Is this scenario possible?
Your first test was run with dotConnect for Oracle on Mono and you have got a low performance. During this first test, Oracle data were cached in RAM. As a result, all further tests were actually performed with data from RAM (not from database). Try running tests with different providers in different orders. Does the problem persist only with dotConnect for Oracle on Mono in this case as well?

Looking forward to your reply.

Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

Re: please confirm

Post by Sao » Wed 11 May 2011 21:15

No. Your scenario is impossible.
Oracle disk performance is ok. I tested in mono 2.8.2 with System.Data.Oracle. Here is log of test launches for the same blob on localhost:

Code: Select all

[oracle@ERR-3:~/err/testblob] $ mono testblob.exe ora
Now is 12.05.2011 2:26:17
cmd.ExecuteScalar(): 00:00:00.2379940; size:0,948768615722656 MB; speed: 3,98652325572349 MB/s
cmd.ExecuteReader(): 00:00:00.0664200; size:0,948768615722656 MB; speed: 14,2843814471945 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe ora
Now is 12.05.2011 2:26:24
cmd.ExecuteScalar(): 00:00:00.0492780; size:0,948768615722656 MB; speed: 19,2533912846028 MB/s
cmd.ExecuteReader(): 00:00:00.0632530; size:0,948768615722656 MB; speed: 14,9995828770597 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe ora
Now is 12.05.2011 2:26:25
cmd.ExecuteScalar(): 00:00:00.0485650; size:0,948768615722656 MB; speed: 19,5360571547958 MB/s
cmd.ExecuteReader(): 00:00:00.0641180; size:0,948768615722656 MB; speed: 14,7972272329557 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe ora
Now is 12.05.2011 2:26:27
cmd.ExecuteScalar(): 00:00:00.0501850; size:0,948768615722656 MB; speed: 18,9054222521203 MB/s
cmd.ExecuteReader(): 00:00:00.0631600; size:0,948768615722656 MB; speed: 15,0216690266412 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe ora
Now is 12.05.2011 2:26:28
cmd.ExecuteScalar(): 00:00:00.0493970; size:0,948768615722656 MB; speed: 19,2070088410765 MB/s
cmd.ExecuteReader(): 00:00:00.0639140; size:0,948768615722656 MB; speed: 14,8444568595716 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe devart 8192
While saving registry data at /etc/mono/2.0/../registry/last-btime: System.UnauthorizedAccessException: Access to the path "/etc/mono/registry/last-btime" is denied.
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x00000] in :0
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share) [0x00000] in :0
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at System.IO.StreamWriter..ctor (System.String path, Boolean append, System.Text.Encoding encoding, Int32 bufferSize) [0x00000] in :0
  at System.IO.StreamWriter..ctor (System.String path, Boolean append, System.Text.Encoding encoding) [0x00000] in :0
  at (wrapper remoting-invoke-with-check) System.IO.StreamWriter:.ctor (string,bool,System.Text.Encoding)
  at Microsoft.Win32.KeyHandler.SaveRegisteredBootTime (System.String path, Int64 btime) [0x00000] in :0
Now is 12.05.2011 2:26:45
cmd.ExecuteScalar(): 00:00:18.8095270; size:0,948768615722656 MB; speed: 0,0504408545585786 MB/s
cmd.ExecuteReader(): 00:00:24.8953610; size:0,948768615722656 MB; speed: 0,0381102573978604 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe devart 8192
While saving registry data at /etc/mono/2.0/../registry/last-btime: System.UnauthorizedAccessException: Access to the path "/etc/mono/registry/last-btime" is denied.
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean anonymous, FileOptions options) [0x00000] in :0
  at System.IO.FileStream..ctor (System.String path, FileMode mode, FileAccess access, FileShare share) [0x00000] in :0
  at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
  at System.IO.StreamWriter..ctor (System.String path, Boolean append, System.Text.Encoding encoding, Int32 bufferSize) [0x00000] in :0
  at System.IO.StreamWriter..ctor (System.String path, Boolean append, System.Text.Encoding encoding) [0x00000] in :0
  at (wrapper remoting-invoke-with-check) System.IO.StreamWriter:.ctor (string,bool,System.Text.Encoding)
  at Microsoft.Win32.KeyHandler.SaveRegisteredBootTime (System.String path, Int64 btime) [0x00000] in :0
Now is 12.05.2011 2:27:31
cmd.ExecuteScalar(): 00:00:24.8638330; size:0,948768615722656 MB; speed: 0,0381585822154877 MB/s
cmd.ExecuteReader(): 00:00:24.9018670; size:0,948768615722656 MB; speed: 0,0381003005004667 MB/s
[oracle@ERR-3:~/err/testblob] $ mono testblob.exe ora
Now is 12.05.2011 2:28:54
cmd.ExecuteScalar(): 00:00:00.0501970; size:0,948768615722656 MB; speed: 18,9009027575882 MB/s
cmd.ExecuteReader(): 00:00:00.0645620; size:0,948768615722656 MB; speed: 14,6954650680378 MB/s
I'm almost sure it is issue of network connection in mono-dotConnect-Oracle

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Tue 17 May 2011 13:29

Could you please try running one more test to help us to localize the issue?
Environment: remote Oracle server, application is run on your SLES 11.1 x64 with:
a) System.Data.OracleClient
b) Devart.Data.Oracle

We assume that OracleClient can connect to Oracle server on localhost without using sockets. So we ask you to test both providers when connection to Oracle server is established via network.

Sao
Posts: 10
Joined: Sat 23 Apr 2011 12:52

sorry for long absence

Post by Sao » Thu 14 Jul 2011 07:36

Could you please try running one more test to help us to localize the issue?
Environment: remote Oracle server, application is run on your SLES 11.1 x64 with:
a) System.Data.OracleClient
b) Devart.Data.Oracle

We assume that OracleClient can connect to Oracle server on localhost without using sockets. So we ask you to test both providers when connection to Oracle server is established via network.
Theoretically it could be.
But I think, that test with the same server from windows (posted Sat 30 Apr 2011 06:37) proves that Oracle can send data with proper speed/

I try to reformulate problem:
I have Oracle server, and program, connecting throu DevArt. From Windows all Ok. From mono/linux strange speed.
I see here no logic, that it could be caused by Oracle network subsystem.

May be you coud give me driver version with some debug output?
Or I could give you access to my server...

Shalex
Site Admin
Posts: 9543
Joined: Thu 14 Aug 2008 12:44

Post by Shalex » Mon 18 Jul 2011 14:38

We think that the problem is caused by compatibility of a particular version of Mono with a particular version of Linux. System.Data.OracleClient works via Oracle client and uses native Linux sockets. Our direct mode uses Mono sockets - probably, the isssue is here. We will try to create a program that tests the speed of sockets so that you can check this speed under your Mono and .NET Framework. We will send the program in a week or so.

Post Reply