LargeObject Read and Write question

LargeObject Read and Write question

Postby baba » Thu 08 Mar 2012 19:17

Hello,

I have got some strange effects.
Could you please confirm :

PgSqlLargeObject.Read(buf, offset, size) (and same for Write).
The doc is ambiguous : I finally found that 'offset', is relative to the largeobject from the beginning, not an offset into the buffer.

All examples provided load the lo in one shot. But in my application, i have to update a progress bar.
If you look in my code below, i define a segment size (4096*10), and i read (or write) the lo segment by segment, plus the last 'incomplete' segment.
It looks like the lo.seek is not relevant.
Can you test on your side a test for a file, let says 100Ko, written segment by segment, idem for the read. I guess you will be surprised.

One remark : I do not find the lo in pg_catalog.pg_largeobject. Is this normal ?

One question : when will it be possible to download the mainstream updates with the memcrypt ??

Seb



public static byte[] GetBytes( Int32 lob) {
byte[] cry = new byte[0];
DBC cc = new DBC();
int fsz;
Int32 rd;
PgSqlLargeObject lobj;
Int32 sz;
Int32 numseg;
Int32 lastsize;
lock ( Lo_lock ) {
try {
cc.BeginTransaction();
lobj = new PgSqlLargeObject(cc, lob, false);
lobj.Open();
if ( !lobj.CanRead ) {
lobj.Close ( );
cc.Commit ( );
lutils.MyTrace ( "GetBytes lob=" + lob + ", CANNOT READ !!!" );
return new byte[ 0 ];
}
sz=(int)lobj.Length;
numseg = sz / SEG_SIZE2;
lastsize = Convert.ToInt32 ( sz % SEG_SIZE2 );
lutils.MyTrace ( "GetBytes POS:"+lobj.Position+" LOB:"+lob+" sz=" + sz + " numseg=" + numseg + " lastsize=" + lastsize + " resultant=" + ( ( numseg * SEG_SIZE2 ) + lastsize ) );
fsz = (int)lobj.Length;
cry = new Byte[fsz];
cry.Initialize ( );
byte[] loc = new byte[ SEG_SIZE2 ];
for ( int i=0; i < numseg; i++ ) {
// rd = lobj.Read ( cry, 0, fsz );
int st = i * SEG_SIZE2;
lutils.MyTrace ( "GetBytes AVANT Pos:" + lobj.Position + " seg:" + i + " " + cry[ st ].ToString ( "X2" ) + " " + cry[ st + 1 ].ToString ( "X2" ) + " "
+ cry[ st + 2 ].ToString ( "X2" ) + " " + cry[ st + 3 ].ToString ( "X2" ) );
loc.Initialize();
lobj.Read ( loc, st, SEG_SIZE2 );
Array.Copy(loc, 0, cry, st, SEG_SIZE2);
lutils.MyTrace ( "GetBytes APRES Pos:"+lobj.Position+" seg:" + i + " " + cry[ st ].ToString ( "X2" ) + " " + cry[ st + 1 ].ToString ( "X2" ) + " "
+ cry[ st + 2 ].ToString ( "X2" ) + " " + cry[ st + 3 ].ToString ( "X2" ) );
}
if ( lastsize > 0 ) {
int st = numseg * SEG_SIZE2;
lutils.MyTrace ( "GetBytes AVANT seg:LAST Pos:" + lobj.Position + " " + cry[ st ].ToString ( "X2" )
+ ( lastsize > 1 ? " " + cry[ st + 1 ].ToString ( "X2" ) : "" )
+ " " + ( lastsize > 2 ? " " + cry[ st + 2 ].ToString ( "X2" ) : "" )
+ " " + ( lastsize > 3 ? " " + cry[ st + 3 ].ToString ( "X2" ) : "" ) );
byte[] loc2 = new byte[ lastsize ];
loc2.Initialize ( );
lobj.Read ( loc2, st, lastsize );
Array.Copy ( loc2, 0, cry, st, lastsize );
lutils.MyTrace ( "GetBytes APRES seg:LAST Pos:" + lobj.Position + " " + cry[ st ].ToString ( "X2" )
+ ( lastsize > 1 ? " " + cry[ st + 1 ].ToString ( "X2" ) : "" )
+ " " + ( lastsize > 2 ? " " + cry[ st + 2 ].ToString ( "X2" ) : "" )
+ " " + ( lastsize > 3 ? " " + cry[ st + 3 ].ToString ( "X2" ) : "" ) );
}
lutils.MyTrace ( "GetBytes lob=" + lob + ", length=" + lobj.Length + ", (int)len=" + fsz );
lobj.Close();
cc.Commit();
}
catch(Exception ee) {
String s = ee.ToString();
s += " ";
return new byte[0];
//throw new Exception("GetBytes");
}
}
cc.Close();
return cry;
}
baba
 
Posts: 22
Joined: Thu 14 Apr 2005 16:40

Postby Pinturiccio » Fri 16 Mar 2012 08:11

baba wrote:The doc is ambiguous : I finally found that 'offset', is relative to the largeobject from the beginning, not an offset into the buffer.

PgSqlLargeObject.Read(buf, offset, size) - offset: The zero-based byte offset in buffer at which to begin storing the data read from the large object.
baba wrote:If you look in my code below, i define a segment size (4096*10), and i read (or write) the lo segment by segment, plus the last 'incomplete' segment.
It looks like the lo.seek is not relevant.

If you read data segment by segment you do not need to use lo.seek(...). Reading the next segment will begin from the position where reading the privious segment has ended.
baba wrote:Can you test on your side a test for a file, let says 100Ko, written segment by segment, idem for the read. I guess you will be surprised.

We have tested reading data segment by segment and all works correctly. I've changed the part of your code where you use the "for" loop:
Code: Select all
for ( int i=0; i < numseg; i++ ) {
int st = i * SEG_SIZE2;
lobj.Read ( loc, 0, SEG_SIZE2 );
Array.Copy(loc, 0, cry, st, SEG_SIZE2);
}
if ( lastsize > 0 ) {
int st = numseg * SEG_SIZE2;
byte[] loc2 = new byte[ lastsize ];
lobj.Read ( loc2, 0, lastsize );
Array.Copy ( loc2, 0, cry, st, lastsize );



baba wrote:One remark : I do not find the lo in pg_catalog.pg_largeobject. Is this normal ?

Try to search your lo in pg_catalog.pg_largeobject_metadata
baba wrote:One question : when will it be possible to download the mainstream updates with the memcrypt ??

We are planning to do it in a month or two.
Pinturiccio
Devart Team
 
Posts: 1982
Joined: Wed 02 Nov 2011 09:44


Return to dotConnect for PostgreSQL