Length problem with unicode surrogate pairs

Discussion of open issues, suggestions and bugs regarding usage of dbExpress drivers for Oracle in Delphi and C++Builder
devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Length problem with unicode surrogate pairs

Post by devartfan » Fri 04 Jun 2010 15:38

Hello,

I have the following problem with the Tsimpledataset component
using the dbexpress components in delphi 2009.
I use an oracle 10g Database using the AL32UTF8 character set, which gave us full unicode support.

1)
For example in the database

CREATE TABLE TEST
(
ID NUMBER,
NAME_40B VARCHAR2(40 BYTE),
NAME_10C VARCHAR2(10 CHAR)
)

2) Enter 10 Unicode Supplementary characters (surrogate pairs) from unicode page 2 in column NAME_40B;

3) Execute statement Update Test set name_10C = name_40B;

The Component Tsimpledataset used for a TDBGrid delivers now 10 characters for name_40b and only 5 characters for name_10c.
This is wrong.

The same problematic appears using TOraQuery from your ODAC components.
Only the Delphi2009 build in dbexpress components, which we will not use shows the correct result here.

We need a solution urgent.

Regards

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Mon 07 Jun 2010 15:25

Hello

Please specify which dbExpress driver and ODAC versions are you using. I have tried to reproduce your issue on dbExress 4.50.20 version and ODAC 6.90.0.58 version - all works correctly.

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Thu 10 Jun 2010 14:23

Hello,

we are using dbxoda440_16.

We used characters from unicode page 2!

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Thu 10 Jun 2010 16:00

Unfortunately I cannot reproduce your issue. I need additional information to do it:
- I need to know all parameter values for your TSQLConnection
- I need to know all parameter values for your TSQLQuery
- I need Unicode text that you are inserting into the database
- I need the SQL query or Delphi code that you are using for inserting your Unicode text into the database
- I need the SQL query or Delphi code that you are using to copy Unicode text from the NAME_40B field to the NAME_10C field
- I need the SQL query that selects data from table for showing it in a DB grid

To fix this issue quickly you can send us a simple example that reproduces this issue by E-mail and perform the following steps:
1. Create a table in the database
2. Insert Unicode text into this table
3. Copy this Unicode text from the NAME_40B field to the NAME_10C field
4. Show the result in a DB grid

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Thu 17 Jun 2010 12:17

Point 1 to 3 are simply done with the tool Toad in nthe newest vesion 10.5.., using the sql editor.
The tool toad also shows the same wrong behavior in the data grids, but i still have no answere from quest soft.
With the length and lengthb function it is possible to query the length
of the fields and you can see that the length's are identical but the display
of the data differs.

Only Point 4 is done with the test program.
Can you give me an email adress, than i can sent you some screenshots,
also the required settings and an example program.

regards

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Fri 18 Jun 2010 12:30

The problem is that third-party utilities like Toad or any other can insert invalid data to the table. And when you try to show this data with the Devart dbExpress driver it will be very hard to determine where the error is: invalid data was stored to database or our dbExpress driver read data incorrectly. Therefore I ask you to perform all four steps in the sample application.

You can send sample application to the E-mail: [email protected]

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Thu 08 Jul 2010 08:23

Regarding this topic:

On Jun 21. we have sent you an example program and some screenshorts to your email adress "[email protected]".

Until now we do not receive any kind of answer.

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Thu 08 Jul 2010 15:09

Hello

I checked all E-mails that we got since the 21st of June and I could not find you E-mail. It seems like spam filter or firewall blocked your E-mail. Please send it to us again. But before sending please archive your sample with password and write this password to the same E-mail. Usually it is the best way to send and receive E-mail through spam filters and firewalls. And if I don't confirm receiving this E-mail during 24 hours please notify us at the forum again.

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Fri 09 Jul 2010 10:57

I've sent the email again, with zipped attachements.
password is unicode1

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Mon 12 Jul 2010 12:45

Hello

I still haven't got you sample by E-mail. Please try to sent it to me again directly to my E-mail: borism at devart.com

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Thu 15 Jul 2010 11:24

Hi,

Our it support tell me, that the email was sent, but the receiver devart
considered the mail by "greylisting" as spam.
Can you set eckelmann.de onto a white list please ?

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Thu 15 Jul 2010 13:48

Hi

I don't know how much does it take to tune up our spam filter. Try to send your sample to this E-mail: devartinfo at gmail.com

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Fri 23 Jul 2010 14:37

Hello,

It takes some time but now our it-support has forwarded my last mail to devartinfo at gmail.com.

bork
Devart Team
Posts: 649
Joined: Fri 12 Mar 2010 07:55

Post by bork » Tue 03 Aug 2010 14:05

Hello

I still haven't got you sample by E-mail to the gmail mailbox. If you have FTP server in your company you can share your sample and we will download it.

devartfan
Posts: 17
Joined: Thu 30 Apr 2009 10:01

Post by devartfan » Wed 04 Aug 2010 10:24

Hi,

I've put now all the help files to our ftp download site.
Additional information is sent to you by separate email

Regards

Post Reply