Hi,
I'm evaluating UniDAC for a project to access oracle and sqlite databases..
I've downloaded the trial (unicode and non-unicode) for Delphi 2007 (btw what is the difference between unicode and non unicode version knowing that D2007 is not unicode?)
I have a sqlite database with (among others) a TEXT field, if I enter low ascii (127 the DBMemo is empty, eg:
Paolo (I see the value)
Paolò (I don't see the value)
And is not a visualization problem, a simple UniTable1.FieldByName('').AsString return an empty string
Enabling unicode has no effect (also I've checked, the text is stored in sqlite db in ansi, not unicode)
Obviously I see the field (in both cases) with SQLite Expert.
Another thing.. The database is created and filled by another application, so any solution that involve changes in the db unfortunately is not an option..
Thanks in advance,
Paolo.
High-ascii chars problem with SQLite
more informations...
I've used a newly created database with the same structure and here I have no problem inserting (and reading) high-ascii chars, so the problem must lie in the database creation or use.
But.. I am able to visualize the data on every tools I've used and also with other delphi libraries for sqlite so if you want I've created a small test-case and uploaded to my site (scada.db3, 4KB). If you want you could test unidac with this sqlite database.
http://www.paolorossi.net/scada.db3
The database contains 1 table with 4 records, the problematic field is 'DESCRIZIONE' .
thanks in advance
paolo.
I've used a newly created database with the same structure and here I have no problem inserting (and reading) high-ascii chars, so the problem must lie in the database creation or use.
But.. I am able to visualize the data on every tools I've used and also with other delphi libraries for sqlite so if you want I've created a small test-case and uploaded to my site (scada.db3, 4KB). If you want you could test unidac with this sqlite database.
http://www.paolorossi.net/scada.db3
The database contains 1 table with 4 records, the problematic field is 'DESCRIZIONE' .
thanks in advance
paolo.
Hello,
I have downloaded your sample database, but in Delphi 2007 and XE with UniDAC, SqlLite Expert, and SQLite2009Pro data from the 'DESCRIZIONE' field is not displayed correctly. So in all applications I could see only 'low-ascii' characters.
I think that there is a problem in the database creation, because if I create a database with 'high-ascii' characters on my PC I can see all characters correctly.
I have downloaded your sample database, but in Delphi 2007 and XE with UniDAC, SqlLite Expert, and SQLite2009Pro data from the 'DESCRIZIONE' field is not displayed correctly. So in all applications I could see only 'low-ascii' characters.
I think that there is a problem in the database creation, because if I create a database with 'high-ascii' characters on my PC I can see all characters correctly.
Hi Alex,
thanx for your time..
To see the field "descrizione" in SQLite Expert you must enable the "Default ANSI code page" setting in Tools->Options->General->Encoding
I've investigated the problem a little further and its all about string encoding, the app that fill the database write the data as ANSI, no conversion to UTF-8 prior to send to the database. I've also opened the file with an hex editor and my toughts were confirmed, the data is stored in a single-byte string.
I've tried unidac with D2007, D2010, DXE and I've noticied a slighty different results: in D2007 the string extracted is empty in >D2010 the string extracted is filled with question mark for the non UTF-8 chars..
I've no sources in trial but I bet that UniDAC tries to convert the string with a UTF8Decode function (this one is changed from D2007 and D2009)
So, given the fact that SQLite documentation states that a TEXT field is UTF-8 or UTF-16 encoded, the app that fill the datatabase is the one to blame..
but:
1) From this page http://www.sqlite.org/version3.html
"SQLite is not particular about the text it receives and is more than happy to process text strings that are not normalized or even well-formed UTF-8 or UTF-16. Thus, programmers who want to store IS08859 data can do so using the UTF-8 interfaces"
So it seems that SQLite does not care about string encodings letting the developers dealing with that.
2) UniDAC filters (mandatory, if I haven't missed something) text data with a UTF-8 decoding function
I think that this could be an optional behaviour (but obviously the default) so a developer can read and write data in the encoding of choice (the developer has the responsibility to encode/decode data correctly)
This has the same mean (I think) the "Encoding" settings in SQLite Expert.
btw, I'm happy with the unidac evaluation so a new order is in the way.. I'll buy the Professional version with sources version so I could patch for myself (for this app of mine) the data fetching routine, obviously waiting for a more general solution...
again... thx for your time
paolo.
thanx for your time..
To see the field "descrizione" in SQLite Expert you must enable the "Default ANSI code page" setting in Tools->Options->General->Encoding
I've investigated the problem a little further and its all about string encoding, the app that fill the database write the data as ANSI, no conversion to UTF-8 prior to send to the database. I've also opened the file with an hex editor and my toughts were confirmed, the data is stored in a single-byte string.
I've tried unidac with D2007, D2010, DXE and I've noticied a slighty different results: in D2007 the string extracted is empty in >D2010 the string extracted is filled with question mark for the non UTF-8 chars..
I've no sources in trial but I bet that UniDAC tries to convert the string with a UTF8Decode function (this one is changed from D2007 and D2009)
So, given the fact that SQLite documentation states that a TEXT field is UTF-8 or UTF-16 encoded, the app that fill the datatabase is the one to blame..
but:
1) From this page http://www.sqlite.org/version3.html
"SQLite is not particular about the text it receives and is more than happy to process text strings that are not normalized or even well-formed UTF-8 or UTF-16. Thus, programmers who want to store IS08859 data can do so using the UTF-8 interfaces"
So it seems that SQLite does not care about string encodings letting the developers dealing with that.
2) UniDAC filters (mandatory, if I haven't missed something) text data with a UTF-8 decoding function
I think that this could be an optional behaviour (but obviously the default) so a developer can read and write data in the encoding of choice (the developer has the responsibility to encode/decode data correctly)
This has the same mean (I think) the "Encoding" settings in SQLite Expert.
btw, I'm happy with the unidac evaluation so a new order is in the way.. I'll buy the Professional version with sources version so I could patch for myself (for this app of mine) the data fetching routine, obviously waiting for a more general solution...
again... thx for your time
paolo.
