Problem with UTf8 string fields
Posted: Mon 12 May 2014 14:01
We use Delphi 7 for building a non unicode application. Strings that the user of the app enters at UI are stored to database into Varchar-fields, defined as UTF8, using an IBCQuery. The user can retrieve the previously saved record from database, the app retrieves data with IBCQuery and displays it at UI again. IBCConnection.Options.UseUnicode is set to true.
I encountered that, when changing windows-region settings, other data is stored to the database fields, the special characters (that are not standard ASCII chars) seem not to be encoded correctly into UTF8.
Example :
Setting1
* Set windows setting "clock language and region/region and language/administrative/Language for non-unicode programs" to "greek (Greece)"
* Set windows setting "clock language and region/region and language/Formats/Format to "greek (Greece)"
Use the application to store some data with special characters, e.g with α , β, γ into database.
->varchar data in database seems to be correctly encoded UTF8. Loading record from database to UI works fine.
Setting2
* change windows setting "clock language and region/region and language/Formats/Format to "Germany (German)"
Use the application to store same data as before (with special characters, α , β, γ) into database.
->Compare the new database record to the one stored before (with setting1): it has OTHER data in the varchar field, it seems NOT to be correctly encoded into UTF8. But loading this record from database to UI works fine.
->Furthermore, when the app loads and displays first record, all special characters are displayed wrongly (just "rubbish").
I don't understand why this happens, why does this windows setting influence what is stored to database? Is there a bug or is it wrong usage of IBDAC?
The following lines show the commands for storing:
Query.Connection.Options.Charset := 'UTF8';
Query.Params.Clear;
Query.SQL.Text := 'UPDATE OR INSERT INTO NEW_TABLE (NAME, NUMBER, COMMENT, UID) Values (:NAME, :NUMBER, :COMMENT, :UID) MATCHING (UID)';
Query.Prepare;
Query.ParamByName('NUMBER').Value := DisplayNumberEdit.Text; //This is an TEdit field at UI with font.Charset=DEFAULT_CHARSET
Query.ParamByName('NAME').Value := NameEdit.Text;
Query.ParamByName('COMMENT').Value := CommentEdit.Text;
Query.ParamByName('UID').Value := fCurrentPerson.Guid;
Query.ExecSQL;
We use firebird server version 2.5. and use "IBdac360d7pro".We also tested with a firebird snapshotbuild for 2.5.3 and newest IBDACDemo, resulting in the same behaviour.
I can provide a small example application with source code to reproduce the problem easily, via email (I've not found out how to attach files in this forum).
I have entered this problem into the "support request form" two weeks ago, but did not get any response.
I encountered that, when changing windows-region settings, other data is stored to the database fields, the special characters (that are not standard ASCII chars) seem not to be encoded correctly into UTF8.
Example :
Setting1
* Set windows setting "clock language and region/region and language/administrative/Language for non-unicode programs" to "greek (Greece)"
* Set windows setting "clock language and region/region and language/Formats/Format to "greek (Greece)"
Use the application to store some data with special characters, e.g with α , β, γ into database.
->varchar data in database seems to be correctly encoded UTF8. Loading record from database to UI works fine.
Setting2
* change windows setting "clock language and region/region and language/Formats/Format to "Germany (German)"
Use the application to store same data as before (with special characters, α , β, γ) into database.
->Compare the new database record to the one stored before (with setting1): it has OTHER data in the varchar field, it seems NOT to be correctly encoded into UTF8. But loading this record from database to UI works fine.
->Furthermore, when the app loads and displays first record, all special characters are displayed wrongly (just "rubbish").
I don't understand why this happens, why does this windows setting influence what is stored to database? Is there a bug or is it wrong usage of IBDAC?
The following lines show the commands for storing:
Query.Connection.Options.Charset := 'UTF8';
Query.Params.Clear;
Query.SQL.Text := 'UPDATE OR INSERT INTO NEW_TABLE (NAME, NUMBER, COMMENT, UID) Values (:NAME, :NUMBER, :COMMENT, :UID) MATCHING (UID)';
Query.Prepare;
Query.ParamByName('NUMBER').Value := DisplayNumberEdit.Text; //This is an TEdit field at UI with font.Charset=DEFAULT_CHARSET
Query.ParamByName('NAME').Value := NameEdit.Text;
Query.ParamByName('COMMENT').Value := CommentEdit.Text;
Query.ParamByName('UID').Value := fCurrentPerson.Guid;
Query.ExecSQL;
We use firebird server version 2.5. and use "IBdac360d7pro".We also tested with a firebird snapshotbuild for 2.5.3 and newest IBDACDemo, resulting in the same behaviour.
I can provide a small example application with source code to reproduce the problem easily, via email (I've not found out how to attach files in this forum).
I have entered this problem into the "support request form" two weeks ago, but did not get any response.
