Problem with UTf8 string fields

[schoellm]

We use Delphi 7 for building a non unicode application. Strings that the user of the app enters at UI are stored to database into Varchar-fields, defined as UTF8, using an IBCQuery. The user can retrieve the previously saved record from database, the app retrieves data with IBCQuery and displays it at UI again. IBCConnection.Options.UseUnicode is set to true.

I encountered that, when changing windows-region settings, other data is stored to the database fields, the special characters (that are not standard ASCII chars) seem not to be encoded correctly into UTF8.
Example :
Setting1
* Set windows setting "clock language and region/region and language/administrative/Language for non-unicode programs" to "greek (Greece)"
* Set windows setting "clock language and region/region and language/Formats/Format to "greek (Greece)"
Use the application to store some data with special characters, e.g with α , β, γ into database.
->varchar data in database seems to be correctly encoded UTF8. Loading record from database to UI works fine.
Setting2
* change windows setting "clock language and region/region and language/Formats/Format to "Germany (German)"
Use the application to store same data as before (with special characters, α , β, γ) into database.
->Compare the new database record to the one stored before (with setting1): it has OTHER data in the varchar field, it seems NOT to be correctly encoded into UTF8. But loading this record from database to UI works fine.
->Furthermore, when the app loads and displays first record, all special characters are displayed wrongly (just "rubbish").

I don't understand why this happens, why does this windows setting influence what is stored to database? Is there a bug or is it wrong usage of IBDAC?

The following lines show the commands for storing:
Query.Connection.Options.Charset := 'UTF8';
Query.Params.Clear;
Query.SQL.Text := 'UPDATE OR INSERT INTO NEW_TABLE (NAME, NUMBER, COMMENT, UID) Values (:NAME, :NUMBER, :COMMENT, :UID) MATCHING (UID)';
Query.Prepare;
Query.ParamByName('NUMBER').Value := DisplayNumberEdit.Text; //This is an TEdit field at UI with font.Charset=DEFAULT_CHARSET
Query.ParamByName('NAME').Value := NameEdit.Text;
Query.ParamByName('COMMENT').Value := CommentEdit.Text;
Query.ParamByName('UID').Value := fCurrentPerson.Guid;
Query.ExecSQL;

We use firebird server version 2.5. and use "IBdac360d7pro".We also tested with a firebird snapshotbuild for 2.5.3 and newest IBDACDemo, resulting in the same behaviour.
I can provide a small example application with source code to reproduce the problem easily, via email (I've not found out how to attach files in this forum).
I have entered this problem into the "support request form" two weeks ago, but did not get any response.

[PavloP]

Behavior of the latest IBDAC versions differs from behavior of the version 3.6. Please download the latest IBDAC version (5.3.7) and check whether the problem is reproduced.

[schoellm]

I have downloaded the version 5.3.7.0 and recompiled my test program, but the behaviour is the same.

[PavloP]

Thank you for the information.
We have reproduced the problem and investigation is in progress.

[ZEuS]

The problem may be due to the incorrect international characters handling in Delphi 7.
Several reports on Quality Central point to similar problems:
http://qc.embarcadero.com/wc/qcmain.aspx?d=2654
http://qc.embarcadero.com/wc/qcmain.aspx?d=2460
Please try the workaround proposed in the last report. Open the project source (the Project -> View Source item in the IDE menu) and add the following line above Application.Initialize:

Code: Select all

SetThreadLocale(LOCALE_SYSTEM_DEFAULT);

In order to use the SetThreadLocale procedure, add the Windows unit to the project USES clause.

[schoellm]

Yes, with this workaround the data gets no longer stored in wrongly converted utf8. The problem that remains is, that data records that were stored wrongly with older program versions, before implementation of the workaround, will show up wrongly in UI afterwards, I guess. We will have to convert them in some way and store them correctly...
Thank you for helping.

[ZEuS]

It is good to see that this problem has been solved. Feel free to contact us if you have any further questions about IBDAC.