I am trying to use dbexpress driver for mysql to read and write utf8mb4 data to mysql. I tried both XE2 and XE8, but the driver is not able to update utf8mb4 data into mysql using the option 'use unicode=true'. Moreover, there is no parameter to set charset for the driver. For example, the chinese character (http://www.scarfboy.com/coding/unicode-tool?s=U%2B232ab) is only available in utf8mb4 but not in utf8.
Is there any workaround or solutions ?
I cannot post the character to the forum. So, please refer to the link for that chinese character.
Unable to process utf8mb4 data
Re: Unable to process utf8mb4 data
Unfortunately, we couldn't reproduce the issue: data in utf8mb4 encoding was inserted and updated correctly.
Please check whether the problem is reproduced on our Query demo project. You can find the demo project in the %DBXMySQLDemos%\Win32\Query directory, where %DBXMySQL% is the path to the installed dbExpress driver for MySQL Demo projects installation path on your computer.
If the problem is not reproduced on our demo project, please send us a small sample to demonstrate the issue, including scripts for creating database objects.
Please check whether the problem is reproduced on our Query demo project. You can find the demo project in the %DBXMySQLDemos%\Win32\Query directory, where %DBXMySQL% is the path to the installed dbExpress driver for MySQL Demo projects installation path on your computer.
If the problem is not reproduced on our demo project, please send us a small sample to demonstrate the issue, including scripts for creating database objects.
-
- Posts: 2
- Joined: Mon 11 Jan 2016 03:44
Re: Unable to process utf8mb4 data
Yes. The problem is exists in the sample project.
Below is the script to create the database.
CREATE SCHEMA `test1` DEFAULT CHARACTER SET utf8 ;
CREATE TABLE `test1`.`dept` (
`name` VARCHAR(100) CHARACTER SET 'utf8mb4' NULL);
ALTER TABLE `test1`.`dept`
CHARACTER SET = utf8 ;
grant all privileges on test1.* to testuser@localhost identified by 'testpassword';
flush privileges;
Please test the character in this link: http://www.scarfboy.com/coding/unicode-tool?s=U%2B232ab
As I cannot post this character in your forum, I think this character also cannot insert into your forum database.
Below is the script to create the database.
CREATE SCHEMA `test1` DEFAULT CHARACTER SET utf8 ;
CREATE TABLE `test1`.`dept` (
`name` VARCHAR(100) CHARACTER SET 'utf8mb4' NULL);
ALTER TABLE `test1`.`dept`
CHARACTER SET = utf8 ;
grant all privileges on test1.* to testuser@localhost identified by 'testpassword';
flush privileges;
Please test the character in this link: http://www.scarfboy.com/coding/unicode-tool?s=U%2B232ab
As I cannot post this character in your forum, I think this character also cannot insert into your forum database.
Re: Unable to process utf8mb4 data
Thank you for the information. The problem investigation is in progress. We will inform you when we have any results.
Re: Unable to process utf8mb4 data
This is a major issue. Current MySQL driver implementation DOES NOT account for this charset (utf8mb4), it is simply not supported. First, there is a constant limit of MaxChars in UTF set to 3. Second, the connection issues command 'SET NAMES utf8' that makes it impossible to use utf8mb4 -- there is an error when the 4-byte UTF symbols are inserted into the utf8mb4 database/table/column. I was able to manually enter utf8mb4 into the SET NAMES command in the source codes and fix some comparisons to include utf8mb4 charset in some of the MySQL units, which stopped this error. However, still, the high surrogates become corrupt (due to the fact that the internal functions do not support them when converting this from Delphi's UTF-16, generating garbage). Please fix ASAP - this IS a bug as it utf8mb4 is the only format that fully supports the entire UTF universe, and the simple UTF8 usage is NOT recommended for MySQL deployments.
Re: Unable to process utf8mb4 data
We have added support for utf8mb4 charset. It will be included in the next build of dbExpress Driver for MySQL that we are going to release within a week.
Re: Unable to process utf8mb4 data
The new dbExpress Driver for MySQL 6.8.11 with support for utf8mb4 charset is already available for download now.