utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Discussion of open issues, suggestions and bugs regarding usage of ODBC Drivers
Post Reply

Stellar
Devart Team
Posts: 496
Joined: Tue 03 Oct 2017 11:00

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Post by Stellar » Tue 03 Mar 2020 12:54

Thank you for the information. Please describe the issue in more detail: in which case you get 'M\x8adic' instead of 'Mèdic'? Does the issue occur when retrieving metadata (table and column names) or when retrieving records from a table?

jordi588
Posts: 3
Joined: Fri 28 Feb 2020 14:35

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Post by jordi588 » Wed 04 Mar 2020 14:33

Hi Stellar, and thanks for the reply!

Well, first, it is not only on the 'M\x8adic' example, it occurs when appearing each single special character of the language I'm using on my application (Catalan). I get each single special character with a weird codification (another example: 'Quir\x97rgic' instead of 'Quirúrgic'). The issue occurs when retrieving records from a table.

What surprise my is that if I install the 'Devart ODBC Driver for ASE' in my local Windows the data is outputed correctly whereas the output data is strangely outputed in the Docker container of the application (which is a Debian distribution).

I tried everything possible on the pyodbc python module configuration without success which causes me the impression that I have to modify something in the Dockerfile related to the driver.

Probably the .so file configuration I get in the Docker container when the Driver is installed?

Thanks for your time!

Stellar
Devart Team
Posts: 496
Joined: Tue 03 Oct 2017 11:00

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Post by Stellar » Mon 16 Mar 2020 11:46

If UTF-8 encoding is set on both client and server side, there shouldn't be any issues with character encoding.
Please try adding the parameter "String Types" with the value "Unicode" to the connection string.

For example:
DRIVER=Devart ODBC Driver for ASE;String Types=Unicode;Data Source=Server_name;Database=test;User ID=sa;Password=pwd

jordi588
Posts: 3
Joined: Fri 28 Feb 2020 14:35

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Post by jordi588 » Fri 03 Jul 2020 07:25

Hi and thanks for your response!

I tried this but it does not work. I know that the external database where I amb querying is a SAP Sybase with cp850 encoding. I tried several connectionString that I pass to pyodbc and I also tried several options regarding encoding which offers pyodbc such as:
conn.setdecoding(pyodbc.SQL_CHAR, encoding='cp850')
conn.setdecoding(pyodbc.SQL_WCHAR, encoding='cp850')
conn.setencoding(encoding='cp850')

With this try I amb able to get M┬èdic instead of the desired Mèdic. Where the character ┬ is coming from?

Thank you!

Stellar
Devart Team
Posts: 496
Joined: Tue 03 Oct 2017 11:00

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Post by Stellar » Fri 24 Jul 2020 08:48

Devart ODBC Driver for ASE doesn't have any settings for specifying a code page (encoding). Our driver automatically determines the encoding of a database and the local encoding. Which encoding does your database and Debian installation use?

Post Reply