Page 1 of 1

utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Posted: Fri 28 Feb 2020 14:37
by jordi588

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Posted: Tue 03 Mar 2020 12:54
by Stellar
Thank you for the information. Please describe the issue in more detail: in which case you get 'M\x8adic' instead of 'Mèdic'? Does the issue occur when retrieving metadata (table and column names) or when retrieving records from a table?

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Posted: Wed 04 Mar 2020 14:33
by jordi588
Hi Stellar, and thanks for the reply!

Well, first, it is not only on the 'M\x8adic' example, it occurs when appearing each single special character of the language I'm using on my application (Catalan). I get each single special character with a weird codification (another example: 'Quir\x97rgic' instead of 'Quirúrgic'). The issue occurs when retrieving records from a table.

What surprise my is that if I install the 'Devart ODBC Driver for ASE' in my local Windows the data is outputed correctly whereas the output data is strangely outputed in the Docker container of the application (which is a Debian distribution).

I tried everything possible on the pyodbc python module configuration without success which causes me the impression that I have to modify something in the Dockerfile related to the driver.

Probably the .so file configuration I get in the Docker container when the Driver is installed?

Thanks for your time!

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Posted: Mon 16 Mar 2020 11:46
by Stellar
If UTF-8 encoding is set on both client and server side, there shouldn't be any issues with character encoding.
Please try adding the parameter "String Types" with the value "Unicode" to the connection string.

For example:
DRIVER=Devart ODBC Driver for ASE;String Types=Unicode;Data Source=Server_name;Database=test;User ID=sa;Password=pwd

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Posted: Fri 03 Jul 2020 07:25
by jordi588
Hi and thanks for your response!

I tried this but it does not work. I know that the external database where I amb querying is a SAP Sybase with cp850 encoding. I tried several connectionString that I pass to pyodbc and I also tried several options regarding encoding which offers pyodbc such as:
conn.setdecoding(pyodbc.SQL_CHAR, encoding='cp850')
conn.setdecoding(pyodbc.SQL_WCHAR, encoding='cp850')
conn.setencoding(encoding='cp850')

With this try I amb able to get M┬èdic instead of the desired Mèdic. Where the character ┬ is coming from?

Thank you!

Re: utf-8 coding problems using python pyodbc to connect to a Sybase Database in a Docker container

Posted: Fri 24 Jul 2020 08:48
by Stellar
Devart ODBC Driver for ASE doesn't have any settings for specifying a code page (encoding). Our driver automatically determines the encoding of a database and the local encoding. Which encoding does your database and Debian installation use?