Delphi7 + Zeos + MySQL : Unicode doesn't work

Freature requests from users for ZeosLib's DBOs

Moderators: gto, cipto_kh, EgonHugeist, mdaems

mefisto112
Fresh Boarder
Fresh Boarder
Posts: 4
Joined: 28.02.2007, 20:22

Delphi7 + Zeos + MySQL : Unicode doesn't work

Postby mefisto112 » 28.02.2007, 20:31

Hi all,
in my Delphi 7 project I have a TntDBGrid (http://www.tntware.com/delphicontrols/unicode/) connected to a ZConnection connected to a MySql5 database with some UTF8 fields.
I can't see and write correctly the characters out of standard ansi (i.e. Russian).
If instead of ZConnection I use MyDac (http://crlab.com/mydac/mydac500d7.exe) everything works fine but this last components aren't free.

Any suggestion?

Mefisto

mefisto112
Fresh Boarder
Fresh Boarder
Posts: 4
Joined: 28.02.2007, 20:22

Postby mefisto112 » 05.03.2007, 13:52

If I add in ZConnection.Properties the string CodePage=UTF8 and I read data using UTF8Decode the UTF8 values from MySql are corrects but I can't still write them with the corresponding UTF8Encode.

And most important, in MyDac everything works fine (without need of conversions) only setting MyConnection.Options.UseUnicode=True.

DevTeam, I think this must be a new feature: insert UTF8Encode/Decode in the right position of TZConnection (sorry, I don't know where) and enable it with a new parameter/properties.

User avatar
mdaems
Zeos Project Manager
Zeos Project Manager
Posts: 2766
Joined: 20.09.2005, 15:28
Location: Brussels, Belgium
Contact:

Postby mdaems » 06.03.2007, 00:12

Hi Mefisto,

As I don't understand all this stuff about UTF8 I can only add this post to the feature requests forum. As long as we don't find somebody who wants to program this, it will stay there, I'm afraid.

Mark

mefisto112
Fresh Boarder
Fresh Boarder
Posts: 4
Joined: 28.02.2007, 20:22

Postby mefisto112 » 05.04.2007, 14:25

Added uses utf8vcl and now everything seems to work: http://sourceforge.net/projects/utf8vcl/

Sakura
Fresh Boarder
Fresh Boarder
Posts: 5
Joined: 27.05.2008, 17:01

Postby Sakura » 28.05.2008, 12:41

mdaems wrote:As I don't understand all this stuff about UTF8

There are three different types of character encoding I'm aware at this moment:
ANSI: single-byte
MBCS: Multi-byte character sets
UNICODE: Two-byte character set

ANSI is using codes of 0-255 to display a maximum of 256 different characters. The "codepage" decides which character is what code.
UNICODE is using exactly two bytes which is very simple (no wonder Microsoft used it for NT4/2000/XP/Vista) it allows 65536 different characters.

UTF8 is one (not *the*) Multibyte encoding. The reason for using MBCS is: In most western languages, characters of code less than 128 are the most used characters in any text. The space ($20), characters 'a' to 'z' and 'A' to 'Z' as well as '0'..'9' are all in that bunch of codes.
So, MBCS - in this case UTF8 - uses one, two, or even three bytes per character making a string length unpredictable without decoding the whole string (good reason for Microsoft(r) "640 kb is enough for everybody" Windows(tm) not to use it :-)). You might refer to codes above 127 as 'escape' codes which trigger subsets and sub-subsets of 256 or (in case of three bytes codes) 65536 character codes.
Windows NT/2000/XP/Vista is using UNICODE for all functions. The 'A' versions of functions are just wrapper functions converting arguments to UNICODE and calling the 'W' function. Most functions that accept strings have such wrapper functions. Example: LoadStringA() LoadStringW()

The Delphi programmer should notably use the two functions:
UTF8Encode() to cast aa UNICODE (WideString) into UTF8String
UTF8Decode() to cast an UTF8String into WideString

Most notably, when dealing with UNICODE you should never ever touch any library function from VCL! These are always using AnsiString's. Use of AnsiString will internally convert into the ANSI charset, means: Whatever character isn't in the 256 character cp1251 charset is changed to a question mark ('?') There is a freeware conversion of the popular (unix) library 'libiconv' available at: www.yunqa.de

The benefits of UTF8 is: Less memory consumption since characters as numbers and letters a-z use less space. UTF8 can use an unlimited number of character codes while ANSI only uses 256 and UNICODE uses 65536.
The downside of UTF8 is: There is no relation between the number of characters and the number of bytes consumed for a string, however maximum is currently three bytes. In databases the CHAR type typically is limited to 255 characters. Using UTF8 you can guarantee only an amount of 255 DIV 3 = 85 characters to be stored into that field.

I hope this little introduction is of some use to you. I'd also like to point you to (Delphi professional or enterprise only!) the VCL source code which notably implements one or two important unicode functions: ${BDS}\source\win32\rtl\common\WideStrings.pas and WideStrUtils.pas

User avatar
EgonHugeist
Zeos Dev Team
Zeos Dev Team
Posts: 1701
Joined: 31.03.2011, 22:38

Postby EgonHugeist » 08.04.2012, 11:39

I made a patch in the testing-egonhugeist branch. To have Unicode support with D7-D2007 uncomment the {.DEFINE WITH_UTF8_CONTROLS} and use UTF8 as ClientCodePage-Property of TZConnection.

Now Zeos assings all string fields as TStringField and returns utf8 strings. So you don't need to encode the Strings from WideStrings.

If you switch PreprepareSQL := True; you don't have to change your custom and need to use AnsiToUtf8 for direct statements.

But you need UTF8 controls like TNT or several SourceForge-Projects who can display the data right.

Best regards
King regards, Michael

You want to help? viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image

Cjowet
Fresh Boarder
Fresh Boarder
Posts: 1
Joined: 30.11.2016, 10:40

Delphi7 Zeos MySQL Unicode doesnt work

Postby Cjowet » 02.12.2016, 08:57

I am using the File Operations plugin to write data to files, but would like to be able to write directly into a MySQL table.
Is there a plugin for this?


Return to “Feature Requests”

Who is online

Users browsing this forum: Bing [Bot] and 1 guest