Page 1 of 1

Delphi7 + Zeos + MySQL : Unicode doesn't work

Posted: 28.02.2007, 20:31
by mefisto112
Hi all,
in my Delphi 7 project I have a TntDBGrid (http://www.tntware.com/delphicontrols/unicode/) connected to a ZConnection connected to a MySql5 database with some UTF8 fields.
I can't see and write correctly the characters out of standard ansi (i.e. Russian).
If instead of ZConnection I use MyDac (http://crlab.com/mydac/mydac500d7.exe) everything works fine but this last components aren't free.

Any suggestion?

Mefisto

Posted: 05.03.2007, 13:52
by mefisto112
If I add in ZConnection.Properties the string CodePage=UTF8 and I read data using UTF8Decode the UTF8 values from MySql are corrects but I can't still write them with the corresponding UTF8Encode.

And most important, in MyDac everything works fine (without need of conversions) only setting MyConnection.Options.UseUnicode=True.

DevTeam, I think this must be a new feature: insert UTF8Encode/Decode in the right position of TZConnection (sorry, I don't know where) and enable it with a new parameter/properties.

Posted: 06.03.2007, 00:12
by mdaems
Hi Mefisto,

As I don't understand all this stuff about UTF8 I can only add this post to the feature requests forum. As long as we don't find somebody who wants to program this, it will stay there, I'm afraid.

Mark

Posted: 05.04.2007, 14:25
by mefisto112
Added uses utf8vcl and now everything seems to work: http://sourceforge.net/projects/utf8vcl/

Posted: 28.05.2008, 12:41
by Sakura
mdaems wrote:As I don't understand all this stuff about UTF8

There are three different types of character encoding I'm aware at this moment:
ANSI: single-byte
MBCS: Multi-byte character sets
UNICODE: Two-byte character set

ANSI is using codes of 0-255 to display a maximum of 256 different characters. The "codepage" decides which character is what code.
UNICODE is using exactly two bytes which is very simple (no wonder Microsoft used it for NT4/2000/XP/Vista) it allows 65536 different characters.

UTF8 is one (not *the*) Multibyte encoding. The reason for using MBCS is: In most western languages, characters of code less than 128 are the most used characters in any text. The space ($20), characters 'a' to 'z' and 'A' to 'Z' as well as '0'..'9' are all in that bunch of codes.
So, MBCS - in this case UTF8 - uses one, two, or even three bytes per character making a string length unpredictable without decoding the whole string (good reason for Microsoft(r) "640 kb is enough for everybody" Windows(tm) not to use it :-)). You might refer to codes above 127 as 'escape' codes which trigger subsets and sub-subsets of 256 or (in case of three bytes codes) 65536 character codes.
Windows NT/2000/XP/Vista is using UNICODE for all functions. The 'A' versions of functions are just wrapper functions converting arguments to UNICODE and calling the 'W' function. Most functions that accept strings have such wrapper functions. Example: LoadStringA() LoadStringW()

The Delphi programmer should notably use the two functions:
UTF8Encode() to cast aa UNICODE (WideString) into UTF8String
UTF8Decode() to cast an UTF8String into WideString

Most notably, when dealing with UNICODE you should never ever touch any library function from VCL! These are always using AnsiString's. Use of AnsiString will internally convert into the ANSI charset, means: Whatever character isn't in the 256 character cp1251 charset is changed to a question mark ('?') There is a freeware conversion of the popular (unix) library 'libiconv' available at: www.yunqa.de

The benefits of UTF8 is: Less memory consumption since characters as numbers and letters a-z use less space. UTF8 can use an unlimited number of character codes while ANSI only uses 256 and UNICODE uses 65536.
The downside of UTF8 is: There is no relation between the number of characters and the number of bytes consumed for a string, however maximum is currently three bytes. In databases the CHAR type typically is limited to 255 characters. Using UTF8 you can guarantee only an amount of 255 DIV 3 = 85 characters to be stored into that field.

I hope this little introduction is of some use to you. I'd also like to point you to (Delphi professional or enterprise only!) the VCL source code which notably implements one or two important unicode functions: ${BDS}\source\win32\rtl\common\WideStrings.pas and WideStrUtils.pas

Posted: 08.04.2012, 11:39
by EgonHugeist
I made a patch in the testing-egonhugeist branch. To have Unicode support with D7-D2007 uncomment the {.DEFINE WITH_UTF8_CONTROLS} and use UTF8 as ClientCodePage-Property of TZConnection.

Now Zeos assings all string fields as TStringField and returns utf8 strings. So you don't need to encode the Strings from WideStrings.

If you switch PreprepareSQL := True; you don't have to change your custom and need to use AnsiToUtf8 for direct statements.

But you need UTF8 controls like TNT or several SourceForge-Projects who can display the data right.

Best regards

Delphi7 Zeos MySQL Unicode doesnt work

Posted: 02.12.2016, 08:57
by Cjowet
I am using the File Operations plugin to write data to files, but would like to be able to write directly into a MySQL table.
Is there a plugin for this?