Zeos 7.1.0 problem with hungarian characters

The official tester's forum for ZeosLib 7.1. Ask for help, post proposals or solutions.
lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 09.10.2013, 15:11

Hi!

Win7 X64 Hungarian
Delphi2010
Zeos 7.1.0 beta

Using Dos852 codepage in varchar field.
With zeos, there are strange characters.
Image
Image

with ibtable, got the correct characters.
Image

I tried utf8, ISOxxxx clientcodepage, but it's not working.

How can I set zconnection to work with this codepage?

Thanks
Balázs

User avatar
EgonHugeist
Zeos Dev Team
Zeos Dev Team
Posts: 1739
Joined: 31.03.2011, 22:38

Re: Zeos 7.1.0 problem with hungarian characters

Postby EgonHugeist » 09.10.2013, 21:05

Not sure what wrong. I need some more Information.

Which default characterset is used for the DB 'em selves? Is it UTf8,NONE... ? This makes significant differences.

Maybe my Codepage setting is wrong? ZPlainFirebirdDriver.pas line 604:

Self.AddCodePage('DOS850', CS_DOS850, ceAnsi, zCP_DOS850); {Latin I (no Euro symbol)} check the constant. is 852 the right CP for windows? I thought so: http://msdn.microsoft.com/en-us/library/cc195066.aspx
King regards, Michael

You want to help? viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image

lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Re: Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 10.10.2013, 10:34

Created the DB with these properties.
Image
Image

Sample database
http://www.proforcom.hu/zeos/proba.fdb

The difference between dos850 and dos852 for example:
ô and ő
Using the ő character.

On delphi7 with zeos 7.0.3 everything is OK.

Regards
Balázs

miab3
Zeos Test Team
Zeos Test Team
Posts: 950
Joined: 11.05.2012, 12:32
Location: Poland

Re: Zeos 7.1.0 problem with hungarian characters

Postby miab3 » 10.10.2013, 11:26

@lbalu

And as you leave ClientCodepage empty?

Michal

lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Re: Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 10.10.2013, 13:38

Same thing with empty clientcodepage.
Displaying wrong characters.

miab3
Zeos Test Team
Zeos Test Team
Posts: 950
Joined: 11.05.2012, 12:32
Location: Poland

Re: Zeos 7.1.0 problem with hungarian characters

Postby miab3 » 10.10.2013, 14:20

@lbalu

Could you attach a SQL-script or database with this table?

Michal

lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Re: Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 10.10.2013, 14:46

Sample database
http://www.proforcom.hu/zeos/proba.fdb

Contain a table, called 'partner', created with ibexpert.

User avatar
EgonHugeist
Zeos Dev Team
Zeos Dev Team
Posts: 1739
Joined: 31.03.2011, 22:38

Re: Zeos 7.1.0 problem with hungarian characters

Postby EgonHugeist » 10.10.2013, 16:05

@Ilbalu

issue confirmed. Did check your example. I can't say why but it seems the Windows-internal ansi to wide mapping doesn't work properly for this codepage. OR???

I could resolve this issue by appling this patch: ZPlainFirebirdSriver.pas, line 589.

Resplace this line with
Self.AddCodePage('DOS852', CS_DOS852, ceAnsi, {$IFDEF MSWINDOWS}zCP_L2_ISO_8859_2{$ELSE}zCP_DOS852{$ENDIF}); {Latin II}

Than it should work. I'm a bit surpriced since this CP should be supported like you could see in my prevous post.

Patch done R2805 \testing-7.1

Also is firebird not able to convert cp 852 corectly to UTF8 (which i can't believe because it works for all CP's except charset 'NONE')OR IBTable did write wrong values(my imression). I can't say whats wrong now. If another user has newly issues with this codepage than IBTable is the trouble maker.

Did you try to write strings and read them back with Zeos only? Can you check this please before you replace my proposed patch?
King regards, Michael

You want to help? viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image

lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Re: Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 10.10.2013, 20:04

First.
tried without the patch.
Now, this is good with zeos, but wrong characters if I look in ibexpert.
Example:
In zeos: Árvíztűrő
In ibexpert: *rv*zt*r*

Second.
Applied the patch, and it is working perfectly.

Thank you.

Problem solved.

User avatar
EgonHugeist
Zeos Dev Team
Zeos Dev Team
Posts: 1739
Joined: 31.03.2011, 22:38

Re: Zeos 7.1.0 problem with hungarian characters

Postby EgonHugeist » 10.10.2013, 20:25

Good news.

The issue isn't resolved 100% for ME.

I internaly do only deal with codepages. Before the patch i used Word(852) as the codepage for CharacterSet DOS852.

Now you write you can successfully read and write the non ASCII chars with Zeos. For me it is not interesting what IBExpert is doing here. I internaly call directly the mapping functions of windows. IBExpert is using a class AFAIK.
And AFAIK does DOS852 not support the € symbol which should be possible now, but this is wrong. See: http://msdn.microsoft.com/en-us/library/cc195066.aspx
Also have i headage with the failing FB string conversion to UTF8. Uge surprice for me.

So can you use another external viewer to compare the strings without the patch? Just to confirm the patch is right or IBTable is wrong. That would be nice to KNOW this patch is right before finaly release the 7.1.2-stable candidate.
King regards, Michael

You want to help? viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image

miab3
Zeos Test Team
Zeos Test Team
Posts: 950
Joined: 11.05.2012, 12:32
Location: Poland

Re: Zeos 7.1.0 problem with hungarian characters

Postby miab3 » 12.10.2013, 09:53

Michael,

Sample database:
http://www.proforcom.hu/zeos/proba.fdb
have set Character Set DOS852 (database not table or fields).
It looks like the ZEOS did not provide this setting to table char fields.

Michal

User avatar
EgonHugeist
Zeos Dev Team
Zeos Dev Team
Posts: 1739
Joined: 31.03.2011, 22:38

Re: Zeos 7.1.0 problem with hungarian characters

Postby EgonHugeist » 12.10.2013, 11:20

Michal

nope it did check this and Zeos does it axactly. I had problems to test the changes by my selves, since i can't type in hungarian chars with my german Keybord. Also copying from the forum did fail because of a different encoding.

As i wrote i can't say if this patch is right or wrong. So i simply wrote a comment and if someone else has problems than we'll restart the discussion again. Hope user Ibalu does a reply and support me a bit like i did it for him...
King regards, Michael

You want to help? viewtopic.php?f=4&t=3671
You found a (possible) bug? Use the new bugtracker dude! http://sourceforge.net/p/zeoslib/tickets/

Image

lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Re: Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 18.10.2013, 12:37

I'm back.
Sorry this late, but working hard.

Removed the patch from Zplainfirebirddriver, the problem is returned in zeos.
1,Downloaded EMS sql manager, opened the sample database.
Image

2,Append new record in ems
Image

3, Back to delphi, and in zeos:
Image

4,Saved the patch again, reopened the delphi project:
Image
and everything is good.

Regards
Balázs

marsupilami
Gold Boarder
Gold Boarder
Posts: 671
Joined: 17.01.2011, 14:17

Re: Zeos 7.1.0 problem with hungarian characters

Postby marsupilami » 18.10.2013, 13:56

Hello Michael,

I am not sure, wether this will help you, but I did some tests with the example database. I used Flamerobin and retrieved the data. Once using UTF8 as the connection character set (Conversion ANSI->Unicode done by Firebird) and once using DOS852 as the connection character set (Conversion ANSI->Unicode done by Flame Robin). The SQL in both cases was:

SELECT cast(NEV as varchar(50) character set octets), nev FROM PARTNER

which in Flame Robin displays the cast as a string of hex characters. So - these are the results:

Code: Select all

        String in Hex                String
DOS852: 4bf576e172692052656ee17461   Kõvári Renáta
  UTF8: 4bf576e172692052656ee17461   K§vßri Renßta


In both cases the Character at the second position is F5. It seems that Flamerobin and firebird do different conversions when they get data declared to be in DOS852:
Firebird treats the data as MS-DOS CP 852, which has the § sign at position F5. (http://en.wikipedia.org/wiki/Code_page_852)
Flame Robin treats the data as ISO 8859-2 or WIN1250, which does have the õ at position F5. (http://en.wikipedia.org/wiki/Windows-1250)
Given this, I think Zeos works correctly without the patch, because in firebird terms the correct character set for F5 to become an õ is ISO8859_2 or WIN1250.

@Ibalu: Your data seems to be encoded in WIN250 (which is used in hungarian Windows for ANSI strings) and not in DOS852 - which was used by MS DOS and only gets used in the windows CMD by now. Could you try to create your database using WIN1252 as the default character set?

lbalu
Fresh Boarder
Fresh Boarder
Posts: 7
Joined: 12.08.2013, 15:00

Re: Zeos 7.1.0 problem with hungarian characters

Postby lbalu » 18.10.2013, 14:55

A new database with cp win1250 is ok (I think this codepage contains ő and ű chars).
Good with zeos and with Ibexpert.

But I've got some dos852 DBs, and on delphi 7 with zeos 7.0.3 there aren't weird characters, so I will use the patch for Delphi 2010.


Return to “ZeosLib 7.1 stable Forum”

Who is online

Users browsing this forum: No registered users and 1 guest