Monday, August 11, 2008

Re: [HACKERS] proposal: UTF8 to_ascii function

Hello

2008/8/11 Andrew Dunstan <andrew@dunslane.net>:
>
>
> Pavel Stehule wrote:
>>
>> Hello,
>>
>> combination functions to_ascii and convert_to is broken now. Problem
>> is in convert_to function. It doesn't support 8bit output encoding.
>>
>> Current workaround:
>>
>> CREATE FUNCTION to_ascii(bytea, name)
>> RETURNS text AS 'to_ascii_encname' LANGUAGE internal;
>>
>> SELECT to_ascii(convert_to('Příliš žlutý kůň', 'latin2'),'latin2');
>>
>> I don't expect column collate for 8.4, so we need to have workable
>> to_ascii function.
>>
>> I propose function to_ascii(text, name) that internally convert text
>> from utf8 encoding when it's necessary.cheers
>>
>>
>>
>
> convert_to is not broken. It returns a bytea, and it is up to you to
> de-escape it if you get the text representation.
>

ok, I talked about combination convert_to and to_ascii. to_ascii
doesn't support bytea, what is probably correct. We cannot use
descape, because it remove 8bit. This issue was noticed more times -
http://archives.postgresql.org/pgsql-general/2008-06/msg00495.php


> We are surely not going to go back to a situation where we have functions
> returning text in any encoding other than the database encoding. That
> becomes a vehicle for storing wrongly encoded data in the database, and we
> have just gone through the exercise of plugging those holes. I privately
> predicted when we did this work that it might motivate people who had been
> abusing convert_to to get proper support for multiple encodings done. That
> is the right way to go, not re-opening holes we have just very deliberately
> plugged.
>

to_ascii isn't related to multiple encodings. And actually there is
only one man who works on it. We will be happy for database collation
in 8.4. So without any change this feature will be broken more than
two years.

Regards
Pavel

>
>
> cheers
>
> andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

No comments: