Wrong translations of formula function names and identifiers [1/3]

erAck · October 16, 2019, 6:35pm

Hi,

I did a short investigation how some function name was translated and
stumbled upon quite some mess in general in the formula module.
Specifically, names in msgctxt "RID_STRLIST_FUNCTION_NAMES":

* Translated function names MUST NOT contain spaces, parentheses,
  hyphens, or anything else that would be an operator in spreadsheet
  formula context. Function names may only contain letters (of any
  alphabet or script of course, not just ASCII) or digits or '.' dot or
  '_' underscore, and must start with a letter.

  * Attached (split into two messages because otherwise rejected due to
    oversize) is l10n-wrong-function-names-1.txt and
    l10n-wrong-function-names-2.txt that hopefully caught all, created
    by the command line

grep -E -A1 'msgid "[A-Z0-9._]+"' translations/source/*/formula/messages.po|grep -B1 'msgstr ".*[-+*/ ()~]'

(all on one line) also repeated on top of the attached list.

* Translated identifiers like "#All" or "#Headers" if they start with
  a '#' hash character MUST start with a '#' character also in the
  translation, this is vital to recognize them as table reference
  identifiers (or error constants possibly).

  * Attached (in a third follow-up message because otherwise rejected
    due to oversize) is l10n-wrong-function-names-3.txt that hopefully
    caught all, created by the command line

grep -E -A1 'msgid "#' translations/source/*/formula/messages.po | grep -B1 'msgstr "[^#"]'

(all on one line) also repeated on top of the attached list.

While these wrong translations display correctly when loading
a document, any attempt to compile such formula expression that does not
meet the criteria will fail.

PLEASE, check the attached lists for your language's translation (the
list is sorted by language codes between translations/source/ and
/formula/messages.po) and correct errors ASAP; corrections should also
end up in the 6-3 branch, Cloph probably can help with that.

Also, (possibly after any change) please load the 6.2 attachment
https://bugs.documentfoundation.org/attachment.cgi?id=150855 of
https://bugs.documentfoundation.org/show_bug.cgi?id=93992 in your
translated UI, to check for duplicated function names.

Thanks
Eike

l10n-wrong-function-names-1.txt (26.5 KB)

erAck · October 16, 2019, 6:37pm

Part 2/3, see attachment.

l10n-wrong-function-names-2.txt (19.9 KB)

erAck · October 16, 2019, 6:39pm

Part 3/3, see attachment.

l10n-wrong-function-names-3.txt (19.9 KB)

Milos_Sramek · October 17, 2019, 7:35am

Hi Eike,
I did not get attachment in any of your three messages. Is this only my
problem?
thanks
Milos

sophi · October 17, 2019, 7:53am

Hi,

Hi Eike,
I did not get attachment in any of your three messages. Is this only my
problem?

Attachments are not allowed on the list,
@Eike maybe add them to an issue?
Cheers
Sophie

erAck · October 17, 2019, 10:49am

Hi,

Attachments are not allowed on the list,

This list configuration not even allowing text/plain attachments gets on
my nerves..

@Eike maybe add them to an issue?

Done: https://bugs.documentfoundation.org/show_bug.cgi?id=128200

Eike

sophi · October 17, 2019, 10:55am

Hi,

Hi,

Attachments are not allowed on the list,

This list configuration not even allowing text/plain attachments gets on
my nerves..

@Eike maybe add them to an issue?

Done: https://bugs.documentfoundation.org/show_bug.cgi?id=128200

Thanks a lot!

@all, please check your translations in the two files uploaded on the
issue. Your language code is after translations/source/ in the path.
Cheers
Sophie

Mihkel · October 17, 2019, 12:25pm

sophi (<sophi@libreoffice.org>), 17. oktoober 2019 kell 10:54:

Hi,
> Hi Eike,
> I did not get attachment in any of your three messages. Is this only my
> problem?

Attachments are not allowed on the list,
@Eike maybe add them to an issue?

Strange, I saw the attachments (all three). Luckily my language wasn't
affected.

Br,
Mihkel
Estonian team

sophi · October 17, 2019, 1:25pm

Hi,

sophi (<sophi@libreoffice.org>), 17. oktoober 2019 kell 10:54:

Hi,

Hi Eike,
I did not get attachment in any of your three messages. Is this only my
problem?

Attachments are not allowed on the list,
@Eike maybe add them to an issue?

Strange, I saw the attachments (all three). Luckily my language wasn't
affected.

I realized that it's also visible fro the archive here:
https://listarchives.libreoffice.org/global/l10n/msg12195.html

Cheers
Sophie

kolbjoern · October 17, 2019, 3:10pm

Thank you for this nice and helpful work.
Two errors in my language (nn). Corrected in Pootle. But I think these errors also occurs in master. How do I search for them in weblate?
All I get is "Not found".
Kolbjørn

Den 17.10.2019 15:25, skreiv sophi:

milupo · October 17, 2019, 4:07pm

Kolbjørn Stuestøl schrieb:

Thank you for this nice and helpful work.
Two errors in my language (nn). Corrected in Pootle. But I think these errors also occurs in master. How do I search for them in weblate?
All I get is "Not found".

Hi Kolbjørn,

use "Substring" as Search type in the search form. Use Source strings and choose Nynorsk only as search language to accelerate searching.

Michael

erAck · October 17, 2019, 5:29pm

Hi Kolbjørn,

But I think these errors also occurs in master.

Definitely. The grep output was from the git submodule 'translations' on
master.

Eike

erAck · October 17, 2019, 5:35pm

Hi,

> sophi (<sophi@libreoffice.org>), 17. oktoober 2019 kell 10:54:
>>> I did not get attachment in any of your three messages. Is this only my
>>> problem?
>> Attachments are not allowed on the list,
>> @Eike maybe add them to an issue?
> Strange, I saw the attachments (all three). Luckily my language wasn't
> affected.

I realized that it's also visible fro the archive here:
https://listarchives.libreoffice.org/global/l10n/msg12195.html

Odd.. however that might be possible..

Maybe something Guilhem (on Bcc) could investigate?

Eike

guilhem · October 17, 2019, 6:33pm

I see, my fix from yesterday was incomplete. While pymime no longer
choked on attachments that are not stripped away after reflowing the
message (such as text/plain), I forgot to update the Content-Type.

The problem was that your mail was multipart/signed, but the detached
OpenPGP signature was stripped away (and replaced with the list footer),
resulting in an invalid message which some MUA rightfully refused to
show properly.

Fixed, the former Content-Type of multipart messages is now overridden
with multipart/mixed when not all attachments are stripped away.

kolbjoern · October 17, 2019, 9:07pm

Den 17.10.2019 18:09, skreiv Michael Wolf:

Kolbjørn Stuestøl schrieb:

Thank you for this nice and helpful work.
Two errors in my language (nn). Corrected in Pootle. But I think these errors also occurs in master. How do I search for them in weblate?
All I get is "Not found".

Hi Kolbjørn,

use "Substring" as Search type in the search form. Use Source strings and choose Nynorsk only as search language to accelerate searching.

Thank you.
It works, although the field for choosing languages do not always turns up. I found the errors and corrected them.
Kolbjørn

milupo · October 17, 2019, 9:49pm

Kolbjørn Stuestøl schrieb:

Thank you.
It works, although the field for choosing languages do not always turns up. I found the errors and corrected them.

If you are in the components view for all languages, e. g. LibreOffice UI - master, and you go to the Search Menu, you will see the Search form without language select boxes.

If you are in the components view of a certain language, e. LibreOffice UI - master Norwegian Nynorsk, and you go to the Search menu, you will see the Search form with language select boxes.

Moreover, if you are in the components view for all languages, you will find a Search and replace feature in the Tools menu which is not available when you are in the components view of a certain language.

Michael

kolbjoern · October 18, 2019, 9:40am

Den 17.10.2019 23:51, skreiv Michael Wolf:

Kolbjørn Stuestøl schrieb:

Thank you.
It works, although the field for choosing languages do not always turns up. I found the errors and corrected them.

If you are in the components view for all languages, e. g. LibreOffice UI - master, and you go to the Search Menu, you will see the Search form without language select boxes.

If you are in the components view of a certain language, e. LibreOffice UI - master Norwegian Nynorsk, and you go to the Search menu, you will see the Search form with language select boxes.

Moreover, if you are in the components view for all languages, you will find a Search and replace feature in the Tools menu which is not available when you are in the components view of a certain language.

Michae

Ah. Thank you.
Kolbjørn

Christian_Lohmaier · October 18, 2019, 10:23am

Hi Eike,

I did a short investigation how some function name was translated and
stumbled upon quite some mess in general in the formula module.
Specifically, names in msgctxt "RID_STRLIST_FUNCTION_NAMES":

Thanks for that!

* Translated function names MUST NOT contain spaces, parentheses,
  hyphens, or anything else that would be an operator in spreadsheet
  formula context. Function names may only contain letters (of any
  alphabet or script of course, not just ASCII) or digits or '.' dot or
  '_' underscore, and must start with a letter.
[…]
    grep -E -A1 'msgid "[A-Z0-9._]+"' translations/source/*/formula/messages.po|grep -B1 'msgstr ".*[-+*/ ()~]'

    (all on one line) also repeated on top of the attached list.

Oh, if that ends up in actual translations repository (as opposed to
translations-weblate intermediate one on github), then the check for
function names in pocheck should be amended according.
Right now it checks for duplicates only it seems...
https://gerrit.libreoffice.org/plugins/gitiles/core/+/master/l10ntools/source/pocheck.cxx#136

(and likely should also be put into a custom check to weblate)

* Translated identifiers like "#All" or "#Headers" if they start with
  a '#' hash character MUST start with a '#' character also in the
  translation, this is vital to recognize them as table reference
  identifiers (or error constants possibly).

Thx - I think you already added the "L10n: preserve the leading '#'
hash character in translations." comment to all relevant strings?

grep -E -A1 'msgid "#' translations/source/*/formula/messages.po | grep -B1 'msgstr "[^#"]'

While these wrong translations display correctly when loading
a document, any attempt to compile such formula expression that does not
meet the criteria will fail.

PLEASE, check the attached lists for your language's translation (the
list is sorted by language codes between translations/source/ and
/formula/messages.po) and correct errors ASAP; corrections should also
end up in the 6-3 branch, Cloph probably can help with that.

As they break functionality, I can at least make sure that no invalid
translations are in the repo, pocheck removes problematic translations

Also, (possibly after any change) please load the 6.2 attachment
https://bugs.documentfoundation.org/attachment.cgi?id=150855 of
https://bugs.documentfoundation.org/show_bug.cgi?id=93992 in your
translated UI, to check for duplicated function names.

ciao
Christian

erAck · October 18, 2019, 11:05am

Hi Guilhem,

The problem was that your mail was multipart/signed, but the detached
OpenPGP signature was stripped away (and replaced with the list footer),
resulting in an invalid message which some MUA rightfully refused to
show properly.

Ah that makes perfectly sense.

Thanks!
Eike

Jean-Baptiste_Holcro · October 19, 2019, 8:17am

Hello,

to prevent bugs, you can probably use this Weblate feature to add checks directly in the translation platform:
https://docs.weblate.org/en/latest/user/checks.html#regular-expression

Or develop it using Python if this is more convenient.

If I understand it correctly your project organization, you will apply this rule on this component:
https://weblate.documentfoundation.org/projects/libo_ui-master/formulamessages/

Jean-Baptiste