Translations from Pootle pushed to git (for 3.4 beta2), bugs to fix

timar · April 19, 2011, 7:13am

Hi,

Sunday night I exported updated po files from Pootle and pushed them to
git to the translations module. LibreOffice 3.4 beta 2 will contain
these translation updates.

Please note, however, that there are still bugs in your translations
that need to be fixed ASAP.

Our friends at OOo project developed a new script that checks for
duplicated translations. There are parts of the code where translations
should be unique. For example in Calc function names, SEARCH and FIND
must not be translated into the same term. These translation errors lead
to mysterious bugs. For more explanation see
http://openoffice.org/bugzilla/show_bug.cgi?id=112478

So now there are 3 checks in place for translations:
gsicheck - standard check for tags etc.
readmecheck - same as gsicheck but for the readme
doublecheck - check if different English strings were translated into
the same at critical parts of the code

Locales with bugs:
as, ast, be, bg, bn, bo, br, brx, bs, ca-XV, ca, cy, dgo, el, en-ZA, eo,
et, fa, fr, gl, gu, he, hi, hr, id, it, ja, ka, kk, kn, kok, ks, ku, lt,
lv, mai, mk, ml, mn, mni, mr, my, nb, ne, nn, nr, nso, oc, om, or, pl,
pt-BR, pt, ro, ru, rw, sa-IN, sat, sd, sh, sk, sl, sq, sr, st, sv, ta,
te, tg, tn, tr, ts, ug, uk, uz, vi

Log files (from all tools) and error files (from gsicheck tool):
http://ftp.fsf.hu/LibreOffice/3.4-beta2-l10nbugs/

Pootle also contains checks for many potential translation bugs. There
are many false positives but it finds many real bugs. Please go through
them, especially 'variables' and 'escapes', because these can be fatal
(Review tab).

Best regards,
Andras

Dwayne_Bailey · April 19, 2011, 8:18am

So now there are 3 checks in place for translations:
gsicheck - standard check for tags etc.
readmecheck - same as gsicheck but for the readme
doublecheck - check if different English strings were translated into
the same at critical parts of the code

In an ideal world I'd love those tools to inject the found errors back into Pootle so that you have one place to find and fix errors and no cryptic strings we need to search for. If anyone would want to try code that please give me a shout and we can work out what needs to be done.

Pootle also contains checks for many potential translation bugs. There
are many false positives but it finds many real bugs. Please go through
them, especially 'variables' and 'escapes', because these can be fatal
(Review tab).

We need your help with these false positives:
1) If a test makes no sense in your language, e.g. startcaps in languages with no capitals, then please report that to the Pootle developers as we can switch these off for your language.
2) Please report false positives that affect your language and that you think could/should be fixed in the code.

Please also try to mark your false positives. This allows you to reduce your error count to zero and will then show real errors clearly. You can do that by clicking on the cross next to the check when you are in review mode.

We haven't had much testing of what happens to false positives when the files are upgraded, you would help us test and find bugs here.

Olivier_Hallot · April 19, 2011, 10:56am

Hello Andras

I finished pt-BR yesterday, so one last piece of my 3.4 help files will miss 3.4.... sigh.... will get it on RC's...

About the check indicated, and looking into other translations, pt-BR has just one (ONE !!!) little false positive. So either I am a genius and extremely carefull guy at my work, or the script failed to pick other errors, as it did for other languages...

Can you double check?

http://ftp.fsf.hu/LibreOffice/3.4-beta2-l10nbugs/pt-BR.log

Thank you
Olivier

timar · April 19, 2011, 12:21pm

Hello Olivier,

About the check indicated, and looking into other translations, pt-BR has
just one (ONE !!!) little false positive. So either I am a genius and
extremely carefull guy at my work, or the script failed to pick other
errors, as it did for other languages...

Can you double check?

http://ftp.fsf.hu/LibreOffice/3.4-beta2-l10nbugs/pt-BR.log

This is not a false positive. You need to translate Heading and Title
to different words, otherwise you'll have troubles with default styles
in the localized product. See for example
http://openoffice.org/bugzilla/show_bug.cgi?id=113296

Anyway, congratulations for your almost error-free translation.

Best regards,
Andras

timar · April 19, 2011, 1:06pm

Hello Dwayne,

2011.04.19. 10:17 keltezéssel, Dwayne Bailey írta:

We need your help with these false positives:
1) If a test makes no sense in your language, e.g. startcaps in
languages with no capitals, then please report that to the Pootle
developers as we can switch these off for your language.
2) Please report false positives that affect your language and that you
think could/should be fixed in the code.

For example:
'XX' cannot be replaced. - „XX” nem cserélhető le.

This Hungarian translation is perfectly valid but it fails
'singlequoting', 'doublequoting', and 'startpunc' check. There are
hundreds of string like this. There might be real errors among them, but...

Anyway, I'll report if a check be dropped for my language.

Please also try to mark your false positives. This allows you to reduce
your error count to zero and will then show real errors clearly. You
can do that by clicking on the cross next to the check when you are in
review mode.

Thanks for the hint, I did not know this. Unfortunately it will take
a while to click through several thousand strings.

We haven't had much testing of what happens to false positives when the
files are upgraded, you would help us test and find bugs here.

OK, maybe I should test it on a smaller test module...

Fixing all failing Pootle checks would definitely increase the quality
of translation. However, some teams will not find time to fix them all.
The bugs I reported in my previous mail are severe bugs which can cause
loss of functionality so I wish that everybody corrects at least those.

Best regards,
Andras

Anton_Meixome · April 19, 2011, 1:10pm

Hi,

Why some languages has a .err and .log files? other as galician not.
Is for different checks?

I have arrange it at galician.

@Olivier, about problem with styles. We also had this problem (in
spanish translation, there is too). I send a link, in hope give a
hand.

http://blog.openoffice.gl/coordinacion/article/un-problema-cos-estilos

Otherwise, In galician translation, rest now a lot of job for
coherence as the imported trasnlation from OOo.org (from versión 3.2)
is deficient and ignores hundreds improvements we was doing.

Antón

timar · April 19, 2011, 1:26pm

2011.04.19. 15:09 keltezéssel, Anton Meixome írta:

Hi,

Why some languages has a .err and .log files? other as galician not.
Is for different checks?

.err files come from gsicheck.
The other two checks (readmecheck and doublecheck) do not produce .err
files.

Best regards,
Andras

sophi · April 19, 2011, 2:01pm

Hi Andras,

Hi,

Sunday night I exported updated po files from Pootle and pushed them to
git to the translations module. LibreOffice 3.4 beta 2 will contain
these translation updates.

Please note, however, that there are still bugs in your translations
that need to be fixed ASAP.

I didn't finished the translation yet, I still need to proof read some files, so I didn't make any check for the moment, as I will still upload files on Pootle.
When do you want us to have the files corrected?

Kind regards
Sophie

Dwayne_Bailey · April 19, 2011, 2:04pm

Hello Dwayne,

2011.04.19. 10:17 keltezéssel, Dwayne Bailey írta:

We need your help with these false positives:
1) If a test makes no sense in your language, e.g. startcaps in
languages with no capitals, then please report that to the Pootle
developers as we can switch these off for your language.
2) Please report false positives that affect your language and that you
think could/should be fixed in the code.

For example:
'XX' cannot be replaced. - „XX” nem cserélhető le.

This Hungarian translation is perfectly valid but it fails
'singlequoting', 'doublequoting', and 'startpunc' check. There are
hundreds of string like this. There might be real errors among them, but...

We already do this kind of adaptation for French guillemets so we could probably easily adapt it for Hungarian. So yes that is the type of false positive I'm interested in hearing about and fixing.

If you can give more examples of those strings so that I can build tests that would be helpful,

Anyway, I'll report if a check be dropped for my language.

Please also try to mark your false positives. This allows you to reduce
your error count to zero and will then show real errors clearly. You
can do that by clicking on the cross next to the check when you are in
review mode.

Thanks for the hint, I did not know this. Unfortunately it will take
a while to click through several thousand strings.

Yes, maybe focus on ones that are less error prone then the quotes.

We haven't had much testing of what happens to false positives when the
files are upgraded, you would help us test and find bugs here.

OK, maybe I should test it on a smaller test module...

Fixing all failing Pootle checks would definitely increase the quality
of translation. However, some teams will not find time to fix them all.
The bugs I reported in my previous mail are severe bugs which can cause
loss of functionality so I wish that everybody corrects at least those.

Yes, those few should be a priority and should be at zero.

timar · April 19, 2011, 2:10pm

Hi Sophie,

I didn't finished the translation yet, I still need to proof read some
files, so I didn't make any check for the moment, as I will still upload
files on Pootle.
When do you want us to have the files corrected?

I'll update translations in git every week, just before tagging
betas/rc's/etc. There is no final deadline, really. The sooner you fix
them, the sooner you'll have an error free localized build.

I'll send the updated error list every week to this mailing list.

Thanks,
Andras

tabe · April 19, 2011, 3:00pm

Hi Andras,

Our friends at OOo project developed a new script that checks for
duplicated translations. There are parts of the code where translations
should be unique. For example in Calc function names, SEARCH and FIND
must not be translated into the same term. These translation errors lead
to mysterious bugs. For more explanation see
http://openoffice.org/bugzilla/show_bug.cgi?id=112478

(snip)

Log files (from all tools) and error files (from gsicheck tool):
http://ftp.fsf.hu/LibreOffice/3.4-beta2-l10nbugs/

Thanks for sharing logs, in which Japanese (ja) locale had only one bug
reported. I admit it is *not* false positive and fixed now.
Good tools

Cheers,
-- Takeshi Abe

sophi · April 19, 2011, 3:11pm

Hi Andras,

Hi Sophie,

I didn't finished the translation yet, I still need to proof read some
files, so I didn't make any check for the moment, as I will still upload
files on Pootle.
When do you want us to have the files corrected?

I'll update translations in git every week, just before tagging
betas/rc's/etc. There is no final deadline, really. The sooner you fix
them, the sooner you'll have an error free localized build.

I'll send the updated error list every week to this mailing list.

Ok, thanks a lot Andras.

Kind regards
Sophie

Olivier_Hallot · April 19, 2011, 3:31pm

Hi

Indeed I missed this since the very begining of OpenOffice 1.0, back in 2002....

Makes a lot of sense and thanks Anton for the precise analysis of this error.

Corrected.

Regards

Olivier

filmsi · April 19, 2011, 9:35pm

This same string is also a problem for Slovenian, but I cannot make up
another word if both English words are localized into the same Slovenian
word.
I don't care what your script says but for Slovenian this is not an error -
it would be an error if those strings had a different translation.

Lp, m.

Christian_Lohmaier · April 19, 2011, 10:08pm

Hi Martin,

This is not a false positive. You need to translate Heading and Title
to different words, [...]

This same string is also a problem for Slovenian, but I cannot make up
another word if both English words are localized into the same Slovenian
word.
I don't care what your script says but for Slovenian this is not an error -

That's a very strange attitude. So you're telling Slovenian users to
screw themselves because the translator cannot come up with a way to
use two different strings?

it would be an error if those strings had a different translation.

I really doubt the Slovenian language doesn't have something that
distinguishes a main title, from a simple heading, but be it.

If you lack a word, then just add another. Make Title "Main Heading"
or something similar.

ciao
Christian

Andrea_Pescetti · April 19, 2011, 10:27pm

Martin Srebotnjak wrote:

> This is not a false positive. You need to translate Heading and Title
> to different words, otherwise you'll have troubles with default styles
... I don't care what your script says but for Slovenian this is not an error -
it would be an error if those strings had a different translation.

OK, but using the same word (which, by mistake, we did in Italian too)
will break styles, as already explained, and anyway lead to unforeseen
behaviour, since the code seems to assume that style names are all
distinct. A very similar issue was fixed as a blocker for OpenOffice.org
3.3.0 Italian
http://openoffice.org/bugzilla/show_bug.cgi?id=115232
and indeed, as Christian suggests in your case too, we decided to use a
periphrasis.

Regards,
Andrea.

Christian_Lohmaier · April 20, 2011, 12:15am

Hi Sophie, Andras, *,

Hi Andras,

Sunday night I exported updated po files from Pootle and pushed them to
git to the translations module. LibreOffice 3.4 beta 2 will contain
these translation updates.

Please note, however, that there are still bugs in your translations
that need to be fixed ASAP.

Yes, as some of then even cause the build to break.
http://tinderbox.libreoffice.org/cgi-bin/gunzip.cgi?tree=libreoffice-3-4&brief-log=1303228818.4722#err44

I didn't finished the translation yet, I still need to proof read some
files, so I didn't make any check for the moment, as I will still upload
files on Pootle.
When do you want us to have the files corrected?

ASAP, so that the build can continue and maybe reveil breakers in
other languages

French (fr/text/shared/00/00000200.xhp) and Welsh/Cymraeg
(mediawiki/help/cy/com.sun.wiki-publisher/wiki.xhp) are causing the
break, but that might just be because they are first to break and thus
others won't even be tried.

So please fix those "Expected Symbol: Close tag for <tagname>" ones as
fast as possible, and have Andras upload them, otherwise chances are
that your language will just be skipped.
(as beta2 is planned for this week-end, you really need to hurry, or
either beta2 will not come at all or your language will probably be
skipped (or reverted to the old translationi) - I'm speculating on
this, as I am not involved in doing the release builds, but those are
the only options I could think of, and my guess is that keeping on
schedule is more important than to have the current status of all
translations)

ciao
Christian

Christian_Lohmaier · April 20, 2011, 1:20am

Hi *,

[broken translations that break the build]
So please fix those "Expected Symbol: Close tag for <tagname>" ones as
fast as possible, and have Andras upload them, otherwise chances are
that your language will just be skipped.

For your convenience, I wrote a awk one-liner to turn the error-log
into clickable URLs that will take you to the translation unit in
question:

all in one single line - example for the fr.err log:

awk -F'\t' '$10!="en-US" { n=split($2, fp, /\\/); up=""; for (i=3;
i<n; ++i)( up=up "/" fp[i] ); if ($6 == "") su = ""; else su = "."
$6; print "https://translations.documentfoundation.org/" $10
"/libo34x_" $4 up ".po/translate/?sfields=locations&search=" $5 su}'
< fr.err

the one-liner explained:

awk -F'\t'
tab is field seperator (I used awk, but of course you could do the
same with perl, sed,....)

'$10!="en-US"
we're not interested in the english original strings when creating the
URL, as the original string is displayed in pootle anyway (and the
10th field in the error-log is the language tag)

{ n=split($2, fp, /\\/); up=""; for (i=3; i<n; ++i)( up=up "/" fp[i] );
assemble the path of the file, i.e. turn
"source\text\schart\01\04050100.xhp" into "/schart/01" (.po is appended later)

if ($6 == "") su = ""; else su = "." $6;
when there is a "sub-unit" (no idea how else you call it - i.e. when
there is par_id<whatever>.<subunit>, also use it.

print "https://translations.documentfoundation.org/" $10 "/libo34x_"
$4 up ".po/translate/?sfields=locations&search=" $5 su}'

assemble & print the URL
https://translations.documentfoundation.org/<languagecode>/<project>/<filepath>/translate/?sfields=location&search=<paragraph-id>[<subunit>]

< fr.err

read the file "fr.err" as input

ciao
Christian

Christian_Lohmaier · April 20, 2011, 1:36am

Hi Dwayne, *,

So now there are 3 checks in place for translations:
gsicheck - standard check for tags etc.

This should be integrated into pootle somehow - and errors in there
should "block" the exporting (syncing to on-disk-files) to avoid
checking in invalid files (that then break the build)

readmecheck - same as gsicheck but for the readme
doublecheck - check if different English strings were translated into
the same at critical parts of the code

In an ideal world I'd love those tools to inject the found errors back into
Pootle so that you have one place to find and fix errors and no cryptic
strings we need to search for.

Yes, indeed. But pootle should be more strict than gsicheck (but of
course it should cover everything that gsicheck finds) While pootle
already catches some of the errors, it doesn't flag this one for
example:

https://translations.documentfoundation.org/fr/libo34x_help/shared/00.po/translate/?sfields=locations&search=par_id3149902.25

(in case sophie is too fast and fixes it in the meantime:
<variable id="FehlendesElement">Dans une fenêtre de fichier de base
de données, cliquez sur l'icône <emph>Requêtes</emph>, puis choisissez
<emph>Éditer - Éditer</emp>. Lorsque les champs de référence
n'existent plus, vous voyez cette boîte de dialogue</variable>

i.e.
<variable>
<emph></emph>
<emph></emp> !!!! non-matching - missing "h" - pootle doesn't flag it
</variable>

It's always better to have some false positives than to miss stuff.

And may I suggest to change the theme to give a bigger font for the
translation-box? I guess that's one of the reasons of missing quotes
or writing an *opening* tag as </tag> and not noticing it.

But even better would be a "commit-hook" that instead of letting you
advance to the next translation unit points you back at the
just-changed string with a message "You very likely introduce an error
here - you mistyped the tag's name or something"

ciao
Christian

sophi · April 20, 2011, 3:34am

Hi Christian,

Hi *,

[broken translations that break the build]
So please fix those "Expected Symbol: Close tag for<tagname>" ones as
fast as possible, and have Andras upload them, otherwise chances are
that your language will just be skipped.

For your convenience, I wrote a awk one-liner to turn the error-log
into clickable URLs that will take you to the translation unit in
question:

thanks a lot, it's very kind but my issue here is more about my own organization than fixing the errors. I'm translating off line using wordorge of gedit, so I need to get my local files and Pootle files in sync.
The process po2sdf -> gsicheck -> correcting po then upload on pootle is quite long and I tend to make it only when I'm over with the most important part of localization.
It seems I don't have no more the time to wait for the end of my translation to gischeck the files, so I need to organize myself differently.

But thanks a lot again really, that will help me to correct the files quickly later today.

Kind regards
Sophie