Web lists-archives.com

Separating in-source translations from */messages into own */srcmessages (or similar)




Hi,

PROBLEM: garbage po and mo files in packages

currently many of the scripts used for creating release tarballs of KDE 
software accidentally also add those po files to the tarballs whose 
translations are already fed back directly into the sources by scripty (with 
data files like appdata, desktop, json, or mimetype.xml). So strings which 
will never be loaded from an external mo or qm catalog.

Those po files then also end up in the binary packages as mo files given the 
generic handling of po files during builds. Do this to see how much of those 
unneeded mo files you might have yourself on your system:

du -ch `find /usr/share/locale \
    -name "*appdata.mo" -o \
    -name "*mimetypes.mo" -o \
    -name "desktop_*.mo" -o \
    -name "json_*.mo"`

which for me reports in total 43M.

find ... | wc -l

is 186 here.

Problem is that many people are not aware of all the different data file types 
which scripty cares for, and i18n in general is more seen as "magic which just 
works (mostly)" (well, shows the great work of the KDE i18n team :) ). And the 
number of such file types is rising now and then, just all the different 
release scripts ("official" and home-grown) are not keeping up.


PROPOSED SOLUTION

To keep things simple, what about separating pot/po files used with strings 
fed back directly into the sources in a separate folder system? Similar to how 
po/pot files with manual/doc strings are hold in a separate docmessages/ 
folder?

So the structure in svn would be
l10n-kf5/{templates,af,ar,...}/messages
l10n-kf5/{templates,af,ar,...}/docmessages
l10n-kf5/{templates,af,ar,...}/srcmessages < new one

This should help to make it more clear which po files should end up as mo (or 
qm) files and which po files are just used for allowing translators to 
translate strings directly in data files.

And it would allow release scripts to not be too clever about which po files 
to get from svn and which not.

Unsure about this:
What better name could there be than "srcmessages"?
Can l10n-kde3/l10n-kde4/l10n-support also be handled the same way, without 
breaking things?
Did I miss anything and those po/mo files should be part of source/binary 
packages?

I assume moving those po/pot file into a separate folder "only" needs moving 
of the files, adaption of scripty and translators making sure their files in 
their checkout of svn are properly moved else well, given any uncommited 
changes. And noone else should be affected, right?

Please comment also in general.


INTERMEDIATE FIXING OF RELEASE SCRIPTS

In the meantime authors of release scripts perhaps should be inspired by the 
approach taken by the script preparing KF5 tarballs, which greps all 
Messages.sh files in the sources for "*.pot" to guess about the names of po 
files that should be added to the tarball. Because this will not include pot 
files only created by scripty on the i18n server. Thus might be a good guess 
(unless someone wrote Messages.sh with too much magic inside).

See git repo kde:sysadmin/release-tools, branch frameworks/5.0, file 
make_rc_tag.sh or
https://quickgit.kde.org/?p=sysadmin%2Frelease-tools.git&a=blob&h=452db8ae4f18931771997d406d63687f9053eed3&hb=f2b88c76b687f55a4123c074d859d0f7b2d2e557&f=make_rc_tag.sh

(Using the term "guess" hints all this is still a fragile approach and perhaps 
could see a better approach, but please let's spare that discussion for 
another upcoming email of mine :) and here concentrate on this one proposal 
for the given problem of garbage po/mo files).

Cheers
Friedrich