File corruption on OSX and LO 4252

users
#1

I encountered a most infuriating problem today when working with LO on
OSX - file corruption when writing to an external disk drive.

As I'm often having to work with different machines, and different OSes,
I have an external FAT formatted LaCie hard drive to store my files on.
This works great on Linux, Windows and OSX.

Until LO 4252, I had never encountered any file corruption issues,
absent power cuts and NFS shares :wink: and I have been using StarOffice,
then OpenOffice and now LO since StarOffice first became available for
OSes other than Windows - that just goes to show how long my teeth are
with regard to this product.

However, in the space of a fortnight, LO 4252 has successfully trashed
two Writer documents that I was working on, irreparably, once about a
fortnight ago, and once today. The first time it happened, I thought it
was a one-off, and I actually hadn't been working on the file for long
when it happened.

Today, I lost two hours work, and working to a very tight deadline
thanks to a client's late order. Needless to say, I was/am not impressed.

The file can not be opened in any of the following :

OOo 3.2.1
OOo 4.0.0
LO 3.6.7
LO 4.1.3
LO 4.2.5

The file can not be unzipped or repaired with zip, and yet it is stated
as weighing in at 73 kb (of binary rubbish).

I love LO, and the whole open source idea, but the product needs a
serious quality rethink if things like this can happen. I have had no
such issues on Linux, which just goes to show that the developers, as a
whole, give little heed to the changes they make in the code and the
effect it can have on other OSes (Windows, Mac) - as they would say,
"how are we supposed to know whether change x or y will have an impact
on a closed source OS ?" - more attention is given to the Windows arena
because that is where the perceived markets are for the taking. This
leaves the little people like Joe Bloggs / Jane Doe, etc, here, to deal
with the issues on OSX, of which there are many, and too few developers
willing or interested to care about fixing them.

Had I stuck to MS Word (yes, I have Office 2011 for Mac), this would not
have happened, of that I am certain. Had I stuck to 4.1.x, this would
not have happened either, but no that is now obsolete...instead we have
a 4.2.5.x as "stable" with deficiencies like the one I encountered, and
as a businessman, I begin to seriously start questioning the
benefit/risk assessment of the whole LO thing.

This is just a word of warning to others, as they say "your mileage may
vary".

Alex

#2

Hi Tom,

Hi :slight_smile:
Surely just go back to the last back-up or find a copy in emails or on
a usb-stick. Any chance of it being in a temp folder or auto-backup
or something?

It is corrupt in all of the places where a "backup/autorecovery" should be.

Re-read my message and you will see that I was working from a USB
external hard disk. There is currently no way to get a real-time system
automatic backup with rollback to an external media that works
seamlessly on 3 different OSes (TimeMachine on OSX, Windows autosystem
checkpoint/restore, Linux ???)

Btw do all the different machines you use run the same version of MS
Office as each other?

Even with different versions of Office, I've never had complete garbage
written to an office file. How does your question relate to the problem
exactly ?

Alex

#3

Note, that this problem is specific to LO 4252 on OSX, as mentioned in
my original post.

We are talking about a specific version of LO, on a specific OS, writing
to its native format to an externally connected USB disk. Unless people
have had the same experience with other versions on the same OS, or
other versions on different OSes, then comments like "are your versions
of Office the same ?" are irrelevant and unhelpful.

Alex

#4

Alex Thurgood wrote

... file corruption when writing to an external disk drive. ... I have an
external FAT formatted LaCie hard drive to store my files on. ... I was
working from a USB external hard disk.

I would first try to reproduce the error without a USB external storage
device being a factor. I cannot recommend working directly from a USB
device. Ever. The risk of failure (in all contexts) seems to go up
considerably when doing so and I myself have suffered from this (although
not with LO). Not an answer I know, and you have my full sympathy and
respect Alex, just a possible avenue for further testing.

I am not discounting an LO problem as your description sounds similar to
this bug: https://bugs.freedesktop.org/show_bug.cgi?id=78260 The example
file does open in Writer (displays garbage), but cannot be opened by an
archive manager.

Best wishes, Owen.

#5

Hi Owen,

I would first try to reproduce the error without a USB external storage
device being a factor. I cannot recommend working directly from a USB
device. Ever. The risk of failure (in all contexts) seems to go up
considerably when doing so and I myself have suffered from this (although
not with LO). Not an answer I know, and you have my full sympathy and
respect Alex, just a possible avenue for further testing.

I was also always sceptical of reading/writing directly from/to external
storage, but this issue had never occurred in any of the preceding 3.x
or 4.1 versions of LO on OSX (or in any other version of
StarOffice/OOo/LO/AOO/NeoOffice that I have had). So, I was naturally
caught "unawares" as they say. After many hundreds of hours of using
previous iterations of LO with external storage, one certainly does not
expect it to behave in a contrary fashion in an allegedly "stable" release.

I can confirm that the problem only occurs on OSX when writing to
external storage. And never occurred with 4.1.x, so this is new to 4.2.5
at least. I can't however state with accuracy that it didn't occur with
4.2.3 (which I also have), but seeing as I was using that before, I'm
pretty (although not absolutely) certain that I didn't encounter the
problem there either.

I am not discounting an LO problem as your description sounds similar to
this bug: https://bugs.freedesktop.org/show_bug.cgi?id=78260 The example
file does open in Writer (displays garbage), but cannot be opened by an
archive manager.

It is certainly similar, to the extent, that the file is corrupted,
however, I can not even open it in any version of LO, OOo, or NeoOffice.
I had someone test the file on Linux last night, and they too were
unable to either parse it as a zip file or open it in any version of LO
that they had.

Thanks for your input Owen.

Alex

#6

​This is not really an answer regarding LO, but a general observation on
using external storage that might be pertinent in your case.

I use external USB drives to store most of my media files (pictures and
videos, with some audio files). That imply a lot of files, with relatively
large size. In this setup, I never had a problem with file corruption.
Except for that one time, which happened exactly as I was doing a backup
copy of the whole drive. Every single files where corrupted. When the copy
ended, all files where there. They all had the correct size, but all of
them where invalid. Some video played for a second, then stopped
encountering invalid content, pictures where completely unrecoverable, you
get the idea.
After that, I tried to find the culprit: it was not the OS (windows 7), as
other drives, and these two drives at other time, worked perfectly fine. It
was not the copy software either (teracopy at the time). The USB drives
themselves didn't report any error, and worked fine afterward (they are
still fine today). Turns out, a faulty USB hub can cause silent copy error:
the software, whatever it is, will see the write operation succeed, when
the content on the drive is garbage.

Long story short: a "passive" USB hub silently corrupted ~1TB of data.

Now, let's go back to your issue. The code for writting files (in
LibreOffice or any other software) is the least difficult part to port from
system to system. There is litteraly nothing to do, as various libraries
(including the standard C library) provide the necessary functions. For
most piece of software, there is also no difference at all in writing to an
internal hard-drive, or to an external one.

I'm not saying that the issue doesn't exist, mind you. Bugs exists, and
weird interactions can always happen. But since the "file writting" part is
probably the simplest thing in the whole program, it would be very
interesting to check if LO is really the culprit. The fact that it only
happened those two times with LO is not conclusive: if it is indeed caused
by external causes, it could have gone unnoticed if, for example, you
weren't manipulating other files on the drive at the time.

#7

Hi :slight_smile:
Surely just go back to the last back-up or find a copy in emails or on a
usb-stick. Any chance of it being in a temp folder or auto-backup or
something?

Btw do all the different machines you use run the same version of MS Office
as each other?
Regards from
Tom :slight_smile:

#8

I am working with OSX 10.9.4

I just created a test writer document in 4.2.4.2, saved it to a FAT-32
fomatted memory stick, opened it in 4.2.5.2, made some modifications and
re-saved it. I then opened it in 4.2.5.2. It opened flawlessly. I don't
have a FAT-32 USB hard drive to test it on, but since LO only sees the
drive through the OS, I don't see what difference it would make.

Just another data point.

#9

How long did you keep the file open for, and how many times did you make
changes to file ?

If you take an existing file from your FAT stick, then "Save As.." with
another name to the FAT stick, do you get any issues ?

If you make a copy of the file from Finder on the Fat stick, then open
that copy, make changes and then resave, do you get any issues ?

How big is your memory stick ?

Just trying to narrow down the problem.

Alex

#10

Owen Genat wrote:

Alex Thurgood wrote

... file corruption when writing to an external disk drive. ... I have an
external FAT formatted LaCie hard drive to store my files on. ... I was
working from a USB external hard disk.

First question which occurs to me - Have you had problems with other drives, or only this one? It could be that the disk is failing, so it would be a good idea to back it up ASAP (if you don't already) just in case. Maybe even stop using that disk until you've tracked down the problem - it might stop happening with a different disk (pointing to a faulty disk) or it might turn out to be something else, but either way it'd be better to be safe! As they say, there are two types of disk - those which have failed, and those which will fail ;o)

I would first try to reproduce the error without a USB external storage
device being a factor. I cannot recommend working directly from a USB
device. Ever. The risk of failure (in all contexts) seems to go up
considerably when doing so and I myself have suffered from this (although
not with LO). Not an answer I know, and you have my full sympathy and
respect Alex, just a possible avenue for further testing.

To add to that, I've found using USB disks through a hub to be rather unreliable at least on the one Windows laptop I regularly use with a hub. It may just be my hardware or some combination of things in my setup, but they'll unexpectedly disconnect (and sometimes reconnect almost immediately), particularly while transferring large amounts of data (e.g. while backing up onto a USB disk). Connecting the disk directly to a USB port on the laptop, without the hub, is usually fine. I've never tracked down whether it's the laptop hardware, hub, drivers (either for host controller or hub) or something else, but have experienced this with a couple of different hubs, several different disks and even my scanner sometimes plays up through the hub while being rock solid on a direct port. Not saying it's necessarily the problem here, but something like that happening at the wrong moment during a write could corrupt a file.

I am not discounting an LO problem as your description sounds similar to
this bug: https://bugs.freedesktop.org/show_bug.cgi?id=78260 The example
file does open in Writer (displays garbage),

If the file is corrupted and Writer can't work out the format, it interprets as plain text. The garbage seen is the raw data from the file, seemingly with hashes for non-printable characters. In the case of the file on that bug report, it looks like there is some other garbage (possibly fragment of a different file) at the beginning (the first "PK" is where the file should start), and data is missing from the end of the file. Looks a crash (or disk disconnect or failure) in the middle of writing the file has messed up the file system entries.

but cannot be opened by an
archive manager.

Because it's not a valid ZIP file.

Best wishes, Owen.

Mark.