How to handle regressions

Hi :slight_smile:
Aww heck that'd be brilliant! If TDF could take Thunderbird under their
wing instead of leaving it with Mozilla.

Mozilla don't seem to appreciate just how many people rely on Thunderbird.
It's the best OpenSource email client around, in the opinion of a huge
percentage of people apparently.

People, and articles have often said that LibreOffice could never compete
with MSO because it lacks an email client. Completely missing the point
that so many great email clients integrate so well with LibreOffice. Maybe
if there was a kinda default one and if that was as great as Thunderbird is
then it'd cut a lot of detractors off at the knees.

Sorry! I do agree with your main points!
Regards from
Tom :slight_smile:

I really do understand but I think you underestimate the complexity of a
project with more than 10,000,000 lines of code. That being said, I have
no additional feedback and we're not going to be forcing anything on
volunteers so . . . we can agree to disagree and know that this is how
the project works-take it or leave it.

Best,
Joel

Fix confirmed (had some spare time this morning)...

It was also mentioned in the bug that it would be back-ported to 4.3 (I
guess not for 4.2, but thats ok)...

Hi,

So beside of what was written I want to explain something. I do not know if I say it marketing-wise... But still...

Q: Why do not get bugs fixed at the moment they are reported?

Hi :slight_smile:
The 4.4.0 is due out fairly soon, within a month or so.

It might be good to test-drive the alpha and beta release on 1 or 2
machines because they are generally stable enough for your own usage.
Obviously don't roll-out until after 4.4.1 at the earliest if your users
are likely to get stumped by the slightest issue. Many of us wait until
much later in the cycle but tbh even the beta-releases are stable. It's a
good idea to test-drive releases, in your own daily work, from as early as
possible so that you can see if anything obvious is broken.

Regards from
Tom :slight_smile:

It might be good to test-drive the alpha and beta release on 1 or 2
machines because they are generally stable enough for your own usage.

I'm doing that now with this release - will change to a beta once it is
available.

Obviously don't roll-out until after 4.4.1 at the earliest

For everyone else I always wait until at least the .3 or .4 release

It's a good idea to test-drive releases, in your own daily work, from
as early as possible so that you can see if anything obvious is
broken.

I know, but time is my biggest problem, and the fact is, this is the
first time that I can remember ever having to roll back after beginning
an upgrade rollout.

Also, as I mentioned before, it is simply not possible to test every
single feature - and reading the release notes is all fine and good -
and I actually do (contrary to what someone seemed to be implying
earlier), but the release notes is one big long page, and for 4.2, the
note about the massively major (for those who use them) change from
pop-up to inline editing of Input Fields consisted of one, single
solitary line easily missed among everything else.

Hence my comment that major changes like this should always be
pre-announced on the users list (since, you know, it will kind of affect
them).

Anyway, I've got it setup now so I can easily install master builds
if/when I need to test something.

Q: Why do not get bugs fixed at the moment they are reported?
A: Before devs see the bug, it goes through the hands of QA. We are
a small team and have a lot of backlog on reported bugs. (More bugs are
reported than we are able to tackle)

All well and good, but has nothing to do with what started the flamefest.

1. It took a few days in the bug to get it confirmed.

2. 3 *months* later, Joel asked a question making it very plain he did
not bother to even read the bug report and subsequent comments

3. 1 month later, Jan-Marek posted a patch (thanks Jan-Marek!)

4. 1 month later the bug reporter asked (reasonably politely) if the fix
would be back-ported to 4.3.

5. The same day Joel responded with a rude "Feel free to submit a
patch...' and more like it.

So, it was confirmed very quickly. At that point, the dev who cause it
should have been contacted to fix it, and that same dev should have
taken ono the responsibility of pushing the fix to at least 4.3, if not
also 4.2 (had it been fixed soon after being confirmed, it would have
made more sense to push it to 4.2).

Q: Why the hack did you tell someone to get some money in its hand to get it fixed?
A: I hear such voices very often and got the same answer: Either
help (so that others does not have to wait so long) or pay. Or continue
hoping, that a dev will tackle your pet bug in its free time

And yet again this comment totally ignores the fact that we are talking
about newly contributed code (not some old/pre-existing/legacy bug), and
that software developers should take ownership of their code.

Q: I have a problem XYZ for months but I did not want to report a bug....
A: For us it is like the bug did not exist (ans answer to the next
question)

I have no problem reporting bugs, and have multiple times, so n/a...

Q: I will argue for an hour, but won't check something gets confirmed in a few moments
A: Everyone knowing how this is done will help you, if you ask...

Apparently some of this is a language thing, so I'll give you some room.

But I would seriously ask that you address your apparent unwillingness
to differentiate between:

a) new bugs that are introduced with new code from existing/current
developers, and that these devs should 'own their code' - meaning be
willing to fix bugs when they are confirmed *and* push the changes to
the releases without resorting to responses suggesting users fix it
themselves and/or pay someone else to do it,

and

b) and old/pre-existing/legacy bugs that were introduced by someone long
gone...

I ask again - are you in disagreement with the above? If so, please, by
all means, attempt to explain how you can logically and rationally be in
opposition to this principle.

Also, I'm confused...

Jan-Marek in the bug comment on August 17th - well before the 'Hard code
freeze' on September 1st for 4.3.2 (released on Sept 22nd - said that
the patch would show up in the daily builds after that.

So, I'm not complaining, I'm just asking - can someone who understands
the dev process explain why this fix did not make it into 4.3.2?

Thanks

I have no clue how my comment was rude, it's pretty standard in open
source world to say that "patches welcome" or some variation.

I'm unsubscribing though as I don't have the time nor energy to continue
this and you seem to have more than enough of both to go around. Enjoy
ranting on the thread.

Best,
Joel

Hi, answer inline,

Liebe Grüße, / Yours,
Florian Reisinger

a) new bugs that are introduced with new code from existing/current
developers, and that these devs should 'own their code' - meaning be
willing to fix bugs when they are confirmed *and* push the changes to
the releases without resorting to responses suggesting users fix it
themselves and/or pay someone else to do it,

and

b) and old/pre-existing/legacy bugs that were introduced by someone long
gone...

I ask again - are you in disagreement with the above? If so, please, by
all means, attempt to explain how you can logically and rationally be in
opposition to this principle.

Bugs are bugs. Bugs are not meant to be introduced. For developers and end users it one not matter, if the bug is in new code or not. Even if it seems to be in new code, how do you know? A bug corrected here can affect something "on the other side" of the codebase, which was fixed around to correct the wrong behavior. So, "CORRECT" might not always be the old state. It might be the new, with some crazy side effects.
(Taking your example: I did not touch how to paste, how could I break it. [BTW: I thought I was too dumb to write into the new input field])

By all of this, why should dev A introduced a regression yesterday be more responsible then dev B which broke something 10 years ago? Both are basically the same, so why treat them different.

Regressions are always bad, but neither dev A nor dev B wanted to break this...

If you want the fix NOW (or backport, for which it is too late for 4.2 I guess) it might be wise to get in touch with someone paid (better words then "you should invest into the fix" or hope someone will do that for free :slight_smile: [Backporting is boring and risky]

Did that help?

Bugs are bugs. Bugs are not meant to be introduced.

I agree... your point?

For developers and end users it one not matter, if the bug is in new
code or not.

True enough.

Even if it seems to be in new code, how do you know?

Maybe one clue is that it is a brand new feature that the developer created?

A bug corrected here can affect something "on the other side" of the
codebase, which was fixed around to correct the wrong behavior.

Irrelevant.

If a developer introduces a new inline editing *feature*, and it is
discovered after the fact that something about this new feature is
broken, it is on the developer who built it to fix it... PERIOD...
regardless of if the broken code was introduced by them or is the result
of some related code suffering from bit-rot somewhere else. It is the
nature of software development (I happen to know enough about it to not
be easily bamboozled by gobbledyspeak).

Anything else would result in a finger-pointing madhouse.

By all of this, why should dev A introduced a regression yesterday be
more responsible then dev B which broke something 10 years ago?

It is a simple matter of logic.

If dev B is still around, then dev A should ping dev B to fix their code.

If dev B is long gone, then dev A bears the burden of either finding and
fixing dev B's code, or reverting their new code with the bug
unless/until someone else comes along and fixes the bug.

Both are basically the same, so why treat them different.

You only treat them different if they are (ie, if one is no longer around).

Regressions are always bad, but neither dev A nor dev B wanted to
break this...

<sigh> and again, no one said they did...

If you want the fix NOW (or backport, for which it is too late for
4.2 I guess) it might be wise to get in touch with someone paid (better
words then "you should invest into the fix" or hope someone will do that
for free :slight_smile: [Backporting is boring and risky]

Did that help?

Yep. You have made it plain that you don't think software developers are
or should be held accountable in any way, shape or form for their code
and/or for the parts of the code they choose to work on.

Which means you are someone I would *never* choose to pay to fix a bug,
because you would never sign the agreement that I would put in front of
you requiring you to actually warrant the code you write, and fix any
bugs (at no cost to me), regardless of when they are discovered (within
reason - say one full release cycle).

Hi :slight_smile:
That all seems reasonable.

It also seems like a good idea to regularly market the QA team in the Users
List. New people with a variety of skills join this Mailing List all the
time and many are looking for a way of contributing back to the project.
It'd be kinda rude not to invite them in!

I think more than 1/week might be tooo often and maybe even considered spam
by some but 1/month might not be often enough to catch people before they
wander off.

The main difference with your post this time was that there wasn't even a
hint of refusing to solve a person's bug unless they paid for it. Quite
the opposite!

Talking with people is quite an art and not one that many of us really
expect from devs, especially when they have been deep into coding - nor
from QA who may have been deep into fast filing - nor from documenters or
translators who may have been deep into their equivalent of code.

However, it's not unreasonable to expect marketeers and leaders to be able
to talk to people without causing offence. Also, leaders kinda need to be
able to solve conflicts, instead of creating them! At least within their
own organisation and their own customers/users!

Regards from
Tom :slight_smile:

@Charles, *,

Tanstaafl wrote

Also, I'm confused...

Jan-Marek in the bug comment on August 17th - well before the 'Hard code
freeze' on September 1st for 4.3.2 (released on Sept 22nd - said that
the patch would show up in the daily builds after that.

So, I'm not complaining, I'm just asking - can someone who understands
the dev process explain why this fix did not make it into 4.3.2?

No worries, it can get a bit confusing about what commits are where and
when. His "would show up in the daily builds" refereed only to master
branch.

To be clear, in the normal flow of things, majority of development is made
by commits onto the master branch. That includes new features, or UX/UI
tweaks or even patches to repair or revert regressions. The in-line field
edits have been available for testing in master since the commit in August.

So where it gets confusing is the master's relationship with prior releases.
At present we have the 4.2 branch nearing its final bug-fix release and then
EOL a month later, and the 4.3 branch with its third bug fix in the works.

While commits to master can and do break things, in order to keep the code
stable, patches committed to master require additional review and
deliberation before being back-ported to a prior branch. As needed some
work will be done directly on the older branches--but most is integrated
into master first and ideally are fully tested there.

Some times if dealing with an obvious regression with simple correction, the
developer will put up the back-port for review immediately, but usually just
one release back. At times the issue has technical dependencies and requires
review and validation, and only then will the back-port be posted and again
reviewed before it is committed.

But occasionally the dev will simply overlook a good opportunity to resolve
an issue on branches other than master--which is understandable as most
developers are forward looking and framing the work to tackle the next
feature or issue in queue.

And that is where the user and QA community comes in, we have to stay up
with the issues on the forum and in BZ and occasionally make comment regards
opportunity that might otherwise be missed.

For this issue--despite being on the 4.2 Most Annoying Bug list--once
patched the back-port to 4.2 or to 4.3 for the in-line field editing was not
pursued. In fact it was only proposed yesterday, meaning it missed 4.3.2
testing and build cycle.

But when committed to the branch should make the 4.3.3 bug-fix cut. And,
technically, it could still make the 4.2.7 final release, but that requires
an exceptional approval by three devs or the Engineering Steering Committee.

Jan-Marek G.'s proposed backport to 4.3 branch for the in-line field edits
are up for code review, each of these is a separate facet of the patch
needed for 4.3 (and possibly 4.2) as already implemented in master since
August.
https://gerrit.libreoffice.org/11779
https://gerrit.libreoffice.org/11780
https://gerrit.libreoffice.org/11781
https://gerrit.libreoffice.org/11782

If there are no technical issues, since the patch put into master in August
was sound they'll likely be approved and committed in time for the next 4.3
release--but available for testing in context on "nightly" builds of the 4.3
branch once committed, i.e. they should be checked by users and QA for
continued proper function.

Hope that was clear enough for all.

Stuart

Than you *very* much for the excellent and detailed explanation.

That clears up a lot of things for me about the process.

Follow-up...

In the bug UI, is there an easy way to get a list of bugs that have been
patched/fixed in master, that a user like me could browse, and pick
certain ones that I could then test and if confirmed fixed, could
request be back-ported (if they haven't been already)?

I would be very happy to participate in this manner, but honestly, the
bug UI is a bit daunting, so anything that makes this easier for non
devs would be very welcome for technically inclined non-devs like yours
truly.

Thanks again for taking the time to explain this so clearly.

Please can we move on... Charles S. (aka Tanstaafl) was given
instructions and has agreed to do what needs to be done and review the
corrected function for his use case with a current build of master
(4.4.0alpha0+)--and respond in the fdo#76565 BZ issue regards the UX
regression.

Every thing else is just hubris or a poor understanding of the project's
timed release development flow. Please let it go--most users on this ML
have NO interest in this thread other than the drama. -- Stuart

Actually, this type of discussion points to a process issue. I imagine it
is one of the reasons for having this site.

It's obvious the functionality list of Libreoffice is becomming
overwhelming. The issue of lost functionality upon which user's have come
to rely combined with the unavailability of previous versions on the
Libreoffice website affects LibreOffice across the board. One can struggle
with the question for the motive for this.

Regardless of the outcome of that discussion, *there is a potential HUGE
conflict of interest looming on the horizon in the form of creating bugs so
that outside companies can charge $$$ to fix them. *

QUESTION:
Does there exist a set of LibreOffice TEST documents that are designed to
test ALL OF THE FUNCTIONALITY of a given version of LibreOffice?
Specifically, one that can automate test, perhaps through macros or self
contained programs, all the functionality of Libreoffice?

SUGGESTION #3: If such a set of documents or test harness does NOT EXIST.
Perhaps the QA team could itemize all the functionality of a version of
Libreoffice and structure a project for contribution by end users to
contribute the tests. i.e: A test suite that each useer could download and
then run and/or contribute more.

SUGGESTION #4: If a test harness is too complicated, could the QA team make
an itemized list of all the functionality of LibreOffice, inherited and
otherwise *and create some sort of method to crowd source the testing in a
manner designed to capture any lost functionality*? i.e.: a project
website for each version LibreOffice with the 100,000 things that
LibreOffice can do where the users can test each one and report the test?

NOTE: This is different than just putting a new version out and seeing who
gets hurt. Remember, not all people live in countries where employers have
to make a case to the government for firing a person. Yes, there are plenty
of disclaimers on the software, but the current process may end up making
LibreOffice the sofware you use only for NON-CRITICAL documents. The
long-term user base effects should be obvious.

Regards,

Hi :slight_smile:
Just to deal with just 1 of those points ...

There is an archive of older releases so people can download out-of-date
versions if they want to.

It's not widely publicised (to avoid confusing people) but when people ask
this mailing list, and presumably other forums too, someone does inevitably
point them in the right direction. It's also possible to find out from the
archives or by searching the site.

Regards from
Tom :slight_smile:

Hello Tom,

This archive that you speak of hosts unsupported and unpatched versions that may have major vulnerabilities in them.

At no point in time do we recommend the use of this repository.

Thanks,

Charles.

It may be useful for testing purposes though.
(Although many QA volunteers nowadays work with bibisect
  https://wiki.documentfoundation.org/Bibisect )

Cheers,
Cor

It's obvious the functionality list of Libreoffice is becomming overwhelming.

I wouldn't be surprised if maintaining the functionality list ended before the days of OOo 1.1.3.
There was functionality broken in 1.1.4 that neither QA nor the developers knew existed in 1.1.2.
(I'm not talking only about things like the functionality in OOo 1.1.3-ZA that was not present in any other version of OOo 1.1.x. )

Now wondering if that game has been removed from LibO.
I've forgotten how it was accessed. :frowning:

Regardless of the outcome of that discussion, *there is a potential HUGE conflict of interest looming on the horizon in the form of creating bugs so that outside companies can charge $$$ to fix them. *

As a potential issue, it exists only as long as organizations and individuals are mislead about FLOSS.
The biggest misconception being that FLOSS is gratis.

QUESTION:
Does there exist a set of LibreOffice TEST documents that are designed to test ALL OF THE FUNCTIONALITY of a given version of LibreOffice?

No.

Specifically, one that can automate test, perhaps through macros or self contained programs, all the functionality of Libreoffice?

Theoretically that can be automated, but due to a11y issues with LibO, it is not currently possible to do so.

SUGGESTION #3: If such a set of documents or test harness does NOT EXIST.
Perhaps the QA team could itemize all the functionality of a version of Libreoffice and structure a project for contribution by end users to contribute the tests.

i.e: A test suite that each user could download and then run and/or contribute more.

In theory, the documents that accompany bug reports are:
* Used to verify that bug fixes do fix the reported issue;
* Construct test documents to be incorporated into QA;

SUGGESTION #4: If a test harness is too complicated,

Assuming the a11y issues are solved (which is required, if LibO is to be deployed in corporate environment in a legal jurisdiction that takes a11y legislation as something more significant than a bone for voters), the stumbling block is the number of platforms each item of functionality can be tested on.

Even basic testing requires 91 platforms. (12 for Windows, 5 for Mac OS X, 10 for BSD distros, 2 for OpenSolaris, 10 each for Ubuntu, Debian, Red Hat, Mint, Arch, Puppy, and one each for Kali,TAILS & DoD.)
If "exotic" platforms are included, then the number of testing platforms increases by at least two orders of magnitude, and maybe even three.

>could the QA team make an itemized list of all the functionality of LibreOffice, inherited and otherwise

Methinks that LibO is like WangWriter during its prime.
C/Support and Sales: "No, you can not construct spreadsheets using this product".
Experienced users: "Yes you can. Here is how to do it."
Wang developers: "That won't work."
Wang tech support: "It does work, regardless of what your specifications claim".

*and create some sort of method to crowd source the testing in a manner designed to capture any lost functionality*?

That is what QA _currently_ does, albeit without the list of all of LibO's current functionality.

It might be a bit too late to compile a complete list of functionality.
What is possible, is to construct a list, based upon:
* Reported regression bugs;
* Deliberately added new functionality;
* Known, existing functionality;
* New functionality added by accident, serendipity, or unintentionally, and discovered by a user;

i.e.: a project website for each version LibreOffice with the 100,000 things that LibreOffice can do where the users can test each one and report the test?

Depending upon how "things" is defined, that figure could be closer to 10,000,000,000 than 100,000.

jonathon

Hi :slight_smile:
+1
Also when users find functionality is broken in a newer release they often
revert to or remain with an older release.

If such users ask for advice from this mailing list it's often been the
advice that many here would give them. We would usually suggest they post
a bug-report and maybe getting involved with helping fix it.

I don't see what other choice they have.

Regards from
Tom :slight_smile: