Engaging users: initial results of the survey

Servus Micheal,

true, what you are saying, but one little thing:

[ given the vast disparity in time-to-file (1 minute) vs.
time-to-fix (5 man days avg) ]

...in correlation with 1 developer investing 5 days to fix it vs.
maybe hundreds, thousands or even hundreds of thousands of users
wasting for example 5 minutes per day due to the bug.

:smiley:

Cheers,
Stefan

Stefan Weigel wrote

[ given the vast disparity in time-to-file (1 minute) vs.
time-to-fix (5 man days avg) ]

...in correlation with 1 developer investing 5 days to fix it vs.
maybe hundreds, thousands or even hundreds of thousands of users
wasting for example 5 minutes per day due to the bug.

+1

Plus the hundreds of thousands that give up on LO *because* of the bug.

The logic that "a bug is not important because not many people report it"
has a big flaw: most people give up without bothering to report... (and in
the case of Bugzilla, reporting requires quite a lot of effort)

My suggestion? Change volunteer model.

In other volunteer based organizations (humanistic, animal protection, etc),
people agree to do work on tasks attributed to them even if that is not what
they feel like doing. The "do whatever you feel like" model doesn't work.

My 2 cents.

I think there's a dangerous perception here: The perception that the LO
developers work on nothing except what they want to work on.

I'm pretty sure that that is very false. They work on LO because they
want to work on LO, and they probably choose what to work on based on
at least the following:

a) How many people report a specific bug
b) How serious that bug is to people's productivity
c) How in line that bug is with current roadmaps
d) Regression bugs probably have higher priority
e) What parts of the system they know the best
f) How much time they have at the moment, and how big the bug is
g) How much fun they think it will be (or more likely, which bug will
be the least annoying to fix)

The logic that "a bug is not important because not many people report
it" has a big flaw: most people give up without bothering to
report... (and in the case of Bugzilla, reporting requires quite a
lot of effort)

This logic may not be true in an absolute sense, but consider that it's
not normally a question of how important the bug is on its own merits,
but how important the bug is compared to other bugs. If 50 people have
reported bug A and only 2 people have reported bug B, while bug B may
still be important, and there may be another 20 people who haven't
reported it, it is not as important as bug A, and there may be another
20 people who haven't reported bug A as well.

In other volunteer based organizations (humanistic, animal
protection, etc), people agree to do work on tasks attributed to them
even if that is not what they feel like doing. The "do whatever you
feel like" model doesn't work.

And I don't think anybody is suggesting that that model is the
predominant one here.

I have pointed out in the past that you cannot expect a developer to
work on your bug, because there is nothing forcing him to work on
anything but what he wants to, but that doesn't mean that is all he
does. It means you can't *force* him to work on what *you* want him to
work on. I'm sure the developers *do* give careful consideration to
what they work on, it just might not be what you feel they should work
on, but they've got a bigger picture than you.

Remember, we do have to keep the developers happy to some extent,
otherwise they leave. This is true even if they are getting paid (I've
left more than one company which was paying me, because I was unhappy
with some other aspect of the situation), and especially true if they
are not getting paid. Also, just because they don't prioritise the bugs
you think are important, doesn't mean they are cherry-picking just the
bugs they like, it probably just means they had other, more important
things to deal with.

Just something to keep in mind.

Paul

Paul-6 wrote

I think there's a dangerous perception here: The perception that the LO
developers work on nothing except what they want to work on.

I didn't mean to say that.
I'm aware that some developers work on whatever is needed and fix the most
urgent bugs/regressions.
But out of 300 developers, there must be people who can fix the "boring"
bugs and the "not important" bugs... Of course you would have to ask these
developers to start with bug #1 and fix it before moving to #2

Michael Meeks once wrote "Developers don't like to be told what to do". I'm
sure they don't. But if nobody does then there is no solution for bugs that
keep lingering...

Paul-6 wrote

I have pointed out in the past that you cannot expect a developer to
work on your bug, because there is nothing forcing him to work on
anything but what he wants to, but that doesn't mean that is all he
does. It means you can't *force* him to work on what *you* want him to
work on. I'm sure the developers *do* give careful consideration to
what they work on, it just might not be what you feel they should work
on, but they've got a bigger picture than you.

First, it's not *my* bug. The bug is the software. The software is not mine.
Second, many times I already have a solution for the problem. I only report
it so that the bug is fixed for the benefit of the community. I even report
bugs that don't affect me at all.
Third, "there is nothing forcing him to work on anything but what he wants
to" is exactly the problem IMO.

Paul-6 wrote

Remember, we do have to keep the developers happy to some extent,
otherwise they leave.

Yes, so do other people. But they are not so important, right?
If you can't tell developers what to do, some bugs will always be there
because they are boring to fix or because they are "not important".

I'm suggesting that a compromise based volunteer model is applied to all,
not just to developers. Then you might start to see a change and a real
community :wink:

Paul,

Just completing Pedro's answers inline...

Paul-6 wrote
> I think there's a dangerous perception here: The perception that
> the LO developers work on nothing except what they want to work on.

I didn't mean to say that.
I'm aware that some developers work on whatever is needed and fix the
most urgent bugs/regressions.
But out of 300 developers, there must be people who can fix the
"boring" bugs and the "not important" bugs... Of course you would
have to ask these developers to start with bug #1 and fix it before
moving to #2

Michael Meeks once wrote "Developers don't like to be told what to
do". I'm sure they don't. But if nobody does then there is no
solution for bugs that keep lingering...

and nobody says the system is perfect. :wink:
But to come back to Paul's objection, yes, developers work on what they
want to work on. Their motivation can be anything from a salary to some
dream they want or yet another thing that keeps them awake at night.
Somewhere in between I'm sure there's a reasonable guy . But "whatever
is needed" is prone to a wide range of interpretation.

Let me give you an example. While "your" bug (good point Pedro, by the
way) wasn't being fixed, some guy called Caolan McNamara, who wrote the
code of the word processing module back in the days of OpenOffice.org
took on the daunting task of rewriting the entire graphical system of
LibreOffice. And mind you, we're talking about over 6Million lines of
code for a suite like LibreOffice. Was it necessary? Hell yes. Was it a
high priority? Absolutely. Did he have the time to focus on the bug
you're mentioning? No.

But to him, this objective was of the highest importance and it was
*sorely* needed. I'm not saying the bug you reported wasn't important.
I'm saying that while you may be complaining, others are cheering.
Other bugs get fixed. See my point?

Paul-6 wrote
> I have pointed out in the past that you cannot expect a developer to
> work on your bug, because there is nothing forcing him to work on
> anything but what he wants to, but that doesn't mean that is all he
> does. It means you can't *force* him to work on what *you* want him
> to work on. I'm sure the developers *do* give careful consideration
> to what they work on, it just might not be what you feel they
> should work on, but they've got a bigger picture than you.

First, it's not *my* bug. The bug is the software. The software is
not mine. Second, many times I already have a solution for the
problem. I only report it so that the bug is fixed for the benefit of
the community. I even report bugs that don't affect me at all.

+1

Third, "there is nothing forcing him to work on anything but what he
wants to" is exactly the problem IMO.

And yet that's how most of the FOSS projects work. But then again, no
system is perfect.

Paul-6 wrote
> Remember, we do have to keep the developers happy to some extent,
> otherwise they leave.

Yes, so do other people. But they are not so important, right?
If you can't tell developers what to do, some bugs will always be
there because they are boring to fix or because they are "not
important".

I'm suggesting that a compromise based volunteer model is applied to
all, not just to developers. Then you might start to see a change and
a real community :wink:

Motivation is a hard thing to assess. Rather than reaching a
compromise in abstracto, I'd say that the compromise is found through
social engineering and everyone's motivation. Let's say that you are
reporting bugs on a regular basis. Some of these bugs are particularly
hairy ones, and it catches developers' attention. It's likely that
after two or three bug reports of that kind, developers, at least some
of them, might be paying attention.

Yet another way to look at it is that the number of volunteers
reporting the bug or making it an issue to tackle over the various
collaborative and communication channels we have around the project.
Basically, this is an invitation to contribute and get recognized. By
contributing, you get recognized, you get bonus points, and your
credibility grows. Mind you, it works the same way for developers. And
because of that, the fact that you, a known contributor points out that
there's a leftover bugfix that may even already have a solution has
more chances to get fixed.

Hope this helps,

CC'ing the website list because it is about the website, but I'm not subscribed, so please CC me on replies if you really want to discuss this complaint.

The problem of course is that there is no queue of bugs-to-fix. We try
to prioritize issues, so that we can see those that are seriously
debilitating and then try to fix those on a best-effort basis.

This prompted me to go file a bug (Feature Request) for something I've been meaning to file for some time now, then I couldn't remember if I'd already done it before, so I wanted to check and see...

Well... I must say, I am horrified by what I found.

The 'new' website is extraordinarily difficult to navigate if you want to do anything other than download the latest version.

My simple goal was to log into the Bug system, check 'My Bugs' and see if I'd reported this yet, and if not, report it...

1. after going to www.libreoffice.org, why do I have to first find and click on 'Main Website' to get to ... the main website? Why isn't www.libreoffice.org the main website?

2. After going to the main website, I clicked on 'Get Help', and then clicked on 'Bug'...

The only option here is to report a bug.

What if I don't want to report a bug, but only want to search for bugs?

Even after I log in, the only next step available is to continue reporting a bug I don't want/need to report!?

This is HORRIBLY BROKEN.

How do I just log into the bug reporting system and search it? Anyone?

1. The survey seems to be a Self seLected Opinion Poll (SLOP), so I'm
taking it with a grain of salt the size of the Sears Tower. There's no
margin of error included in the poll either and based upon the sample as
being from the mailing lists (where people are generally active anyway)
I'd say it's fairly skewed.
2. The conclusions are generic, wishy-washy and are based on guesses
and assumptions with no hard underlying data. How much in contributions
has LibreOffice raised? Does that fit in with what the survey said?
Where is the Quality Assurance in the web site? And why would an end
user be interested in that?
3. User support and quality assurance do not require too much time or
technical knowledge. Remind me not to hire you for either of those
tasks in my business. Those are things that professional companies hire
entire other companies to do.

I'd give this project an F in a freshman statistics class, and would not
base any strategy off of this "survey"

Thanks John, I'll take it from your comment that
1) you are either a survey professional and you only wait for the next
survey to contribute your time designing it

and/or

2) you will contribute the costs of hiring a market research firm the
next time we need a survey.

Allegedly, I and none of the other people who designed the survey are
professional survey designers.

Best,

For me I tried to navigate bugzilla to report continuing bugs where 2 formatting issues occur regularly:

1. Some cells in Calc have mixed size formatting in them or mixed fonts (parts are in Geneva 10 and parts in Geneva 12) I can go in by hand to each cell, select the piece that is the wrong size (12 point) change the font size and save the spreadsheet. At some point in the future (and no it is not consistently the next time) I re-open the file and those corrections are gone, the fonts are back to mixed size. It happens all the time but I can' get a small spreadsheet to demonstrate the problem. The one I am using that has the problem contains 15 years worth of sheep data and records and has a number of separate sheets in it. I've tried to copy out smaller sections and while I can get the small ones to have the mixed font issue when I change the font in them it seems to stay changed.

2. Alternatively I click on the upper left blank cell above the row and column number lines to select the entire spreadsheet. Change the font and or size . Cells with a single font and size in the text change as expected but any cells with mixed sizes do not.

Unfortunately bugzilla is worse than difficult to use and now it's even harder to find out how to do that.

Yet another way to look at it is that the number of volunteers
reporting the bug or making it an issue to tackle over the various
collaborative and communication channels we have around the project.

Eugenie (Oogie) McGuire
Desert Weyr http://www.desertweyr.com/
Paonia, CO USA

You made a survey without a survey statistician on your team. Did you
send out a request for such a person on the mailing lists to advise you
before you put together the survey? Did you have a clear and concise
question that you wanted to answer before you developed the survey
questions? Did you run the questions by an aforementioned professional
in the staff and check for confirmation bias?
I am not a professional statistician, and that's just what I spotted. I
have covered surveys as a journalist in my previous career, though. And
I also am a veteran of setting up business projects. A survey
statistician would have a lot more to say I am sure. And we're not even
starting on the analysis. In fact, I'd throw out the analysis and the
results and start anew. First off, define "users" (end users,
evangelists, business users?) and state the overall purpose of your
survey in a single question.
I regret some of the tone of the previous e-mail (first e-mail prior to
coffee), but there's nothing here to work with. You've got 300
self-selected users with at least two major questions in one survey that
you did not break out by region, sex, profession. You want results, you
need good data underneath.

You made a survey without a survey statistician on your team. Did you
send out a request for such a person on the mailing lists to advise
you before you put together the survey? Did you have a clear and
concise question that you wanted to answer before you developed the
survey questions? Did you run the questions by an aforementioned
professional in the staff and check for confirmation bias?

No. And apparently you have little awareness of how our project works.
But you make a couple of valid points.

I am not a professional statistician, and that's just what I
spotted. I have covered surveys as a journalist in my previous
career, though. And I also am a veteran of setting up business
projects. A survey statistician would have a lot more to say I am
sure. And we're not even starting on the analysis. In fact, I'd
throw out the analysis and the results and start anew. First off,
define "users" (end users, evangelists, business users?) and state
the overall purpose of your survey in a single question.
I regret some of the tone of the previous e-mail (first e-mail prior
to coffee), but there's nothing here to work with. You've got 300
self-selected users with at least two major questions in one survey
that you did not break out by region, sex, profession. You want
results, you need good data underneath.

You know, aside being rather inaccurate, you're welcome to run another
survey. We're always looking for more volunteers. And I'm glad to help
you on this, so please go ahead.

best,

Charles.

Okay, I point out problems and you're response is "you don't like it
you can run out your own survey" and then say I'm inaccurate without
stating why I'm inaccurate with a solicitation for donations in the
previous e-mail. Do you see the major issue here? Flies, honey, vinegar.

I don't know how your project works, but if you're not doing the proper
work beforehand I don't know how it can work. Ask anybody who's run any
successful project. Heck, even the leaders of failed projects can tell
you. They probably have more information.
First, you define your goals. Next you gather and prepare your
resources. You do a test run, maybe more than one and hope you have
enough time. You have people with specific knowledge critique and make
adjustments. Finally you run the project, and afterwards you analyze
and make improvements for the next time. Those principals apply whether
you're running a for profit project or a non profit.
And that would be the bare bones work if I was running a local project.
You're going global, which involves understanding cultural differences
as well. That is not the type of thing I would do with an ad hoc team
with nobody who has any experience in what I was doing in the first place.
Like I said, define the questions, gather the mailing list. And if you
don't have access to anybody with experience in statistics, don't launch
until you do. A badly done survey is worse than none at all.

John,

I'm well aware on how to run a project; and many comments and
critiques I have read so far are valid. Just keep in mind that we're
not going to run just another survey because according to some, it
wasn't granular enough (btw: there are others who would object to your
methodology as being too expensive to organize or as unncessary).
Running the survey again would end up confusing the users who already
answered.

If you seriously would like to get involved, you should - I mean it,
there's no sarcasm.

Best

Charles.

Charles,

  What's expensive about setting up a mailing list dedicated to the
survey and asking internally "what do we want to know about users and
how do we want to find out about it"? Or doing some requests for
volunteers from local university statistic students? Your volunteers
don't necessarily have to start from the community you are surveying
Professionalism is not about cost; it's about preparation. And I
wouldn't recommend another survey. At least, I wouldn't recommend it
yet. If you don't have money, you do have time. Take six months to a
year to get the right elements in place. In the meantime, use less
formal ways to explore what you want to know about users (feedback
forms, mailing lists, etc).

John,

Okay, that sounds different from what I initially understood. :slight_smile:
I thought you wanted to restart the survey. Anyway, thanks for the tip.
We'll take more care about phrasing what we want in the first place
next time. However, I suggest you read the archives of our marketing
list during the month of October to see our discussions.

Best,

Charles.

You didn't provide links to the bugs that you provided, so I really
can't tell if they're crashes every 4th time you start, "I don't like
where the mirror is placed", or something in between problems. And yes
there is a right way to tell the customer that the issue is either not
one that you get to at this moment or that the majority prefer the
solution solved another way (usually it involves a suggestion of an
alternative technique).
But even though you treat the customer as if they are always right, that
does not mean that they are always right. Some things the customer is
wrong on. Some things are out of budget for LO, and some things are
just preferred to be done another way. The choice has been put in your
court. You can use LO or you can find another office suite that meets
your needs depending upon the issue. The best thing about LO is that
you don't have to drop one to take on another. And not even Microsoft
or Adobe would drop everything and start changing code just because one
user wants a change. Any company that did that would soon find itself
bankrupt.

Hi Tanstaafl,

LibreOffice is using freedesktop.org infrastructure for bug
management software and code repository.

Ok, this doesn't help me any, but ok... :wink:

How do I just log into the bug reporting system and search it? Anyone?

This is a list of all LibreOffice bugs
<https://bugs.freedesktop.org/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=open&product=LibreOffice&content=>.

thanks, but I didn't ask for that...

You would eventually discover that when "Getting Involved"...

Nope... clicked that link, still couldn't find a 'Bugzilla Login Page' that I can simply log into (with my user account), then search bugs, check status of or work with bugs I've reported, etc...

Maybe we should rename "Get Help" -> "Bug" to "Report a bug"

That would help for that one issue, but I still don't have a clue how to log into the bug system and work with bugs...

Hi Charles

Tanstaafl wrote

That would help for that one issue, but I still don't have a clue how to
log into the bug system and work with bugs...

You are right that the current documentation won't take you there :slight_smile:

Go to
https://bugs.freedesktop.org/
and click on Log In (second item from the right on the top or bottom menu)

Hope this helps...

CC'ing the website list because it is about the website, but I'm not
subscribed, so please CC me on replies if you really want to discuss this
complaint.

I've also cc'd the QA team (who manage/work on the bugtracker,
including Rob Snelders, the lead dev for the Bug Submission Assistant
("BSA"))

This prompted me to go file a bug (Feature Request) for something I've been
meaning to file for some time now, then I couldn't remember if I'd already
done it before, so I wanted to check and see...
...
The 'new' website is extraordinarily difficult to navigate if you want to do
anything other than download the latest version.

My simple goal was to log into the Bug system, check 'My Bugs' and see if
I'd reported this yet, and if not, report it...

Fair enough. So you were expecting a link to the bugtracker either on
the main page or from the BSA?

(Rob - Thinking a bit broader, it would be nice if we could give users
a single page that would display information about their activity on
Bugzilla, the Ask site, etc. Using some form of SSO like OpenID or at
least shared credentials (e.g. LDAP) could make this easier)

2. After going to the main website, I clicked on 'Get Help', and then
clicked on 'Bug'...

The only option here is to report a bug.

What if I don't want to report a bug, but only want to search for bugs?

Even after I log in, the only next step available is to continue reporting a
bug I don't want/need to report!?

Rob - Any thoughts here?

This is HORRIBLY BROKEN.

How do I just log into the bug reporting system and search it? Anyone?

To log in, navigate to http://bugs.freedesktop.org/ and click on the
"Log In" link. To search the bugs, click on the "Search" link.

I agree that we should make it easier for users to find and use
Bugzilla. Tanstaafl -- we're actually right in the middle of some
backend infrastructure improvements for Bugzilla, but I'll make sure
that we spend some time on the frontend of things as well.

Best,
--R

Hi :slight_smile:
Something that often annoys me is that we seem to ignore all the work
of all the devs who are paid to work on LibreOffice.

I thought something like 10%-20% of devs are employed by various
companies, governments and other organisations? The advantage for the
organisations, companies and governments is that they still get the
suite far cheaper than they would pay in license fees PLUS they get to
choose what bugs their own devs focus on.

If you don't pay a dev then you still get to use a fantastic office
suite for free and a fairly tiny donation goes a loooong way to
improving it further. Of course we can grumble but i think it's
important to sit back and appreciate just what we do get too.

if we worked harder to encourage more companies to use LO and helped
them find ways to employ part-time or full-time devs with the savings
they would make on license fees then it might start to "snowball" even
more quickly.
Regards from
Tom :slight_smile: