New draft of the proposal for in-house developers

Hi all,

following the initial consultation in regards to the first draft I've included some of the recommendations and comments received.

You can find the new version here:

https://nextcloud.documentfoundation.org/s/d5fF4eCK4JHtpHj

The changes are in italic.

Some of the comment received included preferences for mentors instead of developers or to further outsource some tasks.
Those preferences will be covered by duly prepared proposals written by the proponents so they will not be covered in the document I shared.

Other comments received, also from fellow members of the board, stated that my proposal lacked of clear developers and project management procedures so I've added in page 10 what I see, at least initially, as the simplest approach but suggestions for improvements are very welcome.

Please do remind me if I forgot to include your constructive feedback!

Ciao

Paolo

Hi Paolo,

> following the initial consultation in regards to the first draft I've
> included some of the recommendations and comments received.
>
> You can find the new version here:
>
> https://nextcloud.documentfoundation.org/s/d5fF4eCK4JHtpHj
...
> Please do remind me if I forgot to include your constructive feedback!

     Thank you for the new version; I wrote previously:

> It's a bit sad it is a PDF, and without any heading numbering, so
> rather hard to interact with; an ODT that allowed comments / patching
> would be better.

     I'd really like an ODF version that can be easily improved.

     The concern around clarifying management and tasking of the proposed new staff is still there. I link my original comment which seems to still be unaddressed. Having ten people manage two is a problem as we know from previous boards.

     But let me perhaps juxtapose several statements from the document about how this would work - how do you reconcile these ?

: Document v1.5 has:
: "Our development mentor together with the team should propose
: to the BoD projects for internal development"

     So dev mentor + team propose to board for approval ?

     or:

: "The focus of the in-house developers will be set on specific
: areas suggested for them by TDF’s team in consultation with the
: ESC and, in case of unresolved conflicts, the board."

     TDF's team consult with ESC and only in case of conflicts consult the board ?

     or:

: "TDF’s team in collaboration with ESC and the BoD will evaluate
: that tickets and projects taken on by TDF’s developers can be
: considered neutral in terms of benefit that a single commercial
: contributor can derive from it and that do not overlap with
: current paid for projects run by commercial contributors."

     TDF's team collaborate with the ESC and the BoD to evaluate what projects are suitable.

     Simply because it is not tendered does not mean that the BoD should not have clear oversight of where donation money is spent. The BoD should have a clear and decisive role in an open process - based on advice (as I wrote before). And more as I wrote here:

https://www.mail-archive.com/board-discuss@documentfoundation.org/msg05477.html

     Other pieces surprise me by still being there eg.

: "Commercial contributors confirm that tenders issued by TDF
: form a negligible part of their income"
...
: "and the quantity of bugs, features and updates that may require
: tendering or paid for services by the commercial contributors is
: still so vast that it will not affect noticeably their income"

     As I wrote:

     While it is true that there may be an inexhaustible amount of work and TDF will never reduce that by doing some of it, I'm not sure that is really relevant.

     Why did you keep the controversial app-store piece ?

     I was really surprised to see Interoperability mentioned as an area here still - despite the (apparent) consensus on-list that this was not a huge issue.

> Other comments received, also from fellow members of the board, stated
> that my proposal lacked of clear developers and project management
> procedures so I've added in page 10 what I see, at least initially, as
> the simplest approach but suggestions for improvements are very
> welcome.

     The: "Developers management" and "Project management" sections are quite spartan - it would be good to have something concrete, they read a bit like a request for someone else to write this part for you. Unfortunately (see above) that is rather difficult without an editable version.

     Put in a nutshell, if TDF struggles to project manage tenders effectively (which is much easier), how do you think it will manage with the harder design / architectural / personnel-management / scheduling / prioritization / motivation and other problems associated with writing software to time & budget ?

     So - again, there are some good things here particularly avoiding wasteful and/or counter-productive overlaps with things that are getting done anyway.

     Unfortunately overall it seems not to have engaged with all of the feedback. We still lacks clear management, clear project-management, accountability to the community and so on in my view.

     Regards,

         Michael.

Hi Paolo,

thanks for the updated draft and integrating my references to meta bugs.

Another potential focus area might be Base (the database module).

Alex mentioned it in another thread (that had a different main focus) [1] and I've heard from time to time that it isn't in the best shape.

There's tdf#120062 [2] as a meta bug for database related bugs and enhancements.

I'm not using Base myself, though, and don't have any overview of its current status.
I've seen Julien doing some work there recently. Maybe he, Lionel or anybody else might be able to say more on whether it would make sense to consider that as a potential area to be worked on by TDF in-house developers.

Best regards,
Michael

[1] https://listarchives.documentfoundation.org/www/board-discuss/2022/msg00060.html
[2] https://bugs.documentfoundation.org/showdependencytree.cgi?id=120062&hide_resolved=1

[Something went wrong when copy-pasting Lionel's email address to "CC" in the previous email, I've forwarded it to him in private.]

Hi Julien,

thanks a lot for the input!

Michael

PS: I've fixed Lionel's email address now, something went wrong when I copy-pasted it into my previous email.

Hello,

Here are some thoughts about Base.

Some years ago, there was some decision to reduce our Java dependency (a very good thing).

Main point was to replace HSQLDB part by another database (there are good ones like MySQL/MariaDB and Postgresql) but which also allowed embedding, with a compatible license and with a not dead community, so Firebird was chosen.

In addition to Lionel (who is the Base expert for those who don't already know it), there have been 2 people who mainly worked on it: Andrzej J.R. Hunt and Tamas Bunth. The last one had even implemented a tool to migrate automatically from HSQLDB to Firebird.

Andrzej left Firebird part long time ago and Tamas left some years after him (just to be clear, I see no pb here, each one has his life/constraints/desire/whatever)

In parallel, Firebird has been put "in production" as by default embedded database and automatic migration set as by default. A lot of bugtrackers have been created and even if some part has been fixed, there were too much.

So I first put in experimental automatic migration part then Firebird by default + creation part (you can still open a Firebird embedded in non experimental).

Now we use HSQLDB 1.8 which is quite old and Firebird support is not ready, the pb is Lionel has far less availability and there's no one who replaced him. I gave a try to tackle some bugs but I'm not brainy enough to fix harder ones.

Firebird is not the only pb, charts aren't displayed anymore in reports and the whole reports part is dependent on old Java external components.

There are also address books pbs:

- Mac one  (eg : leaks but not only this, Alex may tell more about this I suppose)

- Thunderbird one can't be used anymore after Mork->Sqlite migration.

I also think about Base stumbling on some specific functions added to standard SQL by some databases which can be workaround sometimes but not always.

So yes, hiring 1 or 2 people on Base part could be relevant unless we'd like to abandon Base. Just to put it clearly here too, I'm not speaking for me since I already got a job and above all, wouldn't be able to do this job, I'm rather thinking about Lionel (if he agrees of course! :-)).

I really think a strong decision (hiring people or abandon it) should be made instead of letting it rot.

PS1: I'm adding Robert and Alex here since they're the main QA for Base part and may provide extra info.

PS2: Lionel, don't hesitate to complete (or correct if I made some mistakes) what I said.

Hi *,

So yes, hiring 1 or 2 people on Base part could be relevant unless we'd like to abandon Base.

Only one hint: I have created a database for a company in Germany (Firebird internal → will be moved to PostgreSQL server). Donation of 3000,- € special for Base has been transfered 2022-01-20.

This could be a little start.

There has been much done for Firebird in the last months. But Java dependency will be there with ReportBuilder.

Regards

Robert

Thanks Paolo.

No more feedback on the draft from me. Just that IMO having in-house developers in TDF is a must since we have seen something like “why do you think we (C*a) would want to help you in creating a competing product?” when someone asked for help in our developer’s IRC channel. So please do it as soon as possible. The proposal does not need to be 100% perfect; let’s be down to earth and make it better and better.

Regards,

Franklin

Paolo Vecchi <paolo.vecchi@documentfoundation.org> 於 2022年3月23日 週三 下午10:08寫道:

Thank you for your support for the proposal Franklin.

And thank you again for all your work during last term that contributed in making this proposal possible.

Ciao

Paolo

Hi Paolo,

Hi all,

following the initial consultation in regards to the first draft I've
included some of the recommendations and comments received.

You can find the new version here:

https://nextcloud.documentfoundation.org/s/d5fF4eCK4JHtpHj

The changes are in italic.

Some of the comment received included preferences for mentors instead
of developers or to further outsource some tasks.
Those preferences will be covered by duly prepared proposals written
by the proponents so they will not be covered in the document I shared.

Other comments received, also from fellow members of the board, stated
that my proposal lacked of clear developers and project management
procedures so I've added in page 10 what I see, at least initially, as
the simplest approach but suggestions for improvements are very welcome.

Please do remind me if I forgot to include your constructive feedback!

I'd suggest to add to the benefits of in-house developers, that it
enables TDF to coach e.g. (more) GSoC students.

Regards,
Andreas

Hi Paolo,

The list is too long to follow. I have few questions here that I don’t find them addressed in the document:

Is that hiring annualy or long term?

( Apologize if this is clear to others. But I don’t know how hiring is done in TDF. )

What’s the lost / cost to TDF if someday the board or future board want to dismiss the developer, in case something bad happens or it doesn’t work out?

After hiring in-house developer, TDF might become a scapegoat directly, for not fixing users bugs.
What would the expected response be?

Is there any preventive measure for the unfair situation mentioned by Michael Stahl[1],
in which enterprise users who deployed for free, and eventually they don’t contribute, then endanger the sustenance of the project?

[1] https://listarchives.documentfoundation.org/www/board-discuss/2022/msg00290.html

Sincerely,
Mark

Paolo Vecchi <paolo.vecchi@documentfoundation.org> 於 2022年3月23日 週三 下午10:09寫道:

Hi Mark,

Yes the discussion has been taking place over a long period of time and across many threads so it is difficult to get all the answers in an easy way. To try to make it easier to follow a discussion and the various proposals the “Decidim startup proposal” has been presented for approval in the budget and I hope it will find a full consensus within the board to invest on it. It’s a long term employment project, that’s why I asked the board to not consider it as a budget line (like tenders) but as a long term strategic investment. The cost to TDF could be 0 or quite a lot, like in any organisation, depending on why the board would want to dismiss an employee. Employment contracts allow for “trial periods” as far as I know, not an HR expert, where if we see that the new employee doesn’t fit with the organisation he/she can be fairly dismissed while if the new employee and TDF are both happy then I don’t see why there should be any issue with a long term employment. We do what we can with the resources that are made available by users/donors. Whatever we do there will be complains but I think having the internal resources to tackle issues that otherwise would not progress is an important step forward. What I hope is that people like you will notice that the proposal tries to create opportunities for better interaction and mutual support in tackling difficult issues. I’ve read some of your and Shinji’s presentations and that’s one of the many reasons why native languages are at the top of the list of my proposal, together with a11y, as it seems like the vast majority of the global population isn’t yet well served by LibreOffice. 2 in-house developers will not solve all the problems for all the users especially when, as you and Shinji rightly pointed out in your presentations, you must be a native speaker to understand and fix some issues. The xkcd in page 8 of your AsiaCon 2019 presentation is spot on in this case as even having the top developers in-house there is a limited amount of fixes/algorithms they can push if they don’t have your support. Could you suggest action points and priorities that I can add to the proposal so that we can see how to tackle together some of the issues that are stopping you from contributing and further improving CJK support? It is unfair that millions of LibreOffice users that have the luck of being able to contribute don’t do it as they don’t seem to appreciate the efforts that each one of us put into the community. It would be even more unfair if we weren’t contributing to LibreOffice for the hundreds of millions of users that are not so lucky and would have no other options. Of course unfortunately there will be cases where some try to abuse the system and it would be great if we spot all those cases. Most will be spotted while others will go through but hopefully they will be benefiting the majority of users and not specific business cases where companies/institutions could have contributed to it. Your question lead also to other questions: What about the tenders we pay for with donors money which could also fix enterprise issues/features? Should we reject tenders that are not fixing bugs and features that are clearly not for a personal use of LibreOffice? Should we consider that Japan is a quite wealthy country so language issues should be funded by local enterprises and institutions? As you see the issue could become much more complex than just having a few fixes slipping through the net. Our Next Decade Manifesto does not take in consideration the capacity to contribute of each individual, LibreOffice is free of charge for all without distinction. Funding TDF so that we can all invest in many areas, in and with many communities, is essential and I’m sure that by giving TDF more internal resources to help each others we will also increase the willingness of people to donate (in many ways, not just money) and with a larger user base many organisations will see that is better also for them to invest in improving and supporting LibreOffice and its community. Ciao Paolo







Hi Paolo,

Paolo Vecchi <paolo.vecchi@documentfoundation.org> 於 2022年3月27日 週日 上午12:34寫道:

Hi Mark,

Hi Paolo,

The list is too long to follow. I have few questions here that I don’t find them addressed in the document:

Yes the discussion has been taking place over a long period of time and across many threads so it is difficult to get all the answers in an easy way.

To try to make it easier to follow a discussion and the various proposals the “Decidim startup proposal” has been presented for approval in the budget and I hope it will find a full consensus within the board to invest on it.

Is that hiring annualy or long term?

( Apologize if this is clear to others. But I don’t know how hiring is done in TDF. )

It’s a long term employment project, that’s why I asked the board to not consider it as a budget line (like tenders) but as a long term strategic investment.

I wonder if other alternatives have been considered. Ex, tender for a dedicated developer ( to a company or a person ) that under TDF’s board / ESC command, for 1-2 year term, or other forms of one-time development

service. ( I’m not sure if this is legally feasible. ) The way seems less risky to me.

A short term one can still be strategic move to make TDF contribute more code and spend more money on development.

I advice to take the short term approach if the risk can not be managed.

The similar question is about the number of developer hired in the rationale section.

If TDF have more money next year, will the number increased? If we need more, can the fund be raised?

Why the number is 2 instead of 1?

What’s the lost / cost to TDF if someday the board or future board want to dismiss the developer, in case something bad happens or it doesn’t work out?

The cost to TDF could be 0 or quite a lot, like in any organisation, depending on why the board would want to dismiss an employee.
Employment contracts allow for “trial periods” as far as I know, not an HR expert, where if we see that the new employee doesn’t fit with the organisation he/she can be fairly dismissed while if the new employee and TDF are both happy then I don’t see why there should be any issue with a long term employment.

Bad things like

  • The newly hired developer do not get well with other core developers or contributors, being toxic to the community, or turning other developers or contributors away.
  • The performance or capability doesn’t meet the expectation, the number of bug fixes is low, or the developer unable to handle multiple unrelated areas.

may happen.

After hiring in-house developer, TDF might become a scapegoat directly, for not fixing users bugs.
What would the expected response be?

We do what we can with the resources that are made available by users/donors.
Whatever we do there will be complains but I think having the internal resources to tackle issues that otherwise would not progress is an important step forward.

What I hope is that people like you will notice that the proposal tries to create opportunities for better interaction and mutual support in tackling difficult issues.

I’ve read some of your and Shinji’s presentations and that’s one of the many reasons why native languages are at the top of the list of my proposal, together with a11y, as it seems like the vast majority of the global population isn’t yet well served by LibreOffice.

2 in-house developers will not solve all the problems for all the users especially when, as you and Shinji rightly pointed out in your presentations, you must be a native speaker to understand and fix some issues. The xkcd in page 8 of your AsiaCon 2019 presentation is spot on in this case as even having the top developers in-house there is a limited amount of fixes/algorithms they can push if they don’t have your support.

Could you suggest action points and priorities that I can add to the proposal so that we can see how to tackle together some of the issues that are stopping you from contributing and further improving CJK support?

I’ve fixed most of the issues that I felt important and go back to review tdf#83066 from time to time. I did not fix all of CJK issues for various reasons. Ex, I don’t agree with some expected results. Some issues seem to be corner case or document specific. I assume other people can also contribute if they think the unresolved issues important. Personally, I feel that tdf#75790 is a real feature gap for Japanese users. I tried to pick it up when I was available without good progress.

Is there any preventive measure for the unfair situation mentioned by Michael Stahl[1],
in which enterprise users who deployed for free, and eventually they don’t contribute, then endanger the sustenance of the project?

[1] https://listarchives.documentfoundation.org/www/board-discuss/2022/msg00290.html

It is unfair that millions of LibreOffice users that have the luck of being able to contribute don’t do it as they don’t seem to appreciate the efforts that each one of us put into the community.

It would be even more unfair if we weren’t contributing to LibreOffice for the hundreds of millions of users that are not so lucky and would have no other options.

To be more specific, my concern is the possiblity of turning contributors away.

Besides the situation aforementioned, I would not have become a contributor, if someone had fixed CJK issues for me 8 years ago.

What if you add two developers but lose more or get less?

What if you want to fix more bugs but the number of bug fixed become less?

I felt these should be addressed.

Of course unfortunately there will be cases where some try to abuse the system and it would be great if we spot all those cases. Most will be spotted while others will go through but hopefully they will be benefiting the majority of users and not specific business cases where companies/institutions could have contributed to it.

Your question lead also to other questions:

What about the tenders we pay for with donors money which could also fix enterprise issues/features?

I’m ok if the main focus is the general public or focus areas in foavor of the disadvantages.

Should we reject tenders that are not fixing bugs and features that are clearly not for a personal use of LibreOffice?

I assume the selection of the tender is oversee by the ESC and the board. It should be transparent and benefit the general public or focus areas in favor of the disadvantages instead of enterprises who have the ability to pay for themselves.

Should we consider that Japan is a quite wealthy country so language issues should be funded by local enterprises and institutions?

No, we don’t do that based on whether a country is wealthy or not.

Japanese community as a whole should be treated as the general public.

OTOH, requests from the enterprises should be conidered carefully.

Whether the issue is under CJK category is not the only thing needs to be considered.

Judgement is needed in order not to be abused, that is independent of the country or the language.

Bad things like

  * The newly hired developer do not get well with other core developers
    or contributors, being toxic to the community, or turning other
    developers or contributors away.
  * The performance or capability doesn't meet the expectation, the
    number of bug fixes is low, or the developer unable to handle
    multiple unrelated areas.

may happen.

Being hired as TDF staff is not equal to becoming a government official or a tenured professor. It is not a position protected for life.

To be more specific, my concern is the possiblity of turning contributors away.

Besides the situation aforementioned, I would not have become a contributor, if someone had fixed CJK issues for me 8 years ago.

Such a negative outcome can be avoided by using the same approach as with the other areas that receive contributions from TDF staff. The key is communicating that we need collaborators and will train new volunteers.

Ilmari

Hi Mark

Hi Paolo,

Paolo Vecchi 於 2022年3月27日 週日 上午12:34寫道: Hi Mark, It’s a long term employment project, that’s why I asked the board to not consider it as a budget line (like tenders) but as a long term strategic investment. I wonder if other alternatives have been considered. Ex, tender for a dedicated developer ( to a company or a person ) that under TDF’s board/ ESC command, for 1-2 year term, or other forms of one-time development service. ( I’m not sure if this is legally feasible. ) The way seems less risky to me.

Expert support is not excluded and it might be necessary for complex items where there local expertise like with CJK is necessary.

A short term one can still be strategic move to make TDF contribute more code and spend more money on development.

I advice to take the short term approach if the risk can not be managed.

Our ED and the rest of the team will help in evaluating the risks and I’m sure they’ll be able to make them manageable.

At the end everything we have been doing involves risks and up to know we managed quite well These are ideas that have been circulating and can be developed further. Once the developers settled in and structured their work for themselves in collaboration with the rest of the team we could look at marketing campaigns to attract further funding so that we could even include freshly graduated students/interns that could learn how to work on a complex project like LibreOffice. Because 2 is the first prime number and a good start to create a development team. Only 1 developer could find himself/herself overwhelmed but 2 could start exchanging ideas, tasks and processes to improve their productivity. That would mean those developers won’t finish their trial period. We need team players with passion and not primadonna. The important thing is the process and the attitude they show in problem solving. Some may be more productive in an area more than another and it will be our task to get the best out of them. If things don’t work out then we’ll see that quite quickly. Then we should work together to see how we can make progress happen with the new developers and/or by using specialists that can unblock the situation and allow you to keep contributing. But there are still many areas where contributions have been lacking for various reasons including the fact that they may be too complex to be tackled by a single volunteer. Also in this case the volunteers could help us in identifying how to unblock the situation and allow them to keep contributing. We already have fewer developers contributing to fix a range of bugs so adding 2 may help in reducing complexity for some issues and get more to start contributing. Adding more resources to development and fixing issues that are stopping hundreds of millions of users from using effectively LibreOffice should lead to more users and possibly more contributors. I do agree but sometimes it seems some “enterprise” features slip through the net of the proposed tenders. AFAIK there is no fixed rule for it. I agree with you. Please do come to the Board meetings to ask more questions and provide your feedback. Ciao Paolo

Hi Franklin,

we have seen something like "why do you think we (C*a) would want to
help you in creating a competing product?" when someone asked for help
in our developer's IRC channel.

  I'm flattered by what I assume is this new C*a contraction of Collabora =)

  Collabora helps its competitors every day - primarily by contributing tons of code to LibreOffice that they can use. Also by working collaboratively with other engineers from competing companies. We give feedback, advice and help on the code - as we do for volunteers - typically in some rough proportion to their estimated contribution.

  We're somewhat less sympathetic to silly questions (RTM) or to attempts by others to leech our time away to support their needs & customers without contributing meaningfully (as/when/if that happens).

  As I regularly say: working on the LibreOffice code is fun - and I'd really recommend actively contributing to the code to everyone so they can get a real feel for how that works first-hand.

  Regards,

    Michael.

Hi,

I'm touched that Julien thought of me to be employed to work on Base,
but being fully occupied, and beyond, by my own business, I'm not
available for employ (at reasonable market rates for a programmer).

I do think that the whole scripting ecosystem, and doing SQL
database-driven data entry, analysis, screen-ready and print-ready
reporting etc is a strategically important feature, and my business
became dependent on that. I would gladly enter a consortium or some
other system to pool resources to finance a developer whose job it is
to fix user-blocking bugs, deep issues that only get worse with time
like:

* dependency on HSQLDB1.8
* finishing Firebird integration well,
* dependency on a local copy of an external reporting engine that
   hasn't been updated in ages
* fundamentally fix predictability of the order of events in Base
   scriptable GUI controls (it is a mess of race conditions)
* better PostgreSQL support (such as table/view/... design)

Hi *,

I would gladly enter a consortium or some
other system to pool resources to finance a developer whose job it is
to fix user-blocking bugs,

Next little hint: Donation of 2000,- € to Document Foundation, special Base, 2022-10-11. Have created a database for a second company in Germany.

Regards

Robert