September 2018 - xwiki-devs@xwiki.org

[xwiki-devs] [STAMP/Test] Metrics we need to improve + strategy
by Vincent Massol 17 Oct '18

17 Oct '18

Hi devs (and anyone else interested to improve the tests of XWiki), History ====== It all started when I analyzed our global TPC and found that it was going down globally even though we have the fail-build-on-jacoco-threshold strategy. I sent several email threads: - Loss of TPC: http://markmail.org/message/hqumkdiz7jm76ya6 - TPC evolution: http://markmail.org/message/up2gc2zzbbe4uqgn - Improve our TPC strategy: http://markmail.org/message/grphwta63pp5p4l7 Note: As a consequence of this last thread, I implemented a Jenkins Pipeline to send us a mail when the global TPC of an XWiki module goes down so that we fix it ASAP. This is still a development in progress. A first version is done and running at https://ci.xwiki.org/view/Tools/job/Clover/ but I need to debug it and fix it (it’s not working ATM). As a result of the global TPC going down/stagnating, I have proposed to have 10.7 focused on Tests + BFD. - Initially I proposed to focus on increasing the global TPC by looking at the reports from 1) above (http://markmail.org/message/qjemnip7hjva2rjd) See the last report at https://up1.xwikisas.com/#mJ0loeB6nBrAgYeKA7MGGw (we need to fix the red parts). - Then with the STAMP mid-term review, a bigger urgency surfaced and I asked if we could instead focus on fixing tests as reported by Descartes to increase both coverage and mutation score (ie test quality), since those are 2 metrics/KPIs measured by STAMP and since XWiki participates to STAMP we need to work on them and increase them substantially. See http://markmail.org/message/ejmdkf3hx7drkj52 The results of XWiki 10.7 has been quite poor on test improvements (more focus on BFD than tests, lots of devs on holidays, etc). This forces us to have a different strategy. Full Strategy proposal ================= 1) As many XWiki SAS devs as possible (and anyone else from the community who’s interested ofc! :)) should spend 1 day per week working on improving STAMP metrics * Currently the agreement is that Thomas and myself will do this for the foreseeable future till we get some good-enough metric progress * Some other devs from XWiki SAS will help out for XWiki 10.8 only FTM (Marius, Adel if he can, Simon in the future). The idea is to see where that could get us by using substantial manpower. 2) All committers: More generally the global TPC failure is also already active and dev need to modify modules that see their global TPC go down. 3) All committers: Of course, the jacoco strategy is also active at each module level. STAMP tools ========== There are 4 tools developed by STAMP: * Descartes: Improves quality of tests by increasing their mutation scores. See http://markmail.org/message/bonb5f7f37omnnog and also https://massol.myxwiki.org/xwiki/bin/view/Blog/MutationTestingDescartes * DSpot: Automatically generate new tests, based on existing tests. See https://massol.myxwiki.org/xwiki/bin/view/Blog/TestGenerationDspot * CAMP: Takes a Dockerfile and generates mutations of it, then deploys and execute tests on the software to see if the mutation works or not. Note this is currently not fitting the need of XWiki and thus I’ve been developing another tool as an experiment (which may go back in CAMP one day), based on TestContainers, see https://massol.myxwiki.org/xwiki/bin/view/Blog/EnvironmentTestingExperiment… * EvoCrash: Takes a stack trace from production logs and generates a test that, when executed, reproduces the crash. See https://markmail.org/message/v74g3tsmflquqwra. See also https://github.com/SERG-Delft/EvoCrash Since XWiki is part of the STAMP research project, we need to use those 4 tools to increase the KPIs associated with the tools. See below. Objectives/KPIs/Metrics for STAMP =========================== The STAMP project has defined 9 KPIs that all partners (and thus XWiki) need to work on: 1) K01: Increase test coverage * Global increase by reducing by 40% the non-covered code. For XWiki since we’re at about 70%, this means reaching about 80% before the end of STAMP (ie. before end of 2019) * Increase the coverage contributions of each tool developed by STAMP. Strategy: * Primary goal: ** Increase coverage by executing Descartes and improving our tests. This is http://markmail.org/message/ejmdkf3hx7drkj52 ** Don’t do anything with DSpot. I’ll do that part. Note that the goal is to write a Jenkins pipeline to automatically execute DSpot from time to time and commit the generated tests in a separate test source and have our build execute both src/test/java and this new test source. ** Don’t do anything with TestContainers FTM since I need to finish a first working version. I may need help in the future to implement docker images for more configurations (on Oracle, in a cluster, with LibreOffice, with an external SOLR server, etc). ** For EvoCrash: We’ll count contributions of EvoCrash to coverage in K08. * Secondary goal: ** Increase our global TPC as mentioned above by fixing the modules in red. 2) K02: Reduce flaky tests. * Objective: reduce the number of flaky tests by 20% Strategy: * Record flaky tests in jira * Fix the max number of them 3) K03: Better test quality * Objective: increase mutation score by 20% Strategy: * Same strategy as K01. 4) K04: More configuration-related paths tested * Objective: increase the code coverage of configuration-related paths in our code by 20% (e.g. DB schema creation, cluster)related code, SOLR-related code, LibreOffice-related code, etc). Strategy: * Leave it to FTM. The idea is to measure Clover TPC with the base configuration, then execute all other configurations (with TestContainers) and regenerate the Clover report to see how much the TPC has increased. 5) K05: Reduce system-specific bugs * Objective: 30% improvement Strategy: * Run TestContainers, execute existing tests and find new bugs related to configurations. Record them 6) K06: More configurations/Faster tests * Objective: increase the number of automatically tested configurations by 50% Strategy: * Increase the # of configurations we test with TestContainers. I’ll do that part initially. * Reduce time it takes to deploy the software under a given configuration vs time it used to take when done manually before STAMP. I’ll do this one. I’ve already worked on it in the past year with the dockerization of XWiki. 7) K07: Pending, nothing to do FTM 8) K08: More crash replicating test cases * Objective: increase the number of crash replicating test cases by at least 70% Strategy: * For all issues that are still open and that have stack traces and for all issues closed but without tests, run EvoCrash on them to try to generate a test. * Record and count the number of successful EvoCrash-generated test cases. * Derive a regression test (which can be very different from the negative of the test generated by evocrash!). * Measure the new coverage increase * Note that I haven’t experimented much with this yet myself. 9) K09: Pending, nothing to do FTM. Conclusion ========= Right now, I need your help for the following KPIs: K01, K02, K03, K08. Since there’s a lot to understand in this email, I’m open to: * Organizing a meeting on youtube live to discuss all this * Answering any questions on this thread ofc * Also feel free to ask on IRC/Matrix. Here’s an extract from STAMP which has more details about the KPIs/metrics: https://up1.xwikisas.com/#QJyxqspKXSzuWNOHUuAaEA Thanks -Vincent

4 15

[xwiki-devs] [Proposal] Prevent users from renaming/move pages with XClass definition
by Simon Urli 17 Oct '18

17 Oct '18

Hi all, users might currently break their AWM application by renaming/moving pages containing XClass definition. We need a proper refactoring operation to be able to properly do such move/rename. But this feature might take a while to be completely available. In the meantime I propose that we prevent users from renaming/moving pages containing XClass. What I propose is the following: - Forbid completely *simple users* to rename/move pages containing XClass - Display a warning to *advanced users* when they perform such operation: the same kind of warning we already have when performing edit on XWiki pages WDYT? Simon -- Simon Urli Software Engineer at XWiki SAS simon.urli(a)xwiki.com More about us at http://www.xwiki.com

6 28

[xwiki-devs] State of Velocity 2.0 upgrade
by Thomas Mortagne 15 Oct '18

15 Oct '18

Hi devs, Some of you may know that I started working on upgrading our scripting engine to Velocity 2.0. The corresponding Jira issue is https://jira.xwiki.org/browse/XCOMMONS-1296. Those of you who want to take a look at breakages with standard Velocity 2.0 upgrade (if you directly use Velocity API) can take a look at http://velocity.apache.org/engine/2.0/upgrading.html. The following details are about XWiki Velocity "engine": = The code You can see the current state of the upgrade on the following branches: * https://github.com/xwiki/xwiki-commons/tree/feature-velocity2 * https://github.com/xwiki/xwiki-rendering/tree/feature-velocity2 * https://github.com/xwiki/xwiki-platform/tree/feature-velocity2 = Bad news == Definitive (probably) breakages * it was not easy in Velocity 1.7 but it's now impossible in Velocity 2.0 to use the hack we currently use to make macro "return" something. I mitigated this a bit by working on a #setVariable directive (the name of the helper we currently have in macros.vm) but it's not yet working in all situations and any code that was not going trough #setVariable will be broken anyway * the hypen ( - ) cannot be used in variable names anymore Those changes make impossible to upgrade to Velocity 2.0 in 10.x branch IMO, to big to be done in the middle of a cycle. == Temporary (hopefully) breakages * impossible to have $ or # at the end of a " based string literal (we have that a lot in various regex for example). This looks like an unplanned regression (since it's not documented) so I'm waiting for some kind reaction in the issues I reported (https://issues.apache.org/jira/browse/VELOCITY-897 and https://issues.apache.org/jira/browse/VELOCITY-896) == Not too impacting (hopefully) breakages The Velocity API changed quite a lot, fortunately not in classes we expose publicly (basically all we expose is VelocityContext). Also reminder: directly manipulating even XWiki Velocity API is considered legacy things (better use templates or wiki content and ScriptContext). = Good news == Macros handling The way to manage namespaces and macros has been completely changed (it does mean that lots of API has been broken in the process but we don't expose this API) and the good news is the new design will allow us to (finally) do proper caching of velocity scripts. I also removed a few hacks we had to do to not end up with endless memory leaks and milti-thread issues caused by the old namespace handling system. == Less stuff on our side I was able to move the following to legacy: * an equivalent of our chainable uberspector system has been implemented in Velocity standard * a DeprecatedCheckUberspector has been implemented in Velocity standard == Retro compatibility I was able to add * XWiki will keep supporting $velocityCount and $velocityHasNext (with a warning). It imply using XWikiVelocityContext instead of VelocityContext if you create it directly (which is a very bad idea) but you don't need to worry about that if you get your context from VelocityManager * a new #setVariable directive help supporting a few use case of macro "return" but unfortunately not all yet (Velocity AST is not super obvious to navigate...) == Performances Other than the caching on our side I already mentioned (huge boost planned when implemented) there is some memory and performance improvements claims in Velocity 2.0 release note (but I did not had time to do any testing myself yet). = TODO a) waiting for answer about the regressions I reported b) try find a way to make #setVariable directive works in a macro which is itself called in another macro (but it may not be possible) c) send a mail to discuss when to merge the Velocity 2 branch when at least a) is resolved Thanks, -- Thomas Mortagne

1 1

[xwiki-devs] Proposal for Macro inline edition
by Simon Urli 15 Oct '18

15 Oct '18

Hi everyone, I'm working on the roadmap issues related to the inline edition with WYSIWYG editor for macro content and macro parameters. The first step is to add a flag to allow user specify that a content or a parameter can be edited inline with the WYSIWYG editor. The second step is to allow the CKEditor to detect where the content and/or parameters should be edited. Let's take the exampe of a simple macro without any parameter, which currently produces this code: <div class="box infomessage"> <div class="title"> <span class="icon info"></span> some title </div> Some content </div> We propose (me & Marius) to ask users to add a wrapper with a specific class around the content to tell the editor it should only allow editing this content, e.g.: <div class="box infomessage"> <div class="title"> <span class="icon info"></span> some title </div> <span class="editable-content">Some content</span> </div> About parameters, our idea was to define a new metadata attribute and to ask users to use it for specifying the content is editable, such as for a parameter named foo: <span class="editable-content" data-parameter="foo">my foo parameter value</span> However I don't know right now how the editor would manage cases such as: <span class="editable-content">Some content with <span class="editable-content" data-parameter="myparameter">a parameter</span></span> So: 1. Do you agree on the usage of a class named "editable-content" which would be used as a tag to allow inline edition? 2. WDYT about using a data-parameter and this class for inline editing of parameters? Thanks, Simon -- Simon Urli Software Engineer at XWiki SAS simon.urli(a)xwiki.com More about us at http://www.xwiki.com

5 34

[xwiki-devs] [Documentation] Upgrade
by Ecaterina Moraru (Valica) 12 Oct '18

12 Oct '18

Hi devs, Since I didn't understood a lot of things the first time I've read the Update documentation, I've made a proposal on improving the grouping of the content. You can see: - The original https://www.xwiki.org/xwiki/bin/view/Documentation/AdminGuide/Upgrade - The proposal https://design.xwiki.org/xwiki/bin/view/Proposal/Upgrade/SimplerUpgrade/Dra… Since the content is highly technical, if you find mistakes or want to add stuff, please just edit them yourself. I've just took the existing content and reorganize it. Let me know what you think about this version and if we can replace the current version with the draft. Thank you, Caty

2 4

[xwiki-devs] [Brainstorming] "Code" subspace for apps
by Vincent Massol 01 Oct '18

01 Oct '18

Hi devs, I’ve just had a quick chat with Edy and I found that we had a difference of opinion on the Code subspace practice. On https://dev.xwiki.org/xwiki/bin/view/Community/ApplicationDevelopmentBestPr… we say: "Technical pages must be put in a subspace named Code” Now Edy says that this should be done only for data-generating apps. It’s not my recollection that this rule was only for this case and this is what I’d like to discuss here. For example, I’ve noticed that ActiveInstalls has all technical pages under ActiveInstalls, see https://github.com/xwiki/xwiki-platform/tree/053f0a2757cea18a5916632a58c604… I would fix it to have only the WebHome remain under ActiveInstalls and move all the technical pages under ActiveInstalls.Code. The only case where it could make sense to not have a Code subspace would be when the app has no UI at all. Even in this case, you might argue that we should always have a home to provide a description about the content of the space. So right now I’m personally in favor of continuing the rule we defined in the best practices: "Technical pages must be put in a subspace named Code” WDYT? Thanks -Vincent

5 6

[xwiki-devs] [Proposal] Turn off redirects in page move/rename
by Vincent Massol 01 Oct '18

01 Oct '18

Hi devs, We see more and more wikis with tons of redirects and issues to manage them, see https://jira.xwiki.org/browse/XWIKI-14570 “ Rationale for turning it off by default: - Several users have reported a usability issue about automatic redirects. The way they express it is the following: "I have duplicates pages in my wiki. I don't understand why this is happening. I'm choosing to rename pages and not to copy them but I still get duplicates in the Navigation panel". - Automatic redirects are especially useful for public wikis where users can have bookmark on pages and you don't want to break them. It can also be useful for internal wikis but it's less an issue. - Even for public wikis not all pages need automatic redirects. Technical pages don't need them for example. “ I’ve had Ludovic report that to me too again this morning. Since we’ve been knowing about the problem for a while already and since it’s very easy to change the default behavior, I’d like to propose that we turn it off by default in XWiki 10.8 (ie right now, or if you think it’s really too dangerous in 10.9), and that we implement an option to configure the default behavior later on (https://jira.xwiki.org/browse/XWIKI-13384) WDYT? Thanks -Vincent

5 9

[xwiki-devs] [XWiki Day] BFD#190
by Oana-Lavinia Florean 28 Sep '18

28 Sep '18

Hello devs, This Thursday is BFD#190: http://dev.xwiki.org/xwiki/bin/view/Community/XWikiDays#HBugfixingday Our current status is: * -46 bugs over 120 days (4 months), i.e. we need to close 46 bugs to have created bugs = closed bugs * -96 bugs over 365 days (1 year) * -144 bugs over 500 days (between 1 and 2 years) * -360 bugs over 1600 days (4.3 years) See https://jira.xwiki.org/secure/Dashboard.jspa?selectPageId=10352 Here's the BFD#190 dashboard to follow the progress during the day: https://jira.xwiki.org/secure/Dashboard.jspa?selectPageId=14325 Good luck at fixing bugs, Lavinia

1 1

[xwiki-devs] [ANN] XWiki 10.8 released
by Adel Atallah 24 Sep '18

24 Sep '18

The XWiki development team is proud to announce the availability of XWiki 10.8. This version adds a new tab in the user profile which displays the groups a user is part of, the Menu application is now available in the Administration page, auto-suggestions has been added for pages in several places, the startup speed of Tomcat has been improved and our CAPTCHA implementation has been replaced with a new one. You can download it here: http://www.xwiki.org/xwiki/bin/view/Main/Download Make sure to review the release notes: https://www.xwiki.org/xwiki/bin/view/ReleaseNotes/Data/XWiki/10.8/ Thanks for your support -The XWiki dev team

1 0

[xwiki-devs] [Brainstorming] Auto-suggestion on space, wiki and attachment xproperties
by Adel Atallah 24 Sep '18

24 Sep '18

Hi devs, Working on the auto-suggestion feature [1], I've been integrating it in xproperties holding a reference to a page. Those xproperties are of type String and have been changed to the Page type so that the page displayer is used and shows the auto-suggestion when needed. I now need to implement an auto-suggestion on xproperties holding a reference to a space (e.g [2]), a wiki (e.g [2]) and an attachment (e.g. [3]). I think the best would be to create an Attachment and Wiki xproperty type and create new displayers exactly like the Page type. I don't think we can create a Space xproperty because it's not meant to be visible by the end user. Maybe we could use the Page xproperty type and have a parameter inside to select only spaces. WDYT? Thanks, Adel -------- [1] https://design.xwiki.org/xwiki/bin/view/Proposal/AutocompleteOnReference [2] https://design.xwiki.org/xwiki/bin/view/Proposal/AutocompleteOnReference#HC… [3] https://design.xwiki.org/xwiki/bin/view/Proposal/AutocompleteOnReference#HC…

5 7