[xwiki-devs] [Proposal] Stopping the 4.4M1 release till tests are all passing - Commando mode
Hi devs, We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list. It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work. This is all the more bad that we're ending the 4.x cycle. Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future WDYT? Thanks -Vincent Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
Big +1, obviously. Let's do it! Marius On Thu, Dec 13, 2012 at 6:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Hi, On Thu, Dec 13, 2012 at 6:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
+1 and, as I`ve already mentioned it to Vincent, maybe this gives us the opportunity to improve our release process/scripts to be able to release from a "release branch" instead of blocking "master" for the whole period of the release.
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
Since I`ve already agreed to being RM for 4.3.1, I guess that this was already part of the work, additional to the help from the owners of the problematic modules on which I`m counting on :) Thanks, Eduard
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 12/13/2012 12:00 PM, Eduard Moraru wrote:
Hi,
On Thu, Dec 13, 2012 at 6:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
+1 and, as I`ve already mentioned it to Vincent, maybe this gives us the opportunity to improve our release process/scripts to be able to release from a "release branch" instead of blocking "master" for the whole period of the release.
The release script already does that. I mean, when starting the release it creates a local release branch, but that branch gets discarded at the end. Do you want to create the release branch beforehand, so that devs can stabilize the build in that release branch? I'm against this, since it contradicts another important principle that we're supposed to follow: the build should always work! There is no good reason why there would be a need for stabilization. That's why we have continuous integration.
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
Since I`ve already agreed to being RM for 4.3.1, I guess that this was already part of the work, additional to the help from the owners of the problematic modules on which I`m counting on :)
Thanks, Eduard
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Sergiu Dumitriu http://purl.org/net/sergiu
On Thu, Dec 13, 2012 at 5:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
There are still a blocker issue for 4.3.1, shouldn't we also have that solve first ? There is also a lot of other issue marked to be fix, I suppose Edy will triage, and if some are important, these should be fixed first. I am just afraid, because most of these issue are assigned to Marius, and IMO it is more important to focus on the 4.3.1 release than on the 4.4M1 one.
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO eGuilde sarl - CTO
Hi Denis, On Dec 13, 2012, at 6:06 PM, Denis Gervalle <dgl@softec.lu> wrote:
On Thu, Dec 13, 2012 at 5:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
There are still a blocker issue for 4.3.1, shouldn't we also have that solve first ? There is also a lot of other issue marked to be fix, I suppose Edy will triage, and if some are important, these should be fixed first. I am just afraid, because most of these issue are assigned to Marius, and IMO it is more important to focus on the 4.3.1 release than on the 4.4M1 one.
At this stage it's more important to release 4.3.1 ASAP and we can do a 4.3.2 after for less important bugs if we want but I don't think it'll be needed because we'll be released 4.4 which is a stabilization/bug fix release and same for 4.5 so we'll recommend users to upgrade to 4.4 and then 4.5 and not 4.3.1 or 4.3.2... If you know of some issues that you consider blocking for 4.3.1 please raise them in a different thread and we can decide if they warrant postponing 4.3.1 more. Thanks -Vincent PS: I suggest you be the next release manager so that you can really decide if it's more important to fix the build or fix bugs… ;) There are some committers that don't help much in fixing the build so it's quite easy if you've not been doing this to consider that the build isn't important.. The most important is to have a stable build. That comes first. Even before bugs because it's a prerequisite for fixing bugs.
Let's do it! http://i.qkme.me/3s6iaw.jpg Guillaume On Thu, Dec 13, 2012 at 6:34 PM, Vincent Massol <vincent@massol.net> wrote:
Hi Denis,
On Dec 13, 2012, at 6:06 PM, Denis Gervalle <dgl@softec.lu> wrote:
On Thu, Dec 13, 2012 at 5:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
There are still a blocker issue for 4.3.1, shouldn't we also have that solve first ? There is also a lot of other issue marked to be fix, I suppose Edy will triage, and if some are important, these should be fixed first. I am just afraid, because most of these issue are assigned to Marius, and IMO it is more important to focus on the 4.3.1 release than on the 4.4M1 one.
At this stage it's more important to release 4.3.1 ASAP and we can do a 4.3.2 after for less important bugs if we want but I don't think it'll be needed because we'll be released 4.4 which is a stabilization/bug fix release and same for 4.5 so we'll recommend users to upgrade to 4.4 and then 4.5 and not 4.3.1 or 4.3.2...
If you know of some issues that you consider blocking for 4.3.1 please raise them in a different thread and we can decide if they warrant postponing 4.3.1 more.
Thanks -Vincent
PS: I suggest you be the next release manager so that you can really decide if it's more important to fix the build or fix bugs… ;) There are some committers that don't help much in fixing the build so it's quite easy if you've not been doing this to consider that the build isn't important.. The most important is to have a stable build. That comes first. Even before bugs because it's a prerequisite for fixing bugs.
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Denis you can consider me as a resource, would like to contribute in 4.3.1 as well as other tasks. -arvind On Thu, Dec 13, 2012 at 11:04 PM, Vincent Massol <vincent@massol.net> wrote:
Hi Denis,
On Dec 13, 2012, at 6:06 PM, Denis Gervalle <dgl@softec.lu> wrote:
On Thu, Dec 13, 2012 at 5:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
There are still a blocker issue for 4.3.1, shouldn't we also have that solve first ? There is also a lot of other issue marked to be fix, I suppose Edy will triage, and if some are important, these should be fixed first. I am just afraid, because most of these issue are assigned to Marius, and IMO it is more important to focus on the 4.3.1 release than on the 4.4M1 one.
At this stage it's more important to release 4.3.1 ASAP and we can do a 4.3.2 after for less important bugs if we want but I don't think it'll be needed because we'll be released 4.4 which is a stabilization/bug fix release and same for 4.5 so we'll recommend users to upgrade to 4.4 and then 4.5 and not 4.3.1 or 4.3.2...
If you know of some issues that you consider blocking for 4.3.1 please raise them in a different thread and we can decide if they warrant postponing 4.3.1 more.
Thanks -Vincent
PS: I suggest you be the next release manager so that you can really decide if it's more important to fix the build or fix bugs… ;) There are some committers that don't help much in fixing the build so it's quite easy if you've not been doing this to consider that the build isn't important.. The most important is to have a stable build. That comes first. Even before bugs because it's a prerequisite for fixing bugs.
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On Thu, Dec 13, 2012 at 6:34 PM, Vincent Massol <vincent@massol.net> wrote:
Hi Denis,
On Dec 13, 2012, at 6:06 PM, Denis Gervalle <dgl@softec.lu> wrote:
On Thu, Dec 13, 2012 at 5:42 PM, Vincent Massol <vincent@massol.net> wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
There are still a blocker issue for 4.3.1, shouldn't we also have that solve first ? There is also a lot of other issue marked to be fix, I suppose Edy will triage, and if some are important, these should be fixed first. I am just afraid, because most of these issue are assigned to Marius, and IMO it is more important to focus on the 4.3.1 release than on the 4.4M1 one.
At this stage it's more important to release 4.3.1 ASAP and we can do a 4.3.2 after for less important bugs if we want but I don't think it'll be needed because we'll be released 4.4 which is a stabilization/bug fix release and same for 4.5 so we'll recommend users to upgrade to 4.4 and then 4.5 and not 4.3.1 or 4.3.2...
Have I said something else ?
If you know of some issues that you consider blocking for 4.3.1 please raise them in a different thread and we can decide if they warrant postponing 4.3.1 more.
None that were not already marked so, and seems to have be solved since my last message :)
Thanks -Vincent
PS: I suggest you be the next release manager so that you can really decide if it's more important to fix the build or fix bugs… ;)
The RM of the latest unstable should not have priority over the RM of the latest stable. When I check yesterday, there were a bunch of new issue marked for 4.3.1. that have gone today, so everything is fine now.
There are some committers that don't help much in fixing the build so it's quite easy if you've not been doing this to consider that the build isn't important.. The most important is to have a stable build. That comes first. Even before bugs because it's a prerequisite for fixing bugs.
I completely understand that. At the same time, the 4.3.1 release have been delayed for almost a week for a few not so recent bugs. IMO, bugs on the latest stable should receive priority attention, especially blocker ones. I was unable to be the RM simply because some of them has not receive early attention, and my availability was last week, not this week.
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Denis Gervalle SOFTEC sa - CEO eGuilde sarl - CTO
On 12/13/2012 11:42 AM, Vincent Massol wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
I've looked at the build failures earlier, and most of them seem to be false alarms. A few failed with a SocketTimedOut exception, which means that either the machine (agent) is flaky, or that XWiki is getting too slow (performance issue). Some tests failed with a missing body, which again suggests a network problem (empty response?). Since most agents share the same physical hardware, maybe these failures just mean that we're putting too much load on the CPU/RAM/disk, and we should reduce the number of agents.
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
-- Sergiu Dumitriu http://purl.org/net/sergiu
On Dec 13, 2012, at 6:11 PM, Sergiu Dumitriu <sergiu@xwiki.org> wrote:
On 12/13/2012 11:42 AM, Vincent Massol wrote:
Hi devs,
We have too many test failures on http://ci.xwiki.org/view/Functional%20Tests/ and too many emails sent by Jenkins on the list.
It has become a nightmare and it's impossible to perform a release anymore with a good confidence it's going to work.
This is all the more bad that we're ending the 4.x cycle.
Thus I propose to do the following: * Don't release 4.4M1 till all tests are passing with no more flickers (say the tests should all pass during 10 full builds for example) * Create a Commando unit in charge of solving the flickers. Since I've already discussed this with Marius I propose that Marius and myself be the first 2 members. If anyone else would like to help please reply to this mail and join us. * This commando unit gives itself 1 full week to solve the flickers (ie till the 21st of December). We'll decide what to do next if we fail to achieve our goal after that deadline. * We start by creating a branch for 4.4M1 so that we isolate ourselves from the rest of the devs who continue to work for 4.4RC1 (reminder: only important bug fixes should go in 4.4RC1) * When we have fixed all flickers on the 4.4M1 branch we merge the changes to both master and the stable-4.3 branch * At the end of next week we also propose a strategy so that this mess doesn't happen again in the future
WDYT?
I've looked at the build failures earlier, and most of them seem to be false alarms.
That's the point of this exercise… not having false alarms...
A few failed with a SocketTimedOut exception, which means that either the machine (agent) is flaky, or that XWiki is getting too slow (performance issue).
No I don't believe this is the cause. The wait is pretty long, in general it fails because the page currently loaded doesn't have wait we're looking for. You can check the screenshots for that.
Some tests failed with a missing body, which again suggests a network problem (empty response?).
Since most agents share the same physical hardware, maybe these failures just mean that we're putting too much load on the CPU/RAM/disk, and we should reduce the number of agents.
We need to investigate. If you wish to help it's great! :) So far I see no reason to consider that the problem is the # of agents. Thanks -Vincent
Thanks -Vincent
Note: We need to release 4.3.1 ASAP so this strategy above will not apply to 4.3.1. For 4.3.1 Edy will need to figure out if all the failing tests are real issues or test issues. I think Edy could do this by a combination of running them locally and doing some manual tests where they also fail locally. Edy WDYT?
participants (7)
-
Arvind Gupta -
Denis Gervalle -
Eduard Moraru -
Guillaume Lerouge -
Marius Dumitru Florea -
Sergiu Dumitriu -
Vincent Massol