[xwiki-devs] [VOTE] Change the way we do releases to include staging.
Hi, I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue. Why: #1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need. #2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository". #3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time. #4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal. #5. Everybody else is doing it (is that even a reason?!) Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule. Here's my +1 Caleb
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later).
+1
For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
+1 with the proviso that we need to take that into account when we publish release dates. When we say that 4.1RC1 will be released on 11th of June, I guess it means we need to release RC1 on 11th - 72 hours then?
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1 Thanks -Vincent
On 05/22/2012 04:52 AM, Vincent Massol wrote:
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later).
+1
For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
+1 with the proviso that we need to take that into account when we publish release dates. When we say that 4.1RC1 will be released on 11th of June, I guess it means we need to release RC1 on 11th - 72 hours then?
Sounds good, to prevent issues having their fix-for version set as the released version, it makes sense to release on jira right away but post-date the release so that dates line up. Anyway this is something we can leave open to experimentation until the right decision makes itself obvious. Caleb
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1
Thanks -Vincent _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On Tue, May 22, 2012 at 5:11 PM, Caleb James DeLisle <calebdelisle@lavabit.com> wrote:
On 05/22/2012 04:52 AM, Vincent Massol wrote:
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later).
+1
For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
+1 with the proviso that we need to take that into account when we publish release dates. When we say that 4.1RC1 will be released on 11th of June, I guess it means we need to release RC1 on 11th - 72 hours then?
Sounds good, to prevent issues having their fix-for version set as the released version, it makes sense to release on jira right away but post-date the release so that dates line up. Anyway this is something we can leave open to experimentation until the right decision makes itself obvious.
We should probably move the jira update from push-release.sh to maven-release.sh and executed right after we finished releasing a project. It was already a big fuzzy even before since you could have commons actually released since a long time before you actually release it on jira (not time to finish the full release, etc...).
Caleb
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1
+1
Thanks -Vincent _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
On 05/23/2012 06:51 AM, Thomas Mortagne wrote:
On Tue, May 22, 2012 at 5:11 PM, Caleb James DeLisle <calebdelisle@lavabit.com> wrote:
On 05/22/2012 04:52 AM, Vincent Massol wrote:
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later).
+1
For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
+1 with the proviso that we need to take that into account when we publish release dates. When we say that 4.1RC1 will be released on 11th of June, I guess it means we need to release RC1 on 11th - 72 hours then?
Sounds good, to prevent issues having their fix-for version set as the released version, it makes sense to release on jira right away but post-date the release so that dates line up. Anyway this is something we can leave open to experimentation until the right decision makes itself obvious.
We should probably move the jira update from push-release.sh to maven-release.sh and executed right after we finished releasing a project. It was already a big fuzzy even before since you could have commons actually released since a long time before you actually release it on jira (not time to finish the full release, etc...).
Not quite, since maven-release.sh should just handle the maven release. But we could add a pre-release.sh script that handles Jira.
Caleb
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1
+1
-- Sergiu Dumitriu http://purl.org/net/sergiu/
On Sun, May 27, 2012 at 5:43 AM, Sergiu Dumitriu <sergiu@xwiki.com> wrote:
On 05/23/2012 06:51 AM, Thomas Mortagne wrote:
On Tue, May 22, 2012 at 5:11 PM, Caleb James DeLisle <calebdelisle@lavabit.com> wrote:
On 05/22/2012 04:52 AM, Vincent Massol wrote:
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later).
+1
For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
+1 with the proviso that we need to take that into account when we publish release dates. When we say that 4.1RC1 will be released on 11th of June, I guess it means we need to release RC1 on 11th - 72 hours then?
Sounds good, to prevent issues having their fix-for version set as the released version, it makes sense to release on jira right away but post-date the release so that dates line up. Anyway this is something we can leave open to experimentation until the right decision makes itself obvious.
We should probably move the jira update from push-release.sh to maven-release.sh and executed right after we finished releasing a project. It was already a big fuzzy even before since you could have commons actually released since a long time before you actually release it on jira (not time to finish the full release, etc...).
Not quite, since maven-release.sh should just handle the maven release. But we could add a pre-release.sh script that handles Jira.
It's not that simple. IMO releasing on Jira mean that what your commit will go in the following version and that happen the minute you create the release tag on git.
Caleb
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1
+1
-- Sergiu Dumitriu http://purl.org/net/sergiu/
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne
+1 The RC1 and Final are tested before the actual release, but it is not nice because while testing the latest snapshot (before RC or Final), developers can still commit, meaning that some bugs might be introduced while I was testing, or incomplete commits (work not done) could be reported as bugs. Regards, Sorin B. On Sun, May 27, 2012 at 12:52 PM, Thomas Mortagne <thomas.mortagne@xwiki.com> wrote:
On Sun, May 27, 2012 at 5:43 AM, Sergiu Dumitriu <sergiu@xwiki.com> wrote:
On 05/23/2012 06:51 AM, Thomas Mortagne wrote:
On Tue, May 22, 2012 at 5:11 PM, Caleb James DeLisle <calebdelisle@lavabit.com> wrote:
On 05/22/2012 04:52 AM, Vincent Massol wrote:
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later).
+1
For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
+1 with the proviso that we need to take that into account when we publish release dates. When we say that 4.1RC1 will be released on 11th of June, I guess it means we need to release RC1 on 11th - 72 hours then?
Sounds good, to prevent issues having their fix-for version set as the released version, it makes sense to release on jira right away but post-date the release so that dates line up. Anyway this is something we can leave open to experimentation until the right decision makes itself obvious.
We should probably move the jira update from push-release.sh to maven-release.sh and executed right after we finished releasing a project. It was already a big fuzzy even before since you could have commons actually released since a long time before you actually release it on jira (not time to finish the full release, etc...).
Not quite, since maven-release.sh should just handle the maven release. But we could add a pre-release.sh script that handles Jira.
It's not that simple. IMO releasing on Jira mean that what your commit will go in the following version and that happen the minute you create the release tag on git.
Caleb
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1
+1
-- Sergiu Dumitriu http://purl.org/net/sergiu/
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
Hi, So this way we avoid the current awkward manual test strategy of testing *after* the release is made and, instead, we get to test in the staging process *before* the release is actually made, right? +1 Thanks, Eduard On Tue, May 22, 2012 at 11:55 AM, Caleb James DeLisle < calebdelisle@lavabit.com> wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 05/22/2012 04:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
What do you mean by "moving away from milestone releases in favor of staging"?
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
+1 as well. -- Sergiu Dumitriu http://purl.org/net/sergiu/
Hi guys, I'd like to bring an issue with this VOTE below. When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go. The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs. So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging). Thanks -Vincent PS: Sorry for not realizing this earlier... On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
It seems that if we want to shorten a release cycle, we have 2 options: #1 No RC release #2 No staging I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement. This is an interesting topic which can be discussed. Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution. Thanks, Caleb On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On Jun 8, 2012, at 11:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely
We're not behind schedule at all. Again the dates we've shown to the world are those listed on http://enterprise.xwiki.org/xwiki/bin/view/Main/Roadmap And they say: * RC1 released on 11th * Final released on 18th So we're still good :) Thanks -Vincent
opposed to a change but IMO if we want to change it we need to proceed with extra caution.
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution.
The purpose of the RC releases was as a kind of staging, without actually using real staging. So I'd say that for the next releases we should only have milestones, and staged final releases. However, for 4.1 we should keep the rc-1 release in the plan, since our staging process is new and not thoroughly tested yet. The way I simulated "staging releases" when I had the RM hat was to leave a window between uploading the artifacts in our maven repository and pushing them further, and in this window I did quick smoke tests to see that everything works (with the help of Sorin and the rest of the QA team at XWiki SAS). So, I propose this as the strategy for our next releases: - Each release goes to a staging repository first - Milestone releases spend a little time in staging, enough for a few quick smoke tests; depending on the time of day, this could last from an hour to at most a day - There are no RC releases, there are only RC staging repositories. Currently the staging repositories have random names, but the RM should rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a RC release goes through more intensive tests, including smoke testing, the full MTR, and developers that contributed to the release should test their committed features. Maybe even a Jenkins job could be executed for that release, with all the automated tests. Timing? Since we don't know how many times we have to stage releases, I'd say that we can start building RCs one week before the planned final release date, and if bugs are found, a new RC is build the next day (to let issues accumulate), hoping that there will be a stable successful build before the final release date, without a 72h mandatory waiting period.
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
-- Sergiu Dumitriu http://purl.org/net/sergiu/
On Jun 9, 2012, at 6:28 AM, Sergiu Dumitriu wrote:
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution.
The purpose of the RC releases was as a kind of staging, without actually using real staging. So I'd say that for the next releases we should only have milestones, and staged final releases.
However, for 4.1 we should keep the rc-1 release in the plan, since our staging process is new and not thoroughly tested yet.
The way I simulated "staging releases" when I had the RM hat was to leave a window between uploading the artifacts in our maven repository and pushing them further, and in this window I did quick smoke tests to see that everything works (with the help of Sorin and the rest of the QA team at XWiki SAS).
So, I propose this as the strategy for our next releases: - Each release goes to a staging repository first - Milestone releases spend a little time in staging, enough for a few quick smoke tests; depending on the time of day, this could last from an hour to at most a day - There are no RC releases, there are only RC staging repositories. Currently the staging repositories have random names, but the RM should rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a RC release goes through more intensive tests, including smoke testing, the full MTR, and developers that contributed to the release should test their committed features. Maybe even a Jenkins job could be executed for that release, with all the automated tests.
Timing? Since we don't know how many times we have to stage releases, I'd say that we can start building RCs one week before the planned final release date, and if bugs are found, a new RC is build the next day (to let issues accumulate), hoping that there will be a stable successful build before the final release date, without a 72h mandatory waiting period.
I have one big problem with this strategy that I've already explained several past emails. The current way of doing staging with RC is very powerful because it's a real release and as such is advertised everywhere: on xwiki.org in download, in blog post, on twitter, on wikipedia, freshmeat, etc. Which means we have a lot of users who can download it and try it out. This will not happen with staging. It won't be advertised significantly (it cannot since it's not a real release) thus leading to a less good solution than what we have. Thus the only valid solution for me to reduce release cycles is to do 2 things at the same time: - work on the quality of our releases (i.e. increase our automated test suite and test coverage) - do short release cycle so that if a user finds a problem in say version 4.45 then we release 4.46 the week after or 2 days after if it's critical (this is basically what Jenkins is doing) Right now our cycle is 3 months for each release which means that if some dev cannot make the date for ex he has to wait 3 months for his stuff to be available to end users, which means devs are pushing stuff even if they are half baked (missing tests for example). Thus, with the current situation, I'd very much prefer to not do staging (in the sense of Nexus staging) and instead continue to use milestones and RCs. Now we can still do staging as a quick check for the RM before pushing to the main repo but it shouldn't last more than 1 day, as otherwise it adds up with the RC time and thus leads to even longer release cycles. <brainstorming> Moving forward: what we could do would be to reduce the whole cycle from 3 months to 3 weeks: * 1 week milestone 1 * 1 week milestone 2 * 1 week final (we need less time for stabilization over what we have now since there's less work done in m1/m2 than when they last 4 weeks each) If a developer doesn't have time to push his stuff in m1 or m2 then he'll have to wait only 1 week. Of course we should have Roadmap that last several releases so that we don't do roadmap preparation all the time and so that we can put large features in the roadmap that span more than 1 release. Actually the roadmap would list priority stuff we want to work on but will not say in what exact version we'll have it. The advantage of this is we'll get even more testing/feedback from community since we'll have real releases and users will get features/improvements/bugfixes pushed to them faster. The challenge is to keep good quality but I think we can do that and in any case there's only one way to know if it works: to try it ;) I think that without Sergiu's automated script for releases doing a release was too costly to do this. This means doing 3 releases in 3 weeks (ie. one per week). Is the automated script good enough to allow for this. Also we're going to have automated upgrades very soon now so it seems like upgrading versions is going to be less an issue for users (note that they don't have to upgrade at each version and they can skip releases). WDYT? Do you think the pace is sustainable for developers and that we won't spend too much time in "bureaucracies"? IMO this would help us automate our release process even further so that it becomes a nobrainer and will help us improve ourselves wrt quality/automated tests even further. There's one problem maybe with APIs. They need time to mature but with the young-api strategy that could be ok. <brainstorming> Thanks -Vincent
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
-- Sergiu Dumitriu http://purl.org/net/sergiu/
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 06/09/2012 03:39 AM, Vincent Massol wrote:
On Jun 9, 2012, at 6:28 AM, Sergiu Dumitriu wrote:
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution.
The purpose of the RC releases was as a kind of staging, without actually using real staging. So I'd say that for the next releases we should only have milestones, and staged final releases.
However, for 4.1 we should keep the rc-1 release in the plan, since our staging process is new and not thoroughly tested yet.
The way I simulated "staging releases" when I had the RM hat was to leave a window between uploading the artifacts in our maven repository and pushing them further, and in this window I did quick smoke tests to see that everything works (with the help of Sorin and the rest of the QA team at XWiki SAS).
So, I propose this as the strategy for our next releases: - Each release goes to a staging repository first - Milestone releases spend a little time in staging, enough for a few quick smoke tests; depending on the time of day, this could last from an hour to at most a day - There are no RC releases, there are only RC staging repositories. Currently the staging repositories have random names, but the RM should rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a RC release goes through more intensive tests, including smoke testing, the full MTR, and developers that contributed to the release should test their committed features. Maybe even a Jenkins job could be executed for that release, with all the automated tests.
Timing? Since we don't know how many times we have to stage releases, I'd say that we can start building RCs one week before the planned final release date, and if bugs are found, a new RC is build the next day (to let issues accumulate), hoping that there will be a stable successful build before the final release date, without a 72h mandatory waiting period.
I have one big problem with this strategy that I've already explained several past emails.
The current way of doing staging with RC is very powerful because it's a real release and as such is advertised everywhere: on xwiki.org in download, in blog post, on twitter, on wikipedia, freshmeat, etc. Which means we have a lot of users who can download it and try it out. This will not happen with staging. It won't be advertised significantly (it cannot since it's not a real release) thus leading to a less good solution than what we have.
I've heard you mention this but I've not noticed any example of a user finding an issue in an RC and reporting it, causing it to be fixed for the final. I've only been around for 2 years though. Do you have such an example?
Thus the only valid solution for me to reduce release cycles is to do 2 things at the same time: - work on the quality of our releases (i.e. increase our automated test suite and test coverage)
This is good no matter how we release.
- do short release cycle so that if a user finds a problem in say version 4.45 then we release 4.46 the week after or 2 days after if it's critical (this is basically what Jenkins is doing)
Right now our cycle is 3 months for each release which means that if some dev cannot make the date for ex he has to wait 3 months for his stuff to be available to end users, which means devs are pushing stuff even if they are half baked (missing tests for example).
Thus, with the current situation, I'd very much prefer to not do staging (in the sense of Nexus staging) and instead continue to use milestones and RCs. Now we can still do staging as a quick check for the RM before pushing to the main repo but it shouldn't last more than 1 day, as otherwise it adds up with the RC time and thus leads to even longer release cycles.
This would make sense if "final" was a clone of rc-X. What we do now is build an RC, fix whatever issues we have with it then build "final" and release it without manual testing. I do not like releasing final without testing, especially since we use the name rc-1, leading users to believe that we have a candidate is tested then selected for release.
<brainstorming> Moving forward:
what we could do would be to reduce the whole cycle from 3 months to 3 weeks: * 1 week milestone 1 * 1 week milestone 2 * 1 week final (we need less time for stabilization over what we have now since there's less work done in m1/m2 than when they last 4 weeks each)
I like the general thinking but from 12 weeks to 3 weeks is a huge step, I think we should at least try 6 week cycle and then do post-release brainstorming to see what issues come up with shortened release times.
If a developer doesn't have time to push his stuff in m1 or m2 then he'll have to wait only 1 week.
Of course we should have Roadmap that last several releases so that we don't do roadmap preparation all the time and so that we can put large features in the roadmap that span more than 1 release. Actually the roadmap would list priority stuff we want to work on but will not say in what exact version we'll have it.
This is good, currently we encourage developers to delay the release.
The advantage of this is we'll get even more testing/feedback from community since we'll have real releases and users will get features/improvements/bugfixes pushed to them faster. The challenge is to keep good quality but I think we can do that and in any case there's only one way to know if it works: to try it ;)
If we do manual testing on final before publicizing it then we're already ahead of the current process. Thanks, Caleb
I think that without Sergiu's automated script for releases doing a release was too costly to do this. This means doing 3 releases in 3 weeks (ie. one per week). Is the automated script good enough to allow for this.
Also we're going to have automated upgrades very soon now so it seems like upgrading versions is going to be less an issue for users (note that they don't have to upgrade at each version and they can skip releases).
WDYT? Do you think the pace is sustainable for developers and that we won't spend too much time in "bureaucracies"?
IMO this would help us automate our release process even further so that it becomes a nobrainer and will help us improve ourselves wrt quality/automated tests even further.
There's one problem maybe with APIs. They need time to mature but with the young-api strategy that could be ok. <brainstorming>
Thanks -Vincent
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
-- Sergiu Dumitriu http://purl.org/net/sergiu/
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On Jun 9, 2012, at 9:37 PM, Caleb James DeLisle wrote:
On 06/09/2012 03:39 AM, Vincent Massol wrote:
On Jun 9, 2012, at 6:28 AM, Sergiu Dumitriu wrote:
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution.
The purpose of the RC releases was as a kind of staging, without actually using real staging. So I'd say that for the next releases we should only have milestones, and staged final releases.
However, for 4.1 we should keep the rc-1 release in the plan, since our staging process is new and not thoroughly tested yet.
The way I simulated "staging releases" when I had the RM hat was to leave a window between uploading the artifacts in our maven repository and pushing them further, and in this window I did quick smoke tests to see that everything works (with the help of Sorin and the rest of the QA team at XWiki SAS).
So, I propose this as the strategy for our next releases: - Each release goes to a staging repository first - Milestone releases spend a little time in staging, enough for a few quick smoke tests; depending on the time of day, this could last from an hour to at most a day - There are no RC releases, there are only RC staging repositories. Currently the staging repositories have random names, but the RM should rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a RC release goes through more intensive tests, including smoke testing, the full MTR, and developers that contributed to the release should test their committed features. Maybe even a Jenkins job could be executed for that release, with all the automated tests.
Timing? Since we don't know how many times we have to stage releases, I'd say that we can start building RCs one week before the planned final release date, and if bugs are found, a new RC is build the next day (to let issues accumulate), hoping that there will be a stable successful build before the final release date, without a 72h mandatory waiting period.
I have one big problem with this strategy that I've already explained several past emails.
The current way of doing staging with RC is very powerful because it's a real release and as such is advertised everywhere: on xwiki.org in download, in blog post, on twitter, on wikipedia, freshmeat, etc. Which means we have a lot of users who can download it and try it out. This will not happen with staging. It won't be advertised significantly (it cannot since it's not a real release) thus leading to a less good solution than what we have.
I've heard you mention this but I've not noticed any example of a user finding an issue in an RC and reporting it, causing it to be fixed for the final. I've only been around for 2 years though. Do you have such an example?
I have checked and there are quite a few. Here are some not reported by us in relatively recent releases: - http://jira.xwiki.org/jira/browse/XWIKI-7035 - http://jira.xwiki.org/jira/browse/XWIKI-7645 - http://jira.xwiki.org/jira/browse/XWIKI-7645 - http://jira.xwiki.org/jira/browse/XWIKI-7753 - http://jira.xwiki.org/jira/browse/XWIKI-7773 - http://jira.xwiki.org/jira/browse/XWIKI-7774 - http://jira.xwiki.org/jira/browse/XWIKI-7763
Thus the only valid solution for me to reduce release cycles is to do 2 things at the same time: - work on the quality of our releases (i.e. increase our automated test suite and test coverage)
This is good no matter how we release.
- do short release cycle so that if a user finds a problem in say version 4.45 then we release 4.46 the week after or 2 days after if it's critical (this is basically what Jenkins is doing)
Right now our cycle is 3 months for each release which means that if some dev cannot make the date for ex he has to wait 3 months for his stuff to be available to end users, which means devs are pushing stuff even if they are half baked (missing tests for example).
Thus, with the current situation, I'd very much prefer to not do staging (in the sense of Nexus staging) and instead continue to use milestones and RCs. Now we can still do staging as a quick check for the RM before pushing to the main repo but it shouldn't last more than 1 day, as otherwise it adds up with the RC time and thus leads to even longer release cycles.
This would make sense if "final" was a clone of rc-X. What we do now is build an RC, fix whatever issues we have with it then build "final" and release it without manual testing. I do not like releasing final without testing, especially since we use the name rc-1, leading users to believe that we have a candidate is tested then selected for release.
Yes, I understand, I had the same issue. However: 1) it's not like we have all the time in the world. This is a balance between testing/quality and developing new features/fixing bugs. So if you push one to take the other one. 2) we only include blocker bugs in RC and all devs have their eyes on commits (ie they do reviews) in RC phase 3) Although it can happen, it didn't happen very often. 4) It's not because we test that we find bugs 5) I remember one instance when our final had a big issue in a given domain. So we released a point release a few days after (2-3 days after). All in all since it doesn't happen often and since we can release a point release quickly I think we have a relatively good balance.
<brainstorming> Moving forward:
what we could do would be to reduce the whole cycle from 3 months to 3 weeks: * 1 week milestone 1 * 1 week milestone 2 * 1 week final (we need less time for stabilization over what we have now since there's less work done in m1/m2 than when they last 4 weeks each)
I like the general thinking but from 12 weeks to 3 weeks is a huge step, I think we should at least try 6 week cycle and then do post-release brainstorming to see what issues come up with shortened release times.
Sure, I'm all for doing experiments. That would be: * 2 weeks milestone 1 (note that the point of M1 is to have a milestone to check our progress) * 2 weeks milestone 2 * 1 week RC1 * 1 week Final Now we need to adapt the workload (ie roadmap ) accordingly of course.
If a developer doesn't have time to push his stuff in m1 or m2 then he'll have to wait only 1 week.
Of course we should have Roadmap that last several releases so that we don't do roadmap preparation all the time and so that we can put large features in the roadmap that span more than 1 release. Actually the roadmap would list priority stuff we want to work on but will not say in what exact version we'll have it.
This is good, currently we encourage developers to delay the release.
Well, I'd like to rephrase what I said because I know from experience that it leads to problems. When developers code they're always 80% finished and the hard part if finishing the remaining 20%. If you don't have a deadline you'll find yourself tuning and improving stuff close to infinitely and it'll take whatever time you have available. So we do need to put some deadlines. What we need to do is split large features into smaller ones and assign them to releases. We still need that plan.
The advantage of this is we'll get even more testing/feedback from community since we'll have real releases and users will get features/improvements/bugfixes pushed to them faster. The challenge is to keep good quality but I think we can do that and in any case there's only one way to know if it works: to try it ;)
If we do manual testing on final before publicizing it then we're already ahead of the current process.
Thanks -Vincent
Thanks,
Caleb
I think that without Sergiu's automated script for releases doing a release was too costly to do this. This means doing 3 releases in 3 weeks (ie. one per week). Is the automated script good enough to allow for this.
Also we're going to have automated upgrades very soon now so it seems like upgrading versions is going to be less an issue for users (note that they don't have to upgrade at each version and they can skip releases).
WDYT? Do you think the pace is sustainable for developers and that we won't spend too much time in "bureaucracies"?
IMO this would help us automate our release process even further so that it becomes a nobrainer and will help us improve ourselves wrt quality/automated tests even further.
There's one problem maybe with APIs. They need time to mature but with the young-api strategy that could be ok. <brainstorming>
Thanks -Vincent
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
Indeed users really do use RC versions. I was quite wrong in that regard. Your proposal to experiment with shorter cycle in order to relieve developers of the pressure to "make a release" of sounds good to me. We probably ought to have an official vote on the matter but you will have my +1. I feel bad that we will not be able to have staging although perhaps the week of time for final would allow for 2 days to be reserved for staging.. Another idea which would make the RM's job easier would be to shift the responsibility from the RM to ask the developers if they are ok to release to the developers to report tickets on JIRA (with fix-for the release version) if they are not. Thanks, Caleb On 06/12/2012 02:21 AM, Vincent Massol wrote:
On Jun 9, 2012, at 9:37 PM, Caleb James DeLisle wrote:
On 06/09/2012 03:39 AM, Vincent Massol wrote:
On Jun 9, 2012, at 6:28 AM, Sergiu Dumitriu wrote:
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution.
The purpose of the RC releases was as a kind of staging, without actually using real staging. So I'd say that for the next releases we should only have milestones, and staged final releases.
However, for 4.1 we should keep the rc-1 release in the plan, since our staging process is new and not thoroughly tested yet.
The way I simulated "staging releases" when I had the RM hat was to leave a window between uploading the artifacts in our maven repository and pushing them further, and in this window I did quick smoke tests to see that everything works (with the help of Sorin and the rest of the QA team at XWiki SAS).
So, I propose this as the strategy for our next releases: - Each release goes to a staging repository first - Milestone releases spend a little time in staging, enough for a few quick smoke tests; depending on the time of day, this could last from an hour to at most a day - There are no RC releases, there are only RC staging repositories. Currently the staging repositories have random names, but the RM should rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a RC release goes through more intensive tests, including smoke testing, the full MTR, and developers that contributed to the release should test their committed features. Maybe even a Jenkins job could be executed for that release, with all the automated tests.
Timing? Since we don't know how many times we have to stage releases, I'd say that we can start building RCs one week before the planned final release date, and if bugs are found, a new RC is build the next day (to let issues accumulate), hoping that there will be a stable successful build before the final release date, without a 72h mandatory waiting period.
I have one big problem with this strategy that I've already explained several past emails.
The current way of doing staging with RC is very powerful because it's a real release and as such is advertised everywhere: on xwiki.org in download, in blog post, on twitter, on wikipedia, freshmeat, etc. Which means we have a lot of users who can download it and try it out. This will not happen with staging. It won't be advertised significantly (it cannot since it's not a real release) thus leading to a less good solution than what we have.
I've heard you mention this but I've not noticed any example of a user finding an issue in an RC and reporting it, causing it to be fixed for the final. I've only been around for 2 years though. Do you have such an example?
I have checked and there are quite a few.
Here are some not reported by us in relatively recent releases: - http://jira.xwiki.org/jira/browse/XWIKI-7035 - http://jira.xwiki.org/jira/browse/XWIKI-7645 - http://jira.xwiki.org/jira/browse/XWIKI-7645 - http://jira.xwiki.org/jira/browse/XWIKI-7753 - http://jira.xwiki.org/jira/browse/XWIKI-7773 - http://jira.xwiki.org/jira/browse/XWIKI-7774 - http://jira.xwiki.org/jira/browse/XWIKI-7763
Thus the only valid solution for me to reduce release cycles is to do 2 things at the same time: - work on the quality of our releases (i.e. increase our automated test suite and test coverage)
This is good no matter how we release.
- do short release cycle so that if a user finds a problem in say version 4.45 then we release 4.46 the week after or 2 days after if it's critical (this is basically what Jenkins is doing)
Right now our cycle is 3 months for each release which means that if some dev cannot make the date for ex he has to wait 3 months for his stuff to be available to end users, which means devs are pushing stuff even if they are half baked (missing tests for example).
Thus, with the current situation, I'd very much prefer to not do staging (in the sense of Nexus staging) and instead continue to use milestones and RCs. Now we can still do staging as a quick check for the RM before pushing to the main repo but it shouldn't last more than 1 day, as otherwise it adds up with the RC time and thus leads to even longer release cycles.
This would make sense if "final" was a clone of rc-X. What we do now is build an RC, fix whatever issues we have with it then build "final" and release it without manual testing. I do not like releasing final without testing, especially since we use the name rc-1, leading users to believe that we have a candidate is tested then selected for release.
Yes, I understand, I had the same issue. However: 1) it's not like we have all the time in the world. This is a balance between testing/quality and developing new features/fixing bugs. So if you push one to take the other one. 2) we only include blocker bugs in RC and all devs have their eyes on commits (ie they do reviews) in RC phase 3) Although it can happen, it didn't happen very often. 4) It's not because we test that we find bugs 5) I remember one instance when our final had a big issue in a given domain. So we released a point release a few days after (2-3 days after).
All in all since it doesn't happen often and since we can release a point release quickly I think we have a relatively good balance.
<brainstorming> Moving forward:
what we could do would be to reduce the whole cycle from 3 months to 3 weeks: * 1 week milestone 1 * 1 week milestone 2 * 1 week final (we need less time for stabilization over what we have now since there's less work done in m1/m2 than when they last 4 weeks each)
I like the general thinking but from 12 weeks to 3 weeks is a huge step, I think we should at least try 6 week cycle and then do post-release brainstorming to see what issues come up with shortened release times.
Sure, I'm all for doing experiments. That would be:
* 2 weeks milestone 1 (note that the point of M1 is to have a milestone to check our progress) * 2 weeks milestone 2 * 1 week RC1 * 1 week Final
Now we need to adapt the workload (ie roadmap ) accordingly of course.
If a developer doesn't have time to push his stuff in m1 or m2 then he'll have to wait only 1 week.
Of course we should have Roadmap that last several releases so that we don't do roadmap preparation all the time and so that we can put large features in the roadmap that span more than 1 release. Actually the roadmap would list priority stuff we want to work on but will not say in what exact version we'll have it.
This is good, currently we encourage developers to delay the release.
Well, I'd like to rephrase what I said because I know from experience that it leads to problems. When developers code they're always 80% finished and the hard part if finishing the remaining 20%. If you don't have a deadline you'll find yourself tuning and improving stuff close to infinitely and it'll take whatever time you have available. So we do need to put some deadlines.
What we need to do is split large features into smaller ones and assign them to releases. We still need that plan.
The advantage of this is we'll get even more testing/feedback from community since we'll have real releases and users will get features/improvements/bugfixes pushed to them faster. The challenge is to keep good quality but I think we can do that and in any case there's only one way to know if it works: to try it ;)
If we do manual testing on final before publicizing it then we're already ahead of the current process.
Thanks -Vincent
Thanks,
Caleb
I think that without Sergiu's automated script for releases doing a release was too costly to do this. This means doing 3 releases in 3 weeks (ie. one per week). Is the automated script good enough to allow for this.
Also we're going to have automated upgrades very soon now so it seems like upgrading versions is going to be less an issue for users (note that they don't have to upgrade at each version and they can skip releases).
WDYT? Do you think the pace is sustainable for developers and that we won't spend too much time in "bureaucracies"?
IMO this would help us automate our release process even further so that it becomes a nobrainer and will help us improve ourselves wrt quality/automated tests even further.
There's one problem maybe with APIs. They need time to mature but with the young-api strategy that could be ok. <brainstorming>
Thanks -Vincent
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
> Hi, > > I'd like to add staging to our official release process. > For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). > For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue. > > Why: > > #1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need. > > #2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository". > > #3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time. > > #4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal. > > #5. Everybody else is doing it (is that even a reason?!) > > > Mandatory? > I would rather impress the RM with how easy and helpful staging can be than bind him with rules. > If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. > In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule. > > > Here's my +1 > > Caleb
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
On 06/09/2012 12:28 AM, Sergiu Dumitriu wrote:
On 06/08/2012 05:07 AM, Caleb James DeLisle wrote:
It seems that if we want to shorten a release cycle, we have 2 options:
#1 No RC release #2 No staging
I think it would be a shame to scrap staging over this, especially since my understanding is we want to move toward frequent releases with no release candidates or milestones so staging would be a requirement.
This is an interesting topic which can be discussed.
Right now for 4.1 I think we need a coherent proposal rather than an adhoc chop-and-slice of the agreed upon schedule and dates. Finally, I'm concerned about changing our release process in the middle of a release which is behind it's normal schedule. I'm not completely opposed to a change but IMO if we want to change it we need to proceed with extra caution.
The purpose of the RC releases was as a kind of staging, without actually using real staging. So I'd say that for the next releases we should only have milestones, and staged final releases.
However, for 4.1 we should keep the rc-1 release in the plan, since our staging process is new and not thoroughly tested yet.
+1 also I think changing the rules in the middle of a release misrepresents the seriousness of the project.
The way I simulated "staging releases" when I had the RM hat was to leave a window between uploading the artifacts in our maven repository and pushing them further, and in this window I did quick smoke tests to see that everything works (with the help of Sorin and the rest of the QA team at XWiki SAS).
So, I propose this as the strategy for our next releases: - Each release goes to a staging repository first
+1
- Milestone releases spend a little time in staging, enough for a few quick smoke tests; depending on the time of day, this could last from an hour to at most a day
+1
- There are no RC releases, there are only RC staging repositories. Currently the staging repositories have random names, but the RM should rename each repository with the proper RC name: 4.2-rc-1, 4.2-rc-2 if the first build was busted, and so on. The version written inside the packages is the final one, like 4.2, without a rc-x suffix. While in staging, a RC release goes through more intensive tests, including smoke testing, the full MTR, and developers that contributed to the release should test their committed features. Maybe even a Jenkins job could be executed for that release, with all the automated tests.
+1
Timing? Since we don't know how many times we have to stage releases, I'd say that we can start building RCs one week before the planned final release date, and if bugs are found, a new RC is build the next day (to let issues accumulate), hoping that there will be a stable successful build before the final release date, without a 72h mandatory waiting period.
+1 Since this has become a bit of a brainstorming session I think it makes sense to have another official vote. Thanks, Caleb
Thanks,
Caleb
On 06/08/2012 03:15 AM, Vincent Massol wrote:
Hi guys,
I'd like to bring an issue with this VOTE below.
When I initially read it I didn't realize that this was about doing double-staging: * once with nexus staging * another one with the RC release
So it increases the time we spend for doing releases instead of reducing it which is the direction we would like to go.
The increase is bad because we're already spending too much time just on the release itself while we should reduce it to a minimum so that we can focus on developing new features/improvements/fixing bugs.
So IMO if we really want to go with staging we need to remove the RC phase and go from M2 to Final directly. However if we were to do this we would need to find a way to advertise it as a release on all channels because this is the time when we need to most testers. Right now it seems to me that an official RC is much more powerful than staging
Thus I'd like to retract my vote on this (if it's not possible I'll send a new vote to not do double staging).
Thanks -Vincent
PS: Sorry for not realizing this earlier...
On May 22, 2012, at 10:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
I just realized that I forgot to close this vote, I see 3 +1's from committers and no 0 and no -1 so it looks passed. Thanks, Caleb On 05/22/2012 04:55 AM, Caleb James DeLisle wrote:
Hi,
I'd like to add staging to our official release process. For milestone releases, I propose the staging cycle be for "0 time" (this may be revisited later). For RC or finals, we place the release in staging and immediately call a VOTE to publish the release, this gives our testing team (everybody!) 72 hours to raise a potential issue.
Why:
#1. After some chat on IRC I decided that it is advantageous to move toward a faster release cycle and begin moving away from milestone releases in favor of staging. This will set the stage for the release method we will need.
#2. Staging is easy, I've modified the release script to include staging and with the script, it is a simple matter of about 5 clicks on nexus to "login", "close repository", "release repository".
#3. Staging is safe, the RM need not worry about fat fingers breaking the release, all it costs is time.
#4. The release process should be as close to the same as possible for milestone and RC/final releases. This simplifies scripting of the process, decreases the amount the RM must remember and makes every milestone release a rehearsal.
#5. Everybody else is doing it (is that even a reason?!)
Mandatory? I would rather impress the RM with how easy and helpful staging can be than bind him with rules. If I had followed the existing process to the letter, I would not have had any experience with staging to begin with. In the interest of continuous improvement I would like to make this a strong recommendation, not a strict rule.
Here's my +1
Caleb
_______________________________________________ devs mailing list devs@xwiki.org http://lists.xwiki.org/mailman/listinfo/devs
participants (6)
-
Caleb James DeLisle -
Eduard Moraru -
Sergiu Dumitriu -
Sorin Burjan -
Thomas Mortagne -
Vincent Massol