"Fabio M. Di Nitto" fdinitto@redhat.com writes:
there is a PR open to merge stable1-proposed that will keep running CI on each cherry-pick backport for now, and we feel the time is ready to cut 1.1, we will just merge that PR at once.
So, this sounds like the point of stable1-proposed is to have CI, but I can't see why stable1 couldn't be set up the same way. What do I miss?
"Fabio M. Di Nitto" notifications@github.com also writes:
On 2/19/2018 10:43 AM, wferi wrote:
I don't really understand the master/stable1/stable1-proposed distinction at the moment, because I can't see anything on master which isn't 1.1 material, so we could pretty much release 1.1 straight from master.
https://lists.kronosnet.org/pipermail/devel/2018-January/000049.html
you are right, at the moment master and stable1-* haven´t diverged yet but they will in future .
It's getting an interesting question nearing the 1.1 release. Basically, why complicate history by pretending 1.1 is being developed in parallel with something else? Surely, there'll be a time when some breaking change (not suitable for 1.x) will have already been merged into master and still the need arises to cut a 1.x+1 release. Then we could branch off stable1 from master right before the breaking change and start doing parallel development when we're forced to. Till then, I'd find it perfectly fine to tag new minor releases on the master branch. Is that an oversimplification?
On 2/19/2018 6:24 PM, Ferenc Wágner wrote:
"Fabio M. Di Nitto" fdinitto@redhat.com writes:
there is a PR open to merge stable1-proposed that will keep running CI on each cherry-pick backport for now, and we feel the time is ready to cut 1.1, we will just merge that PR at once.
So, this sounds like the point of stable1-proposed is to have CI, but I can't see why stable1 couldn't be set up the same way. What do I miss?
"Fabio M. Di Nitto" notifications@github.com also writes:
On 2/19/2018 10:43 AM, wferi wrote:
I don't really understand the master/stable1/stable1-proposed distinction at the moment, because I can't see anything on master which isn't 1.1 material, so we could pretty much release 1.1 straight from master.
https://lists.kronosnet.org/pipermail/devel/2018-January/000049.html
you are right, at the moment master and stable1-* haven´t diverged yet but they will in future .
It's getting an interesting question nearing the 1.1 release. Basically, why complicate history by pretending 1.1 is being developed in parallel with something else? Surely, there'll be a time when some breaking change (not suitable for 1.x) will have already been merged into master and still the need arises to cut a 1.x+1 release. Then we could branch off stable1 from master right before the breaking change and start doing parallel development when we're forced to. Till then, I'd find it perfectly fine to tag new minor releases on the master branch. Is that an oversimplification?
This deserves a semi-long explanation as it touches many interconnected points in my head (and please note: MY HEAD ;))
let´s start from CI. right now CI is still limited to build and make check (or variations of check/distcheck/etc) + build rpms.
CI phase1 goal is to have all HA related packages (libqb/knet/corosync/pacemaker) build and make check depending on which dependency is changing here or there. You can see already that most of corosync has been already added to CI and it rebuilds as knet changes. phase1 covers unit testing and integration testing. Catch as early as possible any breakage between and within each component.
CI phase2 goal is to have functional testing integrated in CI. That is effectively run a series of CSTS HA tests tailored to catch operational issues in the stack. This has not been implemented yet and it will take some time to get there.
How does this fit with branching in knet?
all master branches of all projects will build and test together. From a CI implementation it´s very easy to trigger master with master.
when looking at stable branches around, the test matrix combination is slightly more complex. For example, corosync stable can be released at any given async time from knet, hence it needs to be tested on current stable (aka stable1) and next knet stable (stable1-proposed). As a side note, in jenkins it´s just a lot easier to manage that via git branches than tags within a branch btw (at least based on my poor knowledge of jenkins).
Now, that said, the combination of stable1 and stable1-proposed branches fits 2 more purposes (on top of making it easier to test other projects).
stable1 branch is always in a "can be released" state. It is built on top of the last stable release and nothing else. Let say we need to do an emergency release (CVE, or super major bug fix), we can literally just take that fix in stable1 branch and release immediately without having to worry about other changes landing in stable1.
stable1-proposed serves the purpose of staging area for changes that will go into stable1 at some point in time, but it gives more space for testing vs other projects.
As for tagging from master and branching (as you mention above), this is more a matter of personal preference. I don´t like GA releases out of master branches. I prefer to have a dedicated timeline for stable releases/branches.
You are absolutely right that at this point time there is no difference in any branch but there will be soon and having the branches created early (from 1.0) did allow me to setup the whole CI infrastructure ahead of diverging the code. Note here that changing CI is super time consuming and requires lots of testing too. So prepare early vs scramble later is a much better strategy :-)
hope this gives you a better overview :-)
Fabio
"Fabio M. Di Nitto" fdinitto@redhat.com writes:
On 2/19/2018 6:24 PM, Ferenc Wágner wrote:
Basically, why complicate history by pretending 1.1 is being developed in parallel with something else? Surely, there'll be a time when some breaking change (not suitable for 1.x) will have already been merged into master and still the need arises to cut a 1.x+1 release. Then we could branch off stable1 from master right before the breaking change and start doing parallel development when we're forced to. Till then, I'd find it perfectly fine to tag new minor releases on the master branch. Is that an oversimplification?
all master branches of all projects will build and test together. From a CI implementation it´s very easy to trigger master with master.
when looking at stable branches around, the test matrix combination is slightly more complex. For example, corosync stable can be released at any given async time from knet, hence it needs to be tested on current stable (aka stable1) and next knet stable (stable1-proposed).
stable1 branch is always in a "can be released" state. It is built on top of the last stable release and nothing else. Let say we need to do an emergency release (CVE, or super major bug fix), we can literally just take that fix in stable1 branch and release immediately without having to worry about other changes landing in stable1.
Thanks, Fabio, on second read this all makes perfect sense and I find it a well-engineered solution. The seemingly superfluous existence of all these branches is adequately explained by the needs of the CI setup.
What I still don't get is why stable1-proposed couldn't follow master during the 1.1 cycle. Concretely, why did you cherry-pick individual commits from master into stable1-proposed instead of just fast- forwarding stable1-proposed to master after each PR merge (since they all were 1.1 material)? I think that would have created a simpler representation of the development process. I agree that this method is not generally applicable, but why not exploit it if possible?
Another method would be to merge all 1.1 material into stable1-proposed only, and occasionally merge stable1-proposed into master to avoid excess divergence in presence of 2.0 work. This is a little more complicated but also more general workflow, and still avoids cherry- picking.
Yeah, I don't like cherry-picking, I find it hard to follow. Even git can't follow it (for example git branch --contains <hash> does not find cherry-picked "clones" of the <hash> commit). This "damage" could still be fixed to some extent by merging stable1-proposed into master now.
As for tagging from master and branching (as you mention above), this is more a matter of personal preference. I don´t like GA releases out of master branches. I prefer to have a dedicated timeline for stable releases/branches.
I admittedly don't understand this concern fully. Wouldn't the merging workflow resolve it?
On 2/25/2018 5:40 PM, Ferenc Wágner wrote:
"Fabio M. Di Nitto" fdinitto@redhat.com writes:
On 2/19/2018 6:24 PM, Ferenc Wágner wrote:
Basically, why complicate history by pretending 1.1 is being developed in parallel with something else? Surely, there'll be a time when some breaking change (not suitable for 1.x) will have already been merged into master and still the need arises to cut a 1.x+1 release. Then we could branch off stable1 from master right before the breaking change and start doing parallel development when we're forced to. Till then, I'd find it perfectly fine to tag new minor releases on the master branch. Is that an oversimplification?
all master branches of all projects will build and test together. From a CI implementation it´s very easy to trigger master with master.
when looking at stable branches around, the test matrix combination is slightly more complex. For example, corosync stable can be released at any given async time from knet, hence it needs to be tested on current stable (aka stable1) and next knet stable (stable1-proposed).
stable1 branch is always in a "can be released" state. It is built on top of the last stable release and nothing else. Let say we need to do an emergency release (CVE, or super major bug fix), we can literally just take that fix in stable1 branch and release immediately without having to worry about other changes landing in stable1.
Thanks, Fabio, on second read this all makes perfect sense and I find it a well-engineered solution. The seemingly superfluous existence of all these branches is adequately explained by the needs of the CI setup.
glad it doesn´t make sense only in my head :-)
What I still don't get is why stable1-proposed couldn't follow master during the 1.1 cycle. Concretely, why did you cherry-pick individual commits from master into stable1-proposed instead of just fast- forwarding stable1-proposed to master after each PR merge (since they all were 1.1 material)? I think that would have created a simpler representation of the development process. I agree that this method is not generally applicable, but why not exploit it if possible?
There are 3 main reasons why I didn´t just fast forward stable1-proposed:
1) as you say yourself fast forward can only be applied if the branches are the same. It´s a very specific condition and not generally applicable. That means creating special cases around the birth of a process that should be long-lasting and as generic as possible.
2) we had no idea that master would follow so closely 1.1 for the whole release cycle. I don´t expect this to last at all in future.
3) CI / jenkins and fast forward don´t play well together. This is already causing (minor) problems with corosync branch management. Technically speaking jenkins is monitoring both branches (say master and stableX), if both branches are on the same git sha, jenkins won´t trigger the rebuild and test of the fast forward branch (that makes perfect sense because it´s already been tested), but then it also doesn´t trigger the rebuilds of downstream jobs, and this creates a hole in our test matrix.
Another method would be to merge all 1.1 material into stable1-proposed only, and occasionally merge stable1-proposed into master to avoid excess divergence in presence of 2.0 work. This is a little more complicated but also more general workflow, and still avoids cherry- picking.
In my ideal world, all patches should always land in master first, then, based on the nature of the patch, either cherry-picked or backported into stableX-proposed. We can´t expect that all patches can be cherry-picked or merged as-is.
To be honest, merging, cherry-picking.. I don´t have a strong preference, but all I care is that when I do one or the other I don´t have to think too hard on which process I should apply. It has to be generic enough to last.
Yeah, I don't like cherry-picking, I find it hard to follow. Even git can't follow it (for example git branch --contains <hash> does not find cherry-picked "clones" of the <hash> commit). This "damage" could still be fixed to some extent by merging stable1-proposed into master now.
I honestly don´t see that a major problem. It´s relatively easy to figure out if a given sha1 has been incorporated in a branch.
As for tagging from master and branching (as you mention above), this is more a matter of personal preference. I don´t like GA releases out of master branches. I prefer to have a dedicated timeline for stable releases/branches.
I admittedly don't understand this concern fully. Wouldn't the merging workflow resolve it?
It´s not really a concern, just a matter of personal preference.
Cheers Fabio
-
Fabio M. Di Nitto -
wferi＠niif.hu