19 Feb
2018
19 Feb
'18
5:24 p.m.
"Fabio M. Di Nitto" <fdinitto(a)redhat.com> writes:
there is a PR open to merge stable1-proposed that will
keep running CI
on each cherry-pick backport for now, and we feel the time is ready to
cut 1.1, we will just merge that PR at once.
So, this sounds like the point of stable1-proposed is to have CI, but I
can't see why stable1 couldn't be set up the same way. What do I miss?
"Fabio M. Di Nitto" <notifications(a)github.com> also writes:
On 2/19/2018 10:43 AM, wferi wrote:
It's getting an interesting question nearing the 1.1 release.
Basically, why complicate history by pretending 1.1 is being developed
in parallel with something else? Surely, there'll be a time when some
breaking change (not suitable for 1.x) will have already been merged
into master and still the need arises to cut a 1.x+1 release. Then we
could branch off stable1 from master right before the breaking change
and start doing parallel development when we're forced to. Till then,
I'd find it perfectly fine to tag new minor releases on the master
branch. Is that an oversimplification?
--
Regards,
Feri
I don't really understand the
master/stable1/stable1-proposed
distinction at the moment, because I can't see anything on master
which isn't 1.1 material, so we could pretty much release 1.1
straight from master.
https://lists.kronosnet.org/pipermail/devel/2018-January/000049.html
you are right, at the moment master and stable1-* haven´t diverged yet
but they will in future . 
20 Feb
20 Feb
5:59 a.m.
New subject: stable1 backporting process
On 2/19/2018 6:24 PM, Ferenc Wágner wrote:
"Fabio M. Di Nitto"
<fdinitto(a)redhat.com> writes:
This deserves a semi-long explanation as it touches many interconnected
points in my head (and please note: MY HEAD ;))
let´s start from CI. right now CI is still limited to build and make
check (or variations of check/distcheck/etc) + build rpms.
CI phase1 goal is to have all HA related packages
(libqb/knet/corosync/pacemaker) build and make check depending on which
dependency is changing here or there. You can see already that most of
corosync has been already added to CI and it rebuilds as knet changes.
phase1 covers unit testing and integration testing. Catch as early as
possible any breakage between and within each component.
CI phase2 goal is to have functional testing integrated in CI. That is
effectively run a series of CSTS HA tests tailored to catch operational
issues in the stack. This has not been implemented yet and it will take
some time to get there.
How does this fit with branching in knet?
all master branches of all projects will build and test together. From a
CI implementation it´s very easy to trigger master with master.
when looking at stable branches around, the test matrix combination is
slightly more complex. For example, corosync stable can be released at
any given async time from knet, hence it needs to be tested on current
stable (aka stable1) and next knet stable (stable1-proposed). As a side
note, in jenkins it´s just a lot easier to manage that via git branches
than tags within a branch btw (at least based on my poor knowledge of
jenkins).
Now, that said, the combination of stable1 and stable1-proposed branches
fits 2 more purposes (on top of making it easier to test other projects).
stable1 branch is always in a "can be released" state. It is built on
top of the last stable release and nothing else. Let say we need to do
an emergency release (CVE, or super major bug fix), we can literally
just take that fix in stable1 branch and release immediately without
having to worry about other changes landing in stable1.
stable1-proposed serves the purpose of staging area for changes that
will go into stable1 at some point in time, but it gives more space for
testing vs other projects.
As for tagging from master and branching (as you mention above), this is
more a matter of personal preference. I don´t like GA releases out of
master branches. I prefer to have a dedicated timeline for stable
releases/branches.
You are absolutely right that at this point time there is no difference
in any branch but there will be soon and having the branches created
early (from 1.0) did allow me to setup the whole CI infrastructure ahead
of diverging the code. Note here that changing CI is super time
consuming and requires lots of testing too. So prepare early vs scramble
later is a much better strategy :-)
hope this gives you a better overview :-)
Fabio
there is a PR open to merge stable1-proposed that
will keep running CI
on each cherry-pick backport for now, and we feel the time is ready to
cut 1.1, we will just merge that PR at once.
So, this sounds like the point of stable1-proposed is to have CI, but I
can't see why stable1 couldn't be set up the same way. What do I miss?
"Fabio M. Di Nitto" <notifications(a)github.com> also writes:
On 2/19/2018 10:43 AM, wferi wrote:
It's getting an interesting question nearing the 1.1 release.
Basically, why complicate history by pretending 1.1 is being developed
in parallel with something else? Surely, there'll be a time when some
breaking change (not suitable for 1.x) will have already been merged
into master and still the need arises to cut a 1.x+1 release. Then we
could branch off stable1 from master right before the breaking change
and start doing parallel development when we're forced to. Till then,
I'd find it perfectly fine to tag new minor releases on the master
branch. Is that an oversimplification?
I don't really understand the
master/stable1/stable1-proposed
distinction at the moment, because I can't see anything on master
which isn't 1.1 material, so we could pretty much release 1.1
straight from master.
https://lists.kronosnet.org/pipermail/devel/2018-January/000049.html
you are right, at the moment master and stable1-* haven´t diverged yet
but they will in future .
25 Feb
25 Feb
4:40 p.m.
New subject: stable1 backporting process
"Fabio M. Di Nitto" <fdinitto(a)redhat.com> writes:
On 2/19/2018 6:24 PM, Ferenc Wágner wrote:
Thanks, Fabio, on second read this all makes perfect sense and I find it
a well-engineered solution. The seemingly superfluous existence of all
these branches is adequately explained by the needs of the CI setup.
What I still don't get is why stable1-proposed couldn't follow master
during the 1.1 cycle. Concretely, why did you cherry-pick individual
commits from master into stable1-proposed instead of just fast-
forwarding stable1-proposed to master after each PR merge (since they
all were 1.1 material)? I think that would have created a simpler
representation of the development process. I agree that this method is
not generally applicable, but why not exploit it if possible?
Another method would be to merge all 1.1 material into stable1-proposed
only, and occasionally merge stable1-proposed into master to avoid
excess divergence in presence of 2.0 work. This is a little more
complicated but also more general workflow, and still avoids cherry-
picking.
Yeah, I don't like cherry-picking, I find it hard to follow. Even git
can't follow it (for example git branch --contains <hash> does not find
cherry-picked "clones" of the <hash> commit). This "damage"
could still
be fixed to some extent by merging stable1-proposed into master now.
Basically, why complicate history by pretending
1.1 is being developed
in parallel with something else? Surely, there'll be a time when some
breaking change (not suitable for 1.x) will have already been merged
into master and still the need arises to cut a 1.x+1 release. Then we
could branch off stable1 from master right before the breaking change
and start doing parallel development when we're forced to. Till then,
I'd find it perfectly fine to tag new minor releases on the master
branch. Is that an oversimplification?
all master branches of all projects will build and test together. From a
CI implementation it´s very easy to trigger master with master.
when looking at stable branches around, the test matrix combination is
slightly more complex. For example, corosync stable can be released at
any given async time from knet, hence it needs to be tested on current
stable (aka stable1) and next knet stable (stable1-proposed).
stable1 branch is always in a "can be released" state. It is built on
top of the last stable release and nothing else. Let say we need to do
an emergency release (CVE, or super major bug fix), we can literally
just take that fix in stable1 branch and release immediately without
having to worry about other changes landing in stable1. As for tagging from master and branching (as you
mention above), this is
more a matter of personal preference. I don´t like GA releases out of
master branches. I prefer to have a dedicated timeline for stable
releases/branches.
I admittedly don't understand this concern fully. Wouldn't the merging
workflow resolve it?
--
Regards and a happy 1.1,
Feri :)
26 Feb
26 Feb
6:30 a.m.
New subject: stable1 backporting process
On 2/25/2018 5:40 PM, Ferenc Wágner wrote:
It´s not really a concern, just a matter of personal preference.
Cheers
Fabio
"Fabio M. Di Nitto"
<fdinitto(a)redhat.com> writes:
glad it doesn´t make sense only in my head :-)
On 2/19/2018 6:24 PM, Ferenc Wágner wrote:
Thanks, Fabio, on second read this all makes perfect sense and I find it
a well-engineered solution. The seemingly superfluous existence of all
these branches is adequately explained by the needs of the CI setup. Basically, why complicate history by pretending
1.1 is being developed
in parallel with something else? Surely, there'll be a time when some
breaking change (not suitable for 1.x) will have already been merged
into master and still the need arises to cut a 1.x+1 release. Then we
could branch off stable1 from master right before the breaking change
and start doing parallel development when we're forced to. Till then,
I'd find it perfectly fine to tag new minor releases on the master
branch. Is that an oversimplification?
all master branches of all projects will build and test together. From a
CI implementation it´s very easy to trigger master with master.
when looking at stable branches around, the test matrix combination is
slightly more complex. For example, corosync stable can be released at
any given async time from knet, hence it needs to be tested on current
stable (aka stable1) and next knet stable (stable1-proposed).
stable1 branch is always in a "can be released" state. It is built on
top of the last stable release and nothing else. Let say we need to do
an emergency release (CVE, or super major bug fix), we can literally
just take that fix in stable1 branch and release immediately without
having to worry about other changes landing in stable1.
What I still don't get is why stable1-proposed couldn't follow master
during the 1.1 cycle. Concretely, why did you cherry-pick individual
commits from master into stable1-proposed instead of just fast-
forwarding stable1-proposed to master after each PR merge (since they
all were 1.1 material)? I think that would have created a simpler
representation of the development process. I agree that this method is
not generally applicable, but why not exploit it if possible?
There are 3 main reasons why I didn´t just fast forward stable1-proposed:
1) as you say yourself fast forward can only be applied if the branches
are the same. It´s a very specific condition and not generally
applicable. That means creating special cases around the birth of a
process that should be long-lasting and as generic as possible.
2) we had no idea that master would follow so closely 1.1 for the whole
release cycle. I don´t expect this to last at all in future.
3) CI / jenkins and fast forward don´t play well together. This is
already causing (minor) problems with corosync branch management.
Technically speaking jenkins is monitoring both branches (say master and
stableX), if both branches are on the same git sha, jenkins won´t
trigger the rebuild and test of the fast forward branch (that makes
perfect sense because it´s already been tested), but then it also
doesn´t trigger the rebuilds of downstream jobs, and this creates a hole
in our test matrix.
Another method would be to merge all 1.1 material into stable1-proposed
only, and occasionally merge stable1-proposed into master to avoid
excess divergence in presence of 2.0 work. This is a little more
complicated but also more general workflow, and still avoids cherry-
picking.
In my ideal world, all patches should always land in master first, then,
based on the nature of the patch, either cherry-picked or backported
into stableX-proposed. We can´t expect that all patches can be
cherry-picked or merged as-is.
To be honest, merging, cherry-picking.. I don´t have a strong
preference, but all I care is that when I do one or the other I don´t
have to think too hard on which process I should apply. It has to be
generic enough to last.
Yeah, I don't like cherry-picking, I find it hard to follow. Even git
can't follow it (for example git branch --contains <hash> does not find
cherry-picked "clones" of the <hash> commit). This "damage"
could still
be fixed to some extent by merging stable1-proposed into master now.
I honestly don´t see that a major problem. It´s relatively easy to
figure out if a given sha1 has been incorporated in a branch.
As for tagging from master and branching (as you
mention above), this is
more a matter of personal preference. I don´t like GA releases out of
master branches. I prefer to have a dedicated timeline for stable
releases/branches.
I admittedly don't understand this concern fully. Wouldn't the merging
workflow resolve it?
2262
days inactive
2269
days old
3 comments
2 participants
participants (2)
-
Fabio M. Di Nitto -
wferi＠niif.hu