[jenkins] ha-functional-testing-weekly (build 261) completed with 51 failures

51/138 Stages failed: - aliyun rhel10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7025/ failed in deploy_cluster

- aliyun rhel10 zstream:no upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7009/ failed in deploy_cluster

- aliyun rhel10 zstream:no upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7001/ failed in deploy_cluster

- aliyun rhel10 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7028/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- aliyun rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6988/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- aliyun rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6993/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- aliyun rhel9 zstream:yes upstream:stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6971/ failed in deploy_cluster

- aws rhel10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7047/ failed in deploy_cluster

- aws rhel10 zstream:no upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7036/ failed in deploy_cluster

- aws rhel10 zstream:no upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7030/ failed in deploy_cluster

- aws rhel10 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7048/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- aws rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7008/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- aws rhel9 zstream:no upstream:stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6973/ - pacemaker,resource,IPv4,recovery (nodes: 4)

- aws rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7013/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- aws rhel9 zstream:yes upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7003/ - pacemaker,resource,IPv4,recovery (nodes: 3)

- aws rhel9 zstream:yes upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6970/ - pacemaker,resource,IPv4,recovery (nodes: 3)

- aws rhel9 zstream:yes upstream:stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6977/ - pacemaker,resource,IPv4,recovery (nodes: 2)

- azure rhel10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7053/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- azure rhel10 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7055/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- azure rhel8 zstream:no upstream:stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6938/ failed in deploy_cluster

- azure rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7011/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- azure rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7012/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7056/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel10 zstream:no upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7049/ - lvm,snapper,display_snap,nostack,singlenode (nodes: 1) - lvm,raid_sanity,display_raid,raid4,singlenode (nodes: 1)

- gcp rhel10 zstream:no upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7033/ - lvm,snapper,display_snap,nostack,singlenode (nodes: 1) - lvm,raid_sanity,display_raid,raid4,singlenode (nodes: 1) - pacemaker,recovery,RecoveryActiveNode (nodes: 3)

- gcp rhel10 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7057/ - lvm,snapper,display_snap,nostack,singlenode (nodes: 1) - lvm,raid_sanity,display_raid,raid4,singlenode (nodes: 1) - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel8 zstream:yes upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6932/ - pacemaker,recovery,RecoveryActiveNode (nodes: 3)

- gcp rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7010/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel9 zstream:no upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7004/ - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel9 zstream:no upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6957/ - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel9 zstream:no upstream:stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6965/ - lvm,snapper,display_snap,nostack,singlenode (nodes: 1) - lvm,raid_sanity,display_raid,raid4,singlenode (nodes: 1)

- gcp rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7016/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- gcp rhel9 zstream:yes upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6961/ - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- ibmvpc rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6989/ - setup (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- ibmvpc rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6992/ - setup (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4)

- libvirt centos10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7042/ failed in deploy_cluster

- libvirt rhel10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7035/ failed in deploy_cluster

- libvirt rhel10 zstream:no upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7020/ failed in deploy_cluster

- libvirt rhel10 zstream:no upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7015/ failed in deploy_cluster

- libvirt rhel10 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7037/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- libvirt rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6995/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- libvirt rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/6996/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- ocpv centos10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7051/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- ocpv rhel10 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7043/ failed in deploy_cluster

- ocpv rhel10 zstream:no upstream:next-stable tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7029/ failed in deploy_cluster

- ocpv rhel10 zstream:no upstream:none tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7024/ failed in deploy_cluster

- ocpv rhel10 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7044/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- ocpv rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7000/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- ocpv rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7002/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- osp rhel9 zstream:no upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7021/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

- osp rhel9 zstream:yes upstream:main tests:smoke: https://ci.kronosnet.org/job/global/job/ha-functional-testing/7022/ - setup (nodes: 2) - pacemaker,resource,IPv4,recovery (nodes: 2) - pacemaker,recovery,RecoveryActiveNode (nodes: 2) - setup,setup_sbd,sbd-only (nodes: 2) - setup (nodes: 3) - pacemaker,resource,IPv4,recovery (nodes: 3) - pacemaker,recovery,RecoveryActiveNode (nodes: 3) - setup,setup_sbd,sbd-only (nodes: 3) - setup (nodes: 4) - pacemaker,resource,IPv4,recovery (nodes: 4) - pacemaker,recovery,RecoveryActiveNode (nodes: 4) - setup,setup_sbd,sbd-only (nodes: 4)

Run reason: Started by user Fabio M. Di Nitto Total runtime: 12 hr

Full log: https://ci.kronosnet.org/job/global/job/ha-functional-testing-weekly/261/con...