Theme
After one cluster node is updated, the other cannot be: a cluster update pass is running
You updated the nodes of a cluster one at a time, by hand: Apply when idle or Apply now on the card of the first node, on the Updates page of the SuperAdmin UI. That node updated, passed its health check and runs the new version. Then, on a cluster of two:
- The cluster section reads Cluster update in progress: node - (2 of 2), with no node named, and a time under the first node that stands still.
- On the console of the node you updated, the other node is marked Up to date with "There is nothing to install", although it plainly runs the older version, and the same card says that the nodes run different versions.
- On the console of the other node, that node correctly shows Update available, but applying it is refused:
An update is already in progress. A cluster update pass is running.
Per node actions are available again once it finishes.Nothing changes however long you wait.
This page applies to Syncplify Server! 8.4.0 through 8.12.0. Version 8.14.0 and newer do not do this, and they clear the state on their own if an older version left it behind.
Nothing is broken, and nothing is half done
The node you updated is fully updated and healthy. The other node has not been touched: it runs the older version, serves clients, and nothing on it is half applied. A cluster that runs two versions between the update of one node and the next is a supported state. Do not restart services and do not reboot either node: neither helps.
What to do
- On the Updates page of either node, choose Stop the cluster update.
- Open the console of the node that still runs the older version and choose Apply when idle or Apply now on its own card.
That is all. Stopping the cluster update undoes nothing: the node you already updated stays on the new version, and no node is restarted by it. It only closes a record that should have closed by itself.
If the console of the updated node still shows the other node as up to date after step 1, use the console of the other node for step 2, as written. On these versions each console decides what the other nodes need from what its own node needs, and a node that was just updated needs nothing.
When the version you are updating to is 8.14.0 or newer, step 1 is not needed: the first node closes its own record as it starts on the new version, and the other node can be updated right after it, from either console.
Why it happens
Nodes of a cluster are never updated at the same time. A node takes an update lease from every other node before it updates, and writes a small record that every console can read, so the lease is visible everywhere. For a node you update by hand that record names exactly one node: the node itself.
On 8.4.0 through 8.12.0 that record is closed only once every node of the cluster runs the new version. That is the right rule for an update of the whole cluster and the wrong one for a single node: after the node has finished and given the lease back, its record stays open for as long as any other node is behind, and while it is open every action on the other nodes is refused, which is exactly what keeps them behind. The consoles show the open record as a cluster update that is in progress at no node.
The time under the first node in the cluster section is not a countdown. It is how long that node's update took, and it stands still because it is a result.
Two more things these versions show that are not what they seem
- "at most 10 min, then it starts" beside the buttons of an update that waits for an idle moment. It describes what Apply now instead would do, not the wait you are in. An update that waits for idle has no time limit and never ends a session: it waits until the node has had no sessions and no transfers for a full minute, for as long as that takes. Sessions are ended only if you choose Apply now, after the grace period of the update policy (ten minutes by default).
- "Cluster update in progress: node X (1 of 2)" while the first node waits for its turn, even though you asked for that one node only. It is that node holding the update lease for its own update.
From 8.14.0
- The record of a node updated by hand closes as soon as that node is done. The other nodes can be updated right away, from their own cards on any console or with Update cluster, which then updates only the nodes that are behind. Nothing has to be stopped in between.
- Every console marks a node Update available when it runs something older than the newest version the cluster knows of, so the console of the node you just updated shows the others as behind and offers to update them.
- A node updating by hand reads Node update in progress, not a cluster update, and the cluster section always names the node whose turn it is.
- The time of a finished node carries a check mark, and nothing on the card of an update that waits for idle states a time limit.
- A record left open by an older version is closed automatically within seconds of the first node starting on 8.14.0 or newer.
If you contact support
Send a screenshot of the Updates page of both nodes (each console shows its own view), the output of ss-webrest update status on both, and the ss-webrest log of both nodes covering the hour of the update. Run the command as administrator on Windows (from C:\Program Files\Syncplify Server!) or with sudo on Linux (from /opt/Syncplify/Server/bin).
Related: Cluster update stays at the last node, for a cluster update started with Update cluster whose last node never takes its turn.