bahman/scylla
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

storage_service: do not inherit the host_id of a replaced a node

Browse Source 
We want to always be able to distinguish between
the replacing node and the replacee by using different,
unique, host identifiers.

This will allow us to use the host_id authoritatively
to identify the node (rather then its endpoint ip address)
for token mapping and node operations.

Also, it will be used in the following patch to never allow the
replaced node to rejoin the cluster, as its host_id should never
be reused.

Refs #9839
Refs #12040

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
This commit is contained in:
Benny Halevy
2022-11-23 11:59:37 +02:00
parent 7197757750
commit 9942c60d93
1 changed files with 8 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
service/storage_service.cc
View File

@@ -391,9 +391,12 @@ future<> storage_service::join_token_ring(cdc::generation_service& cdc_gen_servi
// Seed the host ID-to-endpoint map with our own ID.
auto local_host_id = _db.local().get_config().host_id;
if (!replacing_a_node_with_diff_ip) {
// Replacing node with a different ip should own the host_id only after
// the replacing node becomes NORMAL status. It is updated in
// handle_state_normal().
auto endpoint = get_broadcast_address();
auto eps = _gossiper.get_endpoint_state_for_endpoint_ptr(endpoint);
if (eps) {
auto replace_host_id = _gossiper.get_host_id(get_broadcast_address());
slogger.info("Host {}/{} is replacing {}/{} using the same address", local_host_id, endpoint, replace_host_id, endpoint);
}
tmptr->update_host_id(local_host_id, get_broadcast_address());
}
@@ -1732,10 +1735,8 @@ storage_service::prepare_replacement_info(std::unordered_set<gms::inet_address>
auto dc_rack = get_dc_rack_for(replace_address);
auto raft_id = get_raft_id_for(_gossiper, replace_address);
// use the replacee's host Id as our own so we receive hints, etc
auto host_id = _gossiper.get_host_id(replace_address);
co_await _sys_ks.local().set_local_host_id(host_id).discard_result();
const_cast<db::config&>(_db.local().get_config()).host_id = host_id; // FIXME -- carry non-cost config on storage service itself
auto replace_host_id = _gossiper.get_host_id(replace_address);
slogger.info("Host {}/{} is replacing {}/{} using a different address", _db.local().get_config().host_id, get_broadcast_address(), replace_host_id, replace_address);
co_await _gossiper.reset_endpoint_state_map();
co_return replacement_info {