bahman/scylla
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

locator: Do not enforce public ip address for broadcast_rpc_address

Browse Source 
Reported by Felipe Cardeneti:

- Create a 2-node Scylla cluster w/ Ec2MultiRegionSnitch
- Check system.peers table

Scylla (uses public address)
```
cqlsh> select peer,data_center,host_id,preferred_ip,rack,rpc_address,schema_version from system.peers;

peer          | data_center | host_id                              | preferred_ip  | rack | rpc_address   | schema_version
---------------+-------------+--------------------------------------+---------------+------+---------------+--------------------------------------
18.216.98.219 |   us-east-2 | d9443741-a12e-4bbb-91ce-9931cece589c | 172.31.43.122 |   2c | 18.216.98.219 | 95c3fca5-c463-3aba-98c6-1c0b3fac5b58

(1 rows)
```

Cassandra (uses local address):
```
cqlsh> SELECT peer,data_center,host_id,preferred_ip,rack,rpc_address,schema_version from system.peers;

peer          | data_center | host_id                              | preferred_ip  | rack       | rpc_address   | schema_version
---------------+-------------+--------------------------------------+---------------+------------+---------------+--------------------------------------
52.15.104.255 |   us-east-2 | 42c0b717-775f-4998-a420-0388fe8b4e70 | 172.31.42.126 | us-east-2c | 172.31.42.126 | 2207c2a9-f598-3971-986b-2926e09e239d

(1 rows)
```

Config diff:
```
cassandra.yaml:rpc_address: 0.0.0.0
cassandra.yaml:broadcast_rpc_address: 172.31.42.126
/etc/scylla/scylla.yaml:broadcast_rpc_address: 172.31.42.126
/etc/scylla/scylla.yaml:rpc_address: 0.0.0.0
```

After this patch, if broadcast_rpc_address is unset, Ec2MultiRegionSnitch
will use the public ip address to set broadcast_rpc_address. If
broadcast_rpc_address is set, Ec2MultiRegionSnitch will not modify it.

Fixes #10236

Closes #10519
This commit is contained in:
Asias He
2022-05-09 10:30:44 +08:00
committed by Kamil Braun
parent f703e8ded5
commit 77b1db475c
4 changed files with 8 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
locator/ec2_multi_region_snitch.cc
View File

@@ -20,7 +20,8 @@ static constexpr const char* PRIVATE_MAC_QUERY = "/latest/meta-data/network/inte
namespace locator {
ec2_multi_region_snitch::ec2_multi_region_snitch(const snitch_config& cfg)
: ec2_snitch(cfg) {}
: ec2_snitch(cfg)
, _broadcast_rpc_address_specified_by_user(cfg.broadcast_rpc_address_specified_by_user) {}
future<> ec2_multi_region_snitch::start() {
_state = snitch_state::initializing;
@@ -61,7 +62,9 @@ future<> ec2_multi_region_snitch::start() {
// value to a public address in cassandra.yaml.
//
utils::fb_utilities::set_broadcast_address(local_public_address);
utils::fb_utilities::set_broadcast_rpc_address(local_public_address);
if (!_broadcast_rpc_address_specified_by_user) {
utils::fb_utilities::set_broadcast_rpc_address(local_public_address);
}
if (!local_public_address.addr().is_ipv6()) {
sstring priv_addr = aws_api_call(AWS_QUERY_SERVER_ADDR, AWS_QUERY_SERVER_PORT, PRIVATE_IP_QUERY_REQ).get0();

1
locator/ec2_multi_region_snitch.hh
View File

@@ -24,5 +24,6 @@ public:
}
private:
sstring _local_private_address;
bool _broadcast_rpc_address_specified_by_user;
};
} // namespace locator

1
locator/snitch_base.hh
View File

@@ -45,6 +45,7 @@ struct snitch_config {
sstring name = "SimpleSnitch";
sstring properties_file_name = "";
unsigned io_cpu_id = 0;
bool broadcast_rpc_address_specified_by_user = false;
// GCE-specific
sstring gce_meta_server_url = "";

1
main.cc
View File

@@ -870,6 +870,7 @@ To start the scylla server proper, simply invoke as: scylla server (or just scyl
supervisor::notify("creating snitch");
snitch_config snitch_cfg;
snitch_cfg.name = cfg->endpoint_snitch();
snitch_cfg.broadcast_rpc_address_specified_by_user = !cfg->broadcast_rpc_address().empty();
sharded<locator::snitch_ptr>& snitch = i_endpoint_snitch::snitch_instance();
snitch.start(snitch_cfg, std::ref(gossiper)).get();
auto stop_snitch = defer_verbose_shutdown("snitch", [&snitch] {