bahman/scylla
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

network_topology_strategy: Estimate initial_tablets if 0 is set

Browse Source 
If user configured zero initial tablets (spoiler: or this value was set
automagically when enabling tablets begind the scenes) we still need
some value to start with and this patch calculates one.

The math is based on topology and RF so that all shards are covered:

initial_tablets = max(nr_shards_in(dc) / RF_in(dc) for dc in datacenters)

The estimation is done when a table is created, not when the keyspace is
created. For that, the keyspace is configured with zero initial tabled,
and table-creation time zero is converted into auto-estimated value.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
This commit is contained in:
Pavel Emelyanov
2023-12-11 16:18:07 +03:00
parent 423234841e
commit 4c4a9679d8
1 changed files with 34 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
locator/network_topology_strategy.cc
View File

@@ -281,8 +281,42 @@ effective_replication_map_ptr network_topology_strategy::make_replication_map(ta
return do_make_replication_map(table, shared_from_this(), std::move(tm), _rep_factor);
}
//
// Try to use as many tablets initially, so that all shards in the current topology
// are covered with at least one tablet. In other words, the value is
//
// initial_tablets = max(nr_shards_in(dc) / RF_in(dc) for dc in datacenters)
//
static unsigned calculate_initial_tablets_from_topology(const schema& s, const topology& topo, const std::unordered_map<sstring, size_t>& rf) {
unsigned initial_tablets = std::numeric_limits<unsigned>::min();
for (const auto& dc : topo.get_datacenter_endpoints()) {
unsigned shards_in_dc = 0;
unsigned rf_in_dc = 1;
for (const auto& ep : dc.second) {
const auto* node = topo.find_node(ep);
if (node != nullptr) {
shards_in_dc += node->get_shard_count();
}
}
if (auto it = rf.find(dc.first); it != rf.end()) {
rf_in_dc = it->second;
}
unsigned tablets_in_dc = (shards_in_dc + rf_in_dc - 1) / rf_in_dc;
initial_tablets = std::max(initial_tablets, tablets_in_dc);
}
rslogger.debug("Estimated {} initial tablets for table {}.{}", initial_tablets, s.ks_name(), s.cf_name());
return initial_tablets;
}
future<tablet_map> network_topology_strategy::allocate_tablets_for_new_table(schema_ptr s, token_metadata_ptr tm) const {
auto tablet_count = get_initial_tablets();
if (tablet_count == 0) {
tablet_count = calculate_initial_tablets_from_topology(*s, tm->get_topology(), _dc_rep_factor);
}
auto aligned_tablet_count = 1ul << log2ceil(tablet_count);
if (tablet_count != aligned_tablet_count) {
rslogger.info("Rounding up tablet count from {} to {} for table {}.{}", tablet_count, aligned_tablet_count, s->ks_name(), s->cf_name());