bahman/scylla
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

topology_coordinator: tablets: Do not fail in a tight loop

Browse Source 
If streaming or cleanup RPC fails, we would retry immediately. That
fills the logs with erorrs. Throttle them by sleeping on error before
the same action is retried.
This commit is contained in:
Tomasz Grabiec
2024-01-19 20:20:28 +01:00
parent a3f6682ba2
commit bdd5bdae14
1 changed files with 10 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
service/storage_service.cc
View File

@@ -1986,8 +1986,16 @@ class topology_coordinator : public endpoint_lifecycle_subscriber {
// Prevent warnings about abandoned failed future. Logged below.
holder->ignore_ready_future();
}
holder = futurize_invoke(action)
.finally([this, g = _async_gate.hold(), gid, name] () noexcept {
holder = futurize_invoke(action).then_wrapped([this, gid, name] (future<> f) {
if (f.failed()) {
auto ep = f.get_exception();
rtlogger.error("{} for tablet {} failed: {}", name, gid, ep);
return seastar::sleep_abortable(std::chrono::seconds(1), _as).then([ep] () mutable {
std::rethrow_exception(ep);
});
}
return f;
}).finally([this, g = _async_gate.hold(), gid, name] () noexcept {
rtlogger.debug("{} for tablet {} resolved.", name, gid);
_tablets_ready = true;
_topo_sm.event.broadcast();