format-selector: Remove .sync() point

The feature listener callbacks are waited upon to finish in the
middle of the cluster joining process. I particular -- before
actually joining the cluster the format should have being selected.
For that there's a .sync() method that locks the semaphore thus
making sure that any update is finished and it's called right after
the wait_for_gossip_to_settle() finishes.

However, features are enabled inside the wait_for_gossip_to_settle()
in a seastar::async() context that's also waited upon to finish. This
waiting makes it possible for any feature listener to .get() any of
its futures that should be resolved until gossip is settled.

Said that, the format selection barrier can be moved -- instead of
waiting on the semaphore, the respective part of the selection code
can be .get()-ed (it all runs in async context). One thing to care
about -- the remainder should continue running with the gate held.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>

db/sstables-format-selector.cc

@@ -24,8 +24,7 @@ static const sstring SSTABLE_FORMAT_PARAM_NAME = "sstable_format";
 void feature_enabled_listener::on_enabled() {
     if (!_started) {
         _started = true;
-        // FIXME -- discarded future
-        (void)_selector.maybe_select_format(_format);
+        _selector.maybe_select_format(_format).get();
     }
 }
 
@@ -44,9 +43,9 @@ future<> sstables_format_selector::maybe_select_format(sstables::sstable_version
     if (new_format > _selected_format) {
         co_await db::system_keyspace::set_scylla_local_param(SSTABLE_FORMAT_PARAM_NAME, to_string(new_format));
         co_await select_format(new_format);
-        units.return_all();
-        co_await _gossiper.add_local_application_state(gms::application_state::SUPPORTED_FEATURES,
-                 gms::versioned_value::supported_features(_features.local().supported_feature_set()));
+        // FIXME discarded future
+        (void)_gossiper.add_local_application_state(gms::application_state::SUPPORTED_FEATURES,
+                 gms::versioned_value::supported_features(_features.local().supported_feature_set())).finally([h = std::move(hg)] {});
     }
 }
 

db/sstables-format-selector.hh

@@ -64,10 +64,6 @@ public:
     future<> stop();
 
     future<> maybe_select_format(sstables::sstable_version_types new_format);
-
-    void sync() {
-        get_units(_sem, 1).get0();
-    }
 };
 
 } // namespace sstables

main.cc

@@ -1289,7 +1289,6 @@ To start the scylla server proper, simply invoke as: scylla server (or just scyl
                 mm.local().passive_announce(std::move(schema_version));
             });
             gossiper.local().wait_for_gossip_to_settle().get();
-            sst_format_selector.sync();
 
             with_scheduling_group(maintenance_scheduling_group, [&] {
                 return ss.local().join_cluster();