Commit Graph

673 Commits

Author SHA1 Message Date
Vlad Zolotarov
33b195760b gms::gossiper: allow the modification of _subscribers while it's being iterated
Introduce a subscribers_list class that exposes 3 methods:
  - push_back(s) - adds a new element s to the back of the list
  - remove(s) - removes an element s from the list
  - for_each(f) - invoke f on each element of the list
  - make a subscriber_list store shared_ptr to a subscriber
    to allow removing (currently it stores a naked pointer to the object).

subscribers_list allows push_back() and remove() to be called while
another thread (e.g. seastar::async()) is in the middle of for_each().

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Simplify subscribers_list::remove() method.
   - load_broadcaster: inherit from enable_shared_from_this instead
     of async_sharded_service.
2015-10-30 00:16:16 +02:00
Vlad Zolotarov
f70aab2fbb locator: added ec2_multi_region_snitch
This snitch in addition to what EC2Snitch does registers
a reconnectable_snitch_helper that will make messenger_service
connect to internal IPs when it connects to the nodes in the same
data center with the current Node.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v4:
   - Added dual license in newly added files.

New in v3:
   - Returned the Apache license.

New in v2:
   - Update the license to the latest version. ;)
2015-10-26 14:10:47 +02:00
Vlad Zolotarov
a182a33d4d locator::snitch_base: added i_endpoint_snitch::set_local_private_addr()
Sets the value of the local private IP address.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-26 14:10:39 +02:00
Vlad Zolotarov
1bb91399cd locator: added reconnectable_snitch_helper
reconnectable_snitch_helper implements i_endpoint_state_change_subscriber
and triggers reconnect using the internal IP to the nodes in the
same data center when one of the following events happen:

   - on_join()
   - on_change() - when INTERNAL_IP state is changed
   - on_alive()

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v4:
   - Added dual license for newly added files.

New in v3:
   - Fix reconnect() logic.
   - Returned the Apache license.
   - Check if the new local address is not already stored in the cache.
   - Get rid of get_ep_addr().

New in v2:
   - Update the license to the latest version. ;)
2015-10-26 14:09:48 +02:00
Vlad Zolotarov
d683d9cfb0 locator::ec2_snitch: moved info parsing and distribution into the separate function
Added load_config() function that reads AWS info and property file
and distributes the read values on all shards.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-26 14:09:26 +02:00
Vlad Zolotarov
703807b67d locator::ec2_snitch: rename ZONE_QUERY_SERVER_ADDR -> AWS_QUERY_SERVER_ADDR
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-26 14:09:26 +02:00
Asias He
142f29483a token_metadata: Implement add_leaving_endpoint 2015-10-20 21:32:30 +08:00
Asias He
937474bf14 abstract_replication_strategy: Make calculate_natural_endpoints public
It is used by storage_service.
2015-10-20 21:32:30 +08:00
Asias He
c96bc8bbd2 token_metadata: Implement calculate_pending_ranges 2015-10-20 21:32:14 +08:00
Asias He
a6065397d9 token_metadata: Implement clone_after_all_left 2015-10-20 20:38:43 +08:00
Vlad Zolotarov
fe5e983152 locator::gossiping_property_file_snitch: do_withicate file descriptor in property_file_was_modified()
file::stat() requires "this" to be valid will the end of the call.

Fixes issue #464

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-18 16:58:27 +03:00
Vlad Zolotarov
15606fc8d3 locator::production_snitch_base: fix production_snitch_base signature
Make production_snitch_base constructor signature consistent with
the rest of production snitches.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-18 16:58:27 +03:00
Vlad Zolotarov
5712aa4d48 locator::production_snitch_base: cleanup
Don't keep the file descriptor beyond load_property_file() scope.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-18 16:58:27 +03:00
Vlad Zolotarov
f113a57ba1 locator::gossiping_property_file_snitch: use empty string as a default config file name
A non-empty default value of a configuration file name was preventing
the db::config::get_conf_dir() to kick in when a default snitch constructor
was used (this is the way it's always used from scylla).

Fixes issue #459

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-18 16:56:12 +03:00
Asias He
e92a759dad locator: Add debug info for abstract_replication_strategy::get_address_ranges
It is useful to check relations between token and its primary range.
2015-10-13 15:46:36 +08:00
Asias He
b1c92f377d token_metadata: Add get_pending_ranges
One version returns only the ranges
   std::vector<range<token>>

Another version returns a map
   std::unordered_map<range<token>, std::unordered_set<inet_address>>
which is converted from
   std::unordered_multimap<range<token>, inet_address>

They are needed by token_metadata::pending_endpoints_for,
storage_service::get_all_ranges_with_strict_sources_for and
storage_service::decommission.
2015-10-13 15:45:55 +08:00
Asias He
b860f6a393 token_metadata: Add get_pending_ranges_mm
Helper for get_pending_ranges.
2015-10-13 15:45:55 +08:00
Asias He
c96d826fe0 token_metadata: Introduce _pending_ranges member 2015-10-13 15:45:55 +08:00
Asias He
da072b8814 token_metadata: Remove duplicated sortedTokens
It is implemented already.
2015-10-13 15:45:55 +08:00
Asias He
d820c83141 locator: Add abstract_replication_strategy::get_pending_address_ranges
Given the current token_metadata and the new token which will be
inserted into the ring after bootstrap, calculate the ranges this new
node will be responsible for.

This is needed by boot_strapper::bootstrap().
2015-10-13 15:45:55 +08:00
Asias He
1adb27e283 token_metadata: Add clone_only_token_map
Needed by get_pending_address_ranges.
2015-10-13 15:45:55 +08:00
Asias He
527edd69ae locator: Add abstract_replication_strategy::get_range_addresses
Needed by range_streamer::get_all_ranges_with_sources_for.
2015-10-13 15:45:55 +08:00
Asias He
044dcf43de locator: Add abstract_replication_strategy::get_address_ranges
Needed by get_pending_address_ranges.
2015-10-13 15:45:55 +08:00
Asias He
3d0d02816d token_metadata: Add get_primary_ranges_for and get_primary_range_for
Given tokens, return ranges the tokens present. For example, with t1 and
t2, it returns ranges:

(token before t1, t1]
(token before t2, t2]
2015-10-13 15:45:55 +08:00
Asias He
542b1394d7 token_metadata: Add get_predecessor
It is used to get the previous token of this token in the ring.
2015-10-13 15:45:55 +08:00
Asias He
ddfd417c13 locator: Make calculate_natural_endpoints take extra token_metadata parameter
When adding/removing a node, we need to use a temporary token_metadata
with pending tokens.
2015-10-13 15:45:55 +08:00
Vlad Zolotarov
38f77bbfe5 locator: add ec2_snitch
This snitch will read the EC2 availability zone and set the DC
and RACK as follows:

If availability zone is "us-east-1d", then
DC="us-east" and  RACK="1d".

If cassandra-rackdc.properties contains "dc_suffix" field then
DC will be appended with its value.

For instance if dc_suffix=_1_cassandra, then in the example above

DC=us-east_1_cassandra

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:20 +03:00
Vlad Zolotarov
afd44a6e08 locator::gossiping_property_file_snitch: initialize i_endpoint_snitch::io_cpu_id() in the constructor
This is just cleaner.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:19 +03:00
Vlad Zolotarov
2febae90c9 locator::production_snitch_base: unify property file parsing facilities
- Move property file parsing code into production_snitch_base class.
   - Make a parsing code more general:
      - Save the parsed keys in the hash table.
      - Check only two types of errors:
         - Repeating keys.
         - Add a set of all supported keys and add the check for a key
           being supported.
   - Added production_snitch_base.cc file.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:19 +03:00
Vlad Zolotarov
2db219f074 locator::gossiping_property_file_snitch: remove extra "namespace locator"
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:19 +03:00
Vlad Zolotarov
0cfdca55f3 locator::gossiping_property_file_snitch: make get_name() public as it should be
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:19 +03:00
Vlad Zolotarov
ba68436f2f locator::gossiping_property_file_snitch: get rid of warn() and err() wrappers
Use logger() accessor instead for a better resemblance with the Origin.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:19 +03:00
Vlad Zolotarov
d196b034e2 locator::snitch_base: Add a default snitch_base::stop() method
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-10-08 20:57:19 +03:00
Avi Kivity
d5cf0fb2b1 Add license notices 2015-09-20 10:43:39 +03:00
Avi Kivity
987294a412 Add missing copyrights 2015-09-20 10:16:11 +03:00
Pekka Enberg
47b84d9660 locator: Fix minor formatting issues
Fix minor formatting issues introduced in commit
b8211c436b
("locator/abstract_replication_strategy: Add validate_options() helper").

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-09-01 09:45:15 +03:00
Pekka Enberg
5a9cff9dc0 locator/abstract_replication_strategy: Add validate_replication_strategy() helper
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-31 11:54:56 +03:00
Pekka Enberg
b8211c436b locator/abstract_replication_strategy: Add validate_options() helper
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-31 11:54:56 +03:00
Pekka Enberg
cff9eb520b locator/abstract_replication_strategy: Add recognized_options() helper
Add a helper function for obtaining a vector of supported replication
strategy options. This is needed for validation.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-08-31 11:54:31 +03:00
Amnon Heiman
6d875f1ec1 token_metadata: Add const when applicable
This patch adds a const version for get_datacenter_endpoints and
get_topology.

It modified the token iterator to use a const version of token_metadata
and it make first_token, first_token_index, tokens_end and ring_range to
be a const method.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Amnon Heiman
3d27fa442b strategy: make some of the function const
This patch adds const for some of the methods that are not actually
changing the object content.

This is useful when using a const reference to the object.

calculate_natural_endpoints and has_sufficient_replicas where set as
const. The override was added where it was missing.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Nadav Har'El
e7ef6149d7 abstract_replication_strategy: add get_primary_ranges() method
Add a abstract_replication_strategy::get_primary_ranges() method, which is
very similar to the existing get_ranges(), except that only the "primary"
owner of each range will return it in its list.

This is needed for the "primary range" repair option, which asks to repair
only the primary range. This option is useful when the user plans to start
a repair on *all* nodes, we shouldn't repair the same token range multiple
times, so each range should be repaired by only one of the nodes.

abstract_replication_strategy::get_primary_ranges() is similar to Origin's
StorageService.getPrimaryRangesForEndpoint().

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-08-22 20:05:32 +03:00
Vlad Zolotarov
5f13ebad30 locator: create/reset_snitch(): improve the error message for a not-supported case
Fixes issue #124

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-16 12:55:33 +03:00
Vlad Zolotarov
6bffb9232e locator: added i_endpoint_snitch::reset_snitch()
Resets the global snitch with the new value

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:44:43 +03:00
Vlad Zolotarov
ef1c7deff4 locator: introduce i_endpoint_snitch::pause_io() and resume_io() methods
resume_io() is different from start() in that it won't try to read to configuration
and will only restart the periodic I/O task (if any).

This also means that resume_io() may not fail while start() will return an
exceptional future if it fails to read the configuration.

pause_io() is a counterpart of resume_io() - it stops the periodic I/O task (if any).
After it returns a ready future - snitch will not try to read any configuration until
either start() or resume_io() are called.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:38:04 +03:00
Vlad Zolotarov
b9389d4907 locator: Get rid of using the global distributed<snitch_ptr> from inside the snitch
- production_snitch_base: Store a distributed object pointer in the snitch.
   - i_endpoint_snitch::init_snitch_obj(): Set the distributed<> mentioned above.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:37:57 +03:00
Vlad Zolotarov
08ccffc701 locator::i_endpoint_snitch: added init_snitch_obj()
Initializes the given distributed<snitch_ptr> object but
not start()s the local snitch instances.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:33:26 +03:00
Vlad Zolotarov
f9b67f60c2 locator::i_endpoint_snitch: Remove not used _snitch_is_ready promise
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:21:25 +03:00
Vlad Zolotarov
a3cda17bfb locator::gossiping_property_file_snitch: simplify the start()
- Call for read_property_file() directly from the start().
   - Immediately return ready future from the start() for non-IO
     CPUs.
   - Remove the not needed invoke_on_all() invocations.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:21:18 +03:00
Vlad Zolotarov
0b17f2ad75 locator: simple and rack_inferring snitches: add "override" qualifier to get_name() method
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-12 16:19:06 +03:00