bahman/scylla
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

Merge "Introduce new in-memory representation for cells" from Paweł

Browse Source 
"
This is the first part of the first step of switching Scylla. It covers
converting cells to the new serialisation format. The actual structure
of the cells doesn't differ much from the original one with a notable
exception of the fact that large values are now fragmented and
linearisation needs to be explicit. Counters and collections still
partially rely on their old, custom serialisation code and their
handling is not optimial (although not significantly worse than it used
to be).

The new in-memory representation allows objects to be of varying size
and makes it possible to provide deserialisation context so that we
don't need to keep in each instance of an IMR type all the information
needed to interpret it. The structure of IMR types is described in C++
using some metaprogramming with the hopes of making it much easier to
modify the serialisation format that it would be in case of open-coded
serialisation functions.

Moreover, IMR types can own memory thanks to a limited support for
destructors and movers (the latter are not exactly the same thing as C++
move constructors hence a different name). This makes it (relatively)
to ensure that there is an upper bound on the size of all allocations.

For now the only thing that is converted to the IMR are atomic_cells
and collections which means that the reduction in the memory footprint
is not as big as it can be, but introducing the IMR is a big step on its
own and also paves the way towards complete elimination of unbounded
memory allocations.

The first part of this patchset contains miscellaneous preparatory
changes to various parts of the Scylla codebase. They are followed by
introduction of the IMR infrastructure. Then structure of cells is
defined and all helper functions are implemented. Next are several
treewide patches that mostly deal with propagating type information to
the cell-related operations. Finally, atomic_cell and collections are
switched to used the new IMR-based cell implementation.

The IMR is described in much more detail in imr/IMR.md added in "imr:
add IMR documentation".

Refs #2031.
Refs #2409.

perf_simple_query -c4, medians of 30 results:

        ./perf_base  ./perf_imr   diff
 read     308790.08   309775.35   0.3%
 write    402127.32   417729.18   3.9%

The same with 1 byte values:
        ./perf_base1  ./perf_imr1   diff
 read      314107.26    314648.96   0.2%
 write     463801.40    433255.96  -6.6%

The memory footprint is reduced, but that is partially due to removal of
small buffer optimisation (whether it will be restored depends on the
exact mesurements of the performance impact). Generally, this series was
not expected to make a huge difference as this would require converting
whole rows to the IMR.

Memory footprint:
Before:
mutation footprint:
 - in cache: 1264
 - in memtable: 986

After:
mutation footprint:
 - in cache: 1104
 - in memtable: 866

Tests: unit (release, debug)
"

* tag 'imr-cells/v3' of https://github.com/pdziepak/scylla: (37 commits)
  tests/mutation: add test for changing column type
  atomic_cell: switch to new IMR-based cell reperesentation
  atomic_cell: explicitly state when atomic_cell is a collection member
  treewide: require type for creating collection_mutation_view
  treewide: require type for comparing cells
  atomic_cell: introduce fragmented buffer value interface
  treewide: require type to compute cell memory usage
  treewide: require type to copy atomic_cell
  treewide: require type info for copying atomic_cell_or_collection
  treewide: require type for creating atomic_cell
  atomic_cell: require column_definition for creating atomic_cell views
  tests: test imr representation of cells
  types: provide information for IMR
  data: introduce cell
  data: introduce type_info
  imr/utils: add imr object holder
  imr: introduce concepts
  imr: add helper for allocating objects
  imr: allow creating lsa migrators for IMR objects
  imr: introduce placeholders
  ...
This commit is contained in:
Avi Kivity
2018-05-31 19:21:15 +03:00
parent bc7503feee 131a47dea3
commit aab6b0ee27
97 changed files with 6529 additions and 969 deletions
Download Patch File Download Diff File Expand all files Collapse all files

205
atomic_cell.cc Normal file
View File

@@ -0,0 +1,205 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#include "atomic_cell.hh"
#include "atomic_cell_or_collection.hh"
#include "types.hh"
/// LSA mirator for cells with irrelevant type
///
///
const data::type_imr_descriptor& no_type_imr_descriptor() {
static thread_local data::type_imr_descriptor state(data::type_info::make_variable_size());
return state;
}
atomic_cell atomic_cell::make_dead(api::timestamp_type timestamp, gc_clock::time_point deletion_time) {
auto& imr_data = no_type_imr_descriptor();
return atomic_cell(
imr_data.type_info(),
imr_object_type::make(data::cell::make_dead(timestamp, deletion_time), &imr_data.lsa_migrator())
);
}
atomic_cell atomic_cell::make_live(const abstract_type& type, api::timestamp_type timestamp, bytes_view value, atomic_cell::collection_member cm) {
auto& imr_data = type.imr_state();
return atomic_cell(
imr_data.type_info(),
imr_object_type::make(data::cell::make_live(imr_data.type_info(), timestamp, value, bool(cm)), &imr_data.lsa_migrator())
);
}
atomic_cell atomic_cell::make_live(const abstract_type& type, api::timestamp_type timestamp, bytes_view value,
gc_clock::time_point expiry, gc_clock::duration ttl, atomic_cell::collection_member cm) {
auto& imr_data = type.imr_state();
return atomic_cell(
imr_data.type_info(),
imr_object_type::make(data::cell::make_live(imr_data.type_info(), timestamp, value, expiry, ttl, bool(cm)), &imr_data.lsa_migrator())
);
}
atomic_cell atomic_cell::make_live_counter_update(api::timestamp_type timestamp, int64_t value) {
auto& imr_data = no_type_imr_descriptor();
return atomic_cell(
imr_data.type_info(),
imr_object_type::make(data::cell::make_live_counter_update(timestamp, value), &imr_data.lsa_migrator())
);
}
atomic_cell atomic_cell::make_live_uninitialized(const abstract_type& type, api::timestamp_type timestamp, size_t size) {
auto& imr_data = no_type_imr_descriptor();
return atomic_cell(
imr_data.type_info(),
imr_object_type::make(data::cell::make_live_uninitialized(imr_data.type_info(), timestamp, size), &imr_data.lsa_migrator())
);
}
static imr::utils::object<data::cell::structure> copy_cell(const data::type_imr_descriptor& imr_data, const uint8_t* ptr)
{
using imr_object_type = imr::utils::object<data::cell::structure>;
// If the cell doesn't own any memory it is trivial and can be copied with
// memcpy.
auto f = data::cell::structure::get_member<data::cell::tags::flags>(ptr);
if (!f.template get<data::cell::tags::external_data>()) {
data::cell::context ctx(f, imr_data.type_info());
// XXX: We may be better off storing the total cell size in memory. Measure!
auto size = data::cell::structure::serialized_object_size(ptr, ctx);
return imr_object_type::make_raw(size, [&] (uint8_t* dst) noexcept {
std::copy_n(ptr, size, dst);
}, &imr_data.lsa_migrator());
}
return imr_object_type::make(data::cell::copy_fn(imr_data.type_info(), ptr), &imr_data.lsa_migrator());
}
atomic_cell::atomic_cell(const abstract_type& type, atomic_cell_view other)
: atomic_cell(type.imr_state().type_info(),
copy_cell(type.imr_state(), other._view.raw_pointer()))
{ }
atomic_cell_or_collection atomic_cell_or_collection::copy(const abstract_type& type) const {
if (!_data.get()) {
return atomic_cell_or_collection();
}
auto& imr_data = type.imr_state();
return atomic_cell_or_collection(
copy_cell(imr_data, _data.get())
);
}
atomic_cell_or_collection::atomic_cell_or_collection(const abstract_type& type, atomic_cell_view acv)
: _data(copy_cell(type.imr_state(), acv._view.raw_pointer()))
{
}
static collection_mutation_view get_collection_mutation_view(const uint8_t* ptr)
{
auto f = data::cell::structure::get_member<data::cell::tags::flags>(ptr);
auto ti = data::type_info::make_collection();
data::cell::context ctx(f, ti);
auto view = data::cell::structure::get_member<data::cell::tags::cell>(ptr).as<data::cell::tags::collection>(ctx);
auto dv = data::cell::variable_value::make_view(view, f.get<data::cell::tags::external_data>());
return collection_mutation_view { dv };
}
collection_mutation_view atomic_cell_or_collection::as_collection_mutation() const {
return get_collection_mutation_view(_data.get());
}
collection_mutation::collection_mutation(const collection_type_impl& type, collection_mutation_view v)
: _data(imr_object_type::make(data::cell::make_collection(v.data), &type.imr_state().lsa_migrator()))
{
}
collection_mutation::collection_mutation(const collection_type_impl& type, bytes_view v)
: _data(imr_object_type::make(data::cell::make_collection(v), &type.imr_state().lsa_migrator()))
{
}
collection_mutation::operator collection_mutation_view() const
{
return get_collection_mutation_view(_data.get());
}
bool atomic_cell_or_collection::equals(const abstract_type& type, const atomic_cell_or_collection& other) const
{
auto ptr_a = _data.get();
auto ptr_b = other._data.get();
if (!ptr_a || !ptr_b) {
return !ptr_a && !ptr_b;
}
if (type.is_atomic()) {
auto a = atomic_cell_view::from_bytes(type.imr_state().type_info(), _data);
auto b = atomic_cell_view::from_bytes(type.imr_state().type_info(), other._data);
if (a.timestamp() != b.timestamp()) {
return false;
}
if (a.is_live()) {
if (!b.is_live()) {
return false;
}
if (a.is_counter_update()) {
if (!b.is_counter_update()) {
return false;
}
return a.counter_update_value() == b.counter_update_value();
}
if (a.is_live_and_has_ttl()) {
if (!b.is_live_and_has_ttl()) {
return false;
}
if (a.ttl() != b.ttl() || a.expiry() != b.expiry()) {
return false;
}
}
return a.value() == b.value();
}
return a.deletion_time() == b.deletion_time();
} else {
return as_collection_mutation().data == other.as_collection_mutation().data;
}
}
size_t atomic_cell_or_collection::external_memory_usage(const abstract_type& t) const
{
if (!_data.get()) {
return 0;
}
auto ctx = data::cell::context(_data.get(), t.imr_state().type_info());
return data::cell::structure::serialized_object_size(_data.get(), ctx);
}
std::ostream& operator<<(std::ostream& os, const atomic_cell_or_collection& c) {
if (!c._data.get()) {
return os << "{ null atomic_cell_or_collection }";
}
using dc = data::cell;
os << "{ ";
if (dc::structure::get_member<dc::tags::flags>(c._data.get()).get<dc::tags::collection>()) {
os << "collection";
} else {
os << "atomic cell";
}
return os << " @" << static_cast<const void*>(c._data.get()) << " }";
}

348
atomic_cell.hh
View File

@@ -30,178 +30,48 @@
#include <cstdint>
#include <iosfwd>
#include <seastar/util/gcc6-concepts.hh>
#include "data/cell.hh"
#include "data/schema_info.hh"
#include "imr/utils.hh"
template<typename T, typename Input>
static inline
void set_field(Input& v, unsigned offset, T val) {
reinterpret_cast<net::packed<T>*>(v.begin() + offset)->raw = net::hton(val);
}
class abstract_type;
class collection_type_impl;
template<typename T>
static inline
T get_field(const bytes_view& v, unsigned offset) {
return net::ntoh(*reinterpret_cast<const net::packed<T>*>(v.begin() + offset));
}
using atomic_cell_value_view = data::value_view;
using atomic_cell_value_mutable_view = data::value_mutable_view;
class atomic_cell_or_collection;
/*
* Represents atomic cell layout. Works on serialized form.
*
* Layout:
*
* <live> := <int8_t:flags><int64_t:timestamp>(<int32_t:expiry><int32_t:ttl>)?<value>
* <dead> := <int8_t: 0><int64_t:timestamp><int32_t:deletion_time>
*/
class atomic_cell_type final {
private:
static constexpr int8_t LIVE_FLAG = 0x01;
static constexpr int8_t EXPIRY_FLAG = 0x02; // When present, expiry field is present. Set only for live cells
static constexpr int8_t COUNTER_UPDATE_FLAG = 0x08; // Cell is a counter update.
static constexpr unsigned flags_size = 1;
static constexpr unsigned timestamp_offset = flags_size;
static constexpr unsigned timestamp_size = 8;
static constexpr unsigned expiry_offset = timestamp_offset + timestamp_size;
static constexpr unsigned expiry_size = 4;
static constexpr unsigned deletion_time_offset = timestamp_offset + timestamp_size;
static constexpr unsigned deletion_time_size = 4;
static constexpr unsigned ttl_offset = expiry_offset + expiry_size;
static constexpr unsigned ttl_size = 4;
friend class counter_cell_builder;
private:
static bool is_counter_update(bytes_view cell) {
return cell[0] & COUNTER_UPDATE_FLAG;
}
static bool is_live(const bytes_view& cell) {
return cell[0] & LIVE_FLAG;
}
static bool is_live_and_has_ttl(const bytes_view& cell) {
return cell[0] & EXPIRY_FLAG;
}
static bool is_dead(const bytes_view& cell) {
return !is_live(cell);
}
// Can be called on live and dead cells
static api::timestamp_type timestamp(const bytes_view& cell) {
return get_field<api::timestamp_type>(cell, timestamp_offset);
}
template<typename BytesContainer>
static void set_timestamp(BytesContainer& cell, api::timestamp_type ts) {
set_field(cell, timestamp_offset, ts);
}
// Can be called on live cells only
private:
template<typename BytesView>
static BytesView do_get_value(BytesView cell) {
auto expiry_field_size = bool(cell[0] & EXPIRY_FLAG) * (expiry_size + ttl_size);
auto value_offset = flags_size + timestamp_size + expiry_field_size;
cell.remove_prefix(value_offset);
return cell;
}
public:
static bytes_view value(bytes_view cell) {
return do_get_value(cell);
}
static bytes_mutable_view value(bytes_mutable_view cell) {
return do_get_value(cell);
}
// Can be called on live counter update cells only
static int64_t counter_update_value(bytes_view cell) {
return get_field<int64_t>(cell, flags_size + timestamp_size);
}
// Can be called only when is_dead() is true.
static gc_clock::time_point deletion_time(const bytes_view& cell) {
assert(is_dead(cell));
return gc_clock::time_point(gc_clock::duration(
get_field<int32_t>(cell, deletion_time_offset)));
}
// Can be called only when is_live_and_has_ttl() is true.
static gc_clock::time_point expiry(const bytes_view& cell) {
assert(is_live_and_has_ttl(cell));
auto expiry = get_field<int32_t>(cell, expiry_offset);
return gc_clock::time_point(gc_clock::duration(expiry));
}
// Can be called only when is_live_and_has_ttl() is true.
static gc_clock::duration ttl(const bytes_view& cell) {
assert(is_live_and_has_ttl(cell));
return gc_clock::duration(get_field<int32_t>(cell, ttl_offset));
}
static managed_bytes make_dead(api::timestamp_type timestamp, gc_clock::time_point deletion_time) {
managed_bytes b(managed_bytes::initialized_later(), flags_size + timestamp_size + deletion_time_size);
b[0] = 0;
set_field(b, timestamp_offset, timestamp);
set_field(b, deletion_time_offset, deletion_time.time_since_epoch().count());
return b;
}
static managed_bytes make_live(api::timestamp_type timestamp, bytes_view value) {
auto value_offset = flags_size + timestamp_size;
managed_bytes b(managed_bytes::initialized_later(), value_offset + value.size());
b[0] = LIVE_FLAG;
set_field(b, timestamp_offset, timestamp);
std::copy_n(value.begin(), value.size(), b.begin() + value_offset);
return b;
}
static managed_bytes make_live_counter_update(api::timestamp_type timestamp, int64_t value) {
auto value_offset = flags_size + timestamp_size;
managed_bytes b(managed_bytes::initialized_later(), value_offset + sizeof(value));
b[0] = LIVE_FLAG | COUNTER_UPDATE_FLAG;
set_field(b, timestamp_offset, timestamp);
set_field(b, value_offset, value);
return b;
}
static managed_bytes make_live(api::timestamp_type timestamp, bytes_view value, gc_clock::time_point expiry, gc_clock::duration ttl) {
auto value_offset = flags_size + timestamp_size + expiry_size + ttl_size;
managed_bytes b(managed_bytes::initialized_later(), value_offset + value.size());
b[0] = EXPIRY_FLAG | LIVE_FLAG;
set_field(b, timestamp_offset, timestamp);
set_field(b, expiry_offset, expiry.time_since_epoch().count());
set_field(b, ttl_offset, ttl.count());
std::copy_n(value.begin(), value.size(), b.begin() + value_offset);
return b;
}
// make_live_from_serializer() is intended for users that need to serialise
// some object or objects to the format used in atomic_cell::value().
// With just make_live() the patter would look like follows:
// 1. allocate a buffer and write to it serialised objects
// 2. pass that buffer to make_live()
// 3. make_live() needs to prepend some metadata to the cell value so it
// allocates a new buffer and copies the content of the original one
//
// The allocation and copy of a buffer can be avoided.
// make_live_from_serializer() allows the user code to specify the timestamp
// and size of the cell value as well as provide the serialiser function
// object, which would write the serialised value of the cell to the buffer
// given to it by make_live_from_serializer().
template<typename Serializer>
GCC6_CONCEPT(requires requires(Serializer serializer, bytes::iterator it) {
serializer(it);
})
static managed_bytes make_live_from_serializer(api::timestamp_type timestamp, size_t size, Serializer&& serializer) {
auto value_offset = flags_size + timestamp_size;
managed_bytes b(managed_bytes::initialized_later(), value_offset + size);
b[0] = LIVE_FLAG;
set_field(b, timestamp_offset, timestamp);
serializer(b.begin() + value_offset);
return b;
}
template<typename ByteContainer>
friend class atomic_cell_base;
/// View of an atomic cell
template<mutable_view is_mutable>
class basic_atomic_cell_view {
protected:
data::cell::basic_atomic_cell_view<is_mutable> _view;
friend class atomic_cell;
};
public:
using pointer_type = std::conditional_t<is_mutable == mutable_view::no, const uint8_t*, uint8_t*>;
protected:
explicit basic_atomic_cell_view(data::cell::basic_atomic_cell_view<is_mutable> v)
: _view(std::move(v)) { }
basic_atomic_cell_view(const data::type_info& ti, pointer_type ptr)
: _view(data::cell::make_atomic_cell_view(ti, ptr))
{ }
template<typename ByteContainer>
class atomic_cell_base {
protected:
ByteContainer _data;
protected:
atomic_cell_base(ByteContainer&& data) : _data(std::forward<ByteContainer>(data)) { }
friend class atomic_cell_or_collection;
public:
operator basic_atomic_cell_view<mutable_view::no>() const noexcept {
return basic_atomic_cell_view<mutable_view::no>(_view);
}
void swap(basic_atomic_cell_view& other) noexcept {
using std::swap;
swap(_view, other._view);
}
bool is_counter_update() const {
return atomic_cell_type::is_counter_update(_data);
return _view.is_counter_update();
}
bool is_live() const {
return atomic_cell_type::is_live(_data);
return _view.is_live();
}
bool is_live(tombstone t, bool is_counter) const {
return is_live() && !is_covered_by(t, is_counter);
@@ -210,119 +80,132 @@ public:
return is_live() && !is_covered_by(t, is_counter) && !has_expired(now);
}
bool is_live_and_has_ttl() const {
return atomic_cell_type::is_live_and_has_ttl(_data);
return _view.is_expiring();
}
bool is_dead(gc_clock::time_point now) const {
return atomic_cell_type::is_dead(_data) || has_expired(now);
return !is_live() || has_expired(now);
}
bool is_covered_by(tombstone t, bool is_counter) const {
return timestamp() <= t.timestamp || (is_counter && t.timestamp != api::missing_timestamp);
}
// Can be called on live and dead cells
api::timestamp_type timestamp() const {
return atomic_cell_type::timestamp(_data);
return _view.timestamp();
}
void set_timestamp(api::timestamp_type ts) {
atomic_cell_type::set_timestamp(_data, ts);
_view.set_timestamp(ts);
}
// Can be called on live cells only
auto value() const {
return atomic_cell_type::value(_data);
data::basic_value_view<is_mutable> value() const {
return _view.value();
}
// Can be called on live cells only
size_t value_size() const {
return _view.value_size();
}
bool is_value_fragmented() const {
return _view.is_value_fragmented();
}
// Can be called on live counter update cells only
int64_t counter_update_value() const {
return atomic_cell_type::counter_update_value(_data);
return _view.counter_update_value();
}
// Can be called only when is_dead(gc_clock::time_point)
gc_clock::time_point deletion_time() const {
return !is_live() ? atomic_cell_type::deletion_time(_data) : expiry() - ttl();
return !is_live() ? _view.deletion_time() : expiry() - ttl();
}
// Can be called only when is_live_and_has_ttl()
gc_clock::time_point expiry() const {
return atomic_cell_type::expiry(_data);
return _view.expiry();
}
// Can be called only when is_live_and_has_ttl()
gc_clock::duration ttl() const {
return atomic_cell_type::ttl(_data);
return _view.ttl();
}
// Can be called on live and dead cells
bool has_expired(gc_clock::time_point now) const {
return is_live_and_has_ttl() && expiry() <= now;
}
bytes_view serialize() const {
return _data;
return _view.serialize();
}
};
class atomic_cell_view final : public atomic_cell_base<bytes_view> {
atomic_cell_view(bytes_view data) : atomic_cell_base(std::move(data)) {}
public:
static atomic_cell_view from_bytes(bytes_view data) { return atomic_cell_view(data); }
class atomic_cell_view final : public basic_atomic_cell_view<mutable_view::no> {
atomic_cell_view(const data::type_info& ti, const uint8_t* data)
: basic_atomic_cell_view<mutable_view::no>(ti, data) {}
template<mutable_view is_mutable>
atomic_cell_view(data::cell::basic_atomic_cell_view<is_mutable> view)
: basic_atomic_cell_view<mutable_view::no>(view) { }
friend class atomic_cell;
public:
static atomic_cell_view from_bytes(const data::type_info& ti, const imr::utils::object<data::cell::structure>& data) {
return atomic_cell_view(ti, data.get());
}
static atomic_cell_view from_bytes(const data::type_info& ti, bytes_view bv) {
return atomic_cell_view(ti, reinterpret_cast<const uint8_t*>(bv.begin()));
}
friend std::ostream& operator<<(std::ostream& os, const atomic_cell_view& acv);
};
class atomic_cell_mutable_view final : public atomic_cell_base<bytes_mutable_view> {
atomic_cell_mutable_view(bytes_mutable_view data) : atomic_cell_base(std::move(data)) {}
class atomic_cell_mutable_view final : public basic_atomic_cell_view<mutable_view::yes> {
atomic_cell_mutable_view(const data::type_info& ti, uint8_t* data)
: basic_atomic_cell_view<mutable_view::yes>(ti, data) {}
public:
static atomic_cell_mutable_view from_bytes(bytes_mutable_view data) { return atomic_cell_mutable_view(data); }
static atomic_cell_mutable_view from_bytes(const data::type_info& ti, imr::utils::object<data::cell::structure>& data) {
return atomic_cell_mutable_view(ti, data.get());
}
friend class atomic_cell;
};
class atomic_cell_ref final : public atomic_cell_base<managed_bytes&> {
public:
atomic_cell_ref(managed_bytes& buf) : atomic_cell_base(buf) {}
};
using atomic_cell_ref = atomic_cell_mutable_view;
class atomic_cell final : public atomic_cell_base<managed_bytes> {
atomic_cell(managed_bytes b) : atomic_cell_base(std::move(b)) {}
class atomic_cell final : public basic_atomic_cell_view<mutable_view::yes> {
using imr_object_type = imr::utils::object<data::cell::structure>;
imr_object_type _data;
atomic_cell(const data::type_info& ti, imr::utils::object<data::cell::structure>&& data)
: basic_atomic_cell_view<mutable_view::yes>(ti, data.get()), _data(std::move(data)) {}
public:
atomic_cell(const atomic_cell&) = default;
class collection_member_tag;
using collection_member = bool_class<collection_member_tag>;
atomic_cell(atomic_cell&&) = default;
atomic_cell& operator=(const atomic_cell&) = default;
atomic_cell& operator=(const atomic_cell&) = delete;
atomic_cell& operator=(atomic_cell&&) = default;
static atomic_cell from_bytes(managed_bytes b) {
return atomic_cell(std::move(b));
void swap(atomic_cell& other) noexcept {
basic_atomic_cell_view<mutable_view::yes>::swap(other);
_data.swap(other._data);
}
atomic_cell(atomic_cell_view other) : atomic_cell_base(managed_bytes{other._data}) {}
operator atomic_cell_view() const {
return atomic_cell_view(_data);
operator atomic_cell_view() const { return atomic_cell_view(_view); }
atomic_cell(const abstract_type& t, atomic_cell_view other);
static atomic_cell make_dead(api::timestamp_type timestamp, gc_clock::time_point deletion_time);
static atomic_cell make_live(const abstract_type& type, api::timestamp_type timestamp, bytes_view value,
collection_member = collection_member::no);
static atomic_cell make_live(const abstract_type& type, api::timestamp_type timestamp, const bytes& value,
collection_member cm = collection_member::no) {
return make_live(type, timestamp, bytes_view(value), cm);
}
static atomic_cell make_dead(api::timestamp_type timestamp, gc_clock::time_point deletion_time) {
return atomic_cell_type::make_dead(timestamp, deletion_time);
}
static atomic_cell make_live(api::timestamp_type timestamp, bytes_view value) {
return atomic_cell_type::make_live(timestamp, value);
}
static atomic_cell make_live(api::timestamp_type timestamp, const bytes& value) {
return make_live(timestamp, bytes_view(value));
}
static atomic_cell make_live_counter_update(api::timestamp_type timestamp, int64_t value) {
return atomic_cell_type::make_live_counter_update(timestamp, value);
}
static atomic_cell make_live(api::timestamp_type timestamp, bytes_view value,
gc_clock::time_point expiry, gc_clock::duration ttl)
static atomic_cell make_live_counter_update(api::timestamp_type timestamp, int64_t value);
static atomic_cell make_live(const abstract_type&, api::timestamp_type timestamp, bytes_view value,
gc_clock::time_point expiry, gc_clock::duration ttl, collection_member = collection_member::no);
static atomic_cell make_live(const abstract_type& type, api::timestamp_type timestamp, const bytes& value,
gc_clock::time_point expiry, gc_clock::duration ttl, collection_member cm = collection_member::no)
{
return atomic_cell_type::make_live(timestamp, value, expiry, ttl);
return make_live(type, timestamp, bytes_view(value), expiry, ttl, cm);
}
static atomic_cell make_live(api::timestamp_type timestamp, const bytes& value,
gc_clock::time_point expiry, gc_clock::duration ttl)
{
return make_live(timestamp, bytes_view(value), expiry, ttl);
}
static atomic_cell make_live(api::timestamp_type timestamp, bytes_view value, ttl_opt ttl) {
static atomic_cell make_live(const abstract_type& type, api::timestamp_type timestamp, bytes_view value, ttl_opt ttl, collection_member cm = collection_member::no) {
if (!ttl) {
return atomic_cell_type::make_live(timestamp, value);
return make_live(type, timestamp, value, cm);
} else {
return atomic_cell_type::make_live(timestamp, value, gc_clock::now() + *ttl, *ttl);
return make_live(type, timestamp, value, gc_clock::now() + *ttl, *ttl, cm);
}
}
template<typename Serializer>
static atomic_cell make_live_from_serializer(api::timestamp_type timestamp, size_t size, Serializer&& serializer) {
return atomic_cell_type::make_live_from_serializer(timestamp, size, std::forward<Serializer>(serializer));
}
static atomic_cell make_live_uninitialized(const abstract_type& type, api::timestamp_type timestamp, size_t size);
friend class atomic_cell_or_collection;
friend std::ostream& operator<<(std::ostream& os, const atomic_cell& ac);
};
@@ -336,33 +219,24 @@ class collection_mutation_view;
// list: tbd, probably ugly
class collection_mutation {
public:
managed_bytes data;
using imr_object_type = imr::utils::object<data::cell::structure>;
imr_object_type _data;
collection_mutation() {}
collection_mutation(managed_bytes b) : data(std::move(b)) {}
collection_mutation(collection_mutation_view v);
collection_mutation(const collection_type_impl&, collection_mutation_view v);
collection_mutation(const collection_type_impl&, bytes_view bv);
operator collection_mutation_view() const;
};
class collection_mutation_view {
public:
bytes_view data;
bytes_view serialize() const { return data; }
static collection_mutation_view from_bytes(bytes_view v) { return { v }; }
atomic_cell_value_view data;
};
inline
collection_mutation::collection_mutation(collection_mutation_view v)
: data(v.data) {
}
inline
collection_mutation::operator collection_mutation_view() const {
return { data };
}
class column_definition;
int compare_atomic_cell_for_merge(atomic_cell_view left, atomic_cell_view right);
void merge_column(const column_definition& def,
void merge_column(const abstract_type& def,
atomic_cell_or_collection& old,
const atomic_cell_or_collection& neww);

12
atomic_cell_hash.hh
View File

@@ -33,13 +33,15 @@ template<>
struct appending_hash<collection_mutation_view> {
template<typename Hasher>
void operator()(Hasher& h, collection_mutation_view cell, const column_definition& cdef) const {
cell.data.with_linearized([&] (bytes_view cell_bv) {
auto ctype = static_pointer_cast<const collection_type_impl>(cdef.type);
auto m_view = ctype->deserialize_mutation_form(cell);
auto m_view = ctype->deserialize_mutation_form(cell_bv);
::feed_hash(h, m_view.tomb);
for (auto&& key_and_value : m_view.cells) {
::feed_hash(h, key_and_value.first);
::feed_hash(h, key_and_value.second, cdef);
}
});
}
};
@@ -51,7 +53,9 @@ struct appending_hash<atomic_cell_view> {
feed_hash(h, cell.timestamp());
if (cell.is_live()) {
if (cdef.is_counter()) {
::feed_hash(h, counter_cell_view(cell));
counter_cell_view::with_linearized(cell, [&] (counter_cell_view ccv) {
::feed_hash(h, ccv);
});
return;
}
if (cell.is_live_and_has_ttl()) {
@@ -86,9 +90,9 @@ struct appending_hash<atomic_cell_or_collection> {
template<typename Hasher>
void operator()(Hasher& h, const atomic_cell_or_collection& c, const column_definition& cdef) const {
if (cdef.is_atomic()) {
feed_hash(h, c.as_atomic_cell(), cdef);
feed_hash(h, c.as_atomic_cell(cdef), cdef);
} else {
feed_hash(h, c.as_collection_mutation(), cdef);
}
}
};
};

64
atomic_cell_or_collection.hh
View File

@@ -25,42 +25,56 @@
#include "schema.hh"
#include "hashing.hh"
#include "imr/utils.hh"
// A variant type that can hold either an atomic_cell, or a serialized collection.
// Which type is stored is determined by the schema.
// Has an "empty" state.
// Objects moved-from are left in an empty state.
class atomic_cell_or_collection final {
managed_bytes _data;
// FIXME: This has made us lose small-buffer optimisation. Unfortunately,
// due to the changed cell format it would be less effective now, anyway.
// Measure the actual impact because any attempts to fix this will become
// irrelevant once rows are converted to the IMR as well, so maybe we can
// live with this like that.
using imr_object_type = imr::utils::object<data::cell::structure>;
imr_object_type _data;
private:
atomic_cell_or_collection(managed_bytes&& data) : _data(std::move(data)) {}
atomic_cell_or_collection(imr::utils::object<data::cell::structure>&& data) : _data(std::move(data)) {}
public:
atomic_cell_or_collection() = default;
atomic_cell_or_collection(atomic_cell_or_collection&&) = default;
atomic_cell_or_collection(const atomic_cell_or_collection&) = delete;
atomic_cell_or_collection& operator=(atomic_cell_or_collection&&) = default;
atomic_cell_or_collection& operator=(const atomic_cell_or_collection&) = delete;
atomic_cell_or_collection(atomic_cell ac) : _data(std::move(ac._data)) {}
atomic_cell_or_collection(const abstract_type& at, atomic_cell_view acv);
static atomic_cell_or_collection from_atomic_cell(atomic_cell data) { return { std::move(data._data) }; }
atomic_cell_view as_atomic_cell() const { return atomic_cell_view::from_bytes(_data); }
atomic_cell_ref as_atomic_cell_ref() { return { _data }; }
atomic_cell_mutable_view as_mutable_atomic_cell() { return atomic_cell_mutable_view::from_bytes(_data); }
atomic_cell_or_collection(collection_mutation cm) : _data(std::move(cm.data)) {}
atomic_cell_view as_atomic_cell(const column_definition& cdef) const { return atomic_cell_view::from_bytes(cdef.type->imr_state().type_info(), _data); }
atomic_cell_ref as_atomic_cell_ref(const column_definition& cdef) { return atomic_cell_mutable_view::from_bytes(cdef.type->imr_state().type_info(), _data); }
atomic_cell_mutable_view as_mutable_atomic_cell(const column_definition& cdef) { return atomic_cell_mutable_view::from_bytes(cdef.type->imr_state().type_info(), _data); }
atomic_cell_or_collection(collection_mutation cm) : _data(std::move(cm._data)) { }
atomic_cell_or_collection copy(const abstract_type&) const;
explicit operator bool() const {
return !_data.empty();
return bool(_data);
}
bool can_use_mutable_view() const {
return !_data.is_fragmented();
static constexpr bool can_use_mutable_view() {
return true;
}
static atomic_cell_or_collection from_collection_mutation(collection_mutation data) {
return std::move(data.data);
}
collection_mutation_view as_collection_mutation() const {
return collection_mutation_view{_data};
}
bytes_view serialize() const {
return _data;
}
bool operator==(const atomic_cell_or_collection& other) const {
return _data == other._data;
}
size_t external_memory_usage() const {
return _data.external_memory_usage();
void swap(atomic_cell_or_collection& other) noexcept {
_data.swap(other._data);
}
static atomic_cell_or_collection from_collection_mutation(collection_mutation data) { return std::move(data._data); }
collection_mutation_view as_collection_mutation() const;
bytes_view serialize() const;
bool equals(const abstract_type& type, const atomic_cell_or_collection& other) const;
size_t external_memory_usage(const abstract_type&) const;
friend std::ostream& operator<<(std::ostream&, const atomic_cell_or_collection&);
};
namespace std {
inline void swap(atomic_cell_or_collection& a, atomic_cell_or_collection& b) noexcept
{
a.swap(b);
}
}

4
cache_flat_mutation_reader.hh
View File

@@ -365,7 +365,7 @@ bool cache_flat_mutation_reader::ensure_population_lower_bound() {
rows_entry::compare less(*_schema);
// FIXME: Avoid the copy by inserting an incomplete clustering row
auto e = alloc_strategy_unique_ptr<rows_entry>(
current_allocator().construct<rows_entry>(*_last_row));
current_allocator().construct<rows_entry>(*_schema, *_last_row));
e->set_continuous(false);
auto insert_result = rows.insert_check(rows.end(), *e, less);
auto inserted = insert_result.second;
@@ -418,7 +418,7 @@ void cache_flat_mutation_reader::maybe_add_to_cache(const clustering_row& cr) {
cr.cells().prepare_hash(*_schema, column_kind::regular_column);
}
auto new_entry = alloc_strategy_unique_ptr<rows_entry>(
current_allocator().construct<rows_entry>(cr.key(), cr.tomb(), cr.marker(), cr.cells()));
current_allocator().construct<rows_entry>(*_schema, cr.key(), cr.tomb(), cr.marker(), cr.cells()));
new_entry->set_continuous(false);
auto it = _next_row.iterators_valid() ? _next_row.get_iterator_in_latest_version()
: mp.clustered_rows().lower_bound(cr.key(), less);

10
configure.py
View File

@@ -299,6 +299,9 @@ scylla_tests = [
'tests/cql_auth_syntax_test',
'tests/querier_cache',
'tests/limiting_data_source_test',
'tests/meta_test',
'tests/imr_test',
'tests/partition_data_test',
]
perf_tests = [
@@ -377,6 +380,7 @@ extra_cxxflags = {}
cassandra_interface = Thrift(source = 'interface/cassandra.thrift', service = 'Cassandra')
scylla_core = (['database.cc',
'atomic_cell.cc',
'schema.cc',
'frozen_schema.cc',
'schema_registry.cc',
@@ -620,6 +624,7 @@ scylla_core = (['database.cc',
'vint-serialization.cc',
'utils/arch/powerpc/crc32-vpmsum/crc32_wrapper.cc',
'querier.cc',
'data/cell.cc',
]
+ [Antlr3Grammar('cql3/Cql.g')]
+ [Thrift('interface/cassandra.thrift', 'Cassandra')]
@@ -727,6 +732,9 @@ pure_boost_tests = set([
'tests/auth_resource_test',
'tests/enum_set_test',
'tests/cql_auth_syntax_test',
'tests/meta_test',
'tests/imr_test',
'tests/partition_data_test',
])
tests_not_using_seastar_test_framework = set([
@@ -777,6 +785,8 @@ deps['tests/allocation_strategy_test'] = ['tests/allocation_strategy_test.cc', '
deps['tests/log_heap_test'] = ['tests/log_heap_test.cc']
deps['tests/anchorless_list_test'] = ['tests/anchorless_list_test.cc']
deps['tests/perf/perf_fast_forward'] += ['release.cc']
deps['tests/meta_test'] = ['tests/meta_test.cc']
deps['tests/imr_test'] = ['tests/imr_test.cc']
warnings = [
'-Wno-mismatched-tags', # clang-only

29
converting_mutation_partition_applier.hh
View File

@@ -39,16 +39,32 @@ private:
return ::is_compatible(new_def.kind, kind) && new_def.type->is_value_compatible_with(*old_type);
}
static void accept_cell(row& dst, column_kind kind, const column_definition& new_def, const data_type& old_type, atomic_cell_view cell) {
if (is_compatible(new_def, old_type, kind) && cell.timestamp() > new_def.dropped_at()) {
dst.apply(new_def, atomic_cell_or_collection(cell));
if (!is_compatible(new_def, old_type, kind) || cell.timestamp() <= new_def.dropped_at()) {
return;
}
auto new_cell = [&] {
if (cell.is_live() && !old_type->is_counter()) {
if (cell.is_live_and_has_ttl()) {
return atomic_cell_or_collection(
atomic_cell::make_live(*new_def.type, cell.timestamp(), cell.value().linearize(), cell.expiry(), cell.ttl())
);
}
return atomic_cell_or_collection(
atomic_cell::make_live(*new_def.type, cell.timestamp(), cell.value().linearize())
);
} else {
return atomic_cell_or_collection(*new_def.type, cell);
}
}();
dst.apply(new_def, std::move(new_cell));
}
static void accept_cell(row& dst, column_kind kind, const column_definition& new_def, const data_type& old_type, collection_mutation_view cell) {
if (!is_compatible(new_def, old_type, kind)) {
return;
}
cell.data.with_linearized([&] (bytes_view cell_bv) {
auto&& ctype = static_pointer_cast<const collection_type_impl>(old_type);
auto old_view = ctype->deserialize_mutation_form(cell);
auto old_view = ctype->deserialize_mutation_form(cell_bv);
collection_type_impl::mutation_view new_view;
if (old_view.tomb.timestamp > new_def.dropped_at()) {
@@ -60,6 +76,7 @@ private:
}
}
dst.apply(new_def, ctype->serialize_mutation_form(std::move(new_view)));
});
}
public:
converting_mutation_partition_applier(
@@ -120,11 +137,11 @@ public:
// Appends the cell to dst upgrading it to the new schema.
// Cells must have monotonic names.
static void append_cell(row& dst, column_kind kind, const column_definition& new_def, const data_type& old_type, const atomic_cell_or_collection& cell) {
static void append_cell(row& dst, column_kind kind, const column_definition& new_def, const column_definition& old_def, const atomic_cell_or_collection& cell) {
if (new_def.is_atomic()) {
accept_cell(dst, kind, new_def, old_type, cell.as_atomic_cell());
accept_cell(dst, kind, new_def, old_def.type, cell.as_atomic_cell(old_def));
} else {
accept_cell(dst, kind, new_def, old_type, cell.as_collection_mutation());
accept_cell(dst, kind, new_def, old_def.type, cell.as_collection_mutation());
}
}
};

73
counters.cc
View File

@@ -78,10 +78,10 @@ std::vector<counter_shard> counter_cell_view::shards_compatible_with_1_7_4() con
return sorted_shards;
}
static bool apply_in_place(atomic_cell_or_collection& dst, atomic_cell_or_collection& src)
static bool apply_in_place(const column_definition& cdef, atomic_cell_mutable_view dst, atomic_cell_mutable_view src)
{
auto dst_ccmv = counter_cell_mutable_view(dst.as_mutable_atomic_cell());
auto src_ccmv = counter_cell_mutable_view(src.as_mutable_atomic_cell());
auto dst_ccmv = counter_cell_mutable_view(dst);
auto src_ccmv = counter_cell_mutable_view(src);
auto dst_shards = dst_ccmv.shards();
auto src_shards = src_ccmv.shards();
@@ -121,10 +121,10 @@ static bool apply_in_place(atomic_cell_or_collection& dst, atomic_cell_or_collec
return true;
}
void counter_cell_view::apply(atomic_cell_or_collection& dst, atomic_cell_or_collection& src)
void counter_cell_view::apply(const column_definition& cdef, atomic_cell_or_collection& dst, atomic_cell_or_collection& src)
{
auto dst_ac = dst.as_atomic_cell();
auto src_ac = src.as_atomic_cell();
auto dst_ac = dst.as_atomic_cell(cdef);
auto src_ac = src.as_atomic_cell(cdef);
if (!dst_ac.is_live() || !src_ac.is_live()) {
if (dst_ac.is_live() || (!src_ac.is_live() && compare_atomic_cell_for_merge(dst_ac, src_ac) < 0)) {
@@ -143,16 +143,21 @@ void counter_cell_view::apply(atomic_cell_or_collection& dst, atomic_cell_or_col
assert(!dst_ac.is_counter_update());
assert(!src_ac.is_counter_update());
with_linearized(dst_ac, [&] (counter_cell_view dst_ccv) {
with_linearized(src_ac, [&] (counter_cell_view src_ccv) {
if (counter_cell_view(dst_ac).shard_count() >= counter_cell_view(src_ac).shard_count()
&& dst.can_use_mutable_view() && src.can_use_mutable_view()) {
if (apply_in_place(dst, src)) {
return;
if (dst_ccv.shard_count() >= src_ccv.shard_count()) {
auto dst_amc = dst.as_mutable_atomic_cell(cdef);
auto src_amc = src.as_mutable_atomic_cell(cdef);
if (!dst_amc.is_value_fragmented() && !src_amc.is_value_fragmented()) {
if (apply_in_place(cdef, dst_amc, src_amc)) {
return;
}
}
}
auto dst_shards = counter_cell_view(dst_ac).shards();
auto src_shards = counter_cell_view(src_ac).shards();
auto dst_shards = dst_ccv.shards();
auto src_shards = src_ccv.shards();
counter_cell_builder result;
combine(dst_shards.begin(), dst_shards.end(), src_shards.begin(), src_shards.end(),
@@ -161,7 +166,9 @@ void counter_cell_view::apply(atomic_cell_or_collection& dst, atomic_cell_or_col
});
auto cell = result.build(std::max(dst_ac.timestamp(), src_ac.timestamp()));
src = std::exchange(dst, atomic_cell_or_collection(cell));
src = std::exchange(dst, atomic_cell_or_collection(std::move(cell)));
});
});
}
stdx::optional<atomic_cell> counter_cell_view::difference(atomic_cell_view a, atomic_cell_view b)
@@ -171,13 +178,15 @@ stdx::optional<atomic_cell> counter_cell_view::difference(atomic_cell_view a, at
if (!b.is_live() || !a.is_live()) {
if (b.is_live() || (!a.is_live() && compare_atomic_cell_for_merge(b, a) < 0)) {
return atomic_cell(a);
return atomic_cell(*counter_type, a);
}
return { };
}
auto a_shards = counter_cell_view(a).shards();
auto b_shards = counter_cell_view(b).shards();
return with_linearized(a, [&] (counter_cell_view a_ccv) {
return with_linearized(b, [&] (counter_cell_view b_ccv) {
auto a_shards = a_ccv.shards();
auto b_shards = b_ccv.shards();
auto a_it = a_shards.begin();
auto a_end = a_shards.end();
@@ -199,18 +208,21 @@ stdx::optional<atomic_cell> counter_cell_view::difference(atomic_cell_view a, at
if (!result.empty()) {
diff = result.build(std::max(a.timestamp(), b.timestamp()));
} else if (a.timestamp() > b.timestamp()) {
diff = atomic_cell::make_live(a.timestamp(), bytes_view());
diff = atomic_cell::make_live(*counter_type, a.timestamp(), bytes_view());
}
return diff;
});
});
}
void transform_counter_updates_to_shards(mutation& m, const mutation* current_state, uint64_t clock_offset) {
// FIXME: allow current_state to be frozen_mutation
auto transform_new_row_to_shards = [clock_offset] (auto& cells) {
cells.for_each_cell([clock_offset] (auto, atomic_cell_or_collection& ac_o_c) {
auto acv = ac_o_c.as_atomic_cell();
auto transform_new_row_to_shards = [&s = *m.schema(), clock_offset] (column_kind kind, auto& cells) {
cells.for_each_cell([&] (column_id id, atomic_cell_or_collection& ac_o_c) {
auto& cdef = s.column_at(kind, id);
auto acv = ac_o_c.as_atomic_cell(cdef);
if (!acv.is_live()) {
return; // continue -- we are in lambda
}
@@ -221,32 +233,35 @@ void transform_counter_updates_to_shards(mutation& m, const mutation* current_st
};
if (!current_state) {
transform_new_row_to_shards(m.partition().static_row());
transform_new_row_to_shards(column_kind::static_column, m.partition().static_row());
for (auto& cr : m.partition().clustered_rows()) {
transform_new_row_to_shards(cr.row().cells());
transform_new_row_to_shards(column_kind::regular_column, cr.row().cells());
}
return;
}
clustering_key::less_compare cmp(*m.schema());
auto transform_row_to_shards = [clock_offset] (auto& transformee, auto& state) {
auto transform_row_to_shards = [&s = *m.schema(), clock_offset] (column_kind kind, auto& transformee, auto& state) {
std::deque<std::pair<column_id, counter_shard>> shards;
state.for_each_cell([&] (column_id id, const atomic_cell_or_collection& ac_o_c) {
auto acv = ac_o_c.as_atomic_cell();
auto& cdef = s.column_at(kind, id);
auto acv = ac_o_c.as_atomic_cell(cdef);
if (!acv.is_live()) {
return; // continue -- we are in lambda
}
counter_cell_view ccv(acv);
counter_cell_view::with_linearized(acv, [&] (counter_cell_view ccv) {
auto cs = ccv.local_shard();
if (!cs) {
return; // continue
}
shards.emplace_back(std::make_pair(id, counter_shard(*cs)));
});
});
transformee.for_each_cell([&] (column_id id, atomic_cell_or_collection& ac_o_c) {
auto acv = ac_o_c.as_atomic_cell();
auto& cdef = s.column_at(kind, id);
auto acv = ac_o_c.as_atomic_cell(cdef);
if (!acv.is_live()) {
return; // continue -- we are in lambda
}
@@ -268,7 +283,7 @@ void transform_counter_updates_to_shards(mutation& m, const mutation* current_st
});
};
transform_row_to_shards(m.partition().static_row(), current_state->partition().static_row());
transform_row_to_shards(column_kind::static_column, m.partition().static_row(), current_state->partition().static_row());
auto& cstate = current_state->partition();
auto it = cstate.clustered_rows().begin();
@@ -278,10 +293,10 @@ void transform_counter_updates_to_shards(mutation& m, const mutation* current_st
++it;
}
if (it == end || cmp(cr.key(), it->key())) {
transform_new_row_to_shards(cr.row().cells());
transform_new_row_to_shards(column_kind::regular_column, cr.row().cells());
continue;
}
transform_row_to_shards(cr.row().cells(), it->row().cells());
transform_row_to_shards(column_kind::regular_column, cr.row().cells(), it->row().cells());
}
}

102
counters.hh
View File

@@ -79,7 +79,7 @@ static_assert(std::is_pod<counter_id>::value, "counter_id should be a POD type")
std::ostream& operator<<(std::ostream& os, const counter_id& id);
template<typename View>
template<mutable_view is_mutable>
class basic_counter_shard_view {
enum class offset : unsigned {
id = 0u,
@@ -88,7 +88,8 @@ class basic_counter_shard_view {
total_size = unsigned(logical_clock) + sizeof(int64_t),
};
private:
typename View::pointer _base;
using pointer_type = std::conditional_t<is_mutable == mutable_view::no, const signed char*, signed char*>;
pointer_type _base;
private:
template<typename T>
T read(offset off) const {
@@ -100,7 +101,7 @@ public:
static constexpr auto size = size_t(offset::total_size);
public:
basic_counter_shard_view() = default;
explicit basic_counter_shard_view(typename View::pointer ptr) noexcept
explicit basic_counter_shard_view(pointer_type ptr) noexcept
: _base(ptr) { }
counter_id id() const { return read<counter_id>(offset::id); }
@@ -111,7 +112,7 @@ public:
static constexpr size_t off = size_t(offset::value);
static constexpr size_t size = size_t(offset::total_size) - off;
typename View::value_type tmp[size];
signed char tmp[size];
std::copy_n(_base + off, size, tmp);
std::copy_n(other._base + off, size, _base + off);
std::copy_n(tmp, size, other._base + off);
@@ -138,7 +139,7 @@ public:
};
};
using counter_shard_view = basic_counter_shard_view<bytes_view>;
using counter_shard_view = basic_counter_shard_view<mutable_view::no>;
std::ostream& operator<<(std::ostream& os, counter_shard_view csv);
@@ -198,7 +199,7 @@ public:
return do_apply(other);
}
static size_t serialized_size() {
static constexpr size_t serialized_size() {
return counter_shard_view::size;
}
void serialize(bytes::iterator& out) const {
@@ -252,15 +253,33 @@ public:
}
atomic_cell build(api::timestamp_type timestamp) const {
return atomic_cell::make_live_from_serializer(timestamp, serialized_size(), [this] (bytes::iterator out) {
serialize(out);
});
// If we can assume that the counter shards never cross fragment boundaries
// the serialisation code gets much simpler.
static_assert(data::cell::maximum_external_chunk_length % counter_shard::serialized_size() == 0);
auto ac = atomic_cell::make_live_uninitialized(*counter_type, timestamp, serialized_size());
auto dst_it = ac.value().begin();
auto dst_current = *dst_it++;
for (auto&& cs : _shards) {
if (dst_current.empty()) {
dst_current = *dst_it++;
}
assert(!dst_current.empty());
auto value_dst = dst_current.data();
cs.serialize(value_dst);
dst_current.remove_prefix(counter_shard::serialized_size());
}
return ac;
}
static atomic_cell from_single_shard(api::timestamp_type timestamp, const counter_shard& cs) {
return atomic_cell::make_live_from_serializer(timestamp, counter_shard::serialized_size(), [&cs] (bytes::iterator out) {
cs.serialize(out);
});
// We don't really need to bother with fragmentation here.
static_assert(data::cell::maximum_external_chunk_length >= counter_shard::serialized_size());
auto ac = atomic_cell::make_live_uninitialized(*counter_type, timestamp, counter_shard::serialized_size());
auto dst = ac.value().first_fragment().begin();
cs.serialize(dst);
return ac;
}
class inserter_iterator : public std::iterator<std::output_iterator_tag, counter_shard> {
@@ -287,28 +306,32 @@ public:
// <counter_id> := <int64_t><int64_t>
// <shard> := <counter_id><int64_t:value><int64_t:logical_clock>
// <counter_cell> := <shard>*
template<typename View>
template<mutable_view is_mutable>
class basic_counter_cell_view {
protected:
atomic_cell_base<View> _cell;
using linearized_value_view = std::conditional_t<is_mutable == mutable_view::no,
bytes_view, bytes_mutable_view>;
using pointer_type = typename linearized_value_view::pointer;
basic_atomic_cell_view<is_mutable> _cell;
linearized_value_view _value;
private:
class shard_iterator : public std::iterator<std::input_iterator_tag, basic_counter_shard_view<View>> {
typename View::pointer _current;
basic_counter_shard_view<View> _current_view;
class shard_iterator : public std::iterator<std::input_iterator_tag, basic_counter_shard_view<is_mutable>> {
pointer_type _current;
basic_counter_shard_view<is_mutable> _current_view;
public:
shard_iterator() = default;
shard_iterator(typename View::pointer ptr) noexcept
shard_iterator(pointer_type ptr) noexcept
: _current(ptr), _current_view(ptr) { }
basic_counter_shard_view<View>& operator*() noexcept {
basic_counter_shard_view<is_mutable>& operator*() noexcept {
return _current_view;
}
basic_counter_shard_view<View>* operator->() noexcept {
basic_counter_shard_view<is_mutable>* operator->() noexcept {
return &_current_view;
}
shard_iterator& operator++() noexcept {
_current += counter_shard_view::size;
_current_view = basic_counter_shard_view<View>(_current);
_current_view = basic_counter_shard_view<is_mutable>(_current);
return *this;
}
shard_iterator operator++(int) noexcept {
@@ -318,7 +341,7 @@ private:
}
shard_iterator& operator--() noexcept {
_current -= counter_shard_view::size;
_current_view = basic_counter_shard_view<View>(_current);
_current_view = basic_counter_shard_view<is_mutable>(_current);
return *this;
}
shard_iterator operator--(int) noexcept {
@@ -335,22 +358,23 @@ private:
};
public:
boost::iterator_range<shard_iterator> shards() const {
auto bv = _cell.value();
auto begin = shard_iterator(bv.data());
auto end = shard_iterator(bv.data() + bv.size());
auto begin = shard_iterator(_value.data());
auto end = shard_iterator(_value.data() + _value.size());
return boost::make_iterator_range(begin, end);
}
size_t shard_count() const {
return _cell.value().size() / counter_shard_view::size;
return _cell.value().size_bytes() / counter_shard_view::size;
}
public:
protected:
// ac must be a live counter cell
explicit basic_counter_cell_view(atomic_cell_base<View> ac) noexcept : _cell(ac) {
explicit basic_counter_cell_view(basic_atomic_cell_view<is_mutable> ac, linearized_value_view vv) noexcept
: _cell(ac), _value(vv)
{
assert(_cell.is_live());
assert(!_cell.is_counter_update());
}
public:
api::timestamp_type timestamp() const { return _cell.timestamp(); }
static data_type total_value_type() { return long_type; }
@@ -381,14 +405,22 @@ public:
}
};
struct counter_cell_view : basic_counter_cell_view<bytes_view> {
struct counter_cell_view : basic_counter_cell_view<mutable_view::no> {
using basic_counter_cell_view::basic_counter_cell_view;
template<typename Function>
static decltype(auto) with_linearized(basic_atomic_cell_view<mutable_view::no> ac, Function&& fn) {
return ac.value().with_linearized([&] (bytes_view value_view) {
counter_cell_view ccv(ac, value_view);
return fn(ccv);
});
}
// Returns counter shards in an order that is compatible with Scylla 1.7.4.
std::vector<counter_shard> shards_compatible_with_1_7_4() const;
// Reversibly applies two counter cells, at least one of them must be live.
static void apply(atomic_cell_or_collection& dst, atomic_cell_or_collection& src);
static void apply(const column_definition& cdef, atomic_cell_or_collection& dst, atomic_cell_or_collection& src);
// Computes a counter cell containing minimal amount of data which, when
// applied to 'b' returns the same cell as 'a' and 'b' applied together.
@@ -397,9 +429,15 @@ struct counter_cell_view : basic_counter_cell_view<bytes_view> {
friend std::ostream& operator<<(std::ostream& os, counter_cell_view ccv);
};
struct counter_cell_mutable_view : basic_counter_cell_view<bytes_mutable_view> {
struct counter_cell_mutable_view : basic_counter_cell_view<mutable_view::yes> {
using basic_counter_cell_view::basic_counter_cell_view;
explicit counter_cell_mutable_view(atomic_cell_mutable_view ac) noexcept
: basic_counter_cell_view<mutable_view::yes>(ac, ac.value().first_fragment())
{
assert(!ac.value().is_fragmented());
}
void set_timestamp(api::timestamp_type ts) { _cell.set_timestamp(ts); }
};

2
cql3/constants.hh
View File

@@ -210,7 +210,7 @@ public:
if (value.is_null()) {
m.set_cell(prefix, column, std::move(make_dead_cell(params)));
} else if (value.is_value()) {
m.set_cell(prefix, column, std::move(make_cell(*value, params)));
m.set_cell(prefix, column, std::move(make_cell(*column.type, *value, params)));
}
}
};

12
cql3/lists.cc
View File

@@ -304,7 +304,7 @@ lists::setter_by_index::execute(mutation& m, const clustering_key_prefix& prefix
if (!value) {
mut.cells.emplace_back(eidx, params.make_dead_cell());
} else {
mut.cells.emplace_back(eidx, params.make_cell(*value));
mut.cells.emplace_back(eidx, params.make_cell(*ltype->value_comparator(), *value, atomic_cell::collection_member::yes));
}
auto smut = ltype->serialize_mutation_form(mut);
m.set_cell(prefix, column, atomic_cell_or_collection::from_collection_mutation(std::move(smut)));
@@ -331,7 +331,7 @@ lists::setter_by_uuid::execute(mutation& m, const clustering_key_prefix& prefix,
list_type_impl::mutation mut;
mut.cells.reserve(1);
mut.cells.emplace_back(to_bytes(*index), params.make_cell(*value));
mut.cells.emplace_back(to_bytes(*index), params.make_cell(*ltype->value_comparator(), *value, atomic_cell::collection_member::yes));
auto smut = ltype->serialize_mutation_form(mut);
m.set_cell(prefix, column,
atomic_cell_or_collection::from_collection_mutation(
@@ -370,7 +370,7 @@ lists::do_append(shared_ptr<term> value,
auto uuid1 = utils::UUID_gen::get_time_UUID_bytes();
auto uuid = bytes(reinterpret_cast<const int8_t*>(uuid1.data()), uuid1.size());
// FIXME: can e be empty?
appended.cells.emplace_back(std::move(uuid), params.make_cell(*e));
appended.cells.emplace_back(std::move(uuid), params.make_cell(*ltype->value_comparator(), *e, atomic_cell::collection_member::yes));
}
m.set_cell(prefix, column, ltype->serialize_mutation_form(appended));
} else {
@@ -379,7 +379,7 @@ lists::do_append(shared_ptr<term> value,
m.set_cell(prefix, column, params.make_dead_cell());
} else {
auto newv = list_value->get_with_protocol_version(cql_serialization_format::internal());
m.set_cell(prefix, column, params.make_cell(std::move(newv)));
m.set_cell(prefix, column, params.make_cell(*column.type, std::move(newv)));
}
}
}
@@ -400,14 +400,14 @@ lists::prepender::execute(mutation& m, const clustering_key_prefix& prefix, cons
mut.cells.reserve(lvalue->get_elements().size());
// We reverse the order of insertion, so that the last element gets the lastest time
// (lists are sorted by time)
auto&& ltype = static_cast<const list_type_impl*>(column.type.get());
for (auto&& v : lvalue->_elements | boost::adaptors::reversed) {
auto&& pt = precision_time::get_next(time);
auto uuid = utils::UUID_gen::get_time_UUID_bytes(pt.millis.time_since_epoch().count(), pt.nanos);
mut.cells.emplace_back(bytes(uuid.data(), uuid.size()), params.make_cell(*v));
mut.cells.emplace_back(bytes(uuid.data(), uuid.size()), params.make_cell(*ltype->value_comparator(), *v, atomic_cell::collection_member::yes));
}
// now reverse again, to get the original order back
std::reverse(mut.cells.begin(), mut.cells.end());
auto&& ltype = static_cast<const list_type_impl*>(column.type.get());
m.set_cell(prefix, column, atomic_cell_or_collection::from_collection_mutation(ltype->serialize_mutation_form(std::move(mut))));
}

15
cql3/maps.cc
View File

@@ -300,10 +300,11 @@ maps::setter_by_key::execute(mutation& m, const clustering_key_prefix& prefix, c
if (!key) {
throw invalid_request_exception("Invalid null map key");
}
auto avalue = value ? params.make_cell(*value) : params.make_dead_cell();
map_type_impl::mutation update = { {}, { { std::move(to_bytes(*key)), std::move(avalue) } } };
// should have been verified as map earlier?
auto ctype = static_pointer_cast<const map_type_impl>(column.type);
auto avalue = value ? params.make_cell(*ctype->get_values_type(), *value, atomic_cell::collection_member::yes) : params.make_dead_cell();
map_type_impl::mutation update;
update.cells.emplace_back(std::move(to_bytes(*key)), std::move(avalue));
// should have been verified as map earlier?
auto col_mut = ctype->serialize_mutation_form(std::move(update));
m.set_cell(prefix, column, std::move(col_mut));
}
@@ -328,10 +329,10 @@ maps::do_put(mutation& m, const clustering_key_prefix& prefix, const update_para
return;
}
for (auto&& e : map_value->map) {
mut.cells.emplace_back(e.first, params.make_cell(e.second));
}
auto ctype = static_pointer_cast<const map_type_impl>(column.type);
for (auto&& e : map_value->map) {
mut.cells.emplace_back(e.first, params.make_cell(*ctype->get_values_type(), e.second, atomic_cell::collection_member::yes));
}
auto col_mut = ctype->serialize_mutation_form(std::move(mut));
m.set_cell(prefix, column, std::move(col_mut));
} else {
@@ -341,7 +342,7 @@ maps::do_put(mutation& m, const clustering_key_prefix& prefix, const update_para
} else {
auto v = map_type_impl::serialize_partially_deserialized_form({map_value->map.begin(), map_value->map.end()},
cql_serialization_format::internal());
m.set_cell(prefix, column, params.make_cell(std::move(v)));
m.set_cell(prefix, column, params.make_cell(*column.type, std::move(v)));
}
}
}

4
cql3/operation.hh
View File

@@ -91,8 +91,8 @@ public:
return params.make_dead_cell();
}
static atomic_cell make_cell(bytes_view value, const update_parameters& params) {
return params.make_cell(value);
static atomic_cell make_cell(const abstract_type& type, bytes_view value, const update_parameters& params) {
return params.make_cell(type, value);
}
static atomic_cell make_counter_update_cell(int64_t delta, const update_parameters& params) {

2
cql3/restrictions/single_column_restriction.hh
View File

@@ -113,7 +113,7 @@ public:
class contains;
protected:
bytes_view_opt get_value(const schema& schema,
std::optional<atomic_cell_value_view> get_value(const schema& schema,
const partition_key& key,
const clustering_key_prefix& ckey,
const row& cells,

58
cql3/restrictions/statement_restrictions.cc
View File

@@ -430,7 +430,7 @@ void statement_restrictions::validate_secondary_index_selections(bool selects_on
}
}
static bytes_view_opt do_get_value(const schema& schema,
static std::optional<atomic_cell_value_view> do_get_value(const schema& schema,
const column_definition& cdef,
const partition_key& key,
const clustering_key_prefix& ckey,
@@ -438,21 +438,21 @@ static bytes_view_opt do_get_value(const schema& schema,
gc_clock::time_point now) {
switch(cdef.kind) {
case column_kind::partition_key:
return key.get_component(schema, cdef.component_index());
return atomic_cell_value_view(key.get_component(schema, cdef.component_index()));
case column_kind::clustering_key:
return ckey.get_component(schema, cdef.component_index());
return atomic_cell_value_view(ckey.get_component(schema, cdef.component_index()));
default:
auto cell = cells.find_cell(cdef.id);
if (!cell) {
return stdx::nullopt;
return std::nullopt;
}
assert(cdef.is_atomic());
auto c = cell->as_atomic_cell();
return c.is_dead(now) ? stdx::nullopt : bytes_view_opt(c.value());
auto c = cell->as_atomic_cell(cdef);
return c.is_dead(now) ? std::nullopt : std::optional<atomic_cell_value_view>(c.value());
}
}
bytes_view_opt single_column_restriction::get_value(const schema& schema,
std::optional<atomic_cell_value_view> single_column_restriction::get_value(const schema& schema,
const partition_key& key,
const clustering_key_prefix& ckey,
const row& cells,
@@ -472,7 +472,12 @@ bool single_column_restriction::EQ::is_satisfied_by(const schema& schema,
auto operand = value(options);
if (operand) {
auto cell_value = get_value(schema, key, ckey, cells, now);
return cell_value && _column_def.type->compare(*operand, *cell_value) == 0;
if (!cell_value) {
return false;
}
return cell_value->with_linearized([&] (bytes_view cell_value_bv) {
return _column_def.type->compare(*operand, cell_value_bv) == 0;
});
}
return false;
}
@@ -491,9 +496,11 @@ bool single_column_restriction::IN::is_satisfied_by(const schema& schema,
return false;
}
auto operands = values(options);
return cell_value->with_linearized([&] (bytes_view cell_value_bv) {
return std::any_of(operands.begin(), operands.end(), [&] (auto&& operand) {
return operand && _column_def.type->compare(*operand, *cell_value) == 0;
return operand && _column_def.type->compare(*operand, cell_value_bv) == 0;
});
});
}
static query::range<bytes_view> to_range(const term_slice& slice, const query_options& options) {
@@ -526,7 +533,9 @@ bool single_column_restriction::slice::is_satisfied_by(const schema& schema,
if (!cell_value) {
return false;
}
return to_range(_slice, options).contains(*cell_value, _column_def.type->as_tri_comparator());
return cell_value->with_linearized([&] (bytes_view cell_value_bv) {
return to_range(_slice, options).contains(cell_value_bv, _column_def.type->as_tri_comparator());
});
}
bool single_column_restriction::contains::is_satisfied_by(const schema& schema,
@@ -552,7 +561,8 @@ bool single_column_restriction::contains::is_satisfied_by(const schema& schema,
auto&& element_type = col_type->is_set() ? col_type->name_comparator() : col_type->value_comparator();
if (_column_def.type->is_multi_cell()) {
auto cell = cells.find_cell(_column_def.id);
auto&& elements = col_type->deserialize_mutation_form(cell->as_collection_mutation()).cells;
return cell->as_collection_mutation().data.with_linearized([&] (bytes_view collection_bv) {
auto&& elements = col_type->deserialize_mutation_form(collection_bv).cells;
auto end = std::remove_if(elements.begin(), elements.end(), [now] (auto&& element) {
return element.second.is_dead(now);
});
@@ -562,7 +572,9 @@ bool single_column_restriction::contains::is_satisfied_by(const schema& schema,
continue;
}
auto found = std::find_if(elements.begin(), end, [&] (auto&& element) {
return element_type->compare(element.second.value(), *val) == 0;
return element.second.value().with_linearized([&] (bytes_view value_bv) {
return element_type->compare(value_bv, *val) == 0;
});
});
if (found == end) {
return false;
@@ -589,16 +601,26 @@ bool single_column_restriction::contains::is_satisfied_by(const schema& schema,
auto found = std::find_if(elements.begin(), end, [&] (auto&& element) {
return map_key_type->compare(element.first, *map_key) == 0;
});
if (found == end || element_type->compare(found->second.value(), *map_value) != 0) {
if (found == end) {
return false;
}
auto cmp = found->second.value().with_linearized([&] (bytes_view value_bv) {
return element_type->compare(value_bv, *map_value);
});
if (cmp != 0) {
return false;
}
}
return true;
});
} else {
auto cell_value = get_value(schema, key, ckey, cells, now);
if (!cell_value) {
return false;
}
auto deserialized = _column_def.type->deserialize(*cell_value);
auto deserialized = cell_value->with_linearized([&] (bytes_view cell_value_bv) {
return _column_def.type->deserialize(cell_value_bv);
});
for (auto&& value : _values) {
auto val = value->bind_and_get(options);
if (!val) {
@@ -669,7 +691,9 @@ bool token_restriction::EQ::is_satisfied_by(const schema& schema,
for (auto&& operand : values(options)) {
if (operand) {
auto cell_value = do_get_value(schema, **cdef, key, ckey, cells, now);
satisfied = cell_value && (*cdef)->type->compare(*operand, *cell_value) == 0;
satisfied = cell_value && cell_value->with_linearized([&] (bytes_view cell_value_bv) {
return (*cdef)->type->compare(*operand, cell_value_bv) == 0;
});
}
if (!satisfied) {
break;
@@ -691,7 +715,9 @@ bool token_restriction::slice::is_satisfied_by(const schema& schema,
if (!cell_value) {
return false;
}
satisfied = range.contains(*cell_value, cdef->type->as_tri_comparator());
satisfied = cell_value->with_linearized([&] (bytes_view cell_value_bv) {
return range.contains(cell_value_bv, cdef->type->as_tri_comparator());
});
if (!satisfied) {
break;
}

4
cql3/sets.cc
View File

@@ -269,7 +269,7 @@ sets::adder::do_add(mutation& m, const clustering_key_prefix& row_key, const upd
}
for (auto&& e : set_value->_elements) {
mut.cells.emplace_back(e, params.make_cell({}));
mut.cells.emplace_back(e, params.make_cell(*set_type->value_comparator(), {}, atomic_cell::collection_member::yes));
}
auto smut = set_type->serialize_mutation_form(mut);
@@ -279,7 +279,7 @@ sets::adder::do_add(mutation& m, const clustering_key_prefix& row_key, const upd
auto v = set_type->serialize_partially_deserialized_form(
{set_value->_elements.begin(), set_value->_elements.end()},
cql_serialization_format::internal());
m.set_cell(row_key, column, params.make_cell(std::move(v)));
m.set_cell(row_key, column, params.make_cell(*column.type, std::move(v)));
} else {
m.set_cell(row_key, column, params.make_dead_cell());
}

8
cql3/statements/batch_statement.cc
View File

@@ -239,18 +239,18 @@ void batch_statement::verify_batch_size(const std::vector<mutation>& mutations)
public:
void accept_partition_tombstone(tombstone) override {}
void accept_static_cell(column_id, atomic_cell_view v) override {
size += v.value().size();
size += v.value().size_bytes();
}
void accept_static_cell(column_id, collection_mutation_view v) override {
size += v.data.size();
size += v.data.size_bytes();
}
void accept_row_tombstone(const range_tombstone&) override {}
void accept_row(position_in_partition_view, const row_tombstone&, const row_marker&, is_dummy, is_continuous) override {}
void accept_row_cell(column_id, atomic_cell_view v) override {
size += v.value().size();
size += v.value().size_bytes();
}
void accept_row_cell(column_id id, collection_mutation_view v) override {
size += v.data.size();
size += v.data.size_bytes();
}
size_t size = 0;

6
cql3/update_parameters.hh
View File

@@ -142,7 +142,7 @@ public:
return atomic_cell::make_dead(_timestamp, _local_deletion_time);
}
atomic_cell make_cell(bytes_view value) const {
atomic_cell make_cell(const abstract_type& type, bytes_view value, atomic_cell::collection_member cm = atomic_cell::collection_member::no) const {
auto ttl = _ttl;
if (ttl.count() <= 0) {
@@ -150,9 +150,9 @@ public:
}
if (ttl.count() > 0) {
return atomic_cell::make_live(_timestamp, value, _local_deletion_time + ttl, ttl);
return atomic_cell::make_live(type, _timestamp, value, _local_deletion_time + ttl, ttl, cm);
} else {
return atomic_cell::make_live(_timestamp, value);
return atomic_cell::make_live(type, _timestamp, value, cm);
}
};

52
data/cell.cc Normal file
View File

@@ -0,0 +1,52 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#include "data/cell.hh"
#include "types.hh"
thread_local imr::alloc::context_factory<data::cell::last_chunk_context> lcc;
thread_local imr::alloc::lsa_migrate_fn<data::cell::external_last_chunk,
imr::alloc::context_factory<data::cell::last_chunk_context>> data::cell::lsa_last_chunk_migrate_fn(lcc);
thread_local imr::alloc::context_factory<data::cell::chunk_context> ecc;
thread_local imr::alloc::lsa_migrate_fn<data::cell::external_chunk,
imr::alloc::context_factory<data::cell::chunk_context>> data::cell::lsa_chunk_migrate_fn(ecc);
int compare_unsigned(data::value_view lhs, data::value_view rhs) noexcept
{
auto it1 = lhs.begin();
auto it2 = rhs.begin();
while (it1 != lhs.end() && it2 != rhs.end()) {
auto r = ::compare_unsigned(*it1, *it2);
if (r) {
return r;
}
++it1;
++it2;
}
if (it1 != lhs.end()) {
return 1;
} else if (it2 != rhs.end()) {
return -1;
}
return 0;
}

891
data/cell.hh Normal file
View File

@@ -0,0 +1,891 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <boost/range/algorithm/copy.hpp>
#include <boost/range/algorithm/for_each.hpp>
#include <seastar/util/variant_utils.hh>
#include "imr/compound.hh"
#include "imr/fundamental.hh"
#include "imr/alloc.hh"
#include "imr/utils.hh"
#include "imr/concepts.hh"
#include "data/schema_info.hh"
#include "data/value_view.hh"
#include "gc_clock.hh"
#include "timestamp.hh"
namespace data {
template<typename T>
class value_writer;
struct cell {
static constexpr size_t maximum_internal_storage_length = value_view::maximum_internal_storage_length;
static constexpr size_t maximum_external_chunk_length = value_view::maximum_external_chunk_length;
struct tags {
class cell;
class atomic_cell;
class collection;
class flags;
class live;
class expiring;
class counter_update;
class external_data;
class ttl;
class expiry;
class empty;
class timestamp;
class value;
class dead;
class counter_update;
class fixed_value;
class variable_value;
class value_size;
class value_data;
class pointer;
class data;
class external_data;
class chunk_back_pointer;
class chunk_next;
class chunk_data;
class last_chunk_size;
};
using flags = imr::flags<
tags::collection,
tags::live,
tags::expiring,
tags::counter_update,
tags::empty,
tags::external_data
>;
/// Variable-length cell value
///
/// This is a definition of the IMR structure of a variable-length value.
/// It is used both by collections, counters and regular cells which type
/// is variable-sized. The data can be stored internally, if its size is
/// smaller or equal maximum_internal_storage_length or externally if it
/// larger.
struct variable_value {
using data_variant = imr::variant<tags::value_data,
imr::member<tags::pointer, imr::tagged_type<tags::pointer, imr::pod<uint8_t*>>>,
imr::member<tags::data, imr::buffer<tags::data>>
>;
using structure = imr::structure<
imr::member<tags::value_size, imr::pod<uint32_t>>,
imr::member<tags::value_data, data_variant>
>;
/// Create writer of a variable-size value
///
/// Returns a function object that can be used as a writer of a variable
/// value. The first argument is expected to be either IMR sizer or
/// serializer and the second is an appropriate IMR allocator helper
/// object.
/// \arg force_internal if set to true stores the value internally
/// regardless of its size (used by collection members).
template<typename FragmentRange>
static value_writer<std::decay_t<FragmentRange>> write(FragmentRange&& value, bool force_internal = false) noexcept;
static auto write(bytes_view value, bool force_internal = false) noexcept;
/// Create writer of an uninitialised variable-size value
static value_writer<empty_fragment_range> write(size_t size, bool force_internal = false) noexcept;
class context {
bool _external_storage;
uint32_t _value_size;
public:
explicit context(bool external_storage, uint32_t value_size) noexcept
: _external_storage(external_storage), _value_size(value_size) { }
template<typename Tag>
auto active_alternative_of() const noexcept {
if (_external_storage) {
return data_variant::index_for<tags::pointer>();
} else {
return data_variant::index_for<tags::data>();
}
}
template<typename Tag>
size_t size_of() const noexcept {
return _value_size;
}
template<typename Tag, typename... Args>
auto context_for(Args&&...) const noexcept {
return *this;
}
};
template<mutable_view is_mutable>
static basic_value_view<is_mutable> do_make_view(structure::basic_view<is_mutable> view, bool external_storage);
static data::value_view make_view(structure::view view, bool external_storage) {
return do_make_view(view, external_storage);
}
static data::value_mutable_view make_view(structure::mutable_view view, bool external_storage) {
return do_make_view(view, external_storage);
}
};
using fixed_value = imr::buffer<tags::fixed_value>;
/// Cell value
///
/// The cell value can be either a deletion time (if the cell is dead),
/// a delta (counter update cell), fixed-size value or variable-sized value.
using value_variant = imr::variant<tags::value,
imr::member<tags::dead, imr::pod<int32_t>>,
imr::member<tags::counter_update, imr::pod<int64_t>>,
imr::member<tags::fixed_value, fixed_value>,
imr::member<tags::variable_value, variable_value::structure>
>;
/// Atomic cell
///
/// Atomic cells can be either regular cells or counters. Moreover, the
/// cell may be live or dead and the regular cells may have expiration time.
/// Counter cells may be either sets of shards or a delta. The former is not
/// fully converted to the IMR yet and still use a custom serilalisation
/// format. The IMR treats such cells the same way it handles regular blobs.
using atomic_cell = imr::structure<
imr::member<tags::timestamp, imr::pod<api::timestamp_type>>,
imr::optional_member<tags::expiring, imr::structure<
imr::member<tags::ttl, imr::pod<int32_t>>,
imr::member<tags::expiry, imr::pod<int32_t>>
>>,
imr::member<tags::value, value_variant>
>;
using atomic_cell_or_collection = imr::variant<tags::cell,
imr::member<tags::atomic_cell, atomic_cell>,
imr::member<tags::collection, variable_value::structure>
>;
/// Top IMR definition of a cell
///
/// A cell in Scylla's data model can be either atomic (a regular cell,
/// a counter or a frozen collection) or an unfrozen collection. As for now
/// only regular cells are fully utilising the IMR. Collections are still
/// using custom serialisation format and from the IMR point of view are
/// just opaque values.
using structure = imr::structure<
imr::member<tags::flags, flags>,
imr::member<tags::cell, atomic_cell_or_collection>
>;
/// An fragment of externally stored value
///
/// If a cell value size is above maximum_internal_storage_length it is
/// stored externally. Moreover, in order to avoid stressing the memory
/// allocators with large allocations values are fragmented in chunks
/// no larger than maximum_external_chunk_length. The size of all chunks,
/// but the last one is always maximum_external_chunk_length.
using external_chunk = imr::structure<
imr::member<tags::chunk_back_pointer, imr::tagged_type<tags::chunk_back_pointer, imr::pod<uint8_t*>>>,
imr::member<tags::chunk_next, imr::pod<uint8_t*>>,
imr::member<tags::chunk_data, imr::buffer<tags::chunk_data>>
>;
using external_last_chunk_size = imr::pod<uint16_t>;
/// The last fragment of an externally stored value
///
/// The size of the last fragment of a value stored externally may vary.
/// Due to the requirements the LSA imposes on migrators we need to store
/// the size inside it so that it can be retrieved when the LSA migrates
/// object.
using external_last_chunk = imr::structure<
imr::member<tags::chunk_back_pointer, imr::tagged_type<tags::chunk_back_pointer, imr::pod<uint8_t*>>>,
imr::member<tags::last_chunk_size, external_last_chunk_size>,
imr::member<tags::chunk_data, imr::buffer<tags::chunk_data>>
>;
class context;
class minimal_context;
/// Value fragment deserialisation context
///
/// This is a deserialization context for all, but last, value fragments.
/// Their size is fixed.
struct chunk_context {
explicit constexpr chunk_context(const uint8_t*) noexcept { }
template<typename Tag>
static constexpr size_t size_of() noexcept {
return cell::maximum_external_chunk_length;
}
template<typename Tag, typename... Args>
auto context_for(Args&&...) const noexcept {
return *this;
}
};
/// Last value fragment deserialisation context
class last_chunk_context {
uint16_t _size;
public:
explicit last_chunk_context(const uint8_t* ptr) noexcept
: _size(external_last_chunk::get_member<tags::last_chunk_size>(ptr).load())
{ }
template<typename Tag>
size_t size_of() const noexcept {
return _size;
}
template<typename Tag, typename... Args>
auto context_for(Args&&...) const noexcept {
return *this;
}
};
template<mutable_view is_mutable>
class basic_atomic_cell_view;
using atomic_cell_view = basic_atomic_cell_view<mutable_view::no>;
using mutable_atomic_cell_view = basic_atomic_cell_view<mutable_view::yes>;
private:
static thread_local imr::alloc::lsa_migrate_fn<external_last_chunk,
imr::alloc::context_factory<last_chunk_context>> lsa_last_chunk_migrate_fn;
static thread_local imr::alloc::lsa_migrate_fn<external_chunk,
imr::alloc::context_factory<chunk_context>> lsa_chunk_migrate_fn;
public:
/// Make a writer that copies a cell
///
/// This function creates a writer that copies a cell. It can be either
/// atomic or a collection.
///
/// \arg ptr needs to remain valid as long as the writer is in use.
/// \returns imr::WriterAllocator for cell::structure.
static auto copy_fn(const type_info& ti, const uint8_t* ptr);
/// Make a writer for a collection
///
/// \arg data needs to remain valid as long as the writer is in use.
/// \returns imr::WriterAllocator for cell::structure.
template<typename FragmentRange, typename = std::enable_if_t<is_fragment_range_v<std::decay_t<FragmentRange>>>>
static auto make_collection(FragmentRange&& data) noexcept {
return [data] (auto&& serializer, auto&& allocations) noexcept {
return serializer
.serialize(imr::set_flag<tags::collection>(),
imr::set_flag<tags::external_data>(data.size_bytes() > maximum_internal_storage_length))
.template serialize_as<tags::collection>(variable_value::write(data), allocations)
.done();
};
}
static auto make_collection(bytes_view data) noexcept {
return make_collection(single_fragment_range(data));
}
/// Make a writer for a dead cell
///
/// This function returns a generic lambda that is a writer for a dead
/// cell with the specified timestamp and deletion time.
///
/// \returns imr::WriterAllocator for cell::structure.
static auto make_dead(api::timestamp_type ts, gc_clock::time_point deletion_time) noexcept {
return [ts, deletion_time] (auto&& serializer, auto&&...) noexcept {
return serializer
.serialize()
.template serialize_as_nested<tags::atomic_cell>()
.serialize(ts)
.skip()
.template serialize_as<tags::dead>(deletion_time.time_since_epoch().count())
.done()
.done();
};
}
static auto make_live_counter_update(api::timestamp_type ts, int64_t delta) noexcept {
return [ts, delta] (auto&& serializer, auto&&...) noexcept {
return serializer
.serialize(imr::set_flag<tags::live>(),
imr::set_flag<tags::counter_update>())
.template serialize_as_nested<tags::atomic_cell>()
.serialize(ts)
.skip()
.template serialize_as<tags::counter_update>(delta)
.done()
.done();
};
}
/// Make a writer for a live non-expiring cell
///
/// \arg value needs to remain valid as long as the writer is in use.
/// \arg force_internal always store the value internally regardless of its
/// size. This is a temporary (hopefully, sorry if you are reading this in
/// 2020) hack to make integration with collections easier.
///
/// \returns imr::WriterAllocator for cell::structure.
template<typename FragmentRange, typename = std::enable_if_t<is_fragment_range_v<std::decay_t<FragmentRange>>>>
static auto make_live(const type_info& ti, api::timestamp_type ts, FragmentRange&& value, bool force_internal = false) noexcept {
return [&ti, ts, value, force_internal] (auto&& serializer, auto&& allocations) noexcept {
auto after_expiring = serializer
.serialize(imr::set_flag<tags::live>(),
imr::set_flag<tags::empty>(value.empty()),
imr::set_flag<tags::external_data>(!force_internal && !ti.is_fixed_size() && value.size_bytes() > maximum_internal_storage_length))
.template serialize_as_nested<tags::atomic_cell>()
.serialize(ts)
.skip();
return [&] {
if (ti.is_fixed_size()) {
return after_expiring.template serialize_as<tags::fixed_value>(value);
} else {
return after_expiring
.template serialize_as<tags::variable_value>(variable_value::write(value, force_internal), allocations);
}
}().done().done();
};
}
static auto make_live(const type_info& ti, api::timestamp_type ts, bytes_view value, bool force_internal = false) noexcept {
return make_live(ti, ts, single_fragment_range(value), force_internal);
}
template<typename FragmentRange, typename = std::enable_if_t<is_fragment_range_v<std::decay_t<FragmentRange>>>>
static auto make_live(const type_info& ti, api::timestamp_type ts, FragmentRange&& value, gc_clock::time_point expiry, gc_clock::duration ttl, bool force_internal = false) noexcept
{
return [&ti, ts, value, expiry, ttl, force_internal] (auto&& serializer, auto&& allocations) noexcept {
auto after_expiring = serializer
.serialize(imr::set_flag<tags::live>(),
imr::set_flag<tags::expiring>(),
imr::set_flag<tags::empty>(value.empty()),
imr::set_flag<tags::external_data>(!force_internal && !ti.is_fixed_size() && value.size_bytes() > maximum_internal_storage_length))
.template serialize_as_nested<tags::atomic_cell>()
.serialize(ts)
.serialize_nested()
.serialize(ttl.count())
.serialize(expiry.time_since_epoch().count())
.done();
return [&] {
if (ti.is_fixed_size()) {
return after_expiring.template serialize_as<tags::fixed_value>(value);
} else {
return after_expiring
.template serialize_as<tags::variable_value>(variable_value::write(value, force_internal), allocations);
}
}().done().done();
};
}
static auto make_live(const type_info& ti, api::timestamp_type ts, bytes_view value, gc_clock::time_point expiry, gc_clock::duration ttl, bool force_internal = false) noexcept {
return make_live(ti, ts, single_fragment_range(value), expiry, ttl, force_internal);
}
/// Make a writer of a live cell with uninitialised value
///
/// This function returns a function object which is a writer of a live
/// cell. The space for value is allocated but not initialised. This can be
/// used if the value is a result of some IMR-independent serialisation
/// (e.g. counters).
///
/// \returns imr::WriterAllocator for cell::structure.
static auto make_live_uninitialized(const type_info& ti, api::timestamp_type ts, size_t size) noexcept {
return [&ti, ts, size] (auto&& serializer, auto&& allocations) noexcept {
auto after_expiring = serializer
.serialize(imr::set_flag<tags::live>(),
imr::set_flag<tags::empty>(!size),
imr::set_flag<tags::external_data>(!ti.is_fixed_size() && size > maximum_internal_storage_length))
.template serialize_as_nested<tags::atomic_cell>()
.serialize(ts)
.skip();
return [&] {
if (ti.is_fixed_size()) {
return after_expiring.template serialize_as<tags::fixed_value>(size, [] (uint8_t*) noexcept { });
} else {
return after_expiring
.template serialize_as<tags::variable_value>(variable_value::write(size, false), allocations);
}
}().done().done();
};
}
template<typename Builder>
static size_t size_of(Builder&& builder, imr::alloc::object_allocator& allocator) noexcept {
return structure::size_when_serialized(std::forward<Builder>(builder), allocator.get_sizer());
}
template<typename Builder>
static size_t serialize(uint8_t* ptr, Builder&& builder, imr::alloc::object_allocator& allocator) noexcept {
return structure::serialize(ptr, std::forward<Builder>(builder), allocator.get_serializer());
}
static atomic_cell_view make_atomic_cell_view(const type_info& ti, const uint8_t* ptr) noexcept;
static mutable_atomic_cell_view make_atomic_cell_view(const type_info& ti, uint8_t* ptr) noexcept;
static void destroy(uint8_t* ptr) noexcept;
};
/// Minimal cell deserialisation context
///
/// This is a minimal deserialisation context that doesn't require the cell
/// type to be known, but allows only some operations to be performed. In
/// particular it is able to provide sufficient information to destroy a cell.
class cell::minimal_context {
protected:
cell::flags::view _flags;
public:
explicit minimal_context(cell::flags::view flags) noexcept
: _flags(flags) { }
template<typename Tag>
bool is_present() const noexcept;
template<typename Tag>
auto active_alternative_of() const noexcept;
template<typename Tag>
size_t size_of() const noexcept;
template<typename Tag>
auto context_for(const uint8_t*) const noexcept {
return *this;
}
};
template<>
inline bool cell::minimal_context::is_present<cell::tags::expiring>() const noexcept {
return _flags.get<tags::expiring>();
}
template<>
inline auto cell::minimal_context::active_alternative_of<cell::tags::cell>() const noexcept {
if (_flags.get<tags::collection>()) {
return cell::atomic_cell_or_collection::index_for<tags::collection>();
} else {
return cell::atomic_cell_or_collection::index_for<tags::atomic_cell>();
}
}
/// Cell deserialisation context
///
/// This class combines schema-dependnent and instance-specific information
/// and provides an appropriate interface for the IMR deserialisation routines
/// to read a cell.
class cell::context : public cell::minimal_context {
type_info _type;
public:
explicit context(const uint8_t* ptr, const type_info& tinfo) noexcept
: context(structure::get_member<tags::flags>(ptr), tinfo) { }
explicit context(cell::flags::view flags, const type_info& tinfo) noexcept
: minimal_context(flags), _type(tinfo) { }
template<typename Tag>
bool is_present() const noexcept {
return minimal_context::is_present<Tag>();
}
template<typename Tag>
auto active_alternative_of() const noexcept {
return minimal_context::active_alternative_of<Tag>();
}
template<typename Tag>
size_t size_of() const noexcept;
template<typename Tag>
auto context_for(const uint8_t*) const noexcept {
return *this;
}
};
template<>
inline auto cell::context::context_for<cell::tags::variable_value>(const uint8_t* ptr) const noexcept {
auto length = variable_value::structure::get_member<tags::value_size>(ptr);
return variable_value::context(_flags.get<tags::external_data>(), length.load());
}
template<>
inline auto cell::context::context_for<cell::tags::collection>(const uint8_t* ptr) const noexcept {
auto length = variable_value::structure::get_member<tags::value_size>(ptr);
return variable_value::context(_flags.get<tags::external_data>(), length.load());
}
template<>
inline auto cell::context::active_alternative_of<cell::tags::value>() const noexcept {
if (_flags.get<tags::live>()) {
if (__builtin_expect(_flags.get<tags::counter_update>(), false)) {
return cell::value_variant::index_for<tags::counter_update>();
}
if (_type.is_fixed_size()) {
return cell::value_variant::index_for<tags::fixed_value>();
} else {
return cell::value_variant::index_for<tags::variable_value>();
}
} else {
return cell::value_variant::index_for<tags::dead>();
}
}
template<>
inline size_t cell::context::size_of<cell::tags::fixed_value>() const noexcept {
return _flags.get<tags::empty>() ? 0 : _type.value_size();
}
/// Atomic cell view
///
/// This is a, possibly mutable, view of an atomic cell. It is a wrapper on top
/// of IMR-generated view that provides more convenient interface which doesn't
/// depend on the actual cell structure.
///
/// \note Instances of this class are being copied and passed by value a lot.
/// It is desireable that it remains small and trivial, so that the compiler
/// can try to keep it in registers at all times. We also should not worry too
/// much about computing the same thing more than once (unless the profiler
/// tells otherwise, of course). Most of the IMR code and its direct users rely
/// heavily on inlining which would allow the compiler remove duplicated
/// computations.
template<mutable_view is_mutable>
class cell::basic_atomic_cell_view {
public:
using view_type = structure::basic_view<is_mutable>;
private:
type_info _type;
view_type _view;
private:
flags::view flags_view() const noexcept {
return _view.template get<tags::flags>();
}
atomic_cell::basic_view<is_mutable> cell_view() const noexcept {
return _view.template get<tags::cell>().template as<tags::atomic_cell>();
}
context make_context() const noexcept {
return context(flags_view(), _type);
}
public:
basic_atomic_cell_view(const type_info& ti, view_type v) noexcept
: _type(ti), _view(std::move(v)) { }
operator basic_atomic_cell_view<mutable_view::no>() const noexcept {
return basic_atomic_cell_view<mutable_view::no>(_type, _view);
}
const uint8_t* raw_pointer() const { return _view.raw_pointer(); }
bytes_view serialize() const noexcept {
assert(!flags_view().template get<tags::external_data>());
auto ptr = raw_pointer();
auto len = structure::serialized_object_size(ptr, make_context());
return bytes_view(reinterpret_cast<const int8_t*>(ptr), len);
}
bool is_live() const noexcept {
return flags_view().template get<tags::live>();
}
bool is_expiring() const noexcept {
return flags_view().template get<tags::expiring>();
}
bool is_counter_update() const noexcept {
return flags_view().template get<tags::counter_update>();
}
api::timestamp_type timestamp() const noexcept {
return cell_view().template get<tags::timestamp>().load();
}
void set_timestamp(api::timestamp_type ts) noexcept {
cell_view().template get<tags::timestamp>().store(ts);
}
gc_clock::time_point expiry() const noexcept {
auto v = cell_view().template get<tags::expiring>().get().template get<tags::expiry>().load();
return gc_clock::time_point(gc_clock::duration(v));
}
gc_clock::duration ttl() const noexcept {
auto v = cell_view().template get<tags::expiring>().get().template get<tags::ttl>().load();
return gc_clock::duration(v);
}
gc_clock::time_point deletion_time() const noexcept {
auto v = cell_view().template get<tags::value>(make_context()).template as<tags::dead>().load();
return gc_clock::time_point(gc_clock::duration(v));
}
int64_t counter_update_value() const noexcept {
return cell_view().template get<tags::value>(make_context()).template as<tags::counter_update>().load();
}
basic_value_view<is_mutable> value() const noexcept {
auto ctx = make_context();
return cell_view().template get<tags::value>(ctx).visit(make_visitor(
[] (fixed_value::basic_view<is_mutable> view) { return basic_value_view<is_mutable>(view, 0, nullptr); },
[&] (variable_value::structure::basic_view<is_mutable> view) {
return variable_value::make_view(view, flags_view().template get<tags::external_data>());
},
[] (...) -> basic_value_view<is_mutable> { abort(); }
), ctx);
}
size_t value_size() const noexcept {
auto ctx = make_context();
return cell_view().template get<tags::value>(ctx).visit(make_visitor(
[] (fixed_value::view view) -> size_t { return view.size(); },
[] (variable_value::structure::view view) -> size_t {
return view.template get<tags::value_size>().load();
},
[] (...) -> size_t { abort(); }
), ctx);
}
bool is_value_fragmented() const noexcept {
return flags_view().template get<tags::external_data>() && value_size() > maximum_external_chunk_length;
}
};
inline auto cell::copy_fn(const type_info& ti, const uint8_t* ptr)
{
// Slow path
return [&ti, ptr] (auto&& serializer, auto&& allocations) noexcept {
auto f = structure::get_member<tags::flags>(ptr);
context ctx(ptr, ti);
if (f.get<tags::collection>()) {
auto view = structure::get_member<tags::cell>(ptr).as<tags::collection>(ctx);
auto dv = variable_value::make_view(view, f.get<tags::external_data>());
return make_collection(dv)(serializer, allocations);
} else {
auto acv = atomic_cell_view(ti, structure::make_view(ptr, ti));
if (acv.is_live()) {
if (acv.is_counter_update()) {
return make_live_counter_update(acv.timestamp(), acv.counter_update_value())(serializer, allocations);
} else if (acv.is_expiring()) {
return make_live(ti, acv.timestamp(), acv.value(), acv.expiry(), acv.ttl())(serializer, allocations);
}
return make_live(ti, acv.timestamp(), acv.value())(serializer, allocations);
} else {
return make_dead(acv.timestamp(), acv.deletion_time())(serializer, allocations);
}
}
};
}
inline cell::atomic_cell_view cell::make_atomic_cell_view(const type_info& ti, const uint8_t* ptr) noexcept {
return atomic_cell_view(ti, structure::make_view(ptr));
}
inline cell::mutable_atomic_cell_view cell::make_atomic_cell_view(const type_info& ti, uint8_t* ptr) noexcept {
return mutable_atomic_cell_view(ti, structure::make_view(ptr));
}
/// Context for external value destruction
///
/// When a cell value is stored externally as a list of fragments we need to
/// know when we reach the last fragment. The way to do that is to read the
/// total value size from the parent cell object and use the fact that the size
/// of all fragments except the last one is cell::maximum_external_chunk_length.
class fragment_chain_destructor_context : public imr::no_context_t {
size_t _total_length;
public:
explicit fragment_chain_destructor_context(size_t total_length) noexcept
: _total_length(total_length) { }
void next_chunk() noexcept { _total_length -= data::cell::maximum_external_chunk_length; }
bool is_last_chunk() const noexcept { return _total_length <= data::cell::maximum_external_chunk_length; }
};
}
namespace imr {
namespace methods {
/// Cell destructor
///
/// If the cell value exceeds certain thresholds its value is stored externally
/// (possibly fragmented). This requires a destructor so that the owned memory
/// can be freed when the cell is destroyed.
/// Note that we don't need to know the actual type of the cell to destroy it,
/// since all the necessary information is stored in each instance. This means
/// that IMR cells can be owned by C++ objects without the problem of passing
/// arguments to C++ destructors.
template<>
struct destructor<data::cell::structure> {
static void run(uint8_t* ptr, ...) {
auto flags = data::cell::structure::get_member<data::cell::tags::flags>(ptr);
if (flags.get<data::cell::tags::external_data>()) {
auto cell_offset = data::cell::structure::offset_of<data::cell::tags::cell>(ptr);
auto variable_value_ptr = [&] {
if (flags.get<data::cell::tags::collection>()) {
return ptr + cell_offset;
} else {
auto ctx = data::cell::minimal_context(flags);
auto offset = data::cell::atomic_cell::offset_of<data::cell::tags::value>(ptr + cell_offset, ctx);
return ptr + cell_offset + offset;
}
}();
imr::methods::destroy<data::cell::variable_value>(variable_value_ptr);
}
}
};
/// Cell mover
template<>
struct mover<data::cell::structure> {
static void run(uint8_t* ptr, ...) {
auto flags = data::cell::structure::get_member<data::cell::tags::flags>(ptr);
if (flags.get<data::cell::tags::external_data>()) {
auto cell_offset = data::cell::structure::offset_of<data::cell::tags::cell>(ptr);
auto variable_value_ptr = [&] {
if (flags.get<data::cell::tags::collection>()) {
return ptr + cell_offset;
} else {
auto ctx = data::cell::minimal_context(flags);
auto offset = data::cell::atomic_cell::offset_of<data::cell::tags::value>(ptr + cell_offset, ctx);
return ptr + cell_offset + offset;
}
}();
variable_value_ptr += data::cell::variable_value::structure::offset_of<data::cell::tags::value_data>(variable_value_ptr);
imr::methods::move<imr::tagged_type<data::cell::tags::pointer, imr::pod<uint8_t*>>>(variable_value_ptr);
}
}
};
template<>
struct destructor<data::cell::variable_value> {
static void run(uint8_t* ptr, ...) {
auto varval = data::cell::variable_value::structure::make_view(ptr);
auto total_length = varval.template get<data::cell::tags::value_size>().load();
if (total_length <= data::cell::maximum_internal_storage_length) {
return;
}
auto ctx = data::fragment_chain_destructor_context(total_length);
auto ptr_view = varval.get<data::cell::tags::value_data>().as<data::cell::tags::pointer>();
if (ctx.is_last_chunk()) {
imr::methods::destroy<data::cell::external_last_chunk>(ptr_view.load());
} else {
imr::methods::destroy<data::cell::external_chunk>(ptr_view.load(), ctx);
}
current_allocator().free(ptr_view.load());
}
};
template<>
struct mover<imr::tagged_type<data::cell::tags::pointer, imr::pod<uint8_t*>>> {
static void run(uint8_t* ptr, ...) {
auto ptr_view = imr::pod<uint8_t*>::make_view(ptr);
auto chk_ptr = ptr_view.load();
auto chk = data::cell::external_last_chunk::make_view(chk_ptr, data::cell::last_chunk_context(chk_ptr));
chk.get<data::cell::tags::chunk_back_pointer>().store(ptr);
}
};
template<>
struct mover<imr::tagged_type<data::cell::tags::chunk_back_pointer, imr::pod<uint8_t*>>> {
static void run(uint8_t* bptr, ...) {
auto bptr_view = imr::pod<uint8_t*>::make_view(bptr);
auto ptr_ptr = bptr_view.load();
auto ptr = imr::pod<uint8_t*>::make_view(ptr_ptr);
ptr.store(bptr);
}
};
/// External chunk destructor
template<>
struct destructor<data::cell::external_chunk> {
static void run(uint8_t* ptr, data::fragment_chain_destructor_context ctx) {
bool first = true;
while (true) {
ctx.next_chunk();
auto echk_view = data::cell::external_chunk::make_view(ptr);
auto ptr_view = echk_view.get<data::cell::tags::chunk_next>();
if (ctx.is_last_chunk()) {
imr::methods::destroy<data::cell::external_last_chunk>(ptr_view.load());
current_allocator().free(ptr_view.load());
if (!first) {
current_allocator().free(ptr);
}
break;
} else {
auto last = ptr;
ptr = ptr_view.load();
if (!first) {
current_allocator().free(last);
} else {
first = false;
}
}
}
}
};
template<>
struct mover<data::cell::external_chunk> {
static void run(uint8_t* ptr, ...) {
auto echk_view = data::cell::external_chunk::make_view(ptr, data::cell::chunk_context(ptr));
auto next_ptr = echk_view.get<data::cell::tags::chunk_next>().load();
auto bptr = imr::pod<uint8_t*>::make_view(next_ptr);
bptr.store(ptr + echk_view.offset_of<data::cell::tags::chunk_next>());
auto back_ptr = echk_view.get<data::cell::tags::chunk_back_pointer>().load();
auto nptr = imr::pod<uint8_t*>::make_view(back_ptr);
nptr.store(ptr);
}
};
}
}
template<>
struct appending_hash<data::value_view> {
template<typename Hasher>
void operator()(Hasher& h, data::value_view v) const {
feed_hash(h, v.size_bytes());
using boost::range::for_each;
for_each(v, [&h] (auto&& chk) {
h.update(reinterpret_cast<const char*>(chk.data()), chk.size());
});
}
};
int compare_unsigned(data::value_view lhs, data::value_view rhs) noexcept;
namespace data {
struct type_imr_descriptor {
using context_factory = imr::alloc::context_factory<imr::utils::object_context<cell::context, data::type_info>, data::type_info>;
using lsa_migrate_fn = imr::alloc::lsa_migrate_fn<imr::utils::object<cell::structure>::structure, context_factory>;
private:
data::type_info _type_info;
lsa_migrate_fn _lsa_migrator;
public:
explicit type_imr_descriptor(data::type_info ti)
: _type_info(ti)
, _lsa_migrator(context_factory(ti))
{ }
const data::type_info& type_info() const { return _type_info; }
const lsa_migrate_fn& lsa_migrator() const { return _lsa_migrator; }
};
}
#include "value_view_impl.hh"
#include "cell_impl.hh"

179
data/cell_impl.hh Normal file
View File

@@ -0,0 +1,179 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "data/cell.hh"
namespace data {
template<typename FragmentRange>
class value_writer {
FragmentRange _value;
typename FragmentRange::const_iterator _value_it;
typename FragmentRange::const_iterator _value_end;
bytes_view _value_current;
size_t _value_size;
bool _force_internal;
private:
// Distinguishes between cell::make_live_uninitialized() and other
// cell::make_live*() variants.
static constexpr bool initialize_value() {
return !std::is_same_v<FragmentRange, empty_fragment_range>;
}
auto write_all_to_destination() {
if constexpr (initialize_value()) {
return [this] (uint8_t* out) noexcept {
auto dst = reinterpret_cast<bytes_mutable_view::pointer>(out);
while (_value_it != _value_end) {
_value_current = *_value_it++;
dst = std::copy_n(_value_current.data(), _value_current.size(), dst);
}
};
} else {
return [] (uint8_t*) noexcept { };
}
}
auto write_to_destination(size_t n) {
if constexpr (initialize_value()) {
return [this, n] (uint8_t* out) mutable noexcept {
auto dst = reinterpret_cast<bytes_mutable_view::pointer>(out);
while (n) {
auto this_size = std::min(_value_current.size(), n);
dst = std::copy_n(_value_current.data(), this_size, dst);
_value_current.remove_prefix(this_size);
if (_value_current.empty()) {
++_value_it;
_value_current = *_value_it;
}
n -= this_size;
}
};
} else {
return [] (uint8_t*) noexcept { };
}
}
public:
value_writer(FragmentRange value, size_t value_size, bool force_internal)
: _value(std::move(value))
, _value_it(_value.begin())
, _value_end(_value.end())
, _value_current(_value.empty() ? bytes_view() : *_value_it)
, _value_size(value_size)
, _force_internal(force_internal)
{ }
template<typename Serializer, typename Allocator>
GCC6_CONCEPT(
requires (imr::is_sizer_for_v<cell::variable_value::structure, Serializer>
&& std::is_same_v<Allocator, imr::alloc::object_allocator::sizer>)
|| (imr::is_serializer_for_v<cell::variable_value::structure, Serializer>
&& std::is_same_v<Allocator, imr::alloc::object_allocator::serializer>)
)
auto operator()(Serializer serializer, Allocator allocations) {
auto after_size = serializer.serialize(_value_size);
if (_force_internal || _value_size <= cell::maximum_internal_storage_length) {
return after_size
.template serialize_as<cell::tags::data>(_value_size, write_all_to_destination())
.done();
}
imr::placeholder<imr::pod<uint8_t*>> next_pointer_phldr;
auto next_pointer_position = after_size.position();
auto cell_ser = after_size.template serialize_as<cell::tags::pointer>(next_pointer_phldr);
auto offset = 0;
auto migrate_fn_ptr = &cell::lsa_chunk_migrate_fn;
while (_value_size - offset > cell::maximum_external_chunk_length) {
imr::placeholder<imr::pod<uint8_t*>> phldr;
auto chunk_ser = allocations.template allocate_nested<cell::external_chunk>(migrate_fn_ptr)
.serialize(next_pointer_position);
next_pointer_position = chunk_ser.position();
next_pointer_phldr.serialize(
chunk_ser.serialize(phldr)
.serialize(cell::maximum_external_chunk_length,
write_to_destination(cell::maximum_external_chunk_length))
.done()
);
next_pointer_phldr = phldr;
offset += cell::maximum_external_chunk_length;
}
size_t left = _value_size - offset;
auto ptr = allocations.template allocate_nested<cell::external_last_chunk>(&cell::lsa_last_chunk_migrate_fn)
.serialize(next_pointer_position)
.serialize(left)
.serialize(left, write_to_destination(left))
.done();
next_pointer_phldr.serialize(ptr);
return cell_ser.done();
}
};
inline value_writer<empty_fragment_range> cell::variable_value::write(size_t value_size, bool force_internal) noexcept
{
GCC6_CONCEPT(static_assert(imr::WriterAllocator<value_writer<empty_fragment_range>, structure>));
return value_writer<empty_fragment_range>(empty_fragment_range(), value_size, force_internal);
}
template<typename FragmentRange>
inline value_writer<std::decay_t<FragmentRange>> cell::variable_value::write(FragmentRange&& value, bool force_internal) noexcept
{
GCC6_CONCEPT(static_assert(imr::WriterAllocator<value_writer<std::decay_t<FragmentRange>>, structure>));
return value_writer<std::decay_t<FragmentRange>>(std::forward<FragmentRange>(value), value.size_bytes(), force_internal);
}
inline auto cell::variable_value::write(bytes_view value, bool force_internal) noexcept
{
return write(single_fragment_range(value), force_internal);
}
template<mutable_view is_mutable>
inline basic_value_view<is_mutable> cell::variable_value::do_make_view(structure::basic_view<is_mutable> view, bool external_storage)
{
auto size = view.template get<tags::value_size>().load();
context ctx(external_storage, size);
return view.template get<tags::value_data>().visit(make_visitor(
[&] (imr::pod<uint8_t*>::view ptr) {
auto ex_ptr = static_cast<uint8_t*>(ptr.load());
if (size > maximum_external_chunk_length) {
auto ex_ctx = chunk_context(ex_ptr);
auto ex_view = external_chunk::make_view(ex_ptr, ex_ctx);
auto next = static_cast<uint8_t*>(ex_view.get<tags::chunk_next>().load());
return basic_value_view<is_mutable>(ex_view.get<tags::chunk_data>(ex_ctx), size - maximum_external_chunk_length, next);
} else {
auto ex_ctx = last_chunk_context(ex_ptr);
auto ex_view = external_last_chunk::make_view(ex_ptr, ex_ctx);
assert(ex_view.get<tags::chunk_data>(ex_ctx).size() == size);
return basic_value_view<is_mutable>(ex_view.get<tags::chunk_data>(ex_ctx), 0, nullptr);
}
},
[] (imr::buffer<tags::data>::basic_view<is_mutable> data) {
return basic_value_view<is_mutable>(data, 0, nullptr);
}
), ctx);
}
}

65
data/schema_info.hh Normal file
View File

@@ -0,0 +1,65 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <cstddef>
namespace data {
/// Type information
///
/// `type_info` keeps the type information relevant for the serialisation code.
/// In particular we need to distinguish between fixed-size and variable-sized
/// types. Collections and counters are considered to be variable-sized types.
///
/// \note Even if the type is fixed-size (e.g. `int32_type`) the value can be
/// empty and its length will be 0. This is a special (and rare) case handled
/// by the cell implementation and ignored by `type_info`.
class type_info {
size_t _fixed_size;
private:
explicit type_info(size_t size) noexcept : _fixed_size(size) { }
public:
static type_info make_fixed_size(size_t size) noexcept {
return type_info { size_t(size) };
}
static type_info make_variable_size() noexcept {
return type_info { 0 };
}
static type_info make_collection() noexcept {
return type_info { 0 };
}
/// Check whether the type is fixed-size.
bool is_fixed_size() const noexcept {
return _fixed_size > 0;
}
/// Get the size of the value of a fixed-size type.
///
/// Valid only if `is_fixed_size()` returns `true`.
size_t value_size() const noexcept {
return _fixed_size;
}
};
}

140
data/value_view.hh Normal file
View File

@@ -0,0 +1,140 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "utils/fragment_range.hh"
namespace data {
/// View of a cell value
///
/// `basic_value_view` is a non-owning reference to a, possibly fragmented,
/// opaque value of a cell. It behaves like an immutable range of fragments.
///
/// Moreover, there are functions that linearise the value in order to ease the
/// integration with the pre-existing code. Nevertheless, using them should be
/// avoided.
///
/// \note For now `basic_value_view` is used by regular atomic cells, counters
/// and collections. This is due to the fact that counters and collections
/// haven't been fully transitioned to the IMR yet and still use custom
/// serialisation formats. Once this is resolved `value_view` can be used
/// exclusively by regular atomic cells.
template<mutable_view is_mutable>
class basic_value_view {
public:
static constexpr size_t maximum_internal_storage_length = 8 * 1024;
static constexpr size_t maximum_external_chunk_length = 8 * 1024;
using fragment_type = std::conditional_t<is_mutable == mutable_view::no,
bytes_view, bytes_mutable_view>;
using raw_pointer_type = std::conditional_t<is_mutable == mutable_view::no,
const uint8_t*, uint8_t*>;
private:
size_t _remaining_size;
fragment_type _first_fragment;
raw_pointer_type _next;
public:
basic_value_view(fragment_type first, size_t remaining_size, raw_pointer_type next)
: _remaining_size(remaining_size), _first_fragment(first), _next(next)
{ }
explicit basic_value_view(fragment_type first)
: basic_value_view(first, 0, nullptr)
{ }
/// Iterator over fragments
class iterator {
fragment_type _view;
raw_pointer_type _next;
size_t _left;
public:
using iterator_category = std::forward_iterator_tag;
using value_type = fragment_type;
using pointer = const fragment_type*;
using reference = const fragment_type&;
using difference_type = std::ptrdiff_t;
iterator(fragment_type bv, size_t total, raw_pointer_type next) noexcept
: _view(bv), _next(next), _left(total) { }
const fragment_type& operator*() const {
return _view;
}
const fragment_type* operator->() const {
return &_view;
}
iterator& operator++();
iterator operator++(int) {
auto it = *this;
operator++();
return it;
}
bool operator==(const iterator& other) const {
return _view.data() == other._view.data();
}
bool operator!=(const iterator& other) const {
return !(*this == other);
}
};
using const_iterator = iterator;
auto begin() const {
return iterator(_first_fragment, _remaining_size, _next);
}
auto end() const {
return iterator(fragment_type(), 0, nullptr);
}
bool operator==(const basic_value_view& other) const noexcept;
bool operator==(bytes_view bv) const noexcept;
/// Total size of the value
size_t size_bytes() const noexcept {
return _first_fragment.size() + _remaining_size;
}
bool empty() const noexcept {
return _first_fragment.empty();
}
bool is_fragmented() const noexcept {
return bool(_next);
}
fragment_type first_fragment() const noexcept {
return _first_fragment;
}
bytes linearize() const;
template<typename Function>
decltype(auto) with_linearized(Function&& fn) const;
};
using value_view = basic_value_view<mutable_view::no>;
using value_mutable_view = basic_value_view<mutable_view::yes>;
}

116
data/value_view_impl.hh Normal file
View File

@@ -0,0 +1,116 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "data/cell.hh"
namespace data {
template<mutable_view is_mutable>
inline typename basic_value_view<is_mutable>::iterator& basic_value_view<is_mutable>::iterator::operator++()
{
if (!_next) {
_view = fragment_type();
} else if (_left > maximum_external_chunk_length) {
cell::chunk_context ctx(_next);
auto v = cell::external_chunk::make_view(_next, ctx);
_next = static_cast<uint8_t*>(v.template get<cell::tags::chunk_next>(ctx).load());
_view = v.template get<cell::tags::chunk_data>(ctx);
_left -= cell::maximum_external_chunk_length;
} else {
cell::last_chunk_context ctx(_next);
auto v = cell::external_last_chunk::make_view(_next, ctx);
_view = v.template get<cell::tags::chunk_data>(ctx);
_next = nullptr;
}
return *this;
}
template<mutable_view is_mutable>
inline bool basic_value_view<is_mutable>::operator==(const basic_value_view& other) const noexcept
{
// We can assume that all values are fragmented exactly in the same way.
auto it1 = begin();
auto it2 = other.begin();
while (it1 != end() && it2 != other.end()) {
if (*it1 != *it2) {
return false;
}
++it1;
++it2;
}
return it1 == end() && it2 == other.end();
}
template<mutable_view is_mutable>
inline bool basic_value_view<is_mutable>::operator==(bytes_view bv) const noexcept
{
bool equal = true;
using boost::range::for_each;
for_each(*this, [&] (bytes_view fragment) {
if (fragment.size() > bv.size()) {
equal = false;
} else {
auto bv_frag = bv.substr(0, fragment.size());
equal = equal && fragment == bv_frag;
bv.remove_prefix(fragment.size());
}
});
return equal && bv.empty();
}
template<mutable_view is_mutable>
inline bytes basic_value_view<is_mutable>::linearize() const
{
bytes b(bytes::initialized_later(), size_bytes());
auto it = b.begin();
for (auto fragment : *this) {
it = boost::copy(fragment, it);
}
return b;
}
template<mutable_view is_mutable>
template<typename Function>
inline decltype(auto) basic_value_view<is_mutable>::with_linearized(Function&& fn) const
{
bytes b;
bytes_view bv;
if (is_fragmented()) {
b = linearize();
bv = b;
} else {
bv = _first_fragment;
}
return fn(bv);
}
inline std::ostream& operator<<(std::ostream& os, value_view vv)
{
using boost::range::for_each;
for_each(vv, [&os] (bytes_view fragment) {
os << fragment;
});
return os;
}
}

4
database.cc
View File

@@ -634,7 +634,7 @@ column_family::find_row(schema_ptr s, const dht::decorated_key& partition_key, c
auto r = p->find_row(*s, clustering_key);
if (r) {
// FIXME: remove copy if only one data source
return make_ready_future<const_row_ptr>(std::make_unique<row>(*r));
return make_ready_future<const_row_ptr>(std::make_unique<row>(*s, column_kind::regular_column, *r));
} else {
return make_ready_future<const_row_ptr>();
}
@@ -3681,7 +3681,7 @@ std::ostream&
operator<<(std::ostream& os, const atomic_cell_view& acv) {
if (acv.is_live()) {
return fprint(os, "atomic_cell{%s;ts=%d;expiry=%d,ttl=%d}",
to_hex(acv.value()),
to_hex(acv.value().linearize()),
acv.timestamp(),
acv.is_live_and_has_ttl() ? acv.expiry().time_since_epoch().count() : -1,
acv.is_live_and_has_ttl() ? acv.ttl().count() : 0);

10
db/schema_tables.cc
View File

@@ -796,7 +796,7 @@ static void delete_schema_version(mutation& m) {
auto&& cell = cells.find_cell(version_col.id);
api::timestamp_type t = api::new_timestamp();
if (cell) {
t = std::max(t, cell->as_atomic_cell().timestamp());
t = std::max(t, cell->as_atomic_cell(version_col).timestamp());
}
cells.apply(version_col, atomic_cell::make_dead(t, gc_clock::now()));
}
@@ -1229,7 +1229,7 @@ make_map_mutation(const Map& map,
for (auto&& entry : map) {
auto te = f(entry);
mut.cells.emplace_back(ktyp->decompose(data_value(te.first)), atomic_cell::make_live(timestamp, vtyp->decompose(data_value(te.second))));
mut.cells.emplace_back(ktyp->decompose(data_value(te.first)), atomic_cell::make_live(*vtyp, timestamp, vtyp->decompose(data_value(te.second)), atomic_cell::collection_member::yes));
}
auto col_mut = column_type->serialize_mutation_form(std::move(mut));
@@ -1238,7 +1238,7 @@ make_map_mutation(const Map& map,
map_type_impl::native_type tmp;
tmp.reserve(map.size());
std::transform(map.begin(), map.end(), std::inserter(tmp, tmp.end()), f);
return atomic_cell::make_live(timestamp, column_type->decompose(make_map_value(column_type, std::move(tmp))));
return atomic_cell::make_live(*column.type, timestamp, column_type->decompose(make_map_value(column_type, std::move(tmp))));
}
}
@@ -1398,7 +1398,7 @@ make_list_mutation(const std::vector<T, Args...>& values,
auto uuid = utils::UUID_gen::get_time_UUID_bytes();
m.cells.emplace_back(
bytes(reinterpret_cast<const int8_t*>(uuid.data()), uuid.size()),
atomic_cell::make_live(timestamp, vtyp->decompose(std::move(dv))));
atomic_cell::make_live(*vtyp, timestamp, vtyp->decompose(std::move(dv)), atomic_cell::collection_member::yes));
}
auto list_mut = column_type->serialize_mutation_form(std::move(m));
@@ -1407,7 +1407,7 @@ make_list_mutation(const std::vector<T, Args...>& values,
list_type_impl::native_type tmp;
tmp.reserve(values.size());
std::transform(values.begin(), values.end(), std::back_inserter(tmp), f);
return atomic_cell::make_live(timestamp, column_type->decompose(make_list_value(column_type, std::move(tmp))));
return atomic_cell::make_live(*column.type, timestamp, column_type->decompose(make_list_value(column_type, std::move(tmp))));
}
}

35
db/view/view.cc
View File

@@ -183,7 +183,7 @@ static bool is_partition_key_empty(
default:
// No multi-cell columns in the view's partition key
auto& c = update.cells().cell_at(base_col->id);
return c.as_atomic_cell().value().empty();
return c.as_atomic_cell(*base_col).value().empty();
}
}
@@ -264,8 +264,9 @@ row_marker view_updates::compute_row_marker(const clustering_row& base_row) cons
auto marker = base_row.marker();
auto col_id = _view_info.base_non_pk_column_in_view_pk();
if (col_id) {
auto& def = _base->regular_column_at(*col_id);
// Note: multi-cell columns can't be part of the primary key.
auto cell = base_row.cells().cell_at(*col_id).as_atomic_cell();
auto cell = base_row.cells().cell_at(*col_id).as_atomic_cell(def);
return cell.is_live_and_has_ttl() ? row_marker(cell.timestamp(), cell.ttl(), cell.expiry()) : row_marker(cell.timestamp());
}
@@ -300,7 +301,7 @@ row_marker view_updates::compute_row_marker(const clustering_row& base_row) cons
return;
}
if (def.is_atomic()) {
maybe_update_expiry_and_ttl(c.as_atomic_cell());
maybe_update_expiry_and_ttl(c.as_atomic_cell(def));
} else {
auto ctype = static_pointer_cast<const collection_type_impl>(def.type);
ctype->for_each_cell(c.as_collection_mutation(), maybe_update_expiry_and_ttl);
@@ -317,7 +318,8 @@ row_marker view_updates::compute_row_marker(const clustering_row& base_row) cons
}
deletable_row& view_updates::get_view_row(const partition_key& base_key, const clustering_row& update) {
auto get_value = boost::adaptors::transformed([&, this] (const column_definition& cdef) {
std::vector<bytes> linearized_values;
auto get_value = boost::adaptors::transformed([&, this] (const column_definition& cdef) -> bytes_view {
auto* base_col = _base->get_column_definition(cdef.name());
assert(base_col);
switch (base_col->kind) {
@@ -327,10 +329,11 @@ deletable_row& view_updates::get_view_row(const partition_key& base_key, const c
return update.key().get_component(*_base, base_col->position());
default:
auto& c = update.cells().cell_at(base_col->id);
if (base_col->is_atomic()) {
return c.as_atomic_cell().value();
auto value_view = base_col->is_atomic() ? c.as_atomic_cell(cdef).value() : c.as_collection_mutation().data;
if (value_view.is_fragmented()) {
return linearized_values.emplace_back(value_view.linearize());
}
return c.as_collection_mutation().data;
return value_view.first_fragment();
}
});
auto& partition = partition_for(partition_key::from_range(_view->partition_key_columns() | get_value));
@@ -366,7 +369,7 @@ void view_updates::create_entry(const partition_key& base_key, const clustering_
auto marker = compute_row_marker(update);
r.apply(marker);
r.apply(update.tomb());
add_cells_to_view(*_base, *_view, update.cells(), r.cells());
add_cells_to_view(*_base, *_view, row(*_base, column_kind::regular_column, update.cells()), r.cells());
}
/**
@@ -388,7 +391,8 @@ void view_updates::do_delete_old_entry(const partition_key& base_key, const clus
// We delete the old row using a shadowable row tombstone, making sure that
// the tombstone deletes everything in the row (or it might still show up).
// Note: multi-cell columns can't be part of the primary key.
auto cell = existing.cells().cell_at(*col_id).as_atomic_cell();
auto& def = _base->regular_column_at(*col_id);
auto cell = existing.cells().cell_at(*col_id).as_atomic_cell(def);
if (cell.is_live()) {
r.apply(shadowable_tombstone(cell.timestamp(), now));
}
@@ -406,7 +410,7 @@ void view_updates::do_delete_old_entry(const partition_key& base_key, const clus
// Unselected columns are used regardless of being live or dead, since we don't know if
// they were used to compute the view entry's row marker.
if (def.is_atomic()) {
set_max_ts(cell.as_atomic_cell());
set_max_ts(cell.as_atomic_cell(def));
} else {
auto ctype = static_pointer_cast<const collection_type_impl>(def.type);
ctype->for_each_cell(cell.as_collection_mutation(), set_max_ts);
@@ -483,11 +487,12 @@ void view_updates::generate_update(
auto* after = update.cells().find_cell(*col_id);
// Note: multi-cell columns can't be part of the primary key.
auto& cdef = _base->regular_column_at(*col_id);
if (existing) {
auto* before = existing->cells().find_cell(*col_id);
if (before && before->as_atomic_cell().is_live()) {
if (after && after->as_atomic_cell().is_live()) {
auto cmp = compare_atomic_cell_for_merge(before->as_atomic_cell(), after->as_atomic_cell());
if (before && before->as_atomic_cell(cdef).is_live()) {
if (after && after->as_atomic_cell(cdef).is_live()) {
auto cmp = compare_atomic_cell_for_merge(before->as_atomic_cell(cdef), after->as_atomic_cell(cdef));
if (cmp == 0) {
update_entry(base_key, update, *existing, now);
} else {
@@ -501,7 +506,7 @@ void view_updates::generate_update(
}
// No existing row or the cell wasn't live
if (after && after->as_atomic_cell().is_live()) {
if (after && after->as_atomic_cell(cdef).is_live()) {
create_entry(base_key, update, now);
}
}
@@ -674,7 +679,7 @@ future<stop_iteration> view_update_builder::on_results() {
if (tombstone && _existing && !_existing->is_end_of_partition()) {
// We don't care if it's a range tombstone, as we're only looking for existing entries that get deleted
if (_existing->is_clustering_row()) {
auto& existing = _existing->as_clustering_row();
auto existing = clustering_row(*_schema, _existing->as_clustering_row());
auto update = clustering_row(existing.key(), row_tombstone(std::move(tombstone)), row_marker(), ::row());
generate_update(std::move(update), { std::move(existing) });
}

16
flat_mutation_reader.cc
View File

@@ -27,12 +27,22 @@
#include <boost/range/adaptor/transformed.hpp>
#include <seastar/util/defer.hh>
static size_t compute_buffer_size(const schema& s, circular_buffer<mutation_fragment>& buffer)
{
return boost::accumulate(
buffer
| boost::adaptors::transformed([&s] (const mutation_fragment& mf) {
return mf.memory_usage(s);
}), size_t(0)
);
}
void flat_mutation_reader::impl::forward_buffer_to(const position_in_partition& pos) {
_buffer.erase(std::remove_if(_buffer.begin(), _buffer.end(), [this, &pos] (mutation_fragment& f) {
return !f.relevant_for_range_assuming_after(*_schema, pos);
}), _buffer.end());
_buffer_size = boost::accumulate(_buffer | boost::adaptors::transformed(std::mem_fn(&mutation_fragment::memory_usage)), size_t(0));
_buffer_size = compute_buffer_size(*_schema, _buffer);
}
void flat_mutation_reader::impl::clear_buffer_to_next_partition() {
@@ -41,7 +51,7 @@ void flat_mutation_reader::impl::clear_buffer_to_next_partition() {
});
_buffer.erase(_buffer.begin(), next_partition_start);
_buffer_size = boost::accumulate(_buffer | boost::adaptors::transformed(std::mem_fn(&mutation_fragment::memory_usage)), size_t(0));
_buffer_size = compute_buffer_size(*_schema, _buffer);
}
flat_mutation_reader flat_mutation_reader::impl::reverse_partitions(flat_mutation_reader::impl& original) {
@@ -77,7 +87,7 @@ flat_mutation_reader flat_mutation_reader::impl::reverse_partitions(flat_mutatio
if (is_buffer_full()) {
return stop_iteration::yes;
}
push_mutation_fragment(*std::exchange(_partition_end, stdx::nullopt));
push_mutation_fragment(std::move(*std::exchange(_partition_end, stdx::nullopt)));
return stop_iteration::no;
}
future<stop_iteration> consume_partition_from_source(db::timeout_clock::time_point timeout) {

4
flat_mutation_reader.hh
View File

@@ -101,7 +101,7 @@ public:
void push_mutation_fragment(Args&&... args) {
seastar::memory::on_alloc_point(); // for exception safety tests
_buffer.emplace_back(std::forward<Args>(args)...);
_buffer_size += _buffer.back().memory_usage();
_buffer_size += _buffer.back().memory_usage(*_schema);
}
void clear_buffer() {
_buffer.erase(_buffer.begin(), _buffer.end());
@@ -132,7 +132,7 @@ public:
mutation_fragment pop_mutation_fragment() {
auto mf = std::move(_buffer.front());
_buffer.pop_front();
_buffer_size -= mf.memory_usage();
_buffer_size -= mf.memory_usage(*_schema);
return mf;
}

6
frozen_mutation.cc
View File

@@ -222,18 +222,18 @@ public:
future<stop_iteration> consume(static_row&& sr) {
_sr = std::move(sr);
_dirty_size += _sr->memory_usage();
_dirty_size += _sr->memory_usage(_schema);
return maybe_flush();
}
future<stop_iteration> consume(clustering_row&& cr) {
_dirty_size += cr.memory_usage();
_dirty_size += cr.memory_usage(_schema);
_crs.emplace_back(std::move(cr));
return maybe_flush();
}
future<stop_iteration> consume(range_tombstone&& rt) {
_dirty_size += rt.memory_usage();
_dirty_size += rt.memory_usage(_schema);
_rts.apply(_schema, std::move(rt));
return maybe_flush();
}

18
idl-compiler.py
View File

@@ -418,12 +418,14 @@ def add_param_writer_basic_type(name, base_state, typ, var_type = "", var_index
set_command = ("_state.f.end(_out);" if not root_node else "state.f.end(_out);") if var_type is not "" else ""
return_command = "{ _out, std::move(_state._parent) }" if var_type is not "" and not root_node else "{ _out, std::move(_state) }"
allow_fragmented = False
if typ in ['bytes', 'sstring']:
typ += '_view'
allow_fragmented = True
else:
typ = 'const ' + typ + '&'
return Template(reindent(4, """
writer = Template(reindent(4, """
after_${base_state}__$name<Output> write_$name$var_type($typ t) && {
$create_variant_state
$set_varient_index
@@ -431,6 +433,20 @@ def add_param_writer_basic_type(name, base_state, typ, var_type = "", var_index
$set_command
return $return_command;
}""")).substitute(locals())
if allow_fragmented:
writer += Template(reindent(4, """
template<typename FragmentedBuffer>
GCC6_CONCEPT(requires FragmentRange<FragmentedBuffer>)
after_${base_state}__$name<Output> write_fragmented_$name$var_type(FragmentedBuffer&& fragments) && {
$set_varient_index
using boost::range::for_each;
for_each(fragments, [&] ($typ t) {
serialize(_out, t);
});
$set_command
return $return_command;
}""")).substitute(locals())
return writer
def add_param_writer_object(name, base_state, typ, var_type = "", var_index = None, root_node = False):
var_type1 = "_" + var_type if var_type != "" else ""

295
imr/alloc.hh Normal file
View File

@@ -0,0 +1,295 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "utils/chunked_vector.hh"
#include "utils/logalloc.hh"
#include "imr/core.hh"
#include "imr/methods.hh"
namespace imr {
namespace alloc {
static const struct no_context_factory_t {
static no_context_t create(const void*) noexcept { return no_context; }
} no_context_factory;
/// Deserialisation context factory
///
/// Deserialisation contexts provide the IMR code with additional information
/// needed to deserialise an IMR object. Often the sources of that information
/// are both the object itself as well as some external state shared by multiple
/// IMR objects of the same type.
/// `context_factory` is a helper class for creating contexts it keeps the
/// shared state (e.g. per-schema information) and when given a pointer to a
/// IMR object creates a deserialisation context for it.
template<typename Context, typename... State>
class context_factory {
std::tuple<State...> _state;
private:
template<size_t... Index>
Context create(const uint8_t* ptr, std::index_sequence<Index...>) const noexcept {
return Context(ptr, std::get<Index>(_state)...);
}
public:
template<typename... Args>
context_factory(Args&&... args) : _state(std::forward<Args>(args)...) { }
context_factory(context_factory&) = default;
context_factory(const context_factory&) = default;
context_factory(context_factory&&) = default;
Context create(const uint8_t* ptr) const noexcept {
return create(ptr, std::index_sequence_for<State...>());
}
};
GCC6_CONCEPT(
template<typename T>
concept bool ContextFactory = requires(const T factory, const uint8_t* ptr) {
{ factory.create(ptr) } noexcept;
};
static_assert(ContextFactory<no_context_factory_t>,
"no_context_factory_t has to meet ContextFactory constraints");
)
/// LSA migrator for IMR objects
///
/// IMR objects may own memory and therefore moving and destroying them may
/// be non-trivial. This class implements an LSA migrator for an IMR objects
/// of type `Structure`. The deserialisation context needed to invoke the mover
/// is going to be created by the provided context factory `CtxFactory`.
template<typename Structure, typename CtxFactory>
GCC6_CONCEPT(requires ContextFactory<CtxFactory>)
class lsa_migrate_fn final : public migrate_fn_type, CtxFactory {
public:
explicit lsa_migrate_fn(CtxFactory context_factory)
: migrate_fn_type(1)
, CtxFactory(std::move(context_factory))
{ }
lsa_migrate_fn(lsa_migrate_fn&&) = delete;
lsa_migrate_fn(const lsa_migrate_fn&) = delete;
lsa_migrate_fn& operator=(lsa_migrate_fn&&) = delete;
lsa_migrate_fn& operator=(const lsa_migrate_fn&) = delete;
virtual void migrate(void* src_ptr, void* dst_ptr, size_t size) const noexcept override {
std::memcpy(dst_ptr, src_ptr, size);
auto dst = static_cast<uint8_t*>(dst_ptr);
methods::move<Structure>(dst, CtxFactory::create(dst));
}
virtual size_t size(const void* obj_ptr) const noexcept override {
auto ptr = static_cast<const uint8_t*>(obj_ptr);
return Structure::serialized_object_size(ptr, CtxFactory::create(ptr));
}
};
// LSA migrator for objects which mover doesn't require a deserialisation context
template<typename Structure>
struct default_lsa_migrate_fn {
static lsa_migrate_fn<Structure, no_context_factory_t> migrate_fn;
};
template<typename Structure>
lsa_migrate_fn<Structure, no_context_factory_t> default_lsa_migrate_fn<Structure>::migrate_fn(no_context_factory);
/// IMR object allocator
///
/// This is a helper class that helps creating IMR objects that may own memory.
/// The serialisation of IMR objects is done in two phases:
/// 1. IMR figures out the size of the object. `sizer` provided by `get_sizer()`
/// records the size of all necessary memory allocations.
/// `allocate_all()` is called and allocates memory for all owned objects.
/// 2. Data is written to the allocated memory. `serializer` returned by
/// `get_serializer()` provides pointers to the allocated buffers and handles
/// their serialisation.
class object_allocator {
union allocation {
static_assert(std::is_trivially_destructible_v<std::pair<size_t, void*>>);
static_assert(std::is_trivially_destructible_v<std::pair<size_t, allocation_strategy::migrate_fn>>);
private:
std::pair<size_t, allocation_strategy::migrate_fn> _allocation_request;
std::pair<size_t, void*> _allocated_object;
public:
explicit allocation(size_t n, allocation_strategy::migrate_fn fn) noexcept
: _allocation_request(std::make_pair(n, fn)) { }
void allocate(allocation_strategy& allocator) {
auto ptr = allocator.alloc(_allocation_request.second, _allocation_request.first, 1);
_allocated_object = std::make_pair(_allocation_request.first, ptr);
}
void free(allocation_strategy& allocator) noexcept {
allocator.free(_allocated_object.second, _allocated_object.first);
}
void set_request_size(size_t n) noexcept {
_allocation_request.first = n;
}
void* pointer() const noexcept { return _allocated_object.second; }
size_t size() const noexcept { return _allocated_object.first; }
};
allocation_strategy& _allocator;
std::vector<allocation> _allocations;
size_t _position = 0;
bool _failed = false;
private:
size_t request(size_t n, allocation_strategy::migrate_fn migrate) noexcept {
auto id = _allocations.size();
try {
_allocations.emplace_back(n, migrate);
} catch (...) {
_failed = true;
}
return id;
}
void set_request_size(size_t id, size_t n) noexcept {
if (__builtin_expect(!_failed, true)) {
_allocations[id].set_request_size(n);
}
}
uint8_t* next_object() noexcept {
return static_cast<uint8_t*>(_allocations[_position++].pointer());
}
public:
class sizer {
object_allocator& _parent;
public:
class continuation {
object_allocator& _parent;
size_t _idx;
public:
continuation(object_allocator& parent, size_t idx) noexcept
: _parent(parent), _idx(idx) { }
uint8_t* run(size_t size) noexcept {
_parent.set_request_size(_idx, size);
return nullptr;
}
};
public:
explicit sizer(object_allocator& parent) noexcept
: _parent(parent) { }
/// Request allocation of an IMR object
///
/// This method request an allocation of an IMR object of type T. The
/// arguments are passed to `T::size_when_serialized`.
///
/// \return null pointer of type `uint8_t*`.
template<typename T, typename... Args>
uint8_t* allocate(migrate_fn_type* migrate_fn, Args&& ... args) noexcept {
auto size = T::size_when_serialized(std::forward<Args>(args)...);
_parent.request(size, migrate_fn);
// We are in the sizing phase and only collect information about
// the size of the required objects. The serializer will return
// the real pointer to the memory buffer requested here, but since
// both sizer and serializer need to expose the same interface we
// need to return something from sizer as well even though the
// value will be ignored.
return nullptr;
}
template<typename T, typename... Args>
auto allocate_nested(migrate_fn_type* migrate_fn, Args&& ... args) noexcept {
auto n = _parent.request(0, migrate_fn);
return T::get_sizer(continuation(_parent, n),
std::forward<Args>(args)...);
}
};
class serializer {
object_allocator& _parent;
public:
class continuation {
uint8_t* _ptr;
public:
explicit continuation(uint8_t* ptr) noexcept : _ptr(ptr) { }
uint8_t* run(uint8_t*) noexcept {
return _ptr;
}
};
public:
explicit serializer(object_allocator& parent) noexcept
: _parent(parent) { }
/// Writes an IMR object to the preallocated buffer
///
/// In the second serialisation phase this method writes an IMR object
/// to the buffer requested in the sizing phase. Arguments are passed
/// to `T::serialize`.
/// \return pointer to the IMR object
template<typename T, typename... Args>
uint8_t* allocate(migrate_fn_type* migrate_fn, Args&& ... args) noexcept {
auto ptr = _parent.next_object();
T::serialize(ptr, std::forward<Args>(args)...);
return ptr;
}
template<typename T, typename... Args>
auto allocate_nested(migrate_fn_type*, Args&& ... args) noexcept {
auto ptr = _parent.next_object();
return T::get_serializer(ptr,
continuation(ptr),
std::forward<Args>(args)...);
}
};
public:
explicit object_allocator(allocation_strategy& allocator = current_allocator())
: _allocator(allocator) { }
size_t requested_allocations_count() const noexcept { return _allocations.size(); }
/// Allocates all buffers requested in the sizing phase.
void allocate_all() {
if (__builtin_expect(_failed, false)) {
throw std::bad_alloc();
}
auto it = _allocations.begin();
try {
// TODO: Send a batch of allocations to the allocation strategy.
while (it != _allocations.end()) {
it->allocate(_allocator);
++it;
}
} catch (...) {
while (it != _allocations.begin()) {
--it;
it->free(_allocator);
}
throw;
}
}
sizer get_sizer() noexcept { return sizer(*this); }
serializer get_serializer() noexcept { return serializer(*this); }
};
}
}

592
imr/compound.hh Normal file
View File

@@ -0,0 +1,592 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <array>
#include <type_traits>
#include <seastar/util/gcc6-concepts.hh>
#include "utils/meta.hh"
#include "imr/core.hh"
namespace imr {
/// Optionally present object
///
/// Represents a value that may be not present. Information whether or not
/// the optional is engaged is not stored and must be provided by external
/// context.
template<typename Tag, typename Type>
struct optional {
using underlying = Type;
public:
template<::mutable_view is_mutable>
class basic_view {
using pointer_type = std::conditional_t<is_mutable == ::mutable_view::no,
const uint8_t*, uint8_t*>;
pointer_type _ptr;
public:
explicit basic_view(pointer_type ptr) noexcept : _ptr(ptr) { }
operator basic_view<::mutable_view::no>() const noexcept {
return basic_view<::mutable_view::no>(_ptr);
}
template<typename Context = no_context_t>
GCC6_CONCEPT(requires requires(const Context& ctx) {
{ ctx.template context_for<Tag>() } noexcept;
})
auto get(const Context& ctx = no_context) noexcept {
return Type::make_view(_ptr, ctx.template context_for<Tag>(_ptr));
}
};
using view = basic_view<::mutable_view::no>;
using mutable_view = basic_view<::mutable_view::yes>;
public:
template<typename Context = no_context_t>
static auto make_view(const uint8_t* in, const Context& ctx = no_context) noexcept {
return view(in);
}
template<typename Context = no_context_t>
static auto make_view(uint8_t* in, const Context& ctx = no_context) noexcept {
return mutable_view(in);
}
public:
template<typename Context>
GCC6_CONCEPT(requires requires(const Context& ctx) {
{ ctx.template is_present<Tag>() } noexcept -> bool;
})
static size_t serialized_object_size(const uint8_t* in, const Context& context) noexcept {
return context.template is_present<Tag>()
? Type::serialized_object_size(in, context)
: 0;
}
template<typename... Args>
static size_t size_when_serialized(Args&&... args) noexcept {
return Type::size_when_serialized(std::forward<Args>(args)...);
}
template<typename... Args>
static size_t serialize(uint8_t* out, Args&&... args) noexcept {
return Type::serialize(out, std::forward<Args>(args)...);
}
template<typename Continuation = no_op_continuation>
static auto get_sizer(Continuation cont = no_op_continuation()) {
return Type::get_sizer(std::move(cont));
}
template<typename Continuation = no_op_continuation>
static auto get_serializer(uint8_t* out, Continuation cont = no_op_continuation()) {
return Type::get_serializer(out, std::move(cont));
}
};
template<typename Tag, typename Type>
struct member {
using tag = Tag;
using type = Type;
};
namespace internal {
template<typename Tag>
struct do_find_member {
template<typename Member>
using type = std::is_same<Tag, typename Member::tag>;
};
template<typename Tag, typename... Members>
static constexpr auto get_member_index = meta::find_if<do_find_member<Tag>::template type, Members...>;
template<typename Tag, typename... Members>
using get_member = meta::get<get_member_index<Tag, Members...>, Members...>;
template<size_t Offset, size_t N, template<size_t> typename Function>
struct do_generate_branch_tree {
template<typename... Args>
static decltype(auto) run(size_t n, Args&&... args) {
if constexpr (N == 1) {
return Function<Offset>::run(std::forward<Args>(args)...);
} else if (N >= 2) {
if (n < Offset + N / 2) {
return do_generate_branch_tree<Offset, N / 2, Function>::run(n, std::forward<Args>(args)...);
} else {
return do_generate_branch_tree<Offset + N / 2, N - N / 2, Function>::run(n, std::forward<Args>(args)...);
}
}
}
};
template<size_t N, template<size_t> typename Function>
using generate_branch_tree = do_generate_branch_tree<0, N, Function>;
}
template<typename Tag, typename... Alternatives>
struct variant {
class alternative_index {
size_t _index;
private:
constexpr explicit alternative_index(size_t idx) noexcept
: _index(idx) { }
friend class variant;
public:
constexpr size_t index() const noexcept { return _index; }
};
template<typename AlternativeTag>
constexpr static alternative_index index_for() noexcept {
return alternative_index(internal::get_member_index<AlternativeTag, Alternatives...>);
}
private:
template<size_t N>
struct alternative_visitor {
template<typename Visitor>
static decltype(auto) run(Visitor&& visitor) {
using member = typename meta::get<N, Alternatives...>;
return visitor(static_cast<member*>(nullptr));
}
};
template<typename Visitor>
static decltype(auto) choose_alternative(alternative_index index, Visitor&& visitor) {
// For large sizeof...(Alternatives) a jump table may be the better option.
return internal::generate_branch_tree<sizeof...(Alternatives), alternative_visitor>::run(index.index(), std::forward<Visitor>(visitor));
}
public:
template<::mutable_view is_mutable>
class basic_view {
using pointer_type = std::conditional_t<is_mutable == ::mutable_view::no,
const uint8_t*, uint8_t*>;
pointer_type _ptr;
public:
explicit basic_view(pointer_type ptr) noexcept
: _ptr(ptr)
{ }
pointer_type raw_pointer() const noexcept { return _ptr; }
operator basic_view<::mutable_view::no>() const noexcept {
return basic_view<::mutable_view::no>(_ptr);
}
template<typename AlternativeTag, typename Context = no_context_t>
auto as(const Context& context = no_context) noexcept {
using member = internal::get_member<AlternativeTag, Alternatives...>;
return member::type::make_view(_ptr, context.template context_for<AlternativeTag>(_ptr));
}
template<typename Visitor, typename Context>
decltype(auto) visit(Visitor&& visitor, const Context& context) {
auto alt_idx = context.template active_alternative_of<Tag>();
return choose_alternative(alt_idx, [&] (auto object) {
using type = std::remove_pointer_t<decltype(object)>;
return visitor(type::type::make_view(_ptr, context.template context_for<typename type::tag>(_ptr)));
});
}
template<typename Visitor, typename Context>
decltype(auto) visit_type(Visitor&& visitor, const Context& context) {
auto alt_idx = context.template active_alternative_of<Tag>();
return choose_alternative(alt_idx, [&] (auto object) {
using type = std::remove_pointer_t<decltype(object)>;
return visitor(static_cast<type*>(nullptr));
});
}
};
using view = basic_view<::mutable_view::no>;
using mutable_view = basic_view<::mutable_view::yes>;
public:
template<typename Context>
static view make_view(const uint8_t* in, const Context& context) noexcept {
return view(in);
}
template<typename Context>
static mutable_view make_view(uint8_t* in, const Context& context) noexcept {
return mutable_view(in);
}
public:
template<typename Context>
GCC6_CONCEPT(requires requires(const Context& ctx) {
{ ctx.template active_alternative_of<Tag>() } noexcept -> alternative_index;
})
static size_t serialized_object_size(const uint8_t* in, const Context& context) noexcept {
return choose_alternative(context.template active_alternative_of<Tag>(), [&] (auto object) noexcept {
using alternative = std::remove_pointer_t<decltype(object)>;
return alternative::type::serialized_object_size(in, context.template context_for<typename alternative::tag>(in));
});
}
template<typename AlternativeTag, typename... Args>
static size_t size_when_serialized(Args&&... args) noexcept {
using member = internal::get_member<AlternativeTag, Alternatives...>;
return member::type::size_when_serialized(std::forward<Args>(args)...);
}
template<typename AlternativeTag, typename... Args>
static size_t serialize(uint8_t* out, Args&&... args) noexcept {
using member = internal::get_member<AlternativeTag, Alternatives...>;
return member::type::serialize(out, std::forward<Args>(args)...);
}
template<typename AlternativeTag, typename Continuation = no_op_continuation>
static auto get_sizer(Continuation cont = no_op_continuation()) {
using member = internal::get_member<AlternativeTag, Alternatives...>;
return member::type::get_sizer(std::move(cont));
}
template<typename AlternativeTag, typename Continuation = no_op_continuation>
static auto get_serializer(uint8_t* out, Continuation cont = no_op_continuation()) {
using member = internal::get_member<AlternativeTag, Alternatives...>;
return member::type::get_serializer(out, std::move(cont));
}
};
template<typename Tag, typename Type>
using optional_member = member<Tag, optional<Tag, Type>>;
template<typename Tag, typename... Types>
using variant_member = member<Tag, variant<Tag, Types...>>;
namespace internal {
template<typename Continuation, typename... Members>
class structure_sizer : Continuation {
size_t _size;
public:
explicit structure_sizer(size_t size, Continuation&& cont) noexcept
: Continuation(std::move(cont)), _size(size) {}
uint8_t* position() const noexcept {
// We are in the sizing phase and there is no object to point to yet.
// The serializer will return a real position in the destination buffer,
// but since sizer and serializer need to expose the same interface we
// need to return something even though the value will be ignored.
return nullptr;
}
auto done() noexcept { return Continuation::run(_size); }
};
template<typename NestedContinuation, typename... Members>
class structure_sizer_continuation : NestedContinuation {
size_t _size;
public:
explicit structure_sizer_continuation(size_t size, NestedContinuation&& cont) noexcept
: NestedContinuation(std::move(cont)), _size(size) {}
structure_sizer<NestedContinuation, Members...> run(size_t size) noexcept {
return structure_sizer<NestedContinuation, Members...>(size + _size,
std::move(*static_cast<NestedContinuation*>(this)));
}
};
template<typename Continuation, typename Member, typename... Members>
class basic_structure_sizer : protected Continuation {
protected:
size_t _size;
using continuation = structure_sizer_continuation<Continuation, Members...>;
public:
explicit basic_structure_sizer(size_t size, Continuation&& cont) noexcept
: Continuation(std::move(cont)), _size(size) {}
uint8_t* position() const noexcept { return nullptr; }
template<typename... Args>
structure_sizer<Continuation, Members...> serialize(Args&& ... args) noexcept {
auto size = Member::type::size_when_serialized(std::forward<Args>(args)...);
return structure_sizer<Continuation, Members...>(size + _size, std::move(*static_cast<Continuation*>(this)));
}
template<typename... Args>
auto serialize_nested(Args&& ... args) noexcept {
return Member::type::get_sizer(continuation(_size, std::move(*static_cast<Continuation*>(this))),
std::forward<Args>(args)...);
}
};
template<typename Continuation, typename Member, typename... Members>
struct structure_sizer<Continuation, Member, Members...>
: basic_structure_sizer<Continuation, Member, Members...> {
using basic_structure_sizer<Continuation, Member, Members...>::basic_structure_sizer;
};
template<typename Continuation, typename Tag, typename Type, typename... Members>
struct structure_sizer<Continuation, optional_member<Tag, Type>, Members...>
: basic_structure_sizer<Continuation, optional_member<Tag, Type>, Members...> {
using basic_structure_sizer<Continuation, optional_member<Tag, Type>, Members...>::basic_structure_sizer;
structure_sizer<Continuation, Members...> skip() noexcept {
return structure_sizer<Continuation, Members...>(this->_size, std::move(*static_cast<Continuation*>(this)));
}
};
template<typename Continuation, typename Tag, typename... Types, typename... Members>
struct structure_sizer<Continuation, variant_member<Tag, Types...>, Members...>
: basic_structure_sizer<Continuation, variant_member<Tag, Types...>, Members...> {
using basic_structure_sizer<Continuation, variant_member<Tag, Types...>, Members...>::basic_structure_sizer;
template<typename... Args>
structure_sizer<Continuation, Members...> serialize(Args&& ... args) noexcept = delete;
template<typename... Args>
auto serialize_nested(Args&& ... args) noexcept = delete;
template<typename AlternativeTag, typename... Args>
structure_sizer<Continuation, Members...> serialize_as(Args&& ... args) noexcept {
using type = variant<Tag, Types...>;
auto size = type::template size_when_serialized<AlternativeTag>(std::forward<Args>(args)...);
return structure_sizer<Continuation, Members...>(size + this->_size, std::move(*static_cast<Continuation*>(this)));
}
template<typename AlternativeTag, typename... Args>
auto serialize_as_nested(Args&& ... args) noexcept {
using type = variant<Tag, Types...>;
using cont_type = typename basic_structure_sizer<Continuation, variant_member<Tag, Types...>, Members...>::continuation;
auto cont = cont_type(this->_size, std::move(*static_cast<Continuation*>(this)));
return type::template get_sizer<AlternativeTag>(std::move(cont),
std::forward<Args>(args)...);
}
};
template<typename Continuation, typename... Members>
class structure_serializer : Continuation {
uint8_t* _out;
public:
explicit structure_serializer(uint8_t* out, Continuation&& cont) noexcept
: Continuation(std::move(cont)), _out(out) {}
uint8_t* position() const noexcept { return _out; }
auto done() noexcept { return Continuation::run(_out); }
};
template<typename NestedContinuation, typename... Members>
struct structure_serializer_continuation : private NestedContinuation {
explicit structure_serializer_continuation(NestedContinuation&& cont) noexcept
: NestedContinuation(std::move(cont)) {}
structure_serializer<NestedContinuation, Members...> run(uint8_t* out) noexcept {
return structure_serializer<NestedContinuation, Members...>(out,
std::move(*static_cast<NestedContinuation*>(this)));
}
};
template<typename Continuation, typename Member, typename... Members>
class basic_structure_serializer : protected Continuation {
protected:
uint8_t* _out;
using continuation = structure_serializer_continuation<Continuation, Members...>;
public:
explicit basic_structure_serializer(uint8_t* out, Continuation&& cont) noexcept
: Continuation(std::move(cont)), _out(out) {}
uint8_t* position() const noexcept { return _out; }
template<typename... Args>
structure_serializer<Continuation, Members...> serialize(Args&& ... args) noexcept {
auto size = Member::type::serialize(_out, std::forward<Args>(args)...);
return structure_serializer<Continuation, Members...>(_out + size, std::move(*static_cast<Continuation*>(this)));
}
template<typename... Args>
auto serialize_nested(Args&& ... args) noexcept {
return Member::type::get_serializer(_out,
continuation(std::move(*static_cast<Continuation*>(this))),
std::forward<Args>(args)...);
}
};
template<typename Continuation, typename Member, typename... Members>
struct structure_serializer<Continuation, Member, Members...>
: basic_structure_serializer<Continuation, Member, Members...> {
using basic_structure_serializer<Continuation, Member, Members...>::basic_structure_serializer;
};
template<typename Continuation, typename Tag, typename Type, typename... Members>
struct structure_serializer<Continuation, optional_member<Tag, Type>, Members...>
: basic_structure_serializer<Continuation, optional_member<Tag, Type>, Members...> {
using basic_structure_serializer<Continuation, optional_member<Tag, Type>, Members...>::basic_structure_serializer;
structure_serializer<Continuation, Members...> skip() noexcept {
return structure_serializer<Continuation, Members...>(this->_out,
std::move(*static_cast<Continuation*>(this)));
}
};
template<typename Continuation, typename Tag, typename... Types, typename... Members>
struct structure_serializer<Continuation, variant_member<Tag, Types...>, Members...>
: basic_structure_serializer<Continuation, variant_member<Tag, Types...>, Members...> {
using basic_structure_serializer<Continuation, variant_member<Tag, Types...>, Members...>::basic_structure_serializer;
template<typename... Args>
structure_serializer<Continuation, Members...> serialize(Args&& ... args) noexcept = delete;
template<typename... Args>
auto serialize_nested(Args&& ... args) noexcept = delete;
template<typename AlternativeTag, typename... Args>
structure_serializer<Continuation, Members...> serialize_as(Args&& ... args) noexcept {
using type = variant<Tag, Types...>;
auto size = type::template serialize<AlternativeTag>(this->_out, std::forward<Args>(args)...);
return structure_serializer<Continuation, Members...>(this->_out + size,
std::move(*static_cast<Continuation*>(this)));
}
template<typename AlternativeTag, typename... Args>
auto serialize_as_nested(Args&& ... args) noexcept {
using type = variant<Tag, Types...>;
using cont_type = typename basic_structure_serializer<Continuation, variant_member<Tag, Types...>, Members...>::continuation;
auto cont = cont_type(std::move(*static_cast<Continuation*>(this)));
return type::template get_serializer<AlternativeTag>(this->_out,
std::move(cont),
std::forward<Args>(args)...);
}
};
}
// Represents a compound type.
template<typename... Members>
struct structure {
template<::mutable_view is_mutable>
class basic_view {
using pointer_type = std::conditional_t<is_mutable == ::mutable_view::no,
const uint8_t*, uint8_t*>;
pointer_type _ptr;
public:
template<typename Context>
explicit basic_view(pointer_type ptr, const Context& context) noexcept : _ptr(ptr) { }
pointer_type raw_pointer() const noexcept { return _ptr; }
operator basic_view<::mutable_view::no>() const noexcept {
return basic_view<::mutable_view::no>(_ptr, no_context);
}
template<typename Tag, typename Context = no_context_t>
auto offset_of(const Context& context = no_context) const noexcept {
static constexpr auto idx = internal::get_member_index<Tag, Members...>;
size_t total_size = 0;
meta::for_each<meta::take<idx, Members...>>([&] (auto ptr) {
using member = std::remove_pointer_t<decltype(ptr)>;
auto offset = _ptr + total_size;
auto this_size = member::type::serialized_object_size(offset, context.template context_for<typename member::tag>(offset));
total_size += this_size;
});
return total_size;
}
template<typename Tag, typename Context = no_context_t>
auto get(const Context& context = no_context) const noexcept {
using member = internal::get_member<Tag, Members...>;
auto offset = _ptr + offset_of<Tag, Context>(context);
return member::type::make_view(offset, context.template context_for<Tag>(offset));
}
};
using view = basic_view<::mutable_view::no>;
using mutable_view = basic_view<::mutable_view::yes>;
public:
template<typename Context = no_context_t>
static view make_view(const uint8_t* in, const Context& context = no_context) noexcept {
return view(in, context);
}
template<typename Context = no_context_t>
static mutable_view make_view(uint8_t* in, const Context& context = no_context) noexcept {
return mutable_view(in, context);
}
public:
template<typename Context = no_context_t>
static size_t serialized_object_size(const uint8_t* in, const Context& context = no_context) noexcept {
size_t total_size = 0;
meta::for_each<Members...>([&] (auto ptr) noexcept {
using member = std::remove_pointer_t<decltype(ptr)>;
auto offset = in + total_size;
auto this_size = member::type::serialized_object_size(offset, context.template context_for<typename member::tag>(offset));
total_size += this_size;
});
return total_size;
}
template<typename Continuation = no_op_continuation>
static internal::structure_sizer<Continuation, Members...> get_sizer(Continuation cont = no_op_continuation()) {
return internal::structure_sizer<Continuation, Members...>(0, std::move(cont));
}
template<typename Continuation = no_op_continuation>
static internal::structure_serializer<Continuation, Members...> get_serializer(uint8_t* out, Continuation cont = no_op_continuation()) {
return internal::structure_serializer<Continuation, Members...>(out, std::move(cont));
}
template<typename Writer, typename... Args>
static size_t size_when_serialized(Writer&& writer, Args&&... args) noexcept {
return std::forward<Writer>(writer)(get_sizer(), std::forward<Args>(args)...);
}
template<typename Writer, typename... Args>
static size_t serialize(uint8_t* out, Writer&& writer, Args&&... args) noexcept {
auto ptr = std::forward<Writer>(writer)(get_serializer(out), std::forward<Args>(args)...);
return ptr - out;
}
template<typename Tag, typename Context = no_context_t>
static size_t offset_of(const uint8_t* in, const Context& context = no_context) noexcept {
static constexpr auto idx = internal::get_member_index<Tag, Members...>;
size_t total_size = 0;
meta::for_each<meta::take<idx, Members...>>([&] (auto ptr) noexcept {
using member = std::remove_pointer_t<decltype(ptr)>;
auto offset = in + total_size;
auto this_size = member::type::serialized_object_size(offset, context.template context_for<typename member::tag>(offset));
total_size += this_size;
});
return total_size;
}
template<typename Tag, typename Context = no_context_t>
static auto get_member(const uint8_t* in, const Context& context = no_context) noexcept {
auto off = offset_of<Tag>(in, context);
using member = internal::get_member<Tag, Members...>;
return member::type::make_view(in + off, context.template context_for<typename member::tag>(in + off));
}
template<typename Tag, typename Context = decltype(no_context)>
static auto get_member(uint8_t* in, const Context& context = no_context) noexcept {
auto off = offset_of<Tag>(in, context);
using member = internal::get_member<Tag, Members...>;
return member::type::make_view(in + off, context.template context_for<typename member::tag>(in + off));
}
};
template<typename Tag, typename T>
struct tagged_type : T { };
}

86
imr/concepts.hh Normal file
View File

@@ -0,0 +1,86 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "imr/alloc.hh"
#include "imr/compound.hh"
#include "imr/fundamental.hh"
namespace imr {
/// Check if a type T is a sizer for Structure.
template<typename Structure, typename T>
struct is_sizer_for : std::false_type { };
template<typename Continuation, typename... Members>
struct is_sizer_for<structure<Members...>,
internal::structure_sizer<Continuation, Members...>>
: std::true_type { };
template<typename Structure, typename T>
constexpr bool is_sizer_for_v = is_sizer_for<Structure, T>::value;
/// Check if a type T is a serializer for Structure.
template<typename Structure, typename T>
struct is_serializer_for : std::false_type { };
template<typename Continuation, typename... Members>
struct is_serializer_for<structure<Members...>,
internal::structure_serializer<Continuation, Members...>>
: std::true_type { };
template<typename Structure, typename T>
constexpr bool is_serializer_for_v = is_serializer_for<Structure, T>::value;
/// The default sizer for Structure.
template<typename Structure>
using default_sizer_t = decltype(Structure::get_sizer());
/// The default serializer for Structure.
template<typename Structure>
using default_serializer_t = decltype(Structure::get_serializer(nullptr));
GCC6_CONCEPT(
/// A simple writer that accepts only sizer or serializer as an argument.
template<typename Writer, typename Structure>
concept bool WriterSimple = requires(Writer writer, default_sizer_t<Structure> sizer,
default_serializer_t<Structure> serializer)
{
writer(sizer);
writer(serializer);
};
/// A writer that accepts both sizer or serializer and a memory allocator.
template<typename Writer, typename Structure>
concept bool WriterAllocator = requires(Writer writer, default_sizer_t<Structure> sizer,
default_serializer_t<Structure> serializer,
imr::alloc::object_allocator::sizer alloc_sizer,
imr::alloc::object_allocator::serializer alloc_serializer)
{
writer(sizer, alloc_sizer);
writer(serializer, alloc_serializer);
};
)
}

63
imr/core.hh Normal file
View File

@@ -0,0 +1,63 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include "utils/fragment_range.hh"
namespace imr {
/// No-op deserialisation context
///
/// This is a dummy deserialisation context to be used when there is no need
/// for one, but the interface expects a context object.
static const struct no_context_t {
template<typename Tag, typename... Args>
const no_context_t& context_for(Args&&...) const noexcept { return *this; }
} no_context;
struct no_op_continuation {
template<typename T>
static T run(T value) noexcept {
return value;
}
};
template<typename T>
class placeholder {
uint8_t* _pointer = nullptr;
public:
placeholder() = default;
explicit placeholder(uint8_t* ptr) noexcept : _pointer(ptr) { }
void set_pointer(uint8_t* ptr) noexcept { _pointer = ptr; }
template<typename... Args>
void serialize(Args&&... args) noexcept {
if (!_pointer) {
// We lose the information whether we are in the sizing or
// serializing phase, hence the need for this run-time check.
return;
}
T::serialize(_pointer, std::forward<Args>(args)...);
}
};
}

309
imr/fundamental.hh Normal file
View File

@@ -0,0 +1,309 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <boost/range/algorithm/for_each.hpp>
#include <seastar/core/align.hh>
#include <seastar/core/bitops.hh>
#include <seastar/util/gcc6-concepts.hh>
#include "bytes.hh"
#include "utils/meta.hh"
#include "imr/core.hh"
namespace imr {
namespace internal {
template<typename T, typename CharT>
GCC6_CONCEPT(requires std::is_pod<T>::value && sizeof(CharT) == 1)
inline T read_pod(const CharT* in) noexcept {
T obj;
std::copy_n(in, sizeof(T), reinterpret_cast<CharT*>(&obj));
return obj;
}
template<typename T, typename CharT>
GCC6_CONCEPT(requires std::is_pod<T>::value && sizeof(CharT) == 1)
inline void write_pod(T obj, CharT* out) noexcept {
std::copy_n(reinterpret_cast<const CharT*>(&obj), sizeof(T), out);
}
}
template<typename Tag>
class set_flag {
bool _value = true;
public:
set_flag() = default;
explicit set_flag(bool v) noexcept : _value(v) { }
bool value() const noexcept { return _value; }
};
/// Set of flags
///
/// Represents a fixed-size set of tagged flags.
template<typename... Tags>
class flags {
static constexpr auto object_size = seastar::align_up<size_t>(sizeof...(Tags), 8) / 8;
private:
template<typename Tag>
static void do_set_or_clear(uint8_t* ptr, bool set) noexcept {
const auto idx = meta::find<Tag, Tags...>;
const auto byte_idx = idx / 8;
const auto bit_idx = idx % 8;
auto value = ptr[byte_idx];
value &= ~uint8_t(1 << bit_idx);
value |= uint8_t(set) << bit_idx;
ptr[byte_idx] = value;
}
template<typename Tag>
static bool do_get(const uint8_t* ptr) noexcept {
const auto idx = meta::find<Tag, Tags...>;
const auto byte_idx = idx / 8;
const auto bit_idx = idx % 8;
return ptr[byte_idx] & (1 << bit_idx);
}
public:
template<::mutable_view is_mutable>
class basic_view {
using pointer_type = std::conditional_t<is_mutable == ::mutable_view::no,
const uint8_t*, uint8_t*>;
pointer_type _ptr;
public:
explicit basic_view(pointer_type ptr) noexcept : _ptr(ptr) { }
operator basic_view<::mutable_view::no>() const noexcept {
return basic_view<::mutable_view::no>(_ptr);
}
template<typename Tag>
bool get() const noexcept {
return do_get<Tag>(_ptr);
}
template<typename Tag>
void set(bool value = true) noexcept {
do_set_or_clear<Tag>(_ptr, value);
}
};
using view = basic_view<::mutable_view::no>;
using mutable_view = basic_view<::mutable_view::yes>;
public:
template<typename Context = no_context_t>
static view make_view(const uint8_t* in, const Context& = no_context) noexcept {
return view(in);
}
template<typename Context = no_context_t>
static mutable_view make_view(uint8_t* in, const Context& = no_context) noexcept {
return mutable_view(in);
}
public:
template<typename Context = no_context_t>
static size_t serialized_object_size(const uint8_t*, const Context& = no_context) noexcept {
return object_size;
}
template<typename... Tags1>
static size_t size_when_serialized(set_flag<Tags1>...) noexcept {
return object_size;
}
template<typename... Tags1>
static size_t serialize(uint8_t* out, set_flag<Tags1>... sfs) noexcept {
std::fill_n(out, object_size, 0);
(do_set_or_clear<Tags1>(out, sfs.value()), ...);
return object_size;
}
static size_t size_when_serialized(placeholder<flags<Tags...>>&) noexcept {
return object_size;
}
static size_t serialize(uint8_t* out, placeholder<flags<Tags...>>& phldr) noexcept {
phldr.set_pointer(out);
return object_size;
}
};
/// POD object
///
/// Represents a fixed-size POD value.
template<typename Type>
GCC6_CONCEPT(requires std::is_pod<Type>::value)
struct pod {
using underlying = Type;
enum : size_t {
size = sizeof(Type),
};
template<::mutable_view is_mutable>
class basic_view {
using pointer_type = std::conditional_t<is_mutable == ::mutable_view::no,
const uint8_t*, uint8_t*>;
pointer_type _ptr;
public:
explicit basic_view(pointer_type ptr) noexcept : _ptr(ptr) { }
operator basic_view<::mutable_view::no>() const noexcept {
return basic_view<::mutable_view::no>(_ptr);
}
Type load() const noexcept {
return internal::read_pod<Type>(_ptr);
}
void store(const Type& object) noexcept {
internal::write_pod(object, _ptr);
}
};
using view = basic_view<::mutable_view::no>;
using mutable_view = basic_view<::mutable_view::yes>;
public:
template<typename Context = no_context_t>
static view make_view(const uint8_t* in, const Context& = no_context) noexcept {
return view(in);
}
template<typename Context = no_context_t>
static mutable_view make_view(uint8_t* in, const Context& = no_context) noexcept {
return mutable_view(in);
}
public:
template<typename Context = no_context_t>
static size_t serialized_object_size(const uint8_t*, const Context& = no_context) noexcept {
return sizeof(Type);
}
static size_t size_when_serialized(const Type&) noexcept {
return sizeof(Type);
}
static size_t serialize(uint8_t* out, const Type& value) noexcept {
internal::write_pod(value, out);
return sizeof(Type);
}
static size_t size_when_serialized(placeholder<pod<Type>>&) noexcept {
return sizeof(Type);
}
static size_t serialize(uint8_t* out, placeholder<pod<Type>>& phldr) noexcept {
phldr.set_pointer(out);
return sizeof(Type);
}
};
/// Buffer
///
/// Represents an opaque buffer. The size of the buffer is not stored and must
/// be provided by external context.
/// A buffer can be created from a bytes_view, a fragments range or a
/// (size, serializer) pair.
template<typename Tag>
struct buffer {
using view = bytes_view;
using mutable_view = bytes_mutable_view;
template<::mutable_view is_mutable>
using basic_view = std::conditional_t<is_mutable == ::mutable_view::no, view, mutable_view>;
template<typename Context>
GCC6_CONCEPT(requires requires(const Context& ctx) {
{ ctx.template size_of<Tag>() } noexcept -> size_t;
})
static view make_view(const uint8_t* in, const Context& context) noexcept {
auto ptr = reinterpret_cast<bytes_view::pointer>(in);
return bytes_view(ptr, context.template size_of<Tag>());
}
template<typename Context>
GCC6_CONCEPT(requires requires(const Context& ctx) {
{ ctx.template size_of<Tag>() } noexcept -> size_t;
})
static mutable_view make_view(uint8_t* in, const Context& context) noexcept {
auto ptr = reinterpret_cast<bytes_mutable_view::pointer>(in);
return bytes_mutable_view(ptr, context.template size_of<Tag>());
}
public:
template<typename Context>
GCC6_CONCEPT(requires requires(const Context& ctx) {
{ ctx.template size_of<Tag>() } noexcept -> size_t;
})
static size_t serialized_object_size(const uint8_t*, const Context& context) noexcept {
return context.template size_of<Tag>();
}
static size_t size_when_serialized(bytes_view src) noexcept {
return src.size();
}
template<typename Serializer>
GCC6_CONCEPT(requires requires (Serializer ser, uint8_t* ptr) {
{ ser(ptr) } noexcept;
})
static size_t size_when_serialized(size_t size, Serializer&&) noexcept {
return size;
}
template<typename FragmentRange, typename = std::enable_if_t<is_fragment_range_v<std::decay_t<FragmentRange>>>>
static size_t size_when_serialized(FragmentRange&& fragments) {
return fragments.size_bytes();
}
static size_t serialize(uint8_t* out, bytes_view src) {
std::copy_n(src.begin(), src.size(),
reinterpret_cast<bytes_view::value_type*>(out));
return src.size();
}
template<typename FragmentRange, typename = std::enable_if_t<is_fragment_range_v<std::decay_t<FragmentRange>>>>
static size_t serialize(uint8_t* out, FragmentRange&& fragments) {
auto dst = reinterpret_cast<bytes_view::value_type*>(out);
using boost::range::for_each;
for_each(fragments, [&] (bytes_view fragment) {
dst = std::copy(fragment.begin(), fragment.end(), dst);
});
return fragments.size_bytes();
}
template<typename Serializer>
GCC6_CONCEPT(requires requires (Serializer ser, uint8_t* ptr) {
{ ser(ptr) } noexcept;
})
static size_t serialize(uint8_t* out, size_t size, Serializer&& serializer) noexcept {
std::forward<Serializer>(serializer)(out);
return size;
}
};
}

503
imr/in_memory_representation.md Normal file
View File

@@ -0,0 +1,503 @@
# In-Memory Representation
## Introduction
Instances of all C++ objects need to have their size fixed and known at compile
time. This is a significant restriction when it comes to minimising the memory
footprint of objects that are kept alive for a long amount of time.
Introducing a serialisation format that allows objects to be of a variable size
enables a range of possible optimisations, but manually writing serialisation
and deserialisation code for all required types would not only be tedious, but
also quite quickly result in an unmaintainable code.
IMR attempts to solve this problem by providing a general infrastructure for
serialisation and deserialisation with a goal of imposing as little limitations
as possible and generating as efficient code as possible.
## Basic concepts
### Contexts
The IMR expects that in many cases multiple objects will share certain
properties in a very predictable way. This can be taken advantage of by not
storing all the information necessary to correctly read IMR objects inside them,
but moving the shared part to an external state.
Fundamental and compound IMR types are designed with this in mind and many of
them require an external context to provide additional information necessary
during deserialisation.
*For example, `imr::buffer<Tag>` doesn't store its size anywhere, but requires
to be provided at deserialisation with a context object that implements
`size_of<Tag>()` member function.*
The IMR itself doesn't care about the source of the information the context
provide. It can come either from an external state that describes properties of
multiple instances of a particular IMR type or the context logic may read parts
of an IMR object itself. It is also worth noting that objects are not tied to
contexts in any way. The user code could use different context object (with
different internal logic) as long as the decisions it makes are the same and
sufficient for that particular use.
*For example, in a structure a `imr::buffer<Tag>` could be preceded by an
integer which determines its size. When a context is created it takes a pointer
to the structure, reads the size field and returns it when asked by the buffer
deserialiser.*
#### Context factories
In many cases a context is going to combine some external state and data stored
in an IMR object to provide information necessary to deserialise it. In order
to make creation of context more convenient context factories were introduced
which are objects that keep the external state and create an actual context
instance when given a pointer to an IMR object.
```c++
context_factory<ContextType, ExternalState> factory(external_state);
auto context = factory.create(pointer_to_an_imr_object);
```
### Views & mutable views
The interface for deserialising an IMR object is built around views. An IMR type
given context and a pointer to an IMR object returns a view object that
provides methods specific for that type. For fundamental types these functions
will usually provide a way of creating an equivalent C++ type. In case of
compound types their views will allow accessing members in a checked or
unchecked manner.
Some IMR types can be updated in place. A mutable view of their instances can
be created which aside from providing an interface for reading the object will
also allow it to be updated.
### Serialisers
IMR objects are serialised in three phases:
1. Computing the object sizes and recording any necessary memory allocations.
2. Allocating memory. This is the only phase that can fail.
3. Writing serialised data.
It is imperative that the decisions made in the first (sizing) and the third
(writing) phase are exactly the same. In order to make that easier to guarantee
the serialisation interfaces of many compound types accept a lambda template
which is first invoked with a sizer and later with a writer helper objects.
#### Placeholders
If all instances of an IMR type are guaranteed to be the same size it is
possible to defer their serialisation. The serialisation function can be given
a placeholder object which would later be set to an actual value.
#### Continuation hooks
Compound types can be nested, e.g. a member of a structure can be another
structure. However, as mentioned in the [Serialisers](#serialisers) section the
basic interface for serialisation is to accept lambda objects. In case of
nested structures this would require creating a nested lambdas which can be
inconvenient and result in a code that is hard to follow.
The solution is to introduce continuation hooks to serialisation helpers. The
outer structure serialisation helper provides a `serialize_nested()` member
function for serialising the inner one. The function returns a serialisation
helper for the nested structure which behaves as the stand-alone one except
that when `done()` is called it returns back a serialisation helper for the
outer structure.
```c++
return serializer
.serialize(...)
.serialize_nested()
.serialize(...)
.done()
.serialize(...)
.done();
```
### IMR types
#### Tags
IMR extensively uses tags to name members of a compound type or match an
instance of a type with an appropriate member function of the deserialisation
context.
Any type can be a tag, but it is customary to use incomplete classes with a
meaningful names for this purpose.
Reusing tags should be avoided in order to avoid clashes.
#### Type concept
Most IMR types follow a similar interface for serialisation and
deserialisation. There may be, however, some differences specific to a
particular type and its purpose.
```c++
struct imr_type {
static auto make_view(const uint_t* imr_object) noexcept;
static auto make_view(uint_t* imr_object) noexcept;
// Returns the size of a serialized IMR object
template<typename Context>
static size_t serialized_object_size(const uint8_t* imr_object, const Context& context) noexcept;
// Returns the size of a serialized object without actually serializing it.
// Arguments depend on the IMR type.
static size_t size_when_serialized(...) noexcept;
// Returns the size of a serialized object. Arguments depend on the actual
// IMR type.
static size_t serialize(uint8_t* out, ...) noexcept;
};
```
## Fundamental types
Type | Fixed size | Needs context | In-place update | Placeholders |
----------------------|:----------:|:-------------:|:--------------------------:|:------------:|
`flags` | Yes | No | Yes | Yes |
`pod` | Yes | No | Yes | Yes |
`buffer` | No | Yes | Yes (length cannot change) | No |
### `flags`
```c++
template<typename Tag>
struct imr::set_flag {
set_flag(bool v = true) noexcept;
};
template<typename Tags...>
struct imr::flags {
struct view {
template<typename Tag>
bool get() const noexcept;
template<typename Tag>
bool set(bool value = true) noexcept;
};
// Sets flags which tags are present in Tags1, the remaining flags are
// cleared.
template<typename... Tags1>
void serialize(imr::set_flag<Tags1>...) noexcept;
};
```
`flags` is a bitfield of named flags. Each declared flag uses exactly one bit
and the whole bitfield uses the smalles number of bytes possible to fit all
required bits.
`flags` is a fixed-size type that can be updated in place and doesn't require
and external context for deserialisation.
### `pod`
```c++
template<typename PODType>
struct imr::pod {
struct view {
PODType load() const noexcept;
void store(PODType object) noexcept;
};
void serialize(PODType object) noexcept;
};
```
`pod` is a POD object using the same representation as that defined by the C++
ABI including all the internal padding in case of compound types. It is
therefore recommended that `pod` is used exclusively to represent C++
fundamental types.
`pod` is a fixed-size type which can be updated in place.
It does not require an external context for deserialisation.
### `buffer`
```c++
template<typename Tag>
struct imr::buffer {
using view = bytes_view;
void serialize(bytes_view) noexcept;
};
```
`buffer` is a type representing an array of bytes of an arbitrary length.
`buffer` is a variable-size type which can be updated in-place as long as the
length of the buffer remains unchanged.
#### Context requirements
Deserialisation of `buffer` objects requires an external context providing the
following member:
```c++
size_t size_of<Tag>() const noexcept;
```
Where `Tag` is the same type as the one used as a template parameter for
`buffer`. `size_of()` is must return the length of the buffer.
## Compound types
### `tagged_type`
```c++
template<typename Tag, typename IMRType>
struct imr::tagged_type : IMRType { };
```
`tagged_type` is a thin wrapper around an IMR type that allows annotating it
with a tag without changing its views or serialisers in any way.
The main purpose of adding tags to the type name is to allow defining custom
methods for only some usage of a particular IMR type (see section
[Methods](#methods)).
### `optional`
```c++
template<typename Tag, typename IMRType>
struct imr::optional {
struct view {
template<typename Context>
IMRType::view get(const Context&);
};
template<typename... Args>
void serialize(Args&&...) noexcept;
};
```
`optional` represents an object that may not be present in some circumstances.
Information whether it does is not stored in any sort of metadata, but the
deserialisation context is expected to provide it.
#### Context requirements
```c++
bool is_present<Tag>() const noexcept;
```
Optionals require the deserialisation context to provide information whether
the underlying object is present.
### `variant`
```c++
template<typename Tag, typename IMRType>
struct imr::alternative;
template<typename Tag, typename... Alternatives>
struct imr::variant {
struct view {
template<typename AlternativeTag>
auto as() const noexcept;
template<typename Visitor>
decltype(auto) accept(Visitor&&) const;
};
template<typename AlternativeTag, typename... Args>
void serialize(Args&&...) noexcept;
};
```
`variant` is a compound type that has multiple alternatives, each being an
IMR type. At any time there is exactly one active alternative and it is the
only one that can be read. Variants do not store information about the active
alternative by themselves but rely on the external context to provide that
information.
#### Context requirements
```c++
imr::variant::alternative_index active_alternative_of<Tag>() const noexcept;
template<typename AlternativeTag>
auto context_for() const noexcept;
```
Variant requires deserialisation context to provide information which one of
its alternatives is active as well as context for deserialisation of that
alternative.
### `structure`
```c++
template<typename Tag, typename IMRType>
struct imr::member;
template<typename Tag, typename Members>
struct imr::structure {
struct view {
template<typename MemberTag>
auto get() noexcept;
template<typename MemberTag>
size_t offset_of() const noexcept;
};
template<typename Writer, typename... Args>
void serialize(Writer&&, Args&&...) noexcept;
template<typename Tag, typename Context>
auto get_member(uint8_t*, const Context&) const noexcept;
};
struct imr::structure_writer {
// For imr::optional:
template<typename... Args>
auto serialize(Args&&...);
auto skip();
// For imr::variant:
template<typename AlternativeTag, typename... Args>
auto serialize_as(Args&&...);
// For other types:
template<typename... Args>
auto serialize(Args&&...);
// After all members:
auto done() noexcept;
};
```
`structure` is a compound type that stores possibly multiple member objects
one after another. All members are always present and their IMR type is fixed.
The interface for serialising structures is based on serialisation helpers.
Methods `serialize()` and `size_when_serialized()` accept a lambda that would
be given a serialisation helper object. That object expects its member function
`serialize()` (there are slightly different ones for optional and variant
structure members) to be invoked with an arguments that would normally be passed
to methods `structure()` and `size_when_serialized()` of the type of the member
that is being serialised (in the order they appear in the structure definition).
After the last member has been serialised `done()` is to be called and its
return value returned from the lambda.
#### Context requirements
```c++
template<typename AlternativeTag>
auto context_for() const noexcept;
```
## Methods
IMR types can define destructor and mover methods. Their goal is to provide
similar functionality to C++ destructors and move constructors and thus enable
owning references inside IMR objects. Movers are also essential in ensuring
that all references are properly updated when LSA moves objects around.
The default mover and destructor for fundamental types does nothing. The
default methods for compound types and containers invokes that method for
all present elements in an unspecified order.
User can define mover and destructor methods for any IMR type which would
override any default methods that this type might have.
Neither movers nor destructors are allowed to fail.
```c++
template<>
struct imr::mover<IMRType> {
static void run(uint8_t* ptr, const Context& ctx) noexcept {
// user-defined mover action
}
};
template<>
struct imr::destructor<IMRType> {
static void run(const uint8_t* ptr, const Context& ctx) noexcept {
// user-defined destructor action
}
};
```
### Mover
When an IMR object is being moved its serialised form is copied to the new
place in memory and then a mover method is called with the address of the new
location passed as an argument.
*Note that, unlike C++, IMR does not call the destructor of the object moved
from. In other words, while moving in C++ involves creating a new object that
steals internal state from another, IMR just copies the object to the new
location and invokes a mover to give it a chance to do necessary updates, it is
however still considered to be the same object as the original one.*
### Destructor
The destructor of an IMR object is called before the storage it occupies is
freed. The address of the object passed to the destructor is either a location
at which the object was originally created or the address that was given to the
latest invocation of the object mover method.
## Memory allocations
IMR objects are allowed to own memory and it is therefore expected that during
serialisation there will be a need to perform allocation. A convenience class
`imr::alloc::object_allocator` exists to aid this process.
The object allocator provides serialisation helpers for both sizing and writing
phases, which act similarily to the structure serialisation helpers and return
a pointer to the owned object. After the sizing phase member function
`allocate_all()` needs to be called which allocates all the reserved buffers.
*Note that in case there is a chain of owned objects (e.g. a list) placeholders
can be used for pointers to avoid recursion.*
```c++
using imr_type1 = ...;
using imr_type2 = ...;
imr::alloc::object_allocator allocator;
auto writer = [&] (auto serializer, auto& allocator) noexcept {
imr::placeholder<imr::pod<void*>> pointer;
auto ret = serializer
.serialize(pointer)
.done();
auto pointer_to_owned_object = allocator.allocate_nested<imr_type2>()
.serialize(...)
.serialize(...)
.done();
pointer.serialize(pointer_to_owned_object);
return ret;
};
auto size = imr_type1::size_when_serialized(writer , allocator.get_size());
auto obj = std::make_unique<uint8_t[]>(size.total_storage());
allocator.allocate_all();
imr_type1::serialize(obj.get(), writer, allocator.get_serializer());
```
### Log-Structured Allocator support
LSA relies on migrator objects to provide logic necessary for moving objects
and obtaining their size. To do these action the information about the IMR type
of the object needs to be preserved as well as at least minimal external
context sufficient for calling the mover.
A helper class template is provided which takes the IMR type and (context
factory)[#context_factories] type as template parameters as well as actual
context factory in constructor and implements `migrate_fn_type` interface.
```c++
imr::alloc::lsa_migrate_fn<IMRType, ContextFactory> migrator(context_factory);
auto ptr = current_allocator().alloc(migrator, size, 1);
```

167
imr/methods.hh Normal file
View File

@@ -0,0 +1,167 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
namespace imr {
namespace methods {
template<template<class> typename Method>
struct trivial_method {
template<typename... Args>
static void run(Args&&...) noexcept { }
};
template<template<class> typename Method, typename T>
using has_trivial_method = std::is_base_of<trivial_method<Method>, Method<T>>;
namespace internal {
template<template<class> typename Method, typename...>
struct generate_method : trivial_method<Method> { };
template<template<class> typename Method, typename Structure, typename... Tags, typename... Types>
struct generate_method<Method, Structure, member<Tags, Types>...> {
template<typename Context, typename... Args>
static void run(uint8_t* ptr, const Context& context, Args&&... args) noexcept {
auto view = Structure::make_view(ptr, context);
meta::for_each<member<Tags, Types>...>([&] (auto member_type) {
using member = std::remove_pointer_t<decltype(member_type)>;
auto member_ptr = ptr + view.template offset_of<typename member::tag>();
Method<typename member::type>::run(member_ptr,
context.template context_for<typename member::tag>(member_ptr),
std::forward<Args>(args)...);
});
}
};
template<template<class> typename Method, typename Tag, typename Type>
struct generate_method<Method, optional<Tag, Type>> {
template<typename Context, typename... Args>
static void run(uint8_t* ptr, const Context& context, Args&&... args) noexcept {
if (context.template is_present<Tag>()) {
Method<Type>::run(ptr,
context.template context_for<Tag>(ptr),
std::forward<Args>(args)...);
}
}
};
template<template<class> typename Method, typename Tag, typename... Members>
struct generate_method<Method, variant<Tag, Members...>> {
template<typename Context, typename... Args>
static void run(uint8_t* ptr, const Context& context, Args&&... args) noexcept {
auto view = variant<Tag, Members...>::make_view(ptr, context);
view.visit_type([&] (auto alternative_type) {
using member = std::remove_pointer_t<decltype(alternative_type)>;
Method<typename member::type>::run(ptr,
context.template context_for<typename member::tag>(ptr),
std::forward<Args>(args)...);
}, context);
}
};
template<template<class> typename Method>
struct member_has_trivial_method {
template<typename T>
struct type;
};
template<template<class> typename Method>
template<typename Tag, typename Type>
struct member_has_trivial_method<Method>::type<member<Tag, Type>> : has_trivial_method<Method, Type> { };
template<template<class> typename Method, typename T>
struct get_method;
template<template<class> typename Method, typename... Members>
struct get_method<Method, structure<Members...>>
: std::conditional_t<meta::all_of<member_has_trivial_method<Method>::template type, Members...>,
trivial_method<Method>,
generate_method<Method, structure<Members...>, Members...>>
{ };
template<template<class> typename Method, typename Tag, typename Type>
struct get_method<Method, optional<Tag, Type>>
: std::conditional_t<has_trivial_method<Method, Type>::value,
trivial_method<Method>,
generate_method<Method, optional<Tag, Type>>>
{ };
template<template<class> typename Method, typename Tag, typename... Members>
struct get_method<Method, variant<Tag, Members...>>
: std::conditional_t<meta::all_of<member_has_trivial_method<Method>::template type, Members...>,
trivial_method<Method>,
generate_method<Method, variant<Tag, Members...>>>
{ };
template<template<class> typename Method, typename Tag, typename Type>
struct get_method<Method, tagged_type<Tag,Type>>
: std::conditional_t<has_trivial_method<Method, Type>::value,
trivial_method<Method>,
Method<Type>>
{ };
}
template<typename T>
struct destructor : trivial_method<destructor> { };
using trivial_destructor = trivial_method<destructor>;
template<typename T>
using is_trivially_destructible = has_trivial_method<destructor, T>;
template<typename... Members>
struct destructor<structure<Members...>> : internal::get_method<destructor, structure<Members...>> { };
template<typename Tag, typename Type>
struct destructor<optional<Tag, Type>> : internal::get_method<destructor, optional<Tag, Type>> { };
template<typename Tag, typename... Members>
struct destructor<variant<Tag, Members...>> : internal::get_method<destructor, variant<Tag, Members...>> { };
template<typename T, typename Context = decltype(no_context)>
void destroy(uint8_t* ptr, const Context& context = no_context) {
destructor<T>::run(ptr, context);
}
template<typename T>
struct mover : trivial_method<mover> { };
using trivial_mover = trivial_method<mover>;
template<typename T>
using is_trivially_movable = has_trivial_method<mover, T>;
template<typename... Members>
struct mover<structure<Members...>> : internal::get_method<mover, structure<Members...>> { };
template<typename Tag, typename Type>
struct mover<optional<Tag, Type>> : internal::get_method<mover, optional<Tag, Type>> { };
template<typename Tag, typename... Members>
struct mover<variant<Tag, Members...>> : internal::get_method<mover, variant<Tag, Members...>> { };
template<typename T, typename Context = decltype(no_context)>
void move(uint8_t* ptr, const Context& context = no_context) {
mover<T>::run(ptr, context);
}
}
}

188
imr/utils.hh Normal file
View File

@@ -0,0 +1,188 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <type_traits>
#include "imr/core.hh"
#include "imr/alloc.hh"
#include "imr/concepts.hh"
namespace imr {
namespace utils {
class basic_object {
public:
struct tags {
class back_pointer { };
class object { };
};
protected:
uint8_t* _data = nullptr;
friend struct methods::mover<imr::tagged_type<tags::back_pointer, imr::pod<basic_object*>>>;
protected:
explicit basic_object(uint8_t* ptr) noexcept : _data(ptr) { }
void set_data(uint8_t* ptr) noexcept { _data = ptr; }
public:
basic_object() = default;
basic_object(basic_object&& other) noexcept : _data(std::exchange(other._data, nullptr)) { }
basic_object(const basic_object&) = delete;
};
template<typename Context, typename... State>
class object_context {
std::tuple<State...> _state;
private:
template<size_t... Index>
Context create(const uint8_t* ptr, std::index_sequence<Index...>) const noexcept {
return Context(ptr, std::get<Index>(_state)...);
}
public:
object_context(const uint8_t*, State... state) : _state { state... } { }
template<typename Tag, typename... Args>
Context context_for(const uint8_t* ptr, Args&&... args) const noexcept {
return create(ptr, std::index_sequence_for<State...>());
}
};
}
namespace methods {
template<>
struct mover<imr::tagged_type<utils::basic_object::tags::back_pointer, imr::pod<utils::basic_object*>>> {
static void run(uint8_t* ptr, ...) {
auto bptr = imr::tagged_type<utils::basic_object::tags::back_pointer, imr::pod<utils::basic_object*>>::make_view(ptr).load();
bptr->_data = ptr;
}
};
}
namespace utils {
/// Unique pointer to an IMR object
///
/// This is an LSA-aware unique-owner pointer to an IMR object.
template<typename Structure>
class object : public basic_object {
public:
using structure = imr::structure<
imr::member<tags::back_pointer, imr::tagged_type<tags::back_pointer, imr::pod<basic_object*>>>,
imr::member<tags::object, Structure>
>;
private:
explicit object(uint8_t* ptr) noexcept
: basic_object(ptr)
{
structure::template get_member<tags::back_pointer>(_data).store(this);
}
public:
object() = default;
object(object&& other) noexcept : basic_object(std::move(other)) {
if (_data) {
structure::template get_member<tags::back_pointer>(_data).store(this);
}
}
object& operator=(object&& other) noexcept {
swap(other);
return *this;
}
~object() {
if (_data) {
imr::methods::destroy<structure>(_data);
current_allocator().free(_data);
}
}
void swap(object& other) noexcept {
std::swap(_data, other._data);
if (_data) {
structure::template get_member<tags::back_pointer>(_data).store(this);
}
if (other._data) {
structure::template get_member<tags::back_pointer>(other._data).store(&other);
}
}
explicit operator bool() const noexcept { return bool(_data); }
uint8_t* get() noexcept { return _data ? _data + structure::template offset_of<tags::object>(_data) : nullptr; }
const uint8_t* get() const noexcept { return _data ? _data + structure::template offset_of<tags::object>(_data) : nullptr; }
/// Creates an IMR object from a raw writer
///
/// This low-level function creates an IMR object owned by `object` using
/// a raw writer (i.e. does not necessarily follow the standard IMR
/// serialisation process). This is useful for fast copying of trivial
/// IMR objects.
///
/// \note This function could be deprecated once the IMR starts supporting
/// copying IMR objects.
template<typename RawWriter>
GCC6_CONCEPT(requires requires (RawWriter wr, uint8_t* ptr) {
{ wr(ptr) } noexcept;
})
static object make_raw(size_t len, RawWriter&& wr, allocation_strategy::migrate_fn migrate = &imr::alloc::default_lsa_migrate_fn<structure>::migrate_fn) {
object obj;
auto ptr = static_cast<uint8_t*>(current_allocator().alloc(migrate, sizeof(void*) + len, 1));
wr(ptr + sizeof(void*));
auto view = structure::make_view(ptr);
view.template get<tags::back_pointer>().store(&obj);
obj.set_data(ptr);
return obj;
}
/// Create an IMR objects
template<typename Writer>
GCC6_CONCEPT(requires WriterAllocator<Writer, Structure>)
static object make(Writer&& object_writer,
allocation_strategy::migrate_fn migrate = &imr::alloc::default_lsa_migrate_fn<structure>::migrate_fn) {
struct alloc_deleter {
void operator()(uint8_t* ptr) {
current_allocator().free(ptr);
}
};
using alloc_unique_ptr = std::unique_ptr<uint8_t[], alloc_deleter>;
auto writer = [&object_writer] (auto&& ser, auto&& alloc) {
return object_writer(ser.serialize(nullptr).serialize_nested(), alloc).done();
};
auto& alloc = current_allocator();
alloc::object_allocator allocator(alloc);
auto obj_size = structure::size_when_serialized(writer, allocator.get_sizer());
auto ptr = alloc_unique_ptr(static_cast<uint8_t*>(alloc.alloc(migrate, obj_size, 1)));
allocator.allocate_all();
structure::serialize(ptr.get(), writer, allocator.get_serializer());
return object(ptr.release());
}
};
}
}

12
memtable.cc
View File

@@ -428,9 +428,11 @@ public:
};
class partition_snapshot_accounter {
const schema& _schema;
flush_memory_accounter& _accounter;
public:
partition_snapshot_accounter(flush_memory_accounter& acct): _accounter(acct) {}
partition_snapshot_accounter(const schema& s, flush_memory_accounter& acct)
: _schema(s), _accounter(acct) {}
// We will be passed mutation fragments here, and they are allocated using the standard
// allocator. So we can't compute the size in memtable precisely. However, precise accounting is
@@ -439,11 +441,11 @@ public:
// allocation. As long as our size read here is lesser or equal to the size in the memtables, we
// are safe, and worst case we will allow a bit fewer requests in.
void operator()(const range_tombstone& rt) {
_accounter.update_bytes_read(rt.memory_usage());
_accounter.update_bytes_read(rt.memory_usage(_schema));
}
void operator()(const static_row& sr) {
_accounter.update_bytes_read(sr.external_memory_usage());
_accounter.update_bytes_read(sr.external_memory_usage(_schema));
}
void operator()(const partition_start& ph) {}
@@ -457,7 +459,7 @@ public:
// and we don't know which one(s) contributed to the generation of this mutation fragment.
//
// We will add the size of the struct here, and that should be good enough.
_accounter.update_bytes_read(sizeof(rows_entry) + cr.external_memory_usage());
_accounter.update_bytes_read(sizeof(rows_entry) + cr.external_memory_usage(_schema));
}
};
@@ -500,7 +502,7 @@ private:
auto cr = query::clustering_key_filter_ranges::get_ranges(*schema(), schema()->full_slice(), key_and_snp->first.key());
auto snp_schema = key_and_snp->second->schema();
auto mpsr = make_partition_snapshot_flat_reader<partition_snapshot_accounter>(snp_schema, std::move(key_and_snp->first), std::move(cr),
std::move(key_and_snp->second), false, region(), read_section(), mtbl(), streamed_mutation::forwarding::no, _flushed_memory);
std::move(key_and_snp->second), false, region(), read_section(), mtbl(), streamed_mutation::forwarding::no, *snp_schema, _flushed_memory);
if (snp_schema->version() != schema()->version()) {
_partition_reader = transform(std::move(mpsr), schema_upgrader(schema()));
} else {

8
memtable.hh
View File

@@ -79,7 +79,7 @@ public:
size_t size_in_allocator(allocation_strategy& allocator) {
auto size = size_in_allocator_without_rows(allocator);
for (auto&& v : _pe.versions()) {
size += v.size_in_allocator(allocator);
size += v.size_in_allocator(*_schema, allocator);
}
return size;
}
@@ -183,14 +183,16 @@ private:
r.for_each_cell([this, &s, kind](column_id id, const atomic_cell_or_collection& item) {
auto& col = s.column_at(kind, id);
if (col.is_atomic()) {
update(item.as_atomic_cell());
update(item.as_atomic_cell(col));
} else {
auto ctype = static_pointer_cast<const collection_type_impl>(col.type);
auto mview = ctype->deserialize_mutation_form(item.as_collection_mutation());
item.as_collection_mutation().data.with_linearized([&] (bytes_view bv) {
auto mview = ctype->deserialize_mutation_form(bv);
update(mview.tomb);
for (auto& entry : mview.cells) {
update(entry.second);
}
});
}
});
}

10
mutation.cc
View File

@@ -37,9 +37,9 @@ mutation::data::data(partition_key&& key_, schema_ptr&& schema)
{ }
mutation::data::data(schema_ptr&& schema, dht::decorated_key&& key, const mutation_partition& mp)
: _schema(std::move(schema))
: _schema(schema)
, _dk(std::move(key))
, _p(mp)
, _p(*schema, mp)
{ }
mutation::data::data(schema_ptr&& schema, dht::decorated_key&& key, mutation_partition&& mp)
@@ -60,7 +60,7 @@ void mutation::set_static_cell(const bytes& name, const data_value& value, api::
if (!column_def->is_static()) {
throw std::runtime_error(sprint("column '%s' is not static", name));
}
partition().static_row().apply(*column_def, atomic_cell::make_live(timestamp, column_def->type->decompose(value), ttl));
partition().static_row().apply(*column_def, atomic_cell::make_live(*column_def->type, timestamp, column_def->type->decompose(value), ttl));
}
void mutation::set_clustered_cell(const clustering_key& key, const bytes& name, const data_value& value,
@@ -69,7 +69,7 @@ void mutation::set_clustered_cell(const clustering_key& key, const bytes& name,
if (!column_def) {
throw std::runtime_error(sprint("no column definition found for '%s'", name));
}
return set_clustered_cell(key, *column_def, atomic_cell::make_live(timestamp, column_def->type->decompose(value), ttl));
return set_clustered_cell(key, *column_def, atomic_cell::make_live(*column_def->type, timestamp, column_def->type->decompose(value), ttl));
}
void mutation::set_clustered_cell(const clustering_key& key, const column_definition& def, atomic_cell_or_collection&& value) {
@@ -83,7 +83,7 @@ void mutation::set_cell(const clustering_key_prefix& prefix, const bytes& name,
if (!column_def) {
throw std::runtime_error(sprint("no column definition found for '%s'", name));
}
return set_cell(prefix, *column_def, atomic_cell::make_live(timestamp, column_def->type->decompose(value), ttl));
return set_cell(prefix, *column_def, atomic_cell::make_live(*column_def->type, timestamp, column_def->type->decompose(value), ttl));
}
void mutation::set_cell(const clustering_key_prefix& prefix, const column_definition& def, atomic_cell_or_collection&& value) {

2
mutation_compactor.hh
View File

@@ -190,7 +190,7 @@ public:
requires CompactedFragmentsConsumer<Consumer>
)
stop_iteration consume(static_row&& sr, Consumer& consumer) {
_last_static_row = sr;
_last_static_row = static_row(_schema, sr);
auto current_tombstone = _range_tombstones.get_partition_tombstone();
bool is_live = sr.cells().compact_and_expire(_schema, column_kind::static_column,
row_tombstone(current_tombstone),

55
mutation_fragment.hh
View File

@@ -54,8 +54,10 @@ public:
: _ck(std::move(ck)), _t(t), _marker(std::move(marker)), _cells(std::move(cells)) {
_t.maybe_shadow(marker);
}
clustering_row(const rows_entry& re)
: clustering_row(re.key(), re.row().deleted_at(), re.row().marker(), re.row().cells()) { }
clustering_row(const schema& s, const clustering_row& other)
: clustering_row(other._ck, other._t, other._marker, row(s, column_kind::regular_column, other._cells)) { }
clustering_row(const schema& s, const rows_entry& re)
: clustering_row(re.key(), re.row().deleted_at(), re.row().marker(), row(s, column_kind::regular_column, re.row().cells())) { }
clustering_row(rows_entry&& re)
: clustering_row(std::move(re.key()), re.row().deleted_at(), re.row().marker(), std::move(re.row().cells())) { }
@@ -111,12 +113,12 @@ public:
position_in_partition_view position() const;
size_t external_memory_usage() const {
return _ck.external_memory_usage() + _cells.external_memory_usage();
size_t external_memory_usage(const schema& s) const {
return _ck.external_memory_usage() + _cells.external_memory_usage(s, column_kind::regular_column);
}
size_t memory_usage() const {
return sizeof(clustering_row) + external_memory_usage();
size_t memory_usage(const schema& s) const {
return sizeof(clustering_row) + external_memory_usage(s);
}
bool equal(const schema& s, const clustering_row& other) const {
@@ -133,7 +135,8 @@ class static_row {
row _cells;
public:
static_row() = default;
explicit static_row(const row& r) : _cells(r) { }
static_row(const schema& s, const static_row& other) : static_row(s, other._cells) { }
explicit static_row(const schema& s, const row& r) : _cells(s, column_kind::static_column, r) { }
explicit static_row(row&& r) : _cells(std::move(r)) { }
row& cells() { return _cells; }
@@ -159,12 +162,12 @@ public:
position_in_partition_view position() const;
size_t external_memory_usage() const {
return _cells.external_memory_usage();
size_t external_memory_usage(const schema& s) const {
return _cells.external_memory_usage(s, column_kind::static_column);
}
size_t memory_usage() const {
return sizeof(static_row) + external_memory_usage();
size_t memory_usage(const schema& s) const {
return sizeof(static_row) + external_memory_usage(s);
}
bool equal(const schema& s, const static_row& other) const {
@@ -190,12 +193,12 @@ public:
position_in_partition_view position() const;
size_t external_memory_usage() const {
size_t external_memory_usage(const schema&) const {
return _key.external_memory_usage();
}
size_t memory_usage() const {
return sizeof(partition_start) + external_memory_usage();
size_t memory_usage(const schema& s) const {
return sizeof(partition_start) + external_memory_usage(s);
}
bool equal(const schema& s, const partition_start& other) const {
@@ -209,12 +212,12 @@ class partition_end final {
public:
position_in_partition_view position() const;
size_t external_memory_usage() const {
size_t external_memory_usage(const schema&) const {
return 0;
}
size_t memory_usage() const {
return sizeof(partition_end) + external_memory_usage();
size_t memory_usage(const schema& s) const {
return sizeof(partition_end) + external_memory_usage(s);
}
bool equal(const schema& s, const partition_end& other) const {
@@ -328,14 +331,14 @@ public:
mutation_fragment(partition_start&& r);
mutation_fragment(partition_end&& r);
mutation_fragment(const mutation_fragment& o)
mutation_fragment(const schema& s, const mutation_fragment& o)
: _kind(o._kind), _data(std::make_unique<data>()) {
switch(_kind) {
case kind::static_row:
new (&_data->_static_row) static_row(o._data->_static_row);
new (&_data->_static_row) static_row(s, o._data->_static_row);
break;
case kind::clustering_row:
new (&_data->_clustering_row) clustering_row(o._data->_clustering_row);
new (&_data->_clustering_row) clustering_row(s, o._data->_clustering_row);
break;
case kind::range_tombstone:
new (&_data->_range_tombstone) range_tombstone(o._data->_range_tombstone);
@@ -349,14 +352,6 @@ public:
}
}
mutation_fragment(mutation_fragment&& other) = default;
mutation_fragment& operator=(const mutation_fragment& other) {
if (this != &other) {
mutation_fragment copy(other);
this->~mutation_fragment();
new (this) mutation_fragment(std::move(copy));
}
return *this;
}
mutation_fragment& operator=(mutation_fragment&& other) noexcept {
if (this != &other) {
this->~mutation_fragment();
@@ -471,9 +466,9 @@ public:
abort();
}
size_t memory_usage() const {
size_t memory_usage(const schema& s) const {
if (!_data->_size_in_bytes) {
_data->_size_in_bytes = sizeof(data) + visit([] (auto& mf) -> size_t { return mf.external_memory_usage(); });
_data->_size_in_bytes = sizeof(data) + visit([&s] (auto& mf) -> size_t { return mf.external_memory_usage(s); });
}
return *_data->_size_in_bytes;
}

143
mutation_partition.cc
View File

@@ -139,14 +139,14 @@ struct reversal_traits<true> {
}
};
mutation_partition::mutation_partition(const mutation_partition& x)
mutation_partition::mutation_partition(const schema& s, const mutation_partition& x)
: _tombstone(x._tombstone)
, _static_row(x._static_row)
, _static_row(s, column_kind::static_column, x._static_row)
, _static_row_continuous(x._static_row_continuous)
, _rows()
, _row_tombstones(x._row_tombstones) {
auto cloner = [] (const auto& x) {
return current_allocator().construct<std::remove_const_t<std::remove_reference_t<decltype(x)>>>(x);
auto cloner = [&s] (const auto& x) {
return current_allocator().construct<rows_entry>(s, x);
};
_rows.clone_from(x._rows, cloner, current_deleter<rows_entry>());
}
@@ -154,14 +154,14 @@ mutation_partition::mutation_partition(const mutation_partition& x)
mutation_partition::mutation_partition(const mutation_partition& x, const schema& schema,
query::clustering_key_filter_ranges ck_ranges)
: _tombstone(x._tombstone)
, _static_row(x._static_row)
, _static_row(schema, column_kind::static_column, x._static_row)
, _static_row_continuous(x._static_row_continuous)
, _rows()
, _row_tombstones(x._row_tombstones, range_tombstone_list::copy_comparator_only()) {
try {
for(auto&& r : ck_ranges) {
for (const rows_entry& e : x.range(schema, r)) {
_rows.insert(_rows.end(), *current_allocator().construct<rows_entry>(e), rows_entry::compare(schema));
_rows.insert(_rows.end(), *current_allocator().construct<rows_entry>(schema, e), rows_entry::compare(schema));
}
for (auto&& rt : x._row_tombstones.slice(schema, r)) {
_row_tombstones.apply(schema, rt);
@@ -210,13 +210,6 @@ mutation_partition::~mutation_partition() {
_rows.clear_and_dispose(current_deleter<rows_entry>());
}
mutation_partition&
mutation_partition::operator=(const mutation_partition& x) {
mutation_partition n(x);
std::swap(*this, n);
return *this;
}
mutation_partition&
mutation_partition::operator=(mutation_partition&& x) noexcept {
if (this != &x) {
@@ -257,8 +250,8 @@ struct mutation_fragment_applier {
_mp.apply_row_tombstone(_s, std::move(rt));
}
void operator()(static_row sr) {
_mp.static_row().apply(_s, column_kind::static_column, std::move(sr.cells()));
void operator()(const static_row& sr) {
_mp.static_row().apply(_s, column_kind::static_column, sr.cells());
}
void operator()(partition_start ps) {
@@ -268,9 +261,10 @@ struct mutation_fragment_applier {
void operator()(partition_end ps) {
}
void operator()(clustering_row cr) {
auto& dr = _mp.clustered_row(_s, std::move(cr.key()));
dr.apply(_s, std::move(cr));
void operator()(const clustering_row& cr) {
auto temp = clustering_row(_s, cr);
auto& dr = _mp.clustered_row(_s, std::move(temp.key()));
dr.apply(_s, std::move(temp));
}
};
@@ -336,7 +330,7 @@ void mutation_partition::apply_monotonically(const schema& s, mutation_partition
if (s.version() == p_schema.version()) {
apply_monotonically(s, std::move(p), no_cache_tracker);
} else {
mutation_partition p2(p);
mutation_partition p2(s, p);
p2.upgrade(p_schema, s);
apply_monotonically(s, std::move(p2), no_cache_tracker);
}
@@ -353,7 +347,7 @@ mutation_partition::apply_weak(const schema& s, mutation_partition_view p, const
void mutation_partition::apply_weak(const schema& s, const mutation_partition& p, const schema& p_schema) {
// FIXME: Optimize
apply_monotonically(s, mutation_partition(p), p_schema);
apply_monotonically(s, mutation_partition(s, p), p_schema);
}
void mutation_partition::apply_weak(const schema& s, mutation_partition&& p) {
@@ -457,7 +451,7 @@ void mutation_partition::insert_row(const schema& s, const clustering_key& key,
}
void mutation_partition::insert_row(const schema& s, const clustering_key& key, const deletable_row& row) {
auto e = current_allocator().construct<rows_entry>(key, row);
auto e = current_allocator().construct<rows_entry>(s, key, row);
_rows.insert(_rows.end(), *e, rows_entry::compare(s));
}
@@ -601,7 +595,7 @@ void write_cell(RowWriter& w, const query::partition_slice& slice, ::atomic_cell
} else {
return std::move(wr).skip_expiry();
}
}().write_value(c.value());
}().write_fragmented_value(c.value());
[&, wr = std::move(after_value)] () mutable {
if (slice.options.contains<query::partition_slice::option::send_ttl>() && c.is_live_and_has_ttl()) {
return std::move(wr).write_ttl(c.ttl());
@@ -627,6 +621,7 @@ void write_cell(RowWriter& w, const query::partition_slice& slice, const data_ty
template<typename RowWriter>
void write_counter_cell(RowWriter& w, const query::partition_slice& slice, ::atomic_cell_view c) {
assert(c.is_live());
counter_cell_view::with_linearized(c, [&] (counter_cell_view ccv) {
auto wr = w.add().write();
[&, wr = std::move(wr)] () mutable {
if (slice.options.contains<query::partition_slice::option::send_timestamp>()) {
@@ -635,9 +630,10 @@ void write_counter_cell(RowWriter& w, const query::partition_slice& slice, ::ato
return std::move(wr).skip_timestamp();
}
}().skip_expiry()
.write_value(counter_cell_view::total_value_type()->decompose(counter_cell_view(c).total_value()))
.write_value(counter_cell_view::total_value_type()->decompose(ccv.total_value()))
.skip_ttl()
.end_qr_cell();
});
}
// Used to return the timestamp of the latest update to the row
@@ -660,17 +656,17 @@ struct appending_hash<row> {
}
auto&& def = s.column_at(kind, id);
if (def.is_atomic()) {
max_ts.update(cell_and_hash->cell.as_atomic_cell().timestamp());
max_ts.update(cell_and_hash->cell.as_atomic_cell(def).timestamp());
if constexpr (query::using_hash_of_hash_v<Hasher>) {
if (cell_and_hash->hash) {
feed_hash(h, *cell_and_hash->hash);
} else {
query::default_hasher cellh;
feed_hash(cellh, cell_and_hash->cell.as_atomic_cell(), def);
feed_hash(cellh, cell_and_hash->cell.as_atomic_cell(def), def);
feed_hash(h, cellh.finalize_uint64());
}
} else {
feed_hash(h, cell_and_hash->cell.as_atomic_cell(), def);
feed_hash(h, cell_and_hash->cell.as_atomic_cell(def), def);
}
} else {
auto&& cm = cell_and_hash->cell.as_collection_mutation();
@@ -735,13 +731,13 @@ static void get_compacted_row_slice(const schema& s,
} else {
auto&& def = s.column_at(kind, id);
if (def.is_atomic()) {
auto c = cell->as_atomic_cell();
auto c = cell->as_atomic_cell(def);
if (!c.is_live()) {
writer.add().skip();
} else if (def.is_counter()) {
write_counter_cell(writer, slice, cell->as_atomic_cell());
write_counter_cell(writer, slice, cell->as_atomic_cell(def));
} else {
write_cell(writer, slice, cell->as_atomic_cell());
write_cell(writer, slice, cell->as_atomic_cell(def));
}
} else {
auto&& mut = cell->as_collection_mutation();
@@ -762,7 +758,7 @@ bool has_any_live_data(const schema& s, column_kind kind, const row& cells, tomb
cells.for_each_cell_until([&] (column_id id, const atomic_cell_or_collection& cell_or_collection) {
const column_definition& def = s.column_at(kind, id);
if (def.is_atomic()) {
auto&& c = cell_or_collection.as_atomic_cell();
auto&& c = cell_or_collection.as_atomic_cell(def);
if (c.is_live(tomb, now, def.is_counter())) {
any_live = true;
return stop_iteration::yes;
@@ -864,11 +860,6 @@ mutation_partition::query_compacted(query::result::partition_writer& pw, const s
}
}
std::ostream&
operator<<(std::ostream& out, const atomic_cell_or_collection& c) {
return out << to_hex(c._data);
}
std::ostream&
operator<<(std::ostream& os, const std::pair<column_id, const atomic_cell_or_collection&>& c) {
return fprint(os, "{column: %s %s}", c.first, c.second);
@@ -1046,9 +1037,9 @@ apply_monotonically(const column_definition& def, cell_and_hash& dst,
// provided via an upper layer
if (def.is_atomic()) {
if (def.is_counter()) {
counter_cell_view::apply(dst.cell, src); // FIXME: Optimize
counter_cell_view::apply(def, dst.cell, src); // FIXME: Optimize
dst.hash = { };
} else if (compare_atomic_cell_for_merge(dst.cell.as_atomic_cell(), src.as_atomic_cell()) < 0) {
} else if (compare_atomic_cell_for_merge(dst.cell.as_atomic_cell(def), src.as_atomic_cell(def)) < 0) {
using std::swap;
swap(dst.cell, src);
dst.hash = std::move(src_hash);
@@ -1062,7 +1053,7 @@ apply_monotonically(const column_definition& def, cell_and_hash& dst,
void
row::apply(const column_definition& column, const atomic_cell_or_collection& value, cell_hash_opt hash) {
auto tmp = value;
auto tmp = value.copy(*column.type);
apply_monotonically(column, std::move(tmp), std::move(hash));
}
@@ -1168,41 +1159,43 @@ row::find_cell(column_id id) const {
return c_a_h ? &c_a_h->cell : nullptr;
}
size_t row::external_memory_usage() const {
size_t row::external_memory_usage(const schema& s, column_kind kind) const {
size_t mem = 0;
if (_type == storage_type::vector) {
mem += _storage.vector.v.external_memory_usage();
column_id id = 0;
for (auto&& c_a_h : _storage.vector.v) {
mem += c_a_h.cell.external_memory_usage();
auto& cdef = s.column_at(kind, id++);
mem += c_a_h.cell.external_memory_usage(*cdef.type);
}
} else {
for (auto&& ce : _storage.set) {
mem += sizeof(cell_entry) + ce.cell().external_memory_usage();
auto& cdef = s.column_at(kind, ce.id());
mem += sizeof(cell_entry) + ce.cell().external_memory_usage(*cdef.type);
}
}
return mem;
}
size_t rows_entry::memory_usage() const {
size_t rows_entry::memory_usage(const schema& s) const {
size_t size = 0;
if (!dummy()) {
size += key().external_memory_usage();
}
return size +
row().cells().external_memory_usage() +
row().cells().external_memory_usage(s, column_kind::regular_column) +
sizeof(rows_entry);
}
size_t mutation_partition::external_memory_usage() const {
size_t mutation_partition::external_memory_usage(const schema& s) const {
size_t sum = 0;
auto& s = static_row();
sum += s.external_memory_usage();
sum += static_row().external_memory_usage(s, column_kind::static_column);
for (auto& clr : clustered_rows()) {
sum += clr.memory_usage();
sum += clr.memory_usage(s);
}
for (auto& rtb : row_tombstones()) {
sum += rtb.memory_usage();
sum += rtb.memory_usage(s);
}
return sum;
@@ -1396,15 +1389,24 @@ rows_entry::rows_entry(rows_entry&& o) noexcept
}
}
row::row(const row& o)
row::row(const schema& s, column_kind kind, const row& o)
: _type(o._type)
, _size(o._size)
{
if (_type == storage_type::vector) {
new (&_storage.vector) vector_storage(o._storage.vector);
auto& other_vec = o._storage.vector;
auto& vec = *new (&_storage.vector) vector_storage;
vec.present = other_vec.present;
vec.v.reserve(other_vec.v.size());
column_id id = 0;
for (auto& cell : other_vec.v) {
auto& cdef = s.column_at(kind, id++);
vec.v.emplace_back(cell_and_hash { cell.cell.copy(*cdef.type), cell.hash });
}
} else {
auto cloner = [] (const auto& x) {
return current_allocator().construct<std::remove_const_t<std::remove_reference_t<decltype(x)>>>(x);
auto cloner = [&] (const auto& x) {
auto& cdef = s.column_at(kind, x.id());
return current_allocator().construct<cell_entry>(*cdef.type, x);
};
new (&_storage.set) map_type;
try {
@@ -1425,9 +1427,9 @@ row::~row() {
}
}
row::cell_entry::cell_entry(const cell_entry& o)
row::cell_entry::cell_entry(const abstract_type& type, const cell_entry& o)
: _id(o._id)
, _cell_and_hash(o._cell_and_hash)
, _cell_and_hash{ o._cell_and_hash.cell.copy(type), o._cell_and_hash.hash }
{ }
row::cell_entry::cell_entry(cell_entry&& o) noexcept
@@ -1506,8 +1508,11 @@ bool row::equal(column_kind kind, const schema& this_schema, const row& other, c
auto cells_equal = [&] (std::pair<column_id, const atomic_cell_or_collection&> c1,
std::pair<column_id, const atomic_cell_or_collection&> c2) {
static_assert(schema::row_column_ids_are_ordered_by_name::value, "Relying on column ids being ordered by name");
return this_schema.column_at(kind, c1.first).name() == other_schema.column_at(kind, c2.first).name()
&& c1.second == c2.second;
auto& at1 = *this_schema.column_at(kind, c1.first).type;
auto& at2 = other_schema.column_at(kind, c2.first).type;
return at1.equals(at2)
&& this_schema.column_at(kind, c1.first).name() == other_schema.column_at(kind, c2.first).name()
&& c1.second.equals(at1, c2.second);
};
return with_both_ranges(other, [&] (auto r1, auto r2) {
return boost::equal(r1, r2, cells_equal);
@@ -1597,7 +1602,7 @@ bool row::compact_and_expire(
bool erase = false;
const column_definition& def = s.column_at(kind, id);
if (def.is_atomic()) {
atomic_cell_view cell = c.as_atomic_cell();
atomic_cell_view cell = c.as_atomic_cell(def);
auto can_erase_cell = [&] {
return cell.deletion_time() < gc_before && can_gc(tombstone(cell.timestamp(), cell.deletion_time()));
};
@@ -1619,14 +1624,16 @@ bool row::compact_and_expire(
} else {
auto&& cell = c.as_collection_mutation();
auto&& ctype = static_pointer_cast<const collection_type_impl>(def.type);
auto m_view = ctype->deserialize_mutation_form(cell);
collection_type_impl::mutation m = m_view.materialize();
cell.data.with_linearized([&] (bytes_view cell_bv) {
auto m_view = ctype->deserialize_mutation_form(cell_bv);
collection_type_impl::mutation m = m_view.materialize(*ctype);
any_live |= m.compact_and_expire(tomb, query_time, can_gc, gc_before);
if (m.cells.empty() && m.tomb <= tomb.tomb()) {
erase = true;
} else {
c = ctype->serialize_mutation_form(m);
}
});
}
return erase;
});
@@ -1668,15 +1675,15 @@ row row::difference(const schema& s, column_kind kind, const row& other) const
}
auto& cdef = s.column_at(kind, c.first);
if (it == other_range.end() || it->first != c.first) {
r.append_cell(c.first, c.second);
r.append_cell(c.first, c.second.copy(*cdef.type));
} else if (cdef.is_counter()) {
auto cell = counter_cell_view::difference(c.second.as_atomic_cell(), it->second.as_atomic_cell());
auto cell = counter_cell_view::difference(c.second.as_atomic_cell(cdef), it->second.as_atomic_cell(cdef));
if (cell) {
r.append_cell(c.first, std::move(*cell));
}
} else if (s.column_at(kind, c.first).is_atomic()) {
if (compare_atomic_cell_for_merge(c.second.as_atomic_cell(), it->second.as_atomic_cell()) > 0) {
r.append_cell(c.first, c.second);
if (compare_atomic_cell_for_merge(c.second.as_atomic_cell(cdef), it->second.as_atomic_cell(cdef)) > 0) {
r.append_cell(c.first, c.second.copy(*cdef.type));
}
} else {
auto ct = static_pointer_cast<const collection_type_impl>(s.column_at(kind, c.first).type);
@@ -1726,7 +1733,7 @@ void mutation_partition::accept(const schema& s, mutation_partition_visitor& v)
_static_row.for_each_cell([&] (column_id id, const atomic_cell_or_collection& cell) {
const column_definition& def = s.static_column_at(id);
if (def.is_atomic()) {
v.accept_static_cell(id, cell.as_atomic_cell());
v.accept_static_cell(id, cell.as_atomic_cell(def));
} else {
v.accept_static_cell(id, cell.as_collection_mutation());
}
@@ -1740,7 +1747,7 @@ void mutation_partition::accept(const schema& s, mutation_partition_visitor& v)
dr.cells().for_each_cell([&] (column_id id, const atomic_cell_or_collection& cell) {
const column_definition& def = s.regular_column_at(id);
if (def.is_atomic()) {
v.accept_row_cell(id, cell.as_atomic_cell());
v.accept_row_cell(id, cell.as_atomic_cell(def));
} else {
v.accept_row_cell(id, cell.as_collection_mutation());
}
@@ -2014,12 +2021,12 @@ public:
}
stop_iteration consume(static_row&& sr, tombstone, bool is_alive) {
_static_row_is_alive = is_alive;
_memory_accounter.update(sr.memory_usage());
_memory_accounter.update(sr.memory_usage(_schema));
return _mutation_consumer->consume(std::move(sr));
}
stop_iteration consume(clustering_row&& cr, row_tombstone, bool is_alive) {
_live_rows += is_alive;
auto stop = _memory_accounter.update_and_check(cr.memory_usage());
auto stop = _memory_accounter.update_and_check(cr.memory_usage(_schema));
if (is_alive) {
// We are considering finishing current read only after consuming a
// live clustering row. While sending a single live row is enough to
@@ -2031,7 +2038,7 @@ public:
return _mutation_consumer->consume(std::move(cr)) || _stop;
}
stop_iteration consume(range_tombstone&& rt) {
_memory_accounter.update(rt.memory_usage());
_memory_accounter.update(rt.memory_usage(_schema));
return _mutation_consumer->consume(std::move(rt));
}

34
mutation_partition.hh
View File

@@ -104,7 +104,7 @@ class row {
: _id(id)
{ }
cell_entry(cell_entry&&) noexcept;
cell_entry(const cell_entry&);
cell_entry(const abstract_type&, const cell_entry&);
column_id id() const { return _id; }
const atomic_cell_or_collection& cell() const { return _cell_and_hash.cell; }
@@ -166,7 +166,7 @@ private:
public:
row();
~row();
row(const row&);
row(const schema&, column_kind, const row&);
row(row&& other) noexcept;
row& operator=(row&& other) noexcept;
size_t size() const { return _size; }
@@ -335,7 +335,7 @@ public:
bool equal(column_kind kind, const schema& this_schema, const row& other, const schema& other_schema) const;
size_t external_memory_usage() const;
size_t external_memory_usage(const schema&, column_kind) const;
cell_hash_opt cell_hash_for(column_id id) const;
@@ -647,8 +647,13 @@ class deletable_row final {
public:
deletable_row() {}
explicit deletable_row(clustering_row&&);
deletable_row(row_tombstone tomb, const row_marker& marker, const row& cells)
: _deleted_at(tomb), _marker(marker), _cells(cells)
deletable_row(const schema& s, const deletable_row& other)
: _deleted_at(other._deleted_at)
, _marker(other._marker)
, _cells(s, column_kind::regular_column, other._cells)
{ }
deletable_row(const schema& s, row_tombstone tomb, const row_marker& marker, const row& cells)
: _deleted_at(tomb), _marker(marker), _cells(s, column_kind::regular_column, cells)
{}
void apply(const schema&, clustering_row);
@@ -737,16 +742,16 @@ public:
rows_entry(const clustering_key& key, deletable_row&& row)
: _key(key), _row(std::move(row))
{ }
rows_entry(const clustering_key& key, const deletable_row& row)
: _key(key), _row(row)
rows_entry(const schema& s, const clustering_key& key, const deletable_row& row)
: _key(key), _row(s, row)
{ }
rows_entry(const clustering_key& key, row_tombstone tomb, const row_marker& marker, const row& row)
: _key(key), _row(tomb, marker, row)
rows_entry(const schema& s, const clustering_key& key, row_tombstone tomb, const row_marker& marker, const row& row)
: _key(key), _row(s, tomb, marker, row)
{ }
rows_entry(rows_entry&& o) noexcept;
rows_entry(const rows_entry& e)
rows_entry(const schema& s, const rows_entry& e)
: _key(e._key)
, _row(e._row)
, _row(s, e._row)
, _flags(e._flags)
{ }
// Valid only if !dummy()
@@ -837,7 +842,7 @@ public:
bool equal(const schema& s, const rows_entry& other) const;
bool equal(const schema& s, const rows_entry& other, const schema& other_schema) const;
size_t memory_usage() const;
size_t memory_usage(const schema&) const;
void on_evicted(cache_tracker&) noexcept;
};
@@ -911,12 +916,11 @@ public:
, _row_tombstones(other._row_tombstones, range_tombstone_list::copy_comparator_only())
{ }
mutation_partition(mutation_partition&&) = default;
mutation_partition(const mutation_partition&);
mutation_partition(const schema& s, const mutation_partition&);
mutation_partition(const mutation_partition&, const schema&, query::clustering_key_filter_ranges);
mutation_partition(mutation_partition&&, const schema&, query::clustering_key_filter_ranges);
~mutation_partition();
static mutation_partition& container_of(rows_type&);
mutation_partition& operator=(const mutation_partition& x);
mutation_partition& operator=(mutation_partition&& x) noexcept;
bool equal(const schema&, const mutation_partition&) const;
bool equal(const schema& this_schema, const mutation_partition& p, const schema& p_schema) const;
@@ -1104,7 +1108,7 @@ public:
bool is_static_row_live(const schema&,
gc_clock::time_point query_time = gc_clock::time_point::min()) const;
size_t external_memory_usage() const;
size_t external_memory_usage(const schema&) const;
private:
template<typename Func>
void for_each_row(const schema& schema, const query::clustering_range& row_range, bool reversed, Func&& func) const;

12
mutation_partition_applier.hh
View File

@@ -39,11 +39,13 @@ public:
}
virtual void accept_static_cell(column_id id, atomic_cell_view cell) override {
_p._static_row.apply(_schema.column_at(column_kind::static_column, id), atomic_cell_or_collection(cell));
auto& cdef = _schema.static_column_at(id);
_p._static_row.apply(_schema.column_at(column_kind::static_column, id), atomic_cell_or_collection(*cdef.type, cell));
}
virtual void accept_static_cell(column_id id, collection_mutation_view collection) override {
_p._static_row.apply(_schema.column_at(column_kind::static_column, id), atomic_cell_or_collection(collection));
auto& ctype = *static_pointer_cast<const collection_type_impl>(_schema.static_column_at(id).type);
_p._static_row.apply(_schema.column_at(column_kind::static_column, id), atomic_cell_or_collection(collection_mutation(ctype, collection)));
}
virtual void accept_row_tombstone(const range_tombstone& rt) override {
@@ -58,10 +60,12 @@ public:
}
virtual void accept_row_cell(column_id id, atomic_cell_view cell) override {
_current_row->cells().apply(_schema.column_at(column_kind::regular_column, id), atomic_cell_or_collection(cell));
auto& cdef = _schema.regular_column_at(id);
_current_row->cells().apply(_schema.column_at(column_kind::regular_column, id), atomic_cell_or_collection(*cdef.type, cell));
}
virtual void accept_row_cell(column_id id, collection_mutation_view collection) override {
_current_row->cells().apply(_schema.column_at(column_kind::regular_column, id), atomic_cell_or_collection(collection));
auto& ctype = *static_pointer_cast<const collection_type_impl>(_schema.regular_column_at(id).type);
_current_row->cells().apply(_schema.column_at(column_kind::regular_column, id), atomic_cell_or_collection(collection_mutation(ctype, collection)));
}
};

13
mutation_partition_serializer.cc
View File

@@ -44,7 +44,7 @@ template<typename Writer>
auto write_live_cell(Writer&& writer, atomic_cell_view c)
{
return std::move(writer).write_created_at(c.timestamp())
.write_value(c.value())
.write_fragmented_value(c.value())
.end_live_cell();
}
@@ -59,7 +59,7 @@ auto write_counter_cell(Writer&& writer, atomic_cell_view c)
.write_delta(delta)
.end_counter_cell_update();
} else {
counter_cell_view ccv(c);
return counter_cell_view::with_linearized(c, [&] (counter_cell_view ccv) {
auto shards = std::move(value).start_value_counter_cell_full()
.start_shards();
if (service::get_local_storage_service().cluster_supports_correct_counter_order()) {
@@ -72,6 +72,7 @@ auto write_counter_cell(Writer&& writer, atomic_cell_view c)
}
}
return std::move(shards).end_shards().end_counter_cell_full();
});
}
}().end_counter_cell();
}
@@ -83,7 +84,7 @@ auto write_expiring_cell(Writer&& writer, atomic_cell_view c)
.write_expiry(c.expiry())
.start_c()
.write_created_at(c.timestamp())
.write_value(c.value())
.write_fragmented_value(c.value())
.end_c()
.end_expiring_cell();
}
@@ -101,8 +102,9 @@ auto write_dead_cell(Writer&& writer, atomic_cell_view c)
template<typename Writer>
auto write_collection_cell(Writer&& collection_writer, collection_mutation_view cmv, const column_definition& def)
{
return cmv.data.with_linearized([&] (bytes_view cmv_bv) {
auto&& ctype = static_pointer_cast<const collection_type_impl>(def.type);
auto m_view = ctype->deserialize_mutation_form(cmv);
auto m_view = ctype->deserialize_mutation_form(cmv_bv);
auto cells_writer = std::move(collection_writer).write_tomb(m_view.tomb).start_elements();
for (auto&& c : m_view.cells) {
auto cell_writer = cells_writer.add().write_key(c.first);
@@ -115,6 +117,7 @@ auto write_collection_cell(Writer&& collection_writer, collection_mutation_view
}
}
return std::move(cells_writer).end_elements().end_collection_cell();
});
}
template<typename Writer>
@@ -125,7 +128,7 @@ auto write_row_cells(Writer&& writer, const row& r, const schema& s, column_kind
auto& def = s.column_at(kind, id);
auto cell_or_collection_writer = column_writer.add().write_id(id);
if (def.is_atomic()) {
auto&& c = cell.as_atomic_cell();
auto&& c = cell.as_atomic_cell(def);
auto cell_writer = std::move(cell_or_collection_writer).start_c_variant();
if (!c.is_live()) {
write_dead_cell(std::move(cell_writer).start_variant_dead_cell(), c).end_variant().end_column();

49
mutation_partition_view.cc
View File

@@ -51,14 +51,19 @@ using atomic_cell_variant = boost::variant<ser::live_cell_view,
ser::counter_cell_view,
ser::unknown_variant_type>;
atomic_cell read_atomic_cell(atomic_cell_variant cv)
atomic_cell read_atomic_cell(const abstract_type& type, atomic_cell_variant cv, atomic_cell::collection_member cm = atomic_cell::collection_member::no)
{
struct atomic_cell_visitor : boost::static_visitor<atomic_cell> {
class atomic_cell_visitor : public boost::static_visitor<atomic_cell> {
const abstract_type& _type;
atomic_cell::collection_member _collection_member;
public:
explicit atomic_cell_visitor(const abstract_type& t, atomic_cell::collection_member cm)
: _type(t), _collection_member(cm) { }
atomic_cell operator()(ser::live_cell_view& lcv) const {
return atomic_cell::make_live(lcv.created_at(), lcv.value());
return atomic_cell::make_live(_type, lcv.created_at(), lcv.value(), _collection_member);
}
atomic_cell operator()(ser::expiring_cell_view& ecv) const {
return atomic_cell::make_live(ecv.c().created_at(), ecv.c().value(), ecv.expiry(), ecv.ttl());
return atomic_cell::make_live(_type, ecv.c().created_at(), ecv.c().value(), ecv.expiry(), ecv.ttl(), _collection_member);
}
atomic_cell operator()(ser::dead_cell_view& dcv) const {
return atomic_cell::make_dead(dcv.tomb().timestamp(), dcv.tomb().deletion_time());
@@ -93,19 +98,19 @@ atomic_cell read_atomic_cell(atomic_cell_variant cv)
throw std::runtime_error("Trying to deserialize cell in unknown state");
}
};
return boost::apply_visitor(atomic_cell_visitor(), cv);
return boost::apply_visitor(atomic_cell_visitor(type, cm), cv);
}
collection_mutation read_collection_cell(ser::collection_cell_view cv)
collection_mutation read_collection_cell(const collection_type_impl& ctype, ser::collection_cell_view cv)
{
collection_type_impl::mutation mut;
mut.tomb = cv.tomb();
auto&& elements = cv.elements();
mut.cells.reserve(elements.size());
for (auto&& e : elements) {
mut.cells.emplace_back(e.key(), read_atomic_cell(e.value()));
mut.cells.emplace_back(e.key(), read_atomic_cell(*ctype.value_comparator(), e.value(), atomic_cell::collection_member::yes));
}
return collection_type_impl::serialize_mutation_form(mut);
return ctype.serialize_mutation_form(mut);
}
template<typename Visitor>
@@ -130,7 +135,7 @@ void read_and_visit_row(ser::row_view rv, const column_mapping& cm, column_kind
// FIXME: Pass view to cell to avoid copy
auto&& outer = current_allocator();
with_allocator(standard_allocator(), [&] {
auto cell = read_atomic_cell(acv);
auto cell = read_atomic_cell(*_col.type(), acv);
with_allocator(outer, [&] {
_visitor.accept_atomic_cell(_id, cell);
});
@@ -143,7 +148,7 @@ void read_and_visit_row(ser::row_view rv, const column_mapping& cm, column_kind
// FIXME: Pass view to cell to avoid copy
auto&& outer = current_allocator();
with_allocator(standard_allocator(), [&] {
auto cell = read_collection_cell(ccv);
auto cell = read_collection_cell(*static_pointer_cast<const collection_type_impl>(_col.type()), ccv);
with_allocator(outer, [&] {
_visitor.accept_collection(_id, cell);
});
@@ -240,40 +245,46 @@ mutation_fragment frozen_mutation_fragment::unfreeze(const schema& s)
return seastar::visit(view.fragment(),
[&] (ser::clustering_row_view crv) {
class clustering_row_builder {
const schema& _s;
mutation_fragment _mf;
public:
clustering_row_builder(clustering_key key, row_tombstone t, row_marker m)
: _mf(mutation_fragment::clustering_row_tag_t(), std::move(key), std::move(t), std::move(m), row()) { }
clustering_row_builder(const schema& s, clustering_key key, row_tombstone t, row_marker m)
: _s(s), _mf(mutation_fragment::clustering_row_tag_t(), std::move(key), std::move(t), std::move(m), row()) { }
void accept_atomic_cell(column_id id, const atomic_cell& ac) {
_mf.as_mutable_clustering_row().cells().append_cell(id, atomic_cell_or_collection(ac));
auto& type = *_s.regular_column_at(id).type;
_mf.as_mutable_clustering_row().cells().append_cell(id, atomic_cell_or_collection(atomic_cell(type, ac)));
}
void accept_collection(column_id id, const collection_mutation& cm) {
_mf.as_mutable_clustering_row().cells().append_cell(id, atomic_cell_or_collection(cm));
auto& ctype = *static_pointer_cast<const collection_type_impl>(_s.regular_column_at(id).type);
_mf.as_mutable_clustering_row().cells().append_cell(id, atomic_cell_or_collection(collection_mutation(ctype, cm)));
}
mutation_fragment get_mutation_fragment() && { return std::move(_mf); }
};
auto cr = crv.row();
auto t = row_tombstone(cr.deleted_at(), shadowable_tombstone(cr.shadowable_deleted_at()));
clustering_row_builder builder(cr.key(), std::move(t), read_row_marker(cr.marker()));
clustering_row_builder builder(s, cr.key(), std::move(t), read_row_marker(cr.marker()));
read_and_visit_row(cr.cells(), s.get_column_mapping(), column_kind::regular_column, builder);
return std::move(builder).get_mutation_fragment();
},
[&] (ser::static_row_view sr) {
class static_row_builder {
const schema& _s;
mutation_fragment _mf;
public:
static_row_builder() : _mf(static_row()) { }
explicit static_row_builder(const schema& s) : _s(s), _mf(static_row()) { }
void accept_atomic_cell(column_id id, const atomic_cell& ac) {
_mf.as_mutable_static_row().cells().append_cell(id, atomic_cell_or_collection(ac));
auto& type = *_s.static_column_at(id).type;
_mf.as_mutable_static_row().cells().append_cell(id, atomic_cell_or_collection(atomic_cell(type, ac)));
}
void accept_collection(column_id id, const collection_mutation& cm) {
_mf.as_mutable_static_row().cells().append_cell(id, atomic_cell_or_collection(cm));
auto& ctype = *static_pointer_cast<const collection_type_impl>(_s.static_column_at(id).type);
_mf.as_mutable_static_row().cells().append_cell(id, atomic_cell_or_collection(collection_mutation(ctype, cm)));
}
mutation_fragment get_mutation_fragment() && { return std::move(_mf); }
};
static_row_builder builder;
static_row_builder builder(s);
read_and_visit_row(sr.cells(), s.get_column_mapping(), column_kind::static_column, builder);
return std::move(builder).get_mutation_fragment();
},

4
mutation_reader.cc
View File

@@ -421,7 +421,7 @@ future<mutation_reader_merger::mutation_fragment_batch> mutation_reader_merger::
while (!_reader_heap.empty() && key(_fragment_heap).equal(*_schema, key(_reader_heap)));
if (_fragment_heap.size() == 1) {
_single_reader = { _fragment_heap.back().reader, mutation_fragment::kind::partition_start };
_current.emplace_back(_fragment_heap.back().fragment);
_current.emplace_back(std::move(_fragment_heap.back().fragment));
_fragment_heap.clear();
return make_ready_future<mutation_fragment_batch>(_current);
}
@@ -951,7 +951,7 @@ future<> foreign_reader::fill_buffer(db::timeout_clock::time_point timeout) {
_end_of_stream = end_of_steam;
for (const auto& mf : *buffer) {
// Need a copy since the mf is on the remote shard.
push_mutation_fragment(mf);
push_mutation_fragment(mutation_fragment(*_schema, mf));
}
});
}

12
partition_builder.hh
View File

@@ -44,12 +44,14 @@ public:
virtual void accept_static_cell(column_id id, atomic_cell_view cell) override {
row& r = _partition.static_row();
r.append_cell(id, atomic_cell_or_collection(cell));
auto& cdef = _schema.static_column_at(id);
r.append_cell(id, atomic_cell_or_collection(*cdef.type, cell));
}
virtual void accept_static_cell(column_id id, collection_mutation_view collection) override {
row& r = _partition.static_row();
r.append_cell(id, atomic_cell_or_collection(collection));
auto& ctype = *static_pointer_cast<const collection_type_impl>(_schema.static_column_at(id).type);
r.append_cell(id, atomic_cell_or_collection(collection_mutation(ctype, collection)));
}
virtual void accept_row_tombstone(const range_tombstone& rt) override {
@@ -65,11 +67,13 @@ public:
virtual void accept_row_cell(column_id id, atomic_cell_view cell) override {
row& r = _current_row->cells();
r.append_cell(id, atomic_cell_or_collection(cell));
auto& cdef = _schema.regular_column_at(id);
r.append_cell(id, atomic_cell_or_collection(*cdef.type, cell));
}
virtual void accept_row_cell(column_id id, collection_mutation_view collection) override {
row& r = _current_row->cells();
r.append_cell(id, atomic_cell_or_collection(collection));
auto& ctype = *static_pointer_cast<const collection_type_impl>(_schema.regular_column_at(id).type);
r.append_cell(id, atomic_cell_or_collection(collection_mutation(ctype, collection)));
}
};

2
partition_snapshot_reader.hh
View File

@@ -212,7 +212,7 @@ class partition_snapshot_flat_reader : public flat_mutation_reader::impl, public
if (_digest_requested) {
e.row().cells().prepare_hash(_schema, column_kind::regular_column);
}
auto result = mutation_fragment(mutation_fragment::clustering_row_tag_t(), e);
auto result = mutation_fragment(mutation_fragment::clustering_row_tag_t(), _schema, e);
while (has_more_rows() && _eq(peek_row().position(), result.as_clustering_row().position())) {
const rows_entry& e = pop_clustering_row();
if (_digest_requested) {

8
partition_snapshot_row_cursor.hh
View File

@@ -361,7 +361,7 @@ public:
if (digest_requested) {
row->row().cells().prepare_hash(_schema, column_kind::regular_column);
}
auto mf = mutation_fragment(clustering_row(*row));
auto mf = mutation_fragment(clustering_row(_schema, *row));
auto& cr = mf.as_mutable_clustering_row();
for (++it; it != _current_row.end(); ++it) {
cr.apply(_schema, *it->it);
@@ -377,7 +377,7 @@ public:
if (v.unique_owner) {
consumer(std::move(v.it->row()));
} else {
consumer(deletable_row(v.it->row()));
consumer(deletable_row(_schema, v.it->row()));
}
}
}
@@ -387,7 +387,7 @@ public:
size_t memory_usage() const {
size_t result = 0;
for (const position_in_version& v : _current_row) {
result += v.it->memory_usage();
result += v.it->memory_usage(_schema);
}
return result;
}
@@ -413,7 +413,7 @@ public:
} else {
// Copy row from older version because rows in evictable versions must
// hold values which are independently complete to be consistent on eviction.
auto e = current_allocator().construct<rows_entry>(*_current_row[0].it);
auto e = current_allocator().construct<rows_entry>(_schema, *_current_row[0].it);
e->set_continuous(latest_i != rows.end() && latest_i->continuous());
_snp.tracker()->insert(*e);
rows.insert_before(latest_i, *e);

34
partition_version.cc
View File

@@ -77,9 +77,9 @@ stop_iteration partition_version::clear_gently(cache_tracker* tracker) noexcept
return _partition.clear_gently(tracker);
}
size_t partition_version::size_in_allocator(allocation_strategy& allocator) const {
size_t partition_version::size_in_allocator(const schema& s, allocation_strategy& allocator) const {
return allocator.object_memory_size_in_allocator(this) +
partition().external_memory_usage();
partition().external_memory_usage(s);
}
namespace {
@@ -108,14 +108,14 @@ concept bool Reducer() {
// the version chain starting from v.
// |map| extracts the part from each version.
// |reduce| Combines parts from the two versions.
template <typename Result, typename Map, typename Reduce>
template <typename Result, typename Map, typename Initial, typename Reduce>
GCC6_CONCEPT(
requires Mapper<Map, mutation_partition, Result>() && Reducer<Reduce, Result>()
)
inline Result squashed(const partition_version_ref& v, Map&& map, Reduce&& reduce) {
inline Result squashed(const partition_version_ref& v, Map&& map, Initial&& initial, Reduce&& reduce) {
const partition_version* this_v = &*v;
partition_version* it = v->last();
Result r = map(it->partition());
Result r = initial(map(it->partition()));
while (it != this_v) {
it = it->prev();
reduce(r, map(it->partition()));
@@ -123,6 +123,16 @@ inline Result squashed(const partition_version_ref& v, Map&& map, Reduce&& reduc
return r;
}
template <typename Result, typename Map, typename Reduce>
GCC6_CONCEPT(
requires Mapper<Map, mutation_partition, Result>() && Reducer<Reduce, Result>()
)
inline Result squashed(const partition_version_ref& v, Map&& map, Reduce&& reduce) {
return squashed<Result>(v, map,
[] (auto&& o) -> decltype(auto) { return std::forward<decltype(o)>(o); },
reduce);
}
}
::static_row partition_snapshot::static_row(bool digest_requested) const {
@@ -133,6 +143,7 @@ inline Result squashed(const partition_version_ref& v, Map&& map, Reduce&& reduc
}
return mp.static_row();
},
[this] (const row& r) { return row(*_schema, column_kind::static_column, r); },
[this] (row& a, const row& b) { a.apply(*_schema, column_kind::static_column, b); }));
}
@@ -149,6 +160,7 @@ tombstone partition_snapshot::partition_tombstone() const {
mutation_partition partition_snapshot::squashed() const {
return ::squashed<mutation_partition>(version(),
[] (const mutation_partition& mp) -> const mutation_partition& { return mp; },
[this] (const mutation_partition& mp) { return mutation_partition(*_schema, mp); },
[this] (mutation_partition& a, const mutation_partition& b) { a.apply(*_schema, b, *_schema); });
}
@@ -222,7 +234,7 @@ partition_entry partition_entry::make_evictable(const schema& s, mutation_partit
}
partition_entry partition_entry::make_evictable(const schema& s, const mutation_partition& mp) {
return make_evictable(s, mutation_partition(mp));
return make_evictable(s, mutation_partition(s, mp));
}
partition_entry::~partition_entry() {
@@ -300,7 +312,7 @@ partition_version& partition_entry::add_version(const schema& s, cache_tracker*
void partition_entry::apply(const schema& s, const mutation_partition& mp, const schema& mp_schema)
{
apply(s, mutation_partition(mp), mp_schema);
apply(s, mutation_partition(s, mp), mp_schema);
}
void partition_entry::apply(const schema& s, mutation_partition&& mp, const schema& mp_schema)
@@ -385,7 +397,7 @@ public:
// due to the fact that all rows are continuous.
for (version& v : _current_row) {
if (!v.can_move) {
consumer(deletable_row(v.current_row->row()));
consumer(deletable_row(_schema, v.current_row->row()));
} else {
consumer(std::move(v.current_row->row()));
}
@@ -477,7 +489,7 @@ coroutine partition_entry::apply_to_incomplete(const schema& s, partition_entry&
auto current = &*src_snp->version();
while (current) {
dirty_size += allocator.object_memory_size_in_allocator(current)
+ current->partition().static_row().external_memory_usage();
+ current->partition().static_row().external_memory_usage(s, column_kind::static_column);
dst.partition().apply(current->partition().partition_tombstone());
if (static_row_continuous) {
row& static_row = dst.partition().static_row();
@@ -488,7 +500,7 @@ coroutine partition_entry::apply_to_incomplete(const schema& s, partition_entry&
static_row.apply(s, column_kind::static_column, current->partition().static_row());
}
}
dirty_size += current->partition().row_tombstones().external_memory_usage();
dirty_size += current->partition().row_tombstones().external_memory_usage(s);
range_tombstone_list& tombstones = dst.partition().row_tombstones();
// FIXME: defer while applying range tombstones
if (can_move) {
@@ -541,7 +553,7 @@ mutation_partition partition_entry::squashed(schema_ptr from, schema_ptr to)
mutation_partition mp(to);
mp.set_static_row_continuous(_version->partition().static_row_continuous());
for (auto&& v : _version->all_elements()) {
auto older = v.partition();
auto older = mutation_partition(*from, v.partition());
if (from->version() != to->version()) {
older.upgrade(*from, *to);
}

2
partition_version.hh
View File

@@ -174,7 +174,7 @@ public:
bool is_referenced_from_entry() const;
partition_version_ref& back_reference() { return *_backref; }
size_t size_in_allocator(allocation_strategy& allocator) const;
size_t size_in_allocator(const schema& s, allocation_strategy& allocator) const;
};
using partition_version_range = anchorless_list_base_hook<partition_version>::range;

6
range_tombstone.hh
View File

@@ -157,12 +157,12 @@ public:
return true;
}
size_t external_memory_usage() const {
size_t external_memory_usage(const schema&) const {
return start.external_memory_usage() + end.external_memory_usage();
}
size_t memory_usage() const {
return sizeof(range_tombstone) + external_memory_usage();
size_t memory_usage(const schema& s) const {
return sizeof(range_tombstone) + external_memory_usage(s);
}
private:
void move_assign(range_tombstone&& rt) {

4
range_tombstone_list.hh
View File

@@ -187,10 +187,10 @@ public:
friend std::ostream& operator<<(std::ostream& out, const range_tombstone_list&);
bool equal(const schema&, const range_tombstone_list&) const;
size_t external_memory_usage() const {
size_t external_memory_usage(const schema& s) const {
size_t result = 0;
for (auto& rtb : _tombstones) {
result += rtb.memory_usage();
result += rtb.memory_usage(s);
}
return result;
}

2
row_cache.hh
View File

@@ -100,7 +100,7 @@ public:
cache_entry(schema_ptr s, const dht::decorated_key& key, const mutation_partition& p)
: _schema(std::move(s))
, _key(key)
, _pe(partition_entry::make_evictable(*_schema, mutation_partition(p)))
, _pe(partition_entry::make_evictable(*_schema, mutation_partition(*_schema, p)))
{ }
cache_entry(schema_ptr s, dht::decorated_key&& key, mutation_partition&& p)

2
schema_upgrader.hh
View File

@@ -35,7 +35,7 @@ private:
const column_definition& col = _prev->column_at(kind, id);
const column_definition* new_col = _new->get_column_definition(col.name());
if (new_col) {
converting_mutation_partition_applier::append_cell(new_row, kind, *new_col, col.type, std::move(cell));
converting_mutation_partition_applier::append_cell(new_row, kind, *new_col, col, std::move(cell));
}
});
return new_row;

1
serializer.hh
View File

@@ -32,6 +32,7 @@
#include "boost/variant/variant.hpp"
#include "bytes_ostream.hh"
#include "utils/input_stream.hh"
#include "utils/fragment_range.hh"
namespace ser {
using size_type = uint32_t;

4
service/storage_proxy.cc
View File

@@ -2257,7 +2257,7 @@ private:
static primary_key get_last_reconciled_row(const schema& s, const mutation_and_live_row_count& m_a_rc, const query::read_command& cmd, uint32_t limit, bool is_reversed) {
const auto& m = m_a_rc.mut;
auto mp = m.partition();
auto mp = mutation_partition(s, m.partition());
auto&& ranges = cmd.slice.row_ranges(s, m.key());
mp.compact_for_query(s, cmd.timestamp, ranges, is_reversed, limit);
@@ -2516,7 +2516,7 @@ public:
for (const version& v : z.get<0>()) {
auto diff = v.par
? m.partition().difference(schema, v.par->mut().unfreeze(schema).partition())
: m.partition();
: mutation_partition(*schema, m.partition());
auto it = _diffs[m.token()].find(v.from);
std::experimental::optional<mutation> mdiff;
if (!diff.empty()) {

6
sstables/disk_types.hh
View File

@@ -29,6 +29,7 @@
#include <unordered_map>
#include <type_traits>
#include <deque>
#include "atomic_cell.hh"
namespace sstables {
@@ -67,6 +68,11 @@ struct disk_string_view {
bytes_view value;
};
template<typename SizeType>
struct disk_data_value_view {
atomic_cell_value_view value;
};
template <typename Size, typename Members>
struct disk_array {
static_assert(std::is_integral<Size>::value, "Length type must be convertible to integer");

30
sstables/mp_row_consumer.hh
View File

@@ -49,14 +49,16 @@ struct new_mutation {
tombstone tomb;
};
inline atomic_cell make_atomic_cell(api::timestamp_type timestamp,
inline atomic_cell make_atomic_cell(const abstract_type& type,
api::timestamp_type timestamp,
bytes_view value,
gc_clock::duration ttl,
gc_clock::time_point expiration) {
gc_clock::time_point expiration,
atomic_cell::collection_member cm) {
if (ttl != gc_clock::duration::zero()) {
return atomic_cell::make_live(timestamp, value, expiration, ttl);
return atomic_cell::make_live(type, timestamp, value, expiration, ttl, cm);
} else {
return atomic_cell::make_live(timestamp, value);
return atomic_cell::make_live(type, timestamp, value, cm);
}
}
@@ -525,20 +527,28 @@ public:
virtual proceed consume_cell(bytes_view col_name, bytes_view value, int64_t timestamp, int32_t ttl, int32_t expiration) override {
return do_consume_cell(col_name, timestamp, ttl, expiration, [&] (auto&& col) {
auto ac = make_atomic_cell(api::timestamp_type(timestamp),
value,
gc_clock::duration(ttl),
gc_clock::time_point(gc_clock::duration(expiration)));
bool is_multi_cell = col.collection_extra_data.size();
if (is_multi_cell != col.cdef->is_multi_cell()) {
return;
}
if (is_multi_cell) {
auto ctype = static_pointer_cast<const collection_type_impl>(col.cdef->type);
auto ac = make_atomic_cell(*ctype->value_comparator(),
api::timestamp_type(timestamp),
value,
gc_clock::duration(ttl),
gc_clock::time_point(gc_clock::duration(expiration)),
atomic_cell::collection_member::yes);
update_pending_collection(col.cdef, to_bytes(col.collection_extra_data), std::move(ac));
return;
}
auto ac = make_atomic_cell(*col.cdef->type,
api::timestamp_type(timestamp),
value,
gc_clock::duration(ttl),
gc_clock::time_point(gc_clock::duration(expiration)),
atomic_cell::collection_member::no);
if (col.is_static) {
_in_progress->as_mutable_static_row().set_cell(*(col.cdef), std::move(ac));
return;
@@ -961,7 +971,7 @@ public:
if (timestamp <= column_def.dropped_at()) {
return proceed::yes;
}
auto ac = make_atomic_cell(timestamp, value, ttl, local_deletion_time);
auto ac = make_atomic_cell(*column_def.type, timestamp, value, ttl, local_deletion_time, atomic_cell::collection_member::no);
_cells.push_back({*column_id, atomic_cell_or_collection(std::move(ac))});
return proceed::yes;
}

47
sstables/sstables.cc
View File

@@ -378,6 +378,15 @@ inline void write(sstable_version_types v, file_writer& out, const disk_string_v
write(v, out, len, s.value);
}
template<typename SizeType>
inline void write(sstable_version_types ver, file_writer& out, const disk_data_value_view<SizeType>& v) {
SizeType length;
check_truncate_and_assign(length, v.value.size_bytes());
write(ver, out, length);
using boost::range::for_each;
for_each(v.value, [&] (bytes_view fragment) { write(ver, out, fragment); });
}
// We cannot simply read the whole array at once, because we don't know its
// full size. We know the number of elements, but if we are talking about
// disk_strings, for instance, we have no idea how much of the stream each
@@ -1712,6 +1721,16 @@ void write_cell_value(file_writer& out, const abstract_type& type, bytes_view va
}
}
void write_cell_value(file_writer& out, const abstract_type& type, atomic_cell_value_view value) {
if (!value.empty()) {
if (!type.value_length_if_fixed()) {
write_vint(out, value.size_bytes());
}
using boost::range::for_each;
for_each(value, [&] (bytes_view fragment) { write(sstable_version_types::mc, out, fragment); });
}
}
static inline void update_cell_stats(column_stats& c_stats, api::timestamp_type timestamp) {
c_stats.update_timestamp(timestamp);
c_stats.cells_count++;
@@ -1770,19 +1789,20 @@ void sstable::write_cell(file_writer& out, atomic_cell_view cell, const column_d
column_mask mask = column_mask::counter;
write(_version, out, mask, int64_t(0), timestamp);
counter_cell_view ccv(cell);
counter_cell_view::with_linearized(cell, [&] (counter_cell_view ccv) {
write_counter_value(ccv, out, _version, [v = _version] (file_writer& out, uint32_t value) {
return write(v, out, value);
});
_c_stats.update_local_deletion_time(std::numeric_limits<int>::max());
});
} else if (cell.is_live_and_has_ttl()) {
// expiring cell
column_mask mask = column_mask::expiration;
uint32_t ttl = cell.ttl().count();
uint32_t expiration = cell.expiry().time_since_epoch().count();
disk_string_view<uint32_t> cell_value { cell.value() };
disk_data_value_view<uint32_t> cell_value { cell.value() };
_c_stats.update_local_deletion_time(expiration);
// tombstone histogram is updated with expiration time because if ttl is longer
@@ -1795,7 +1815,7 @@ void sstable::write_cell(file_writer& out, atomic_cell_view cell, const column_d
// regular cell
column_mask mask = column_mask::none;
disk_string_view<uint32_t> cell_value { cell.value() };
disk_data_value_view<uint32_t> cell_value { cell.value() };
_c_stats.update_local_deletion_time(std::numeric_limits<int>::max());
@@ -1884,8 +1904,9 @@ void sstable::write_range_tombstone(file_writer& out,
}
void sstable::write_collection(file_writer& out, const composite& clustering_key, const column_definition& cdef, collection_mutation_view collection) {
collection.data.with_linearized([&] (bytes_view collection_bv) {
auto t = static_pointer_cast<const collection_type_impl>(cdef.type);
auto mview = t->deserialize_mutation_form(collection);
auto mview = t->deserialize_mutation_form(collection_bv);
const bytes& column_name = cdef.name();
if (mview.tomb) {
write_range_tombstone(out, clustering_key, composite::eoc::start, clustering_key, composite::eoc::end, { column_name }, mview.tomb);
@@ -1894,6 +1915,7 @@ void sstable::write_collection(file_writer& out, const composite& clustering_key
index_and_write_column_name(out, clustering_key, { column_name, cp.first });
write_cell(out, cp.second, cdef);
}
});
}
// This function is about writing a clustered_row to data file according to SSTables format.
@@ -1919,7 +1941,7 @@ void sstable::write_clustered_row(file_writer& out, const schema& schema, const
return;
}
assert(column_definition.is_regular());
atomic_cell_view cell = c.as_atomic_cell();
atomic_cell_view cell = c.as_atomic_cell(column_definition);
std::vector<bytes_view> column_name = { column_definition.name() };
index_and_write_column_name(out, clustering_key, column_name);
write_cell(out, cell, column_definition);
@@ -1939,7 +1961,7 @@ void sstable::write_static_row(file_writer& out, const schema& schema, const row
const auto& column_name = column_definition.name();
auto sp = composite::static_prefix(schema);
index_and_write_column_name(out, sp, { bytes_view(column_name) });
atomic_cell_view cell = c.as_atomic_cell();
atomic_cell_view cell = c.as_atomic_cell(column_definition);
write_cell(out, cell, column_definition);
});
}
@@ -2904,10 +2926,11 @@ void sstable_writer_m::write_cell(file_writer& writer, atomic_cell_view cell, co
if (has_value) {
if (cdef.is_counter()) {
assert(!cell.is_counter_update());
counter_cell_view ccv(cell);
counter_cell_view::with_linearized(cell, [&] (counter_cell_view ccv) {
write_counter_value(ccv, writer, sstable_version_types::mc, [] (file_writer& out, uint32_t value) {
return write_vint(out, value);
});
});
} else {
write_cell_value(writer, *cdef.type, cell.value());
}
@@ -2954,7 +2977,8 @@ void sstable_writer_m::write_liveness_info(file_writer& writer, const row_marker
void sstable_writer_m::write_collection(file_writer& writer, const column_definition& cdef,
collection_mutation_view collection, const row_time_properties& properties, bool has_complex_deletion) {
auto& ctype = *static_pointer_cast<const collection_type_impl>(cdef.type);
auto mview = ctype.deserialize_mutation_form(collection);
collection.data.with_linearized([&] (bytes_view collection_bv) {
auto mview = ctype.deserialize_mutation_form(collection_bv);
if (has_complex_deletion) {
auto dt = to_deletion_time(mview.tomb);
write_delta_deletion_time(writer, dt);
@@ -2972,6 +2996,7 @@ void sstable_writer_m::write_collection(file_writer& writer, const column_defini
++_c_stats.cells_count;
write_cell(writer, cell, cdef, properties, cell_path);
}
});
}
void sstable_writer_m::write_cells(file_writer& writer, column_kind kind, const row& row_body,
@@ -2987,7 +3012,7 @@ void sstable_writer_m::write_cells(file_writer& writer, column_kind kind, const
write_collection(writer, column_definition, c.as_collection_mutation(), properties, has_complex_deletion);
return;
}
atomic_cell_view cell = c.as_atomic_cell();
atomic_cell_view cell = c.as_atomic_cell(column_definition);
++_c_stats.cells_count;
++_c_stats.column_count;
write_cell(writer, cell, column_definition, properties);
@@ -3071,12 +3096,14 @@ static bool row_has_complex_deletion(const schema& s, const row& r) {
return stop_iteration::no;
}
auto t = static_pointer_cast<const collection_type_impl>(cdef.type);
auto mview = t->deserialize_mutation_form(c.as_collection_mutation());
return c.as_collection_mutation().data.with_linearized([&] (bytes_view c_bv) {
auto mview = t->deserialize_mutation_form(c_bv);
if (mview.tomb) {
result = true;
}
return stop_iteration(static_cast<bool>(mview.tomb));
});
});
return result;
}

1
test.py
View File

@@ -110,6 +110,7 @@ boost_tests = [
'limiting_data_source_test',
'sstable_test',
'sstable_3_x_test',
'meta_test',
]
other_tests = [

6
tests/batchlog_manager_test.cc
View File

@@ -39,8 +39,8 @@
#include "message/messaging_service.hh"
static atomic_cell make_atomic_cell(bytes value) {
return atomic_cell::make_live(0, std::move(value));
static atomic_cell make_atomic_cell(data_type dt, bytes value) {
return atomic_cell::make_live(*dt, 0, std::move(value));
};
SEASTAR_TEST_CASE(test_execute_batch) {
@@ -57,7 +57,7 @@ SEASTAR_TEST_CASE(test_execute_batch) {
auto c_key = clustering_key::from_exploded(*s, {int32_type->decompose(1)});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(100)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(100)));
using namespace std::chrono_literals;

2
tests/cell_locker_test.cc
View File

@@ -133,7 +133,7 @@ SEASTAR_TEST_CASE(test_disjoint_mutations) {
});
auto m3 = mutation(s, partition_key::from_single_value(*s, to_bytes("1")));
m3.partition() = m1.partition();
m3.partition() = mutation_partition(*s, m1.partition());
auto l1 = cl.lock_cells(m1.decorated_key(), partition_cells_range(m1.partition()), no_timeout).get0();
auto l2 = cl.lock_cells(m2.decorated_key(), partition_cells_range(m2.partition()), no_timeout).get0();

119
tests/counter_test.cc
View File

@@ -59,41 +59,52 @@ std::vector<counter_id> generate_ids(unsigned count) {
SEASTAR_TEST_CASE(test_counter_cell) {
return seastar::async([] {
auto cdef = column_definition("name", counter_type, column_kind::regular_column);
auto id = generate_ids(3);
counter_cell_builder b1;
b1.add_shard(counter_shard(id[0], 5, 1));
b1.add_shard(counter_shard(id[1], -4, 1));
auto c1 = atomic_cell_or_collection(b1.build(0));
auto cv = counter_cell_view(c1.as_atomic_cell());
atomic_cell_or_collection c2;
counter_cell_view::with_linearized(c1.as_atomic_cell(cdef), [&] (counter_cell_view cv) {
BOOST_REQUIRE_EQUAL(cv.total_value(), 1);
verify_shard_order(cv);
counter_cell_builder b2;
b2.add_shard(counter_shard(*cv.get_shard(id[0])).update(2, 1));
b2.add_shard(counter_shard(id[2], 1, 1));
auto c2 = atomic_cell_or_collection(b2.build(0));
c2 = atomic_cell_or_collection(b2.build(0));
});
cv = counter_cell_view(c2.as_atomic_cell());
counter_cell_view::with_linearized(c2.as_atomic_cell(cdef), [&] (counter_cell_view cv) {
BOOST_REQUIRE_EQUAL(cv.total_value(), 8);
verify_shard_order(cv);
});
counter_cell_view::apply(c1, c2);
cv = counter_cell_view(c1.as_atomic_cell());
counter_cell_view::apply(cdef, c1, c2);
counter_cell_view::with_linearized(c1.as_atomic_cell(cdef), [&] (counter_cell_view cv) {
BOOST_REQUIRE_EQUAL(cv.total_value(), 4);
verify_shard_order(cv);
});
});
}
SEASTAR_TEST_CASE(test_apply) {
return seastar::async([] {
auto verify_apply = [] (atomic_cell_or_collection dst, atomic_cell_or_collection src, int64_t value) {
counter_cell_view::apply(dst, src);
auto cdef = column_definition("name", counter_type, column_kind::regular_column);
auto cv = counter_cell_view(dst.as_atomic_cell());
auto verify_apply = [&] (const atomic_cell_or_collection& a, const atomic_cell_or_collection& b, int64_t value) {
auto dst = a.copy(*cdef.type);
auto src = b.copy(*cdef.type);
counter_cell_view::apply(cdef, dst, src);
counter_cell_view::with_linearized(dst.as_atomic_cell(cdef), [&] (counter_cell_view cv) {
BOOST_REQUIRE_EQUAL(cv.total_value(), value);
BOOST_REQUIRE_EQUAL(cv.timestamp(), std::max(dst.as_atomic_cell().timestamp(), src.as_atomic_cell().timestamp()));
BOOST_REQUIRE_EQUAL(cv.timestamp(), std::max(dst.as_atomic_cell(cdef).timestamp(), src.as_atomic_cell(cdef).timestamp()));
});
};
auto id = generate_ids(5);
@@ -103,7 +114,9 @@ SEASTAR_TEST_CASE(test_apply) {
b1.add_shard(counter_shard(id[4], 1, 3));
auto c1 = atomic_cell_or_collection(b1.build(1));
auto c2 = counter_cell_builder::from_single_shard(2, counter_shard(id[2], 8, 3));
auto c2 = atomic_cell_or_collection(
counter_cell_builder::from_single_shard(2, counter_shard(id[2], 8, 3))
);
verify_apply(c1, c2, 12);
verify_apply(c2, c1, 12);
@@ -116,7 +129,9 @@ SEASTAR_TEST_CASE(test_apply) {
verify_apply(c1, c3, 15);
verify_apply(c3, c1, 15);
auto c4 = counter_cell_builder::from_single_shard(0, counter_shard(id[2], 8, 1));
auto c4 = atomic_cell_or_collection(
counter_cell_builder::from_single_shard(0, counter_shard(id[2], 8, 1))
);
verify_apply(c1, c4, 6);
verify_apply(c4, c1, 6);
@@ -130,7 +145,9 @@ SEASTAR_TEST_CASE(test_apply) {
verify_apply(c1, c5, 21);
verify_apply(c5, c1, 21);
auto c6 = counter_cell_builder::from_single_shard(3, counter_shard(id[2], 8, 1));
auto c6 = atomic_cell_or_collection(
counter_cell_builder::from_single_shard(3, counter_shard(id[2], 8, 1))
);
verify_apply(c1, c6, 6);
verify_apply(c6, c1, 6);
@@ -152,8 +169,8 @@ atomic_cell_view get_counter_cell(mutation& m) {
const auto& cells = mp.clustered_rows().begin()->row().cells();
BOOST_REQUIRE_EQUAL(cells.size(), 1);
stdx::optional<atomic_cell_view> acv;
cells.for_each_cell([&] (column_id, const atomic_cell_or_collection& ac_o_c) {
acv = ac_o_c.as_atomic_cell();
cells.for_each_cell([&] (column_id id, const atomic_cell_or_collection& ac_o_c) {
acv = ac_o_c.as_atomic_cell(m.schema()->regular_column_at(id));
});
BOOST_REQUIRE(bool(acv));
return *acv;
@@ -164,8 +181,8 @@ atomic_cell_view get_static_counter_cell(mutation& m) {
const auto& cells = mp.static_row();
BOOST_REQUIRE_EQUAL(cells.size(), 1);
stdx::optional<atomic_cell_view> acv;
cells.for_each_cell([&] (column_id, const atomic_cell_or_collection& ac_o_c) {
acv = ac_o_c.as_atomic_cell();
cells.for_each_cell([&] (column_id id, const atomic_cell_or_collection& ac_o_c) {
acv = ac_o_c.as_atomic_cell(m.schema()->static_column_at(id));
});
BOOST_REQUIRE(bool(acv));
return *acv;
@@ -223,15 +240,17 @@ SEASTAR_TEST_CASE(test_counter_mutations) {
m.apply(m2);
auto ac = get_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
counter_cell_view ccv { ac };
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), -102);
verify_shard_order(ccv);
});
ac = get_static_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 20);
verify_shard_order(ccv);
});
m.apply(m3);
ac = get_counter_cell(m);
@@ -251,28 +270,32 @@ SEASTAR_TEST_CASE(test_counter_mutations) {
m = mutation(s, m1.decorated_key(), m1.partition().difference(s, m2.partition()));
ac = get_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 2);
verify_shard_order(ccv);
});
ac = get_static_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 11);
verify_shard_order(ccv);
});
m = mutation(s, m1.decorated_key(), m2.partition().difference(s, m1.partition()));
ac = get_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), -105);
verify_shard_order(ccv);
});
ac = get_static_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 9);
verify_shard_order(ccv);
});
m = mutation(s, m1.decorated_key(), m1.partition().difference(s, m3.partition()));
BOOST_REQUIRE_EQUAL(m.partition().clustered_rows().calculate_size(), 0);
@@ -334,19 +357,19 @@ SEASTAR_TEST_CASE(test_counter_update_mutations) {
auto c1 = atomic_cell::make_live_counter_update(api::new_timestamp(), 5);
auto s1 = atomic_cell::make_live_counter_update(api::new_timestamp(), 4);
mutation m1(s, pk);
m1.set_clustered_cell(ck, col, c1);
m1.set_static_cell(scol, s1);
m1.set_clustered_cell(ck, col, std::move(c1));
m1.set_static_cell(scol, std::move(s1));
auto c2 = atomic_cell::make_live_counter_update(api::new_timestamp(), 9);
auto s2 = atomic_cell::make_live_counter_update(api::new_timestamp(), 8);
mutation m2(s, pk);
m2.set_clustered_cell(ck, col, c2);
m2.set_static_cell(scol, s2);
m2.set_clustered_cell(ck, col, std::move(c2));
m2.set_static_cell(scol, std::move(s2));
auto c3 = atomic_cell::make_dead(api::new_timestamp() / 2, gc_clock::now());
mutation m3(s, pk);
m3.set_clustered_cell(ck, col, c3);
m3.set_static_cell(scol, c3);
m3.set_clustered_cell(ck, col, atomic_cell(*counter_type, c3));
m3.set_static_cell(scol, std::move(c3));
auto m12 = m1;
m12.apply(m2);
@@ -384,19 +407,19 @@ SEASTAR_TEST_CASE(test_transfer_updates_to_shards) {
auto c1 = atomic_cell::make_live_counter_update(api::new_timestamp(), 5);
auto s1 = atomic_cell::make_live_counter_update(api::new_timestamp(), 4);
mutation m1(s, pk);
m1.set_clustered_cell(ck, col, c1);
m1.set_static_cell(scol, s1);
m1.set_clustered_cell(ck, col, std::move(c1));
m1.set_static_cell(scol, std::move(s1));
auto c2 = atomic_cell::make_live_counter_update(api::new_timestamp(), 9);
auto s2 = atomic_cell::make_live_counter_update(api::new_timestamp(), 8);
mutation m2(s, pk);
m2.set_clustered_cell(ck, col, c2);
m2.set_static_cell(scol, s2);
m2.set_clustered_cell(ck, col, std::move(c2));
m2.set_static_cell(scol, std::move(s2));
auto c3 = atomic_cell::make_dead(api::new_timestamp() / 2, gc_clock::now());
mutation m3(s, pk);
m3.set_clustered_cell(ck, col, c3);
m3.set_static_cell(scol, c3);
m3.set_clustered_cell(ck, col, atomic_cell(*counter_type, c3));
m3.set_static_cell(scol, std::move(c3));
auto m0 = m1;
transform_counter_updates_to_shards(m0, nullptr, 0);
@@ -408,30 +431,34 @@ SEASTAR_TEST_CASE(test_transfer_updates_to_shards) {
auto ac = get_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
auto ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 5);
verify_shard_order(ccv);
});
ac = get_static_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 4);
verify_shard_order(ccv);
});
m = m2;
transform_counter_updates_to_shards(m, &m0, 0);
ac = get_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 14);
verify_shard_order(ccv);
});
ac = get_static_counter_cell(m);
BOOST_REQUIRE(ac.is_live());
ccv = counter_cell_view(ac);
counter_cell_view::with_linearized(ac, [&] (counter_cell_view ccv) {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 12);
verify_shard_order(ccv);
});
m = m3;
transform_counter_updates_to_shards(m, &m0, 0);
@@ -452,6 +479,8 @@ SEASTAR_TEST_CASE(test_sanitize_corrupted_cells) {
std::uniform_int_distribution<int64_t> value_dist(-1024 * 1024, 1024 * 1024);
for (auto i = 0; i < 100; i++) {
auto cdef = column_definition("name", counter_type, column_kind::regular_column);
auto shard_count = shard_count_dist(gen);
auto ids = generate_ids(shard_count);
@@ -488,13 +517,14 @@ SEASTAR_TEST_CASE(test_sanitize_corrupted_cells) {
auto c2 = atomic_cell_or_collection(b2.build(0));
// Compare
auto cv1 = counter_cell_view(c1.as_atomic_cell());
auto cv2 = counter_cell_view(c2.as_atomic_cell());
counter_cell_view::with_linearized(c1.as_atomic_cell(cdef), [&] (counter_cell_view cv1) {
counter_cell_view::with_linearized(c2.as_atomic_cell(cdef), [&] (counter_cell_view cv2) {
BOOST_REQUIRE_EQUAL(cv1, cv2);
BOOST_REQUIRE_EQUAL(cv1.total_value(), cv2.total_value());
verify_shard_order(cv1);
verify_shard_order(cv2);
});
});
}
});
}
@@ -537,6 +567,8 @@ SEASTAR_TEST_CASE(test_counter_id_order_1_7_4) {
SEASTAR_TEST_CASE(test_shards_compatible_with_1_7_4) {
return seastar::async([] {
auto cdef = column_definition("name", counter_type, column_kind::regular_column);
auto ids = generate_ids(16);
counter_cell_builder ccb;
@@ -545,7 +577,7 @@ SEASTAR_TEST_CASE(test_shards_compatible_with_1_7_4) {
}
auto ac = atomic_cell_or_collection(ccb.build(0));
auto cv = counter_cell_view(ac.as_atomic_cell());
counter_cell_view::with_linearized(ac.as_atomic_cell(cdef), [&] (counter_cell_view cv) {
verify_shard_order(cv);
@@ -557,6 +589,7 @@ SEASTAR_TEST_CASE(test_shards_compatible_with_1_7_4) {
}
previous = cs.id();
}
});
});
}

7
tests/cql_test_env.cc
View File

@@ -226,14 +226,13 @@ public:
}
bytes actual;
if (!col_def->type->is_multi_cell()) {
auto c = cell->as_atomic_cell();
auto c = cell->as_atomic_cell(*col_def);
assert(c.is_live());
actual = { c.value().begin(), c.value().end() };
actual = c.value().linearize();
} else {
auto c = cell->as_collection_mutation();
auto type = dynamic_pointer_cast<const collection_type_impl>(col_def->type);
actual = type->to_value(type->deserialize_mutation_form(c),
cql_serialization_format::internal());
actual = type->to_value(c, cql_serialization_format::internal());
}
assert(col_def->type->equal(actual, exp));
});

10
tests/flat_mutation_reader_assertions.hh
View File

@@ -115,12 +115,13 @@ public:
if (!cell) {
BOOST_FAIL(sprint("Expected static row with column %s, but it is not present", columns[i].name));
}
auto cmp = compare_unsigned(columns[i].value, cell->as_atomic_cell().value());
auto& cdef = _reader.schema()->static_column_at(columns[i].id);
auto cmp = compare_unsigned(columns[i].value, cell->as_atomic_cell(cdef).value().linearize());
if (cmp != 0) {
BOOST_FAIL(sprint("Expected static row with column %s having value %s, but it has value %s",
columns[i].name,
columns[i].value,
cell->as_atomic_cell().value()));
cell->as_atomic_cell(cdef).value()));
}
}
return *this;
@@ -148,12 +149,13 @@ public:
if (!cell) {
BOOST_FAIL(sprint("Expected row with column %s, but it is not present", columns[i].name));
}
auto cmp = compare_unsigned(columns[i].value, cell->as_atomic_cell().value());
auto& cdef = _reader.schema()->regular_column_at(columns[i].id);
auto cmp = compare_unsigned(columns[i].value, cell->as_atomic_cell(cdef).value().linearize());
if (cmp != 0) {
BOOST_FAIL(sprint("Expected row with column %s having value %s, but it has value %s",
columns[i].name,
columns[i].value,
cell->as_atomic_cell().value()));
cell->as_atomic_cell(cdef).value()));
}
}
return *this;

2
tests/flat_mutation_reader_test.cc
View File

@@ -82,7 +82,7 @@ struct mock_consumer {
}
result consume_end_of_stream() {
_result._consume_end_of_stream_called = true;
return _result;
return std::move(_result);
}
};

719
tests/imr_test.cc Normal file
View File

@@ -0,0 +1,719 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#define BOOST_TEST_MODULE imr
#include <boost/test/unit_test.hpp>
#include <algorithm>
#include <random>
#include <boost/range/irange.hpp>
#include <boost/range/algorithm/copy.hpp>
#include <boost/range/algorithm/generate.hpp>
#include <seastar/util/variant_utils.hh>
#include "imr/fundamental.hh"
#include "imr/compound.hh"
#include "imr/methods.hh"
#include "random-utils.hh"
static constexpr auto random_test_iteration_count = 20;
class A;
class B;
class C;
class D;
BOOST_AUTO_TEST_SUITE(fundamental);
template<typename FillAB, typename FillBC>
struct generate_flags_type;
template<size_t... IdxAB, size_t... IdxBC>
struct generate_flags_type<std::index_sequence<IdxAB...>, std::index_sequence<IdxBC...>> {
using type = imr::flags<A, std::integral_constant<size_t, IdxAB>...,
B, std::integral_constant<ssize_t, IdxBC>..., C>;
};
BOOST_AUTO_TEST_CASE(test_flags) {
using flags_type = generate_flags_type<std::make_index_sequence<7>, std::make_index_sequence<8>>::type;
static constexpr size_t expected_size = 3;
BOOST_CHECK_EQUAL(flags_type::size_when_serialized(), expected_size);
BOOST_CHECK_EQUAL(flags_type::size_when_serialized(imr::set_flag<A>(),
imr::set_flag<B>(),
imr::set_flag<C>()), expected_size);
uint8_t buffer[expected_size];
std::fill_n(buffer, expected_size, 0xbe);
BOOST_CHECK_EQUAL(flags_type::serialize(buffer, imr::set_flag<B>()), expected_size);
auto mview = flags_type::make_view(buffer);
BOOST_CHECK(!mview.get<A>());
BOOST_CHECK(mview.get<B>());
BOOST_CHECK(!mview.get<C>());
mview.set<A>();
mview.set<B>(false);
BOOST_CHECK(mview.get<A>());
BOOST_CHECK(!mview.get<B>());
BOOST_CHECK(!mview.get<C>());
flags_type::view view = mview;
mview.set<C>();
BOOST_CHECK(view.get<A>());
BOOST_CHECK(!view.get<B>());
BOOST_CHECK(view.get<C>());
BOOST_CHECK_EQUAL(flags_type::serialized_object_size(buffer), expected_size);
int some_context;
BOOST_CHECK_EQUAL(flags_type::serialized_object_size(buffer, some_context), expected_size);
std::fill_n(buffer, expected_size, 0xff);
BOOST_CHECK_EQUAL(flags_type::serialize(buffer), expected_size);
BOOST_CHECK(!mview.get<A>());
BOOST_CHECK(!mview.get<B>());
BOOST_CHECK(!mview.get<C>());
}
struct test_pod_type {
int32_t x;
uint64_t y;
friend bool operator==(const test_pod_type& a, const test_pod_type& b) {
return a.x == b.x && a.y == b.y;
}
friend std::ostream& operator<<(std::ostream& os, const test_pod_type& obj) {
return os << "test_pod_type { x: " << obj.x << ", y: " << obj.y << " }";
}
};
BOOST_AUTO_TEST_CASE(test_pod) {
auto generate_object = [] {
std::uniform_int_distribution<decltype(test_pod_type::x)> dist_x;
std::uniform_int_distribution<decltype(test_pod_type::y)> dist_y;
return test_pod_type { dist_x(tests::random::gen), dist_y(tests::random::gen) };
};
using pod_type = imr::pod<test_pod_type>;
uint8_t buffer[pod_type::size];
for (auto i = 0; i < random_test_iteration_count; i++) {
auto obj = generate_object();
BOOST_CHECK_EQUAL(pod_type::size_when_serialized(obj), pod_type::size);
BOOST_CHECK_EQUAL(pod_type::serialize(buffer, obj), pod_type::size);
BOOST_CHECK_EQUAL(pod_type::serialized_object_size(buffer), pod_type::size);
int some_context;
BOOST_CHECK_EQUAL(pod_type::serialized_object_size(buffer, some_context), pod_type::size);
auto mview = pod_type::make_view(buffer);
pod_type::view view = mview;
BOOST_CHECK_EQUAL(mview.load(), obj);
BOOST_CHECK_EQUAL(view.load(), obj);
auto obj2 = generate_object();
mview.store(obj2);
BOOST_CHECK_EQUAL(mview.load(), obj2);
BOOST_CHECK_EQUAL(view.load(), obj2);
}
}
class test_buffer_context {
size_t _size;
public:
explicit test_buffer_context(size_t sz) : _size(sz) { }
template<typename Tag>
size_t size_of() const noexcept;
};
template<>
size_t test_buffer_context::size_of<A>() const noexcept {
return _size;
}
BOOST_AUTO_TEST_CASE(test_buffer) {
using buffer_type = imr::buffer<A>;
auto test = [] (auto serialize) {
auto data = tests::random::get_bytes();
auto size = data.size();
auto buffer = std::make_unique<uint8_t[]>(size);
serialize(buffer.get(), size, data);
const auto ctx = test_buffer_context(size);
BOOST_CHECK_EQUAL(buffer_type::serialized_object_size(buffer.get(), ctx), size);
BOOST_CHECK(boost::range::equal(buffer_type::make_view(buffer.get(), ctx), data));
BOOST_CHECK(boost::range::equal(buffer_type::make_view(const_cast<const uint8_t*>(buffer.get()), ctx), data));
BOOST_CHECK_EQUAL(buffer_type::make_view(buffer.get(), ctx).size(), size);
};
for (auto i = 0; i < random_test_iteration_count; i++) {
test([] (uint8_t* out, size_t size, const bytes& data) {
BOOST_CHECK_EQUAL(buffer_type::size_when_serialized(data), size);
BOOST_CHECK_EQUAL(buffer_type::serialize(out, data), size);
});
test([] (uint8_t* out, size_t size, const bytes& data) {
auto serializer = [&data] (uint8_t* out) noexcept {
boost::range::copy(data, out);
};
BOOST_CHECK_EQUAL(buffer_type::size_when_serialized(size, serializer), size);
BOOST_CHECK_EQUAL(buffer_type::serialize(out, size, serializer), size);
});
}
}
BOOST_AUTO_TEST_SUITE_END();
BOOST_AUTO_TEST_SUITE(compound);
struct test_optional_context {
template<typename Tag>
bool is_present() const noexcept;
template<typename Tag, typename... Args>
decltype(auto) context_for(Args&&...) const noexcept { return *this; }
};
template<>
bool test_optional_context::is_present<A>() const noexcept {
return true;
}
template<>
bool test_optional_context::is_present<B>() const noexcept {
return false;
}
BOOST_AUTO_TEST_CASE(test_optional) {
using optional_type1 = imr::optional<A, imr::pod<uint32_t>>;
using optional_type2 = imr::optional<B, imr::pod<uint32_t>>;
for (auto i = 0; i < random_test_iteration_count; i++) {
auto value = tests::random::get_int<uint32_t>();
auto expected_size = imr::pod<uint32_t>::size_when_serialized(value);
auto buffer = std::make_unique<uint8_t[]>(expected_size);
BOOST_CHECK_EQUAL(optional_type1::size_when_serialized(value), expected_size);
BOOST_CHECK_EQUAL(optional_type1::serialize(buffer.get(), value), expected_size);
BOOST_CHECK_EQUAL(optional_type1::serialized_object_size(buffer.get(), test_optional_context()), expected_size);
BOOST_CHECK_EQUAL(optional_type2::serialized_object_size(buffer.get(), test_optional_context()), 0);
auto view = optional_type1::make_view(buffer.get());
BOOST_CHECK_EQUAL(view.get().load(), value);
}
}
static constexpr auto data_size = 128;
using variant_type = imr::variant<A,
imr::member<B, imr::pod<uint64_t>>,
imr::member<C, imr::buffer<C>>,
imr::member<D, imr::pod<int64_t>>>;
struct test_variant_context {
unsigned _alternative_idx;
public:
template<typename Tag>
size_t size_of() const noexcept;
template<typename Tag>
auto active_alternative_of() const noexcept;
template<typename Tag, typename... Args>
decltype(auto) context_for(Args&&...) const noexcept { return *this; }
};
template<>
size_t test_variant_context::size_of<C>() const noexcept {
return data_size;
}
template<>
auto test_variant_context::active_alternative_of<A>() const noexcept {
switch (_alternative_idx) {
case 0:
return variant_type::index_for<B>();
case 1:
return variant_type::index_for<C>();
case 2:
return variant_type::index_for<D>();
default:
BOOST_FAIL("should not reach");
abort();
}
}
BOOST_AUTO_TEST_CASE(test_variant) {
for (auto i = 0; i < random_test_iteration_count; i++) {
unsigned alternative_idx = tests::random::get_int<unsigned>(2);
uint64_t uinteger = tests::random::get_int<uint64_t>();
int64_t integer = tests::random::get_int<int64_t>();
bytes data = tests::random::get_bytes(data_size);
const size_t expected_size = alternative_idx == 0
? imr::pod<uint64_t>::size_when_serialized(uinteger)
: (alternative_idx == 1 ? data_size : sizeof(int64_t));
auto buffer = std::make_unique<uint8_t[]>(expected_size);
if (!alternative_idx) {
BOOST_CHECK_EQUAL(variant_type::size_when_serialized<B>(uinteger), expected_size);
BOOST_CHECK_EQUAL(variant_type::serialize<B>(buffer.get(), uinteger), expected_size);
} else if (alternative_idx == 1) {
BOOST_CHECK_EQUAL(variant_type::size_when_serialized<C>(data), expected_size);
BOOST_CHECK_EQUAL(variant_type::serialize<C>(buffer.get(), data), expected_size);
} else {
BOOST_CHECK_EQUAL(variant_type::size_when_serialized<D>(integer), expected_size);
BOOST_CHECK_EQUAL(variant_type::serialize<D>(buffer.get(), integer), expected_size);
}
auto ctx = test_variant_context { alternative_idx };
BOOST_CHECK_EQUAL(variant_type::serialized_object_size(buffer.get(), ctx), expected_size);
auto view = variant_type::make_view(buffer.get(), ctx);
bool visitor_was_called = false;
view.visit(make_visitor(
[&] (imr::pod<uint64_t>::view val) {
visitor_was_called = true;
if (alternative_idx == 0) {
BOOST_CHECK_EQUAL(val.load(), uinteger);
} else {
BOOST_FAIL("wrong variant alternative (B)");
}
},
[&] (imr::buffer<C>::view buf) {
visitor_was_called = true;
if (alternative_idx == 1) {
BOOST_CHECK(boost::equal(data, buf));
} else {
BOOST_FAIL("wrong variant alternative (C)");
}
},
[&] (imr::pod<int64_t>::view val) {
visitor_was_called = true;
if (alternative_idx == 2) {
BOOST_CHECK_EQUAL(val.load(), integer);
} else {
BOOST_FAIL("wrong variant alternative (D)");
}
}
), ctx);
BOOST_CHECK(visitor_was_called);
}
}
BOOST_AUTO_TEST_CASE(test_structure_with_fixed) {
using S = imr::structure<imr::member<A, imr::pod<uint8_t>>,
imr::member<B, imr::pod<int64_t>>,
imr::member<C, imr::pod<uint32_t>>>;
static constexpr auto expected_size = sizeof(uint8_t) + sizeof(uint64_t)
+ sizeof(uint32_t);
for (auto i = 0; i < random_test_iteration_count; i++) {
auto a = tests::random::get_int<uint8_t>();
auto b = tests::random::get_int<uint64_t>();
auto c = tests::random::get_int<uint32_t>();
auto writer = [&] (auto&& serializer) noexcept {
return serializer
.serialize(a)
.serialize(b)
.serialize(c)
.done();
};
uint8_t buffer[expected_size];
BOOST_CHECK_EQUAL(S::size_when_serialized(writer), expected_size);
BOOST_CHECK_EQUAL(S::serialize(buffer, writer), expected_size);
BOOST_CHECK_EQUAL(S::serialized_object_size(buffer), expected_size);
auto mview = S::make_view(buffer);
BOOST_CHECK_EQUAL(mview.get<A>().load(), a);
BOOST_CHECK_EQUAL(mview.get<B>().load(), b);
BOOST_CHECK_EQUAL(mview.get<C>().load(), c);
auto view = S::make_view(const_cast<const uint8_t*>(buffer));
BOOST_CHECK_EQUAL(view.get<A>().load(), a);
BOOST_CHECK_EQUAL(view.get<B>().load(), b);
BOOST_CHECK_EQUAL(view.get<C>().load(), c);
a = tests::random::get_int<uint8_t>();
b = tests::random::get_int<uint64_t>();
c = tests::random::get_int<uint32_t>();
mview.get<A>().store(a);
mview.get<B>().store(b);
mview.get<C>().store(c);
BOOST_CHECK_EQUAL(view.get<A>().load(), a);
BOOST_CHECK_EQUAL(view.get<B>().load(), b);
BOOST_CHECK_EQUAL(view.get<C>().load(), c);
}
}
class test_structure_context {
bool _b_is_present;
size_t _c_size_of;
public:
test_structure_context(bool b_is_present, size_t c_size_of) noexcept
: _b_is_present(b_is_present), _c_size_of(c_size_of) { }
template<typename Tag>
bool is_present() const noexcept;
template<typename Tag>
size_t size_of() const noexcept;
template<typename Tag, typename... Args>
decltype(auto) context_for(Args&&...) const noexcept { return *this; }
};
template<>
bool test_structure_context::is_present<B>() const noexcept {
return _b_is_present;
}
template<>
size_t test_structure_context::size_of<C>() const noexcept {
return _c_size_of;
}
BOOST_AUTO_TEST_CASE(test_structure_with_context) {
using S = imr::structure<imr::member<A, imr::flags<B, C>>,
imr::optional_member<B, imr::pod<uint16_t>>,
imr::member<C, imr::buffer<C>>>;
for (auto i = 0; i < random_test_iteration_count; i++) {
auto b_value = tests::random::get_int<uint16_t>();
auto c_data = tests::random::get_bytes();
const auto expected_size = 1 + imr::pod<uint16_t>::size_when_serialized(b_value)
+ c_data.size();
auto writer = [&] (auto&& serializer) noexcept {
return serializer
.serialize(imr::set_flag<B>())
.serialize(b_value)
.serialize(c_data)
.done();
};
BOOST_CHECK_EQUAL(S::size_when_serialized(writer), expected_size);
auto buffer = std::make_unique<uint8_t[]>(expected_size);
BOOST_CHECK_EQUAL(S::serialize(buffer.get(), writer), expected_size);
auto ctx = test_structure_context(true, c_data.size());
BOOST_CHECK_EQUAL(S::serialized_object_size(buffer.get(), ctx), expected_size);
auto mview = S::make_view(buffer.get(), ctx);
BOOST_CHECK(mview.get<A>().get<B>());
BOOST_CHECK(!mview.get<A>().get<C>());
BOOST_CHECK_EQUAL(mview.get<B>().get().load(), b_value);
BOOST_CHECK(boost::range::equal(mview.get<C>(ctx), c_data));
auto view = S::view(mview);
BOOST_CHECK(view.get<A>().get<B>());
BOOST_CHECK(!view.get<A>().get<C>());
BOOST_CHECK_EQUAL(view.get<B>().get().load(), b_value);
BOOST_CHECK(boost::range::equal(view.get<C>(ctx), c_data));
}
}
BOOST_AUTO_TEST_CASE(test_structure_get_element_without_view) {
using S = imr::structure<imr::member<A, imr::flags<B, C>>,
imr::member<B, imr::pod<uint64_t>>,
imr::optional_member<C, imr::pod<uint16_t>>>;
auto uinteger = tests::random::get_int<uint64_t>();
static constexpr auto expected_size = 1 + sizeof(uint64_t);
auto writer = [&] (auto&& serializer) noexcept {
return serializer
.serialize(imr::set_flag<B>())
.serialize(uinteger)
.skip()
.done();
};
BOOST_CHECK_EQUAL(S::size_when_serialized(writer), expected_size);
uint8_t buffer[expected_size];
BOOST_CHECK_EQUAL(S::serialize(buffer, writer), expected_size);
auto fview = S::get_member<A>(buffer);
BOOST_CHECK(fview.get<B>());
BOOST_CHECK(!fview.get<C>());
auto uview = S::get_member<B>(buffer);
BOOST_CHECK_EQUAL(uview.load(), uinteger);
// FIXME test offset
}
BOOST_AUTO_TEST_CASE(test_nested_structure) {
using S1 = imr::structure<imr::optional_member<B, imr::pod<uint16_t>>,
imr::member<C, imr::buffer<C>>,
imr::member<A, imr::pod<uint8_t>>>;
using S = imr::structure<imr::member<A, imr::pod<uint16_t>>,
imr::member<B, S1>,
imr::member<C, imr::pod<uint32_t>>>;
for (auto i = 0; i < random_test_iteration_count; i++) {
auto b1_value = tests::random::get_int<uint16_t>();
auto c1_data = tests::random::get_bytes();
auto a1_value = tests::random::get_int<uint8_t>();
const auto expected_size1 = imr::pod<uint16_t>::size_when_serialized(b1_value)
+ c1_data.size() + sizeof(uint8_t);
auto a_value = tests::random::get_int<uint16_t>();
auto c_value = tests::random::get_int<uint32_t>();
const auto expected_size = sizeof(uint16_t) + expected_size1 + sizeof(uint32_t);
auto writer1 = [&] (auto&& serializer) noexcept {
return serializer
.serialize(b1_value)
.serialize(c1_data)
.serialize(a1_value)
.done();
};
auto writer = [&] (auto&& serializer) noexcept {
return serializer
.serialize(a_value)
.serialize(writer1)
.serialize(c_value)
.done();
};
BOOST_CHECK_EQUAL(S::size_when_serialized(writer), expected_size);
auto buffer = std::make_unique<uint8_t[]>(expected_size);
BOOST_CHECK_EQUAL(S::serialize(buffer.get(), writer), expected_size);
auto ctx = test_structure_context(true, c1_data.size());
BOOST_CHECK_EQUAL(S::serialized_object_size(buffer.get(), ctx), expected_size);
auto view = S::make_view(buffer.get(), ctx);
BOOST_CHECK_EQUAL(view.get<A>().load(), a_value);
BOOST_CHECK_EQUAL(view.get<B>(ctx).get<B>().get().load(), b1_value);
BOOST_CHECK(boost::range::equal(view.get<B>(ctx).get<C>(ctx), c1_data));
BOOST_CHECK_EQUAL(view.get<C>(ctx).load(), c_value);
}
}
BOOST_AUTO_TEST_SUITE_END();
struct object_with_destructor {
static size_t destruction_count;
static uint64_t last_destroyed_one;
static void reset() {
destruction_count = 0;
last_destroyed_one = 0;
}
uint64_t value;
};
size_t object_with_destructor::destruction_count = 0;
uint64_t object_with_destructor::last_destroyed_one = 0;
struct object_without_destructor {
uint64_t value;
};
namespace imr {
namespace methods {
template<>
struct destructor<pod<object_with_destructor>> {
template<typename... Args>
static void run(uint8_t* ptr, Args&&...) noexcept {
object_with_destructor::destruction_count++;
auto view = imr::pod<object_with_destructor>::make_view(ptr);
object_with_destructor::last_destroyed_one = view.load().value;
}
};
}
}
BOOST_AUTO_TEST_SUITE(methods);
BOOST_AUTO_TEST_CASE(test_simple_destructor) {
object_with_destructor::reset();
using O1 = imr::pod<object_with_destructor>;
using O2 = imr::pod<object_without_destructor>;
BOOST_CHECK(!imr::methods::is_trivially_destructible<O1>::value);
BOOST_CHECK(imr::methods::is_trivially_destructible<O2>::value);
static constexpr auto expected_size = sizeof(object_with_destructor);
uint8_t buffer[expected_size];
auto value = tests::random::get_int<uint64_t>();
BOOST_CHECK_EQUAL(O1::serialize(buffer, object_with_destructor { value }), expected_size);
imr::methods::destroy<O1>(buffer);
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 1);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, value);
imr::methods::destroy<O2>(buffer);
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 1);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, value);
}
BOOST_AUTO_TEST_CASE(test_structure_destructor) {
object_with_destructor::reset();
using S = imr::structure<imr::member<A, imr::pod<object_with_destructor>>,
imr::member<B, imr::pod<object_without_destructor>>,
imr::member<C, imr::pod<object_with_destructor>>>;
using S1 = imr::structure<imr::member<A, imr::pod<object_without_destructor>>,
imr::member<B, imr::pod<object_without_destructor>>,
imr::member<C, imr::pod<object_without_destructor>>>;
BOOST_CHECK(!imr::methods::is_trivially_destructible<S>::value);
BOOST_CHECK(imr::methods::is_trivially_destructible<S1>::value);
static constexpr auto expected_size = sizeof(object_with_destructor) * 3;
uint8_t buffer[expected_size];
auto a = tests::random::get_int<uint64_t>();
auto b = tests::random::get_int<uint64_t>();
auto c = tests::random::get_int<uint64_t>();
BOOST_CHECK_EQUAL(S::serialize(buffer, [&] (auto serializer) noexcept {
return serializer
.serialize(object_with_destructor { a })
.serialize(object_without_destructor { b })
.serialize(object_with_destructor { c })
.done();
}), expected_size);
imr::methods::destroy<S>(buffer);
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 2);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, c);
imr::methods::destroy<S1>(buffer);
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 2);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, c);
}
BOOST_AUTO_TEST_CASE(test_optional_destructor) {
object_with_destructor::reset();
using O1 = imr::optional<A, imr::pod<object_with_destructor>>;
using O2 = imr::optional<B, imr::pod<object_with_destructor>>;
using O3 = imr::optional<A, imr::pod<object_without_destructor>>;
BOOST_CHECK(!imr::methods::is_trivially_destructible<O1>::value);
BOOST_CHECK(!imr::methods::is_trivially_destructible<O2>::value);
BOOST_CHECK(imr::methods::is_trivially_destructible<O3>::value);
static constexpr auto expected_size = sizeof(object_with_destructor);
uint8_t buffer[expected_size];
auto value = tests::random::get_int<uint64_t>();
BOOST_CHECK_EQUAL(O1::serialize(buffer, object_with_destructor { value }), expected_size);
imr::methods::destroy<O2>(buffer, compound::test_optional_context());
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 0);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, 0);
imr::methods::destroy<O1>(buffer, compound::test_optional_context());
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 1);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, value);
imr::methods::destroy<O3>(buffer, compound::test_optional_context());
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 1);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, value);
}
using V = imr::variant<A,
imr::member<B, imr::pod<object_with_destructor>>,
imr::member<C, imr::pod<object_without_destructor>>>;
struct test_variant_context {
bool _alternative_b;
public:
template<typename Tag>
auto active_alternative_of() const noexcept;
template<typename Tag>
decltype(auto) context_for(...) const noexcept { return *this; }
};
template<>
auto test_variant_context::active_alternative_of<A>() const noexcept {
if (_alternative_b) {
return V::index_for<B>();
} else {
return V::index_for<C>();
}
}
BOOST_AUTO_TEST_CASE(test_variant_destructor) {
object_with_destructor::reset();
using V1 = imr::variant<A, imr::member<B, imr::pod<object_without_destructor>>>;
BOOST_CHECK(!imr::methods::is_trivially_destructible<V>::value);
BOOST_CHECK(imr::methods::is_trivially_destructible<V1>::value);
static constexpr auto expected_size = sizeof(object_with_destructor);
uint8_t buffer[expected_size];
auto value = tests::random::get_int<uint64_t>();
BOOST_CHECK_EQUAL(V::serialize<B>(buffer, object_with_destructor { value }), expected_size);
imr::methods::destroy<V>(buffer, test_variant_context { false });
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 0);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, 0);
imr::methods::destroy<V>(buffer, test_variant_context { true });
BOOST_CHECK_EQUAL(object_with_destructor::destruction_count, 1);
BOOST_CHECK_EQUAL(object_with_destructor::last_destroyed_one, value);
}
BOOST_AUTO_TEST_SUITE_END();

4
tests/memory_footprint.cc
View File

@@ -102,7 +102,7 @@ static mutation make_cs_mutation() {
mutation m(s, partition_key::from_single_value(*s, bytes_type->from_string("4b343050393536353531")));
for (auto&& col : s->regular_columns()) {
m.set_clustered_cell(clustering_key::make_empty(), col,
atomic_cell::make_live(1, bytes_type->from_string("8f75da6b3dcec90c8a404fb9a5f6b0621e62d39c69ba5758e5f41b78311fbb26cc7a")));
atomic_cell::make_live(*bytes_type, 1, bytes_type->from_string("8f75da6b3dcec90c8a404fb9a5f6b0621e62d39c69ba5758e5f41b78311fbb26cc7a")));
}
return m;
}
@@ -150,7 +150,7 @@ static mutation make_mutation(mutation_settings settings) {
auto ck = clustering_key::from_single_value(*s, bytes_type->decompose(data_value(random_bytes(settings.clustering_key_size))));
for (auto&& col : s->regular_columns()) {
m.set_clustered_cell(ck, col,
atomic_cell::make_live(1,
atomic_cell::make_live(*bytes_type, 1,
bytes_type->decompose(data_value(random_bytes(settings.data_size)))));
}
}

12
tests/memtable_test.cc
View File

@@ -512,7 +512,7 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
{
auto rd = mt->make_flat_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(!row.cells().cell_hash_for(0));
}
@@ -521,14 +521,14 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
slice.options.set<query::partition_slice::option::with_digest>();
auto rd = mt->make_flat_reader(s, query::full_partition_range, slice);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
{
auto rd = mt->make_flat_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
@@ -538,7 +538,7 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
{
auto rd = mt->make_flat_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(!row.cells().cell_hash_for(0));
}
@@ -547,14 +547,14 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
slice.options.set<query::partition_slice::option::with_digest>();
auto rd = mt->make_flat_reader(s, query::full_partition_range, slice);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
{
auto rd = mt->make_flat_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
});

220
tests/meta_test.cc Normal file
View File

@@ -0,0 +1,220 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#define BOOST_TEST_MODULE meta
#include <boost/test/unit_test.hpp>
#include <seastar/util/log.hh>
#include "utils/meta.hh"
namespace internal {
template<typename T>
struct check_constexpr {
template<T N>
struct check {
enum : T {
value = N,
};
};
};
template<typename T>
struct first_argument { };
template<typename R, typename T, typename... Ts>
struct first_argument<R(T, Ts...)> {
using type = T;
};
}
#define INTERNAL_STATIC_CHECK_EQUAL(expected, actual, actual_str) \
BOOST_CHECK_MESSAGE(internal::check_constexpr<std::decay_t<decltype(actual)>>::check<(actual)>::value == (expected), \
actual_str " expected to be equal " #expected " [actual: " << (actual) << ", expected: " << (expected) << "]")
#define INTERNAL_STATIC_CHECK_SAME(expr, expected, actual, actual_str) \
BOOST_CHECK_MESSAGE(expr, actual_str " expected to be the same as " #expected \
" [actual: " << seastar::pretty_type_name(typeid(typename internal::first_argument<void(actual)>::type)) << ", expected: " \
<< seastar::pretty_type_name(typeid(internal::first_argument<void(expected)>::type)) << "]")
#define STATIC_CHECK_EQUAL(expected, ...) \
INTERNAL_STATIC_CHECK_EQUAL(expected, (__VA_ARGS__), #__VA_ARGS__)
#define STATIC_CHECK_SAME(expected, ...) \
INTERNAL_STATIC_CHECK_SAME((std::is_same<__VA_ARGS__, typename internal::first_argument<void(expected)>::type>::value), expected, (__VA_ARGS__), #__VA_ARGS__)
class A { };
class B { };
class C { };
class D { };
BOOST_AUTO_TEST_CASE(find) {
STATIC_CHECK_EQUAL(0, meta::find<A, A, B, C, D>);
STATIC_CHECK_EQUAL(1, meta::find<B, A, B, C, D>);
STATIC_CHECK_EQUAL(2, meta::find<C, A, B, C, D>);
STATIC_CHECK_EQUAL(3, meta::find<D, A, B, C, D>);
STATIC_CHECK_EQUAL(0, meta::find<A, A>);
STATIC_CHECK_EQUAL(0, meta::find<A, A, A>);
STATIC_CHECK_EQUAL(1, meta::find<A, B, A, A>);
STATIC_CHECK_EQUAL(0, meta::find<A, meta::list<A, B, C, D>>);
STATIC_CHECK_EQUAL(1, meta::find<B, meta::list<A, B, C, D>>);
STATIC_CHECK_EQUAL(2, meta::find<C, meta::list<A, B, C, D>>);
STATIC_CHECK_EQUAL(3, meta::find<D, meta::list<A, B, C, D>>);
STATIC_CHECK_EQUAL(0, meta::find<A, meta::list<A>>);
STATIC_CHECK_EQUAL(0, meta::find<A, meta::list<A, A>>);
STATIC_CHECK_EQUAL(1, meta::find<A, meta::list<B, A, A>>);
STATIC_CHECK_EQUAL(1, meta::find<meta::list<A>, meta::list<B>, meta::list<A>>);
STATIC_CHECK_EQUAL(1, meta::find<meta::list<A>, meta::list<meta::list<B>, meta::list<A>>>);
}
BOOST_AUTO_TEST_CASE(get) {
STATIC_CHECK_SAME(A, meta::get<0, A, B, C, D>);
STATIC_CHECK_SAME(B, meta::get<1, A, B, C, D>);
STATIC_CHECK_SAME(C, meta::get<2, A, B, C, D>);
STATIC_CHECK_SAME(D, meta::get<3, A, B, C, D>);
STATIC_CHECK_SAME(A, meta::get<0, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME(B, meta::get<1, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME(C, meta::get<2, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME(D, meta::get<3, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME(A, meta::get<0, meta::list<A>>);
STATIC_CHECK_SAME(meta::list<A>, meta::get<0, meta::list<meta::list<A>>>);
}
BOOST_AUTO_TEST_CASE(take) {
STATIC_CHECK_SAME(meta::list<A>, meta::take<1, A, B, C, D>);
STATIC_CHECK_SAME((meta::list<A, B>), meta::take<2, A, B, C, D>);
STATIC_CHECK_SAME((meta::list<A, B, C>), meta::take<3, A, B, C, D>);
STATIC_CHECK_SAME((meta::list<A, B, C, D>), meta::take<4, A, B, C, D>);
STATIC_CHECK_SAME(meta::list<A>, meta::take<1, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME((meta::list<A, B>), meta::take<2, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME((meta::list<A, B, C>), meta::take<3, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME((meta::list<A, B, C, D>), meta::take<4, meta::list<A, B, C, D>>);
STATIC_CHECK_SAME(meta::list<A>, meta::take<1, meta::list<A>>);
STATIC_CHECK_SAME(meta::list<meta::list<A>>, meta::take<1, meta::list<meta::list<A>>>);
STATIC_CHECK_SAME((meta::list<meta::list<A, B>>), meta::take<1, meta::list<meta::list<A, B>>>);
}
BOOST_AUTO_TEST_CASE(size) {
STATIC_CHECK_EQUAL(0, meta::size<>);
STATIC_CHECK_EQUAL(1, meta::size<A>);
STATIC_CHECK_EQUAL(2, meta::size<A, B>);
STATIC_CHECK_EQUAL(3, meta::size<A, B, C>);
STATIC_CHECK_EQUAL(4, meta::size<A, B, C, D>);
STATIC_CHECK_EQUAL(0, meta::size<meta::list<>>);
STATIC_CHECK_EQUAL(1, meta::size<meta::list<A>>);
STATIC_CHECK_EQUAL(2, meta::size<meta::list<A, B>>);
STATIC_CHECK_EQUAL(3, meta::size<meta::list<A, B, C>>);
STATIC_CHECK_EQUAL(4, meta::size<meta::list<A, B, C, D>>);
STATIC_CHECK_EQUAL(1, meta::size<meta::list<meta::list<A, B>>>);
STATIC_CHECK_EQUAL(3, meta::size<meta::list<A, B>, C, D>);
STATIC_CHECK_EQUAL(3, meta::size<meta::list<meta::list<A, B>, C, D>>);
}
class constexpr_count_all_fn {
size_t _n = 0;
public:
constexpr constexpr_count_all_fn() = default;
template<typename T>
constexpr void operator()(T) { _n++; }
constexpr size_t get() { return _n; }
};
template<typename... Ts>
constexpr size_t constexpr_count_all()
{
constexpr_count_all_fn constexpr_fn;
meta::for_each<Ts...>(constexpr_fn);
return constexpr_fn.get();
}
BOOST_AUTO_TEST_CASE(for_each) {
STATIC_CHECK_EQUAL(0, constexpr_count_all<>());
STATIC_CHECK_EQUAL(4, constexpr_count_all<A, B, C, D>());
size_t n = 0;
meta::for_each<A, B, C, D>([&] (auto&& ptr) {
using type = std::remove_pointer_t<std::decay_t<decltype(ptr)>>;
switch (n) {
case 0: STATIC_CHECK_SAME(A, type); break;
case 1: STATIC_CHECK_SAME(B, type); break;
case 2: STATIC_CHECK_SAME(C, type); break;
case 3: STATIC_CHECK_SAME(D, type); break;
default: BOOST_FAIL("should not reach"); break;
}
n++;
});
BOOST_CHECK_EQUAL(4, n);
STATIC_CHECK_EQUAL(0, constexpr_count_all<meta::list<>>());
STATIC_CHECK_EQUAL(4, constexpr_count_all<meta::list<A, B, C, D>>());
n = 0;
meta::for_each<meta::list<A, B, C, D>>([&] (auto ptr) {
using type = std::remove_pointer_t<decltype(ptr)>;
switch (n) {
case 0: STATIC_CHECK_SAME(A, type); break;
case 1: STATIC_CHECK_SAME(B, type); break;
case 2: STATIC_CHECK_SAME(C, type); break;
case 3: STATIC_CHECK_SAME(D, type); break;
default: BOOST_FAIL("should not reach"); break;
}
n++;
});
BOOST_CHECK_EQUAL(4, n);
n = 0;
meta::for_each<meta::take<2, A, B, C, D>>([&] (auto ptr) {
using type = std::remove_pointer_t<decltype(ptr)>;
switch (n) {
case 0: STATIC_CHECK_SAME(A, type); break;
case 1: STATIC_CHECK_SAME(B, type); break;
default: BOOST_FAIL("should not reach"); break;
}
n++;
});
BOOST_CHECK_EQUAL(2, n);
n = 0;
using list = meta::list<A, B, C, D>;
meta::for_each<meta::take<meta::size<list> - 1, list>>([&] (auto ptr) {
using type = std::remove_pointer_t<decltype(ptr)>;
switch (n) {
case 0: STATIC_CHECK_SAME(A, type); break;
case 1: STATIC_CHECK_SAME(B, type); break;
case 2: STATIC_CHECK_SAME(C, type); break;
default: BOOST_FAIL("should not reach"); break;
}
n++;
});
BOOST_CHECK_EQUAL(3, n);
}

6
tests/mutation_query_test.cc
View File

@@ -141,11 +141,13 @@ SEASTAR_TEST_CASE(test_cells_are_expired_according_to_query_timestamp) {
m1.set_clustered_cell(clustering_key::from_single_value(*s, bytes("A")),
*s->get_column_definition("v1"),
atomic_cell::make_live(api::timestamp_type(1), bytes("A:v1"), now + 1s, 1s));
atomic_cell::make_live(*s->get_column_definition("v1")->type,
api::timestamp_type(1), bytes("A:v1"), now + 1s, 1s));
m1.set_clustered_cell(clustering_key::from_single_value(*s, bytes("B")),
*s->get_column_definition("v1"),
atomic_cell::make_live(api::timestamp_type(1), bytes("B:v1")));
atomic_cell::make_live(*s->get_column_definition("v1")->type,
api::timestamp_type(1), bytes("B:v1")));
auto src = make_source({m1});

7
tests/mutation_source_test.cc
View File

@@ -1454,7 +1454,7 @@ public:
return random_counter_cell();
}
if (col.is_atomic()) {
return atomic_cell::make_live(timestamp_dist(_gen), _blobs[value_blob_index_dist(_gen)]);
return atomic_cell::make_live(*col.type, timestamp_dist(_gen), _blobs[value_blob_index_dist(_gen)]);
}
static thread_local std::uniform_int_distribution<int> element_dist{1, 13};
static thread_local std::uniform_int_distribution<int64_t> uuid_ts_dist{-12219292800000L, -12219292800000L + 1000};
@@ -1463,18 +1463,19 @@ public:
m.cells.reserve(num_cells);
std::unordered_set<bytes> unique_cells;
unique_cells.reserve(num_cells);
auto ctype = static_pointer_cast<const collection_type_impl>(col.type);
for (auto i = 0; i < num_cells; ++i) {
auto uuid = utils::UUID_gen::min_time_UUID(uuid_ts_dist(_gen)).serialize();
if (unique_cells.emplace(uuid).second) {
m.cells.emplace_back(
bytes(reinterpret_cast<const int8_t*>(uuid.data()), uuid.size()),
atomic_cell::make_live(timestamp_dist(_gen), _blobs[value_blob_index_dist(_gen)]));
atomic_cell::make_live(*ctype->value_comparator(), timestamp_dist(_gen), _blobs[value_blob_index_dist(_gen)]));
}
}
std::sort(m.cells.begin(), m.cells.end(), [] (auto&& c1, auto&& c2) {
return timeuuid_type->as_less_comparator()(c1.first, c2.first);
});
return static_pointer_cast<const collection_type_impl>(col.type)->serialize_mutation_form(m);
return ctype->serialize_mutation_form(m);
};
auto get_dead_cell = [&] () -> atomic_cell_or_collection{
if (col.is_atomic() || col.is_counter()) {

242
tests/mutation_test.cc
View File

@@ -63,7 +63,21 @@ static sstring some_column_family("cf");
static db::nop_large_partition_handler nop_lp_handler;
static atomic_cell make_atomic_cell(bytes value) {
return atomic_cell::make_live(0, std::move(value));
return atomic_cell::make_live(*bytes_type, 0, std::move(value));
}
static atomic_cell make_atomic_cell() {
return atomic_cell::make_live(*bytes_type, 0, bytes_view());
}
template<typename T>
static atomic_cell make_atomic_cell(data_type dt, T value) {
return atomic_cell::make_live(*dt, 0, dt->decompose(std::move(value)));
};
template<typename T>
static atomic_cell make_collection_member(data_type dt, T value) {
return atomic_cell::make_live(*dt, 0, dt->decompose(std::move(value)));
};
static mutation_partition get_partition(memtable& mt, const partition_key& key) {
@@ -100,16 +114,17 @@ SEASTAR_TEST_CASE(test_mutation_is_applied) {
auto c_key = clustering_key::from_exploded(*s, {int32_type->decompose(2)});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(3)));
auto c = make_atomic_cell(int32_type, 3);
m.set_clustered_cell(c_key, r1_col, std::move(c));
mt->apply(std::move(m));
auto p = get_partition(*mt, key);
row& r = p.clustered_row(*s, c_key).cells();
auto i = r.find_cell(r1_col.id);
BOOST_REQUIRE(i);
auto cell = i->as_atomic_cell();
auto cell = i->as_atomic_cell(r1_col);
BOOST_REQUIRE(cell.is_live());
BOOST_REQUIRE(int32_type->equal(cell.value(), int32_type->decompose(3)));
BOOST_REQUIRE(int32_type->equal(cell.value().linearize(), int32_type->decompose(3)));
});
}
@@ -172,6 +187,23 @@ SEASTAR_TEST_CASE(test_row_tombstone_updates) {
return make_ready_future<>();
}
collection_type_impl::mutation make_collection_mutation(tombstone t, bytes key, atomic_cell cell)
{
collection_type_impl::mutation m;
m.tomb = t;
m.cells.emplace_back(std::move(key), std::move(cell));
return m;
}
collection_type_impl::mutation make_collection_mutation(tombstone t, bytes key1, atomic_cell cell1, bytes key2, atomic_cell cell2)
{
collection_type_impl::mutation m;
m.tomb = t;
m.cells.emplace_back(std::move(key1), std::move(cell1));
m.cells.emplace_back(std::move(key2), std::move(cell2));
return m;
}
SEASTAR_TEST_CASE(test_map_mutations) {
return seastar::async([] {
auto my_map_type = map_type_impl::get_instance(int32_type, utf8_type, true);
@@ -180,19 +212,19 @@ SEASTAR_TEST_CASE(test_map_mutations) {
auto mt = make_lw_shared<memtable>(s);
auto key = partition_key::from_exploded(*s, {to_bytes("key1")});
auto& column = *s->get_column_definition("s1");
map_type_impl::mutation mmut1{{}, {{int32_type->decompose(101), make_atomic_cell(utf8_type->decompose(sstring("101")))}}};
auto mmut1 = make_collection_mutation({}, int32_type->decompose(101), make_collection_member(utf8_type, sstring("101")));
mutation m1(s, key);
m1.set_static_cell(column, my_map_type->serialize_mutation_form(mmut1));
mt->apply(m1);
map_type_impl::mutation mmut2{{}, {{int32_type->decompose(102), make_atomic_cell(utf8_type->decompose(sstring("102")))}}};
auto mmut2 = make_collection_mutation({}, int32_type->decompose(102), make_collection_member(utf8_type, sstring("102")));
mutation m2(s, key);
m2.set_static_cell(column, my_map_type->serialize_mutation_form(mmut2));
mt->apply(m2);
map_type_impl::mutation mmut3{{}, {{int32_type->decompose(103), make_atomic_cell(utf8_type->decompose(sstring("103")))}}};
auto mmut3 = make_collection_mutation({}, int32_type->decompose(103), make_collection_member(utf8_type, sstring("103")));
mutation m3(s, key);
m3.set_static_cell(column, my_map_type->serialize_mutation_form(mmut3));
mt->apply(m3);
map_type_impl::mutation mmut2o{{}, {{int32_type->decompose(102), make_atomic_cell(utf8_type->decompose(sstring("102 override")))}}};
auto mmut2o = make_collection_mutation({}, int32_type->decompose(102), make_collection_member(utf8_type, sstring("102 override")));
mutation m2o(s, key);
m2o.set_static_cell(column, my_map_type->serialize_mutation_form(mmut2o));
mt->apply(m2o);
@@ -202,7 +234,8 @@ SEASTAR_TEST_CASE(test_map_mutations) {
auto i = r.find_cell(column.id);
BOOST_REQUIRE(i);
auto cell = i->as_collection_mutation();
auto muts = my_map_type->deserialize_mutation_form(cell);
auto cell_b = cell.data.linearize();
auto muts = my_map_type->deserialize_mutation_form(cell_b);
BOOST_REQUIRE(muts.cells.size() == 3);
// FIXME: more strict tests
});
@@ -216,19 +249,19 @@ SEASTAR_TEST_CASE(test_set_mutations) {
auto mt = make_lw_shared<memtable>(s);
auto key = partition_key::from_exploded(*s, {to_bytes("key1")});
auto& column = *s->get_column_definition("s1");
map_type_impl::mutation mmut1{{}, {{int32_type->decompose(101), make_atomic_cell({})}}};
auto mmut1 = make_collection_mutation({}, int32_type->decompose(101), make_atomic_cell());
mutation m1(s, key);
m1.set_static_cell(column, my_set_type->serialize_mutation_form(mmut1));
mt->apply(m1);
map_type_impl::mutation mmut2{{}, {{int32_type->decompose(102), make_atomic_cell({})}}};
auto mmut2 = make_collection_mutation({}, int32_type->decompose(102), make_atomic_cell());
mutation m2(s, key);
m2.set_static_cell(column, my_set_type->serialize_mutation_form(mmut2));
mt->apply(m2);
map_type_impl::mutation mmut3{{}, {{int32_type->decompose(103), make_atomic_cell({})}}};
auto mmut3 = make_collection_mutation({}, int32_type->decompose(103), make_atomic_cell());
mutation m3(s, key);
m3.set_static_cell(column, my_set_type->serialize_mutation_form(mmut3));
mt->apply(m3);
map_type_impl::mutation mmut2o{{}, {{int32_type->decompose(102), make_atomic_cell({})}}};
auto mmut2o = make_collection_mutation({}, int32_type->decompose(102), make_atomic_cell());
mutation m2o(s, key);
m2o.set_static_cell(column, my_set_type->serialize_mutation_form(mmut2o));
mt->apply(m2o);
@@ -238,7 +271,8 @@ SEASTAR_TEST_CASE(test_set_mutations) {
auto i = r.find_cell(column.id);
BOOST_REQUIRE(i);
auto cell = i->as_collection_mutation();
auto muts = my_set_type->deserialize_mutation_form(cell);
auto cell_b = cell.data.linearize();
auto muts = my_set_type->deserialize_mutation_form(cell_b);
BOOST_REQUIRE(muts.cells.size() == 3);
// FIXME: more strict tests
});
@@ -253,19 +287,19 @@ SEASTAR_TEST_CASE(test_list_mutations) {
auto key = partition_key::from_exploded(*s, {to_bytes("key1")});
auto& column = *s->get_column_definition("s1");
auto make_key = [] { return timeuuid_type->decompose(utils::UUID_gen::get_time_UUID()); };
collection_type_impl::mutation mmut1{{}, {{make_key(), make_atomic_cell(int32_type->decompose(101))}}};
auto mmut1 = make_collection_mutation({}, make_key(), make_collection_member(int32_type, 101));
mutation m1(s, key);
m1.set_static_cell(column, my_list_type->serialize_mutation_form(mmut1));
mt->apply(m1);
collection_type_impl::mutation mmut2{{}, {{make_key(), make_atomic_cell(int32_type->decompose(102))}}};
auto mmut2 = make_collection_mutation({}, make_key(), make_collection_member(int32_type, 102));
mutation m2(s, key);
m2.set_static_cell(column, my_list_type->serialize_mutation_form(mmut2));
mt->apply(m2);
collection_type_impl::mutation mmut3{{}, {{make_key(), make_atomic_cell(int32_type->decompose(103))}}};
auto mmut3 = make_collection_mutation({}, make_key(), make_collection_member(int32_type, 103));
mutation m3(s, key);
m3.set_static_cell(column, my_list_type->serialize_mutation_form(mmut3));
mt->apply(m3);
collection_type_impl::mutation mmut2o{{}, {{make_key(), make_atomic_cell(int32_type->decompose(102))}}};
auto mmut2o = make_collection_mutation({}, make_key(), make_collection_member(int32_type, 102));
mutation m2o(s, key);
m2o.set_static_cell(column, my_list_type->serialize_mutation_form(mmut2o));
mt->apply(m2o);
@@ -275,7 +309,8 @@ SEASTAR_TEST_CASE(test_list_mutations) {
auto i = r.find_cell(column.id);
BOOST_REQUIRE(i);
auto cell = i->as_collection_mutation();
auto muts = my_list_type->deserialize_mutation_form(cell);
auto cell_b = cell.data.linearize();
auto muts = my_list_type->deserialize_mutation_form(cell_b);
BOOST_REQUIRE(muts.cells.size() == 4);
// FIXME: more strict tests
});
@@ -302,7 +337,7 @@ SEASTAR_TEST_CASE(test_multiple_memtables_one_partition) {
auto insert_row = [&] (int32_t c1, int32_t r1) {
auto c_key = clustering_key::from_exploded(*s, {int32_type->decompose(c1)});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(r1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, r1));
cf.apply(std::move(m));
return cf.flush();
};
@@ -318,9 +353,9 @@ SEASTAR_TEST_CASE(test_multiple_memtables_one_partition) {
BOOST_REQUIRE(r);
auto i = r->find_cell(r1_col.id);
BOOST_REQUIRE(i);
auto cell = i->as_atomic_cell();
auto cell = i->as_atomic_cell(r1_col);
BOOST_REQUIRE(cell.is_live());
BOOST_REQUIRE(int32_type->equal(cell.value(), int32_type->decompose(r1)));
BOOST_REQUIRE(int32_type->equal(cell.value().linearize(), int32_type->decompose(r1)));
}
};
verify_row(1001, 2001);
@@ -436,7 +471,7 @@ SEASTAR_TEST_CASE(test_multiple_memtables_multiple_partitions) {
auto key = partition_key::from_exploded(*s, {int32_type->decompose(p1)});
auto c_key = clustering_key::from_exploded(*s, {int32_type->decompose(c1)});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, atomic_cell::make_live(ts++, int32_type->decompose(r1)));
m.set_clustered_cell(c_key, r1_col, atomic_cell::make_live(*int32_type, ts++, int32_type->decompose(r1)));
cf.apply(std::move(m));
shadow[p1][c1] = r1;
};
@@ -458,7 +493,7 @@ SEASTAR_TEST_CASE(test_multiple_memtables_multiple_partitions) {
auto c1 = value_cast<int32_t>(int32_type->deserialize(re.key().explode(*s)[0]));
auto cell = re.row().cells().find_cell(r1_col.id);
if (cell) {
result[p1][c1] = value_cast<int32_t>(int32_type->deserialize(cell->as_atomic_cell().value()));
result[p1][c1] = value_cast<int32_t>(int32_type->deserialize(cell->as_atomic_cell(r1_col).value().linearize()));
}
}
return true;
@@ -479,10 +514,12 @@ SEASTAR_TEST_CASE(test_cell_ordering) {
auto assert_order = [] (atomic_cell_view first, atomic_cell_view second) {
if (compare_atomic_cell_for_merge(first, second) >= 0) {
BOOST_FAIL(sprint("Expected %s < %s", first, second));
BOOST_TEST_MESSAGE(sprint("Expected %s < %s", first, second));
abort();
}
if (compare_atomic_cell_for_merge(second, first) <= 0) {
BOOST_FAIL(sprint("Expected %s < %s", second, first));
BOOST_TEST_MESSAGE(sprint("Expected %s < %s", second, first));
abort();
}
};
@@ -492,57 +529,57 @@ SEASTAR_TEST_CASE(test_cell_ordering) {
};
assert_equal(
atomic_cell::make_live(0, bytes("value")),
atomic_cell::make_live(0, bytes("value")));
atomic_cell::make_live(*bytes_type, 0, bytes("value")),
atomic_cell::make_live(*bytes_type, 0, bytes("value")));
assert_order(
atomic_cell::make_live(1, bytes("value")),
atomic_cell::make_live(1, bytes("value"), expiry_1, ttl_1));
atomic_cell::make_live(*bytes_type, 1, bytes("value")),
atomic_cell::make_live(*bytes_type, 1, bytes("value"), expiry_1, ttl_1));
assert_equal(
atomic_cell::make_dead(1, expiry_1),
atomic_cell::make_dead(1, expiry_1));
assert_order(
atomic_cell::make_live(1, bytes()),
atomic_cell::make_live(1, bytes(), expiry_2, ttl_2));
atomic_cell::make_live(*bytes_type, 1, bytes()),
atomic_cell::make_live(*bytes_type, 1, bytes(), expiry_2, ttl_2));
// Origin doesn't compare ttl (is it wise?)
assert_equal(
atomic_cell::make_live(1, bytes("value"), expiry_1, ttl_1),
atomic_cell::make_live(1, bytes("value"), expiry_1, ttl_2));
atomic_cell::make_live(*bytes_type, 1, bytes("value"), expiry_1, ttl_1),
atomic_cell::make_live(*bytes_type, 1, bytes("value"), expiry_1, ttl_2));
assert_order(
atomic_cell::make_live(0, bytes("value1")),
atomic_cell::make_live(0, bytes("value2")));
atomic_cell::make_live(*bytes_type, 0, bytes("value1")),
atomic_cell::make_live(*bytes_type, 0, bytes("value2")));
assert_order(
atomic_cell::make_live(0, bytes("value12")),
atomic_cell::make_live(0, bytes("value2")));
atomic_cell::make_live(*bytes_type, 0, bytes("value12")),
atomic_cell::make_live(*bytes_type, 0, bytes("value2")));
// Live cells are ordered first by timestamp...
assert_order(
atomic_cell::make_live(0, bytes("value2")),
atomic_cell::make_live(1, bytes("value1")));
atomic_cell::make_live(*bytes_type, 0, bytes("value2")),
atomic_cell::make_live(*bytes_type, 1, bytes("value1")));
// ..then by value
assert_order(
atomic_cell::make_live(1, bytes("value1"), expiry_2, ttl_2),
atomic_cell::make_live(1, bytes("value2"), expiry_1, ttl_1));
atomic_cell::make_live(*bytes_type, 1, bytes("value1"), expiry_2, ttl_2),
atomic_cell::make_live(*bytes_type, 1, bytes("value2"), expiry_1, ttl_1));
// ..then by expiry
assert_order(
atomic_cell::make_live(1, bytes(), expiry_1, ttl_1),
atomic_cell::make_live(1, bytes(), expiry_2, ttl_1));
atomic_cell::make_live(*bytes_type, 1, bytes(), expiry_1, ttl_1),
atomic_cell::make_live(*bytes_type, 1, bytes(), expiry_2, ttl_1));
// Dead wins
assert_order(
atomic_cell::make_live(1, bytes("value")),
atomic_cell::make_live(*bytes_type, 1, bytes("value")),
atomic_cell::make_dead(1, expiry_1));
// Dead wins with expiring cell
assert_order(
atomic_cell::make_live(1, bytes("value"), expiry_2, ttl_2),
atomic_cell::make_live(*bytes_type, 1, bytes("value"), expiry_2, ttl_2),
atomic_cell::make_dead(1, expiry_1));
// Deleted cells are ordered first by timestamp
@@ -621,7 +658,7 @@ SEASTAR_TEST_CASE(test_partition_with_live_data_in_static_row_is_present_in_the_
mutation m(s, partition_key::from_single_value(*s, "key1"));
m.partition().static_row().apply(*s->get_column_definition("sc1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("sc1:value")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("sc1:value")))));
auto slice = partition_slice_builder(*s)
.with_no_static_columns()
@@ -647,7 +684,7 @@ SEASTAR_TEST_CASE(test_query_result_with_one_regular_column_missing) {
mutation m(s, partition_key::from_single_value(*s, "key1"));
m.set_clustered_cell(clustering_key::from_single_value(*s, bytes("ck:A")),
*s->get_column_definition("v1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v1:value")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v1:value")))));
auto slice = partition_slice_builder(*s).build();
@@ -678,12 +715,12 @@ SEASTAR_TEST_CASE(test_row_counting) {
auto ckey1 = clustering_key::from_single_value(*s, bytes_type->decompose(data_value(bytes("A"))));
auto ckey2 = clustering_key::from_single_value(*s, bytes_type->decompose(data_value(bytes("B"))));
m.set_clustered_cell(ckey1, col_v, atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v:value")))));
m.set_clustered_cell(ckey1, col_v, atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v:value")))));
BOOST_REQUIRE_EQUAL(1, m.live_row_count());
m.partition().static_row().apply(*s->get_column_definition("sc1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("sc1:value")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("sc1:value")))));
BOOST_REQUIRE_EQUAL(1, m.live_row_count());
@@ -704,8 +741,8 @@ SEASTAR_TEST_CASE(test_row_counting) {
BOOST_REQUIRE_EQUAL(0, m.live_row_count());
m.set_clustered_cell(ckey1, col_v, atomic_cell::make_live(4, bytes_type->decompose(data_value(bytes("v:value")))));
m.set_clustered_cell(ckey2, col_v, atomic_cell::make_live(4, bytes_type->decompose(data_value(bytes("v:value")))));
m.set_clustered_cell(ckey1, col_v, atomic_cell::make_live(*bytes_type, 4, bytes_type->decompose(data_value(bytes("v:value")))));
m.set_clustered_cell(ckey2, col_v, atomic_cell::make_live(*bytes_type, 4, bytes_type->decompose(data_value(bytes("v:value")))));
BOOST_REQUIRE_EQUAL(2, m.live_row_count());
});
@@ -781,7 +818,7 @@ SEASTAR_TEST_CASE(test_apply_monotonically_is_monotonic) {
size_t fail_offset = 0;
while (true) {
mutation m = target;
mutation_partition m2 = second.partition();
auto m2 = mutation_partition(*m.schema(), second.partition());
alloc.fail_after(fail_offset++);
try {
m.partition().apply_monotonically(*m.schema(), std::move(m2), no_cache_tracker);
@@ -822,40 +859,40 @@ SEASTAR_TEST_CASE(test_mutation_diff) {
m1.partition().apply(tombstone { 1, gc_clock::now() });
m1.set_clustered_cell(ckey1, *s->get_column_definition("v1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v1:value1")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v1:value1")))));
m1.set_clustered_cell(ckey1, *s->get_column_definition("v2"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v2:value2")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v2:value2")))));
m1.partition().clustered_row(*s, ckey2).apply(row_marker(3));
m1.set_clustered_cell(ckey2, *s->get_column_definition("v2"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v2:value4")))));
map_type_impl::mutation mset1 {{}, {{int32_type->decompose(1), make_atomic_cell({})}, {int32_type->decompose(2), make_atomic_cell({})}}};
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v2:value4")))));
auto mset1 = make_collection_mutation({}, int32_type->decompose(1), make_atomic_cell(), int32_type->decompose(2), make_atomic_cell());
m1.set_clustered_cell(ckey2, *s->get_column_definition("v3"),
my_set_type->serialize_mutation_form(mset1));
mutation m2(s, partition_key::from_single_value(*s, "key1"));
m2.set_clustered_cell(ckey1, *s->get_column_definition("v1"),
atomic_cell::make_live(1, bytes_type->decompose(data_value(bytes("v1:value1a")))));
atomic_cell::make_live(*bytes_type, 1, bytes_type->decompose(data_value(bytes("v1:value1a")))));
m2.set_clustered_cell(ckey1, *s->get_column_definition("v2"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v2:value2")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v2:value2")))));
m2.set_clustered_cell(ckey2, *s->get_column_definition("v1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v1:value3")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v1:value3")))));
m2.set_clustered_cell(ckey2, *s->get_column_definition("v2"),
atomic_cell::make_live(3, bytes_type->decompose(data_value(bytes("v2:value4a")))));
map_type_impl::mutation mset2 {{}, {{int32_type->decompose(1), make_atomic_cell({})}, {int32_type->decompose(3), make_atomic_cell({})}}};
atomic_cell::make_live(*bytes_type, 3, bytes_type->decompose(data_value(bytes("v2:value4a")))));
auto mset2 = make_collection_mutation({}, int32_type->decompose(1), make_atomic_cell(), int32_type->decompose(3), make_atomic_cell());
m2.set_clustered_cell(ckey2, *s->get_column_definition("v3"),
my_set_type->serialize_mutation_form(mset2));
mutation m3(s, partition_key::from_single_value(*s, "key1"));
m3.set_clustered_cell(ckey1, *s->get_column_definition("v1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v1:value1")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v1:value1")))));
m3.set_clustered_cell(ckey2, *s->get_column_definition("v1"),
atomic_cell::make_live(2, bytes_type->decompose(data_value(bytes("v1:value3")))));
atomic_cell::make_live(*bytes_type, 2, bytes_type->decompose(data_value(bytes("v1:value3")))));
m3.set_clustered_cell(ckey2, *s->get_column_definition("v2"),
atomic_cell::make_live(3, bytes_type->decompose(data_value(bytes("v2:value4a")))));
map_type_impl::mutation mset3 {{}, {{int32_type->decompose(1), make_atomic_cell({})}}};
atomic_cell::make_live(*bytes_type, 3, bytes_type->decompose(data_value(bytes("v2:value4a")))));
auto mset3 = make_collection_mutation({}, int32_type->decompose(1), make_atomic_cell());
m3.set_clustered_cell(ckey2, *s->get_column_definition("v3"),
my_set_type->serialize_mutation_form(mset3));
@@ -870,7 +907,8 @@ SEASTAR_TEST_CASE(test_mutation_diff) {
BOOST_REQUIRE(m2_1.find_row(*s, ckey2));
BOOST_REQUIRE(m2_1.find_row(*s, ckey2)->find_cell(2));
auto cmv = m2_1.find_row(*s, ckey2)->find_cell(2)->as_collection_mutation();
auto cm = my_set_type->deserialize_mutation_form(cmv);
auto cmv_b = cmv.data.linearize();
auto cm = my_set_type->deserialize_mutation_form(cmv_b);
BOOST_REQUIRE(cm.cells.size() == 1);
BOOST_REQUIRE(cm.cells.front().first == int32_type->decompose(3));
@@ -887,7 +925,8 @@ SEASTAR_TEST_CASE(test_mutation_diff) {
BOOST_REQUIRE(!m1_2.find_row(*s, ckey2)->find_cell(0));
BOOST_REQUIRE(!m1_2.find_row(*s, ckey2)->find_cell(1));
cmv = m1_2.find_row(*s, ckey2)->find_cell(2)->as_collection_mutation();
cm = my_set_type->deserialize_mutation_form(cmv);
cmv_b = cmv.data.linearize();
cm = my_set_type->deserialize_mutation_form(cmv_b);
BOOST_REQUIRE(cm.cells.size() == 1);
BOOST_REQUIRE(cm.cells.front().first == int32_type->decompose(2));
@@ -924,29 +963,29 @@ SEASTAR_TEST_CASE(test_large_blobs) {
auto key = partition_key::from_exploded(*s, {to_bytes("key1")});
mutation m(s, key);
m.set_static_cell(s1_col, make_atomic_cell(bytes_type->decompose(data_value(blob1))));
m.set_static_cell(s1_col, make_atomic_cell(bytes_type, data_value(blob1)));
mt->apply(std::move(m));
auto p = get_partition(*mt, key);
row& r = p.static_row();
auto i = r.find_cell(s1_col.id);
BOOST_REQUIRE(i);
auto cell = i->as_atomic_cell();
auto cell = i->as_atomic_cell(s1_col);
BOOST_REQUIRE(cell.is_live());
BOOST_REQUIRE(bytes_type->equal(cell.value(), bytes_type->decompose(data_value(blob1))));
BOOST_REQUIRE(bytes_type->equal(cell.value().linearize(), bytes_type->decompose(data_value(blob1))));
// Stress managed_bytes::linearize and scatter by merging a value into the cell
mutation m2(s, key);
m2.set_static_cell(s1_col, atomic_cell::make_live(7, bytes_type->decompose(data_value(blob2))));
m2.set_static_cell(s1_col, atomic_cell::make_live(*bytes_type, 7, bytes_type->decompose(data_value(blob2))));
mt->apply(std::move(m2));
auto p2 = get_partition(*mt, key);
row& r2 = p2.static_row();
auto i2 = r2.find_cell(s1_col.id);
BOOST_REQUIRE(i2);
auto cell2 = i2->as_atomic_cell();
auto cell2 = i2->as_atomic_cell(s1_col);
BOOST_REQUIRE(cell2.is_live());
BOOST_REQUIRE(bytes_type->equal(cell2.value(), bytes_type->decompose(data_value(blob2))));
BOOST_REQUIRE(bytes_type->equal(cell2.value().linearize(), bytes_type->decompose(data_value(blob2))));
});
}
@@ -1150,6 +1189,34 @@ SEASTAR_TEST_CASE(test_mutation_upgrade) {
});
}
SEASTAR_THREAD_TEST_CASE(test_mutation_upgrade_type_change) {
auto make_builder = [] {
return schema_builder("ks", "cf")
.with_column("pk", bytes_type, column_kind::partition_key)
.with_column("ck", bytes_type, column_kind::clustering_key);
};
auto s1 = make_builder()
.with_column("v1", int32_type)
.build();
auto s2 = make_builder()
.with_column("v1", bytes_type)
.build();
auto pk = partition_key::from_singular(*s1, data_value(bytes("key1")));
auto ck1 = clustering_key::from_singular(*s1, data_value(bytes("A")));
mutation m(s1, pk);
m.set_clustered_cell(ck1, "v1", data_value(int32_t(0x1234abcd)), 1);
m.upgrade(s2);
mutation m2(s2, pk);
m2.set_clustered_cell(ck1, "v1", data_value(from_hex("1234abcd")), 1);
assert_that(m).is_equal_to(m2);
}
SEASTAR_TEST_CASE(test_querying_expired_cells) {
return seastar::async([] {
auto s = schema_builder("ks", "cf")
@@ -1193,12 +1260,12 @@ SEASTAR_TEST_CASE(test_querying_expired_cells) {
{
mutation m(s, pk);
m.set_clustered_cell(ckey1, *s->get_column_definition("v1"), atomic_cell::make_live(api::new_timestamp(), v1.serialize(), t1, ttl));
m.set_clustered_cell(ckey1, *s->get_column_definition("v2"), atomic_cell::make_live(api::new_timestamp(), v2.serialize(), t2, ttl));
m.set_clustered_cell(ckey1, *s->get_column_definition("v3"), atomic_cell::make_live(api::new_timestamp(), v3.serialize(), t3, ttl));
m.set_static_cell(*s->get_column_definition("s1"), atomic_cell::make_live(api::new_timestamp(), v1.serialize(), t1, ttl));
m.set_static_cell(*s->get_column_definition("s2"), atomic_cell::make_live(api::new_timestamp(), v2.serialize(), t2, ttl));
m.set_static_cell(*s->get_column_definition("s3"), atomic_cell::make_live(api::new_timestamp(), v3.serialize(), t3, ttl));
m.set_clustered_cell(ckey1, *s->get_column_definition("v1"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v1.serialize(), t1, ttl));
m.set_clustered_cell(ckey1, *s->get_column_definition("v2"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v2.serialize(), t2, ttl));
m.set_clustered_cell(ckey1, *s->get_column_definition("v3"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v3.serialize(), t3, ttl));
m.set_static_cell(*s->get_column_definition("s1"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v1.serialize(), t1, ttl));
m.set_static_cell(*s->get_column_definition("s2"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v2.serialize(), t2, ttl));
m.set_static_cell(*s->get_column_definition("s3"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v3.serialize(), t3, ttl));
assert_that(results_at_time(m, t0))
.has_only(a_row()
@@ -1229,8 +1296,8 @@ SEASTAR_TEST_CASE(test_querying_expired_cells) {
{
mutation m(s, pk);
m.set_clustered_cell(ckey1, *s->get_column_definition("v1"), atomic_cell::make_live(api::new_timestamp(), v1.serialize(), t1, ttl));
m.set_static_cell(*s->get_column_definition("s1"), atomic_cell::make_live(api::new_timestamp(), v1.serialize(), t3, ttl));
m.set_clustered_cell(ckey1, *s->get_column_definition("v1"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v1.serialize(), t1, ttl));
m.set_static_cell(*s->get_column_definition("s1"), atomic_cell::make_live(*bytes_type, api::new_timestamp(), v1.serialize(), t3, ttl));
assert_that(results_at_time(m, t2))
.has_only(a_row().with_column("s1", v1).and_only_that());
@@ -1251,7 +1318,7 @@ SEASTAR_TEST_CASE(test_tombstone_purge) {
const column_definition& col = *s->get_column_definition("value");
mutation m(s, key);
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(int32_type, 1));
tombstone tomb(api::new_timestamp(), gc_clock::now() - std::chrono::seconds(1));
m.partition().apply(tomb);
BOOST_REQUIRE(!m.partition().empty());
@@ -1293,7 +1360,7 @@ SEASTAR_TEST_CASE(test_slicing_mutation) {
auto test_slicing = [&] (query::clustering_row_ranges ranges, std::vector<int> expected_rows) {
mutation_partition mp1(m.partition(), *s, ranges);
auto mp_temp = m.partition();
auto mp_temp = mutation_partition(*s, m.partition());
mutation_partition mp2(std::move(mp_temp), *s, ranges);
BOOST_REQUIRE(mp1.equal(*s, mp2));
@@ -1397,19 +1464,20 @@ SEASTAR_TEST_CASE(test_collection_cell_diff) {
{{"p", utf8_type}}, {}, {{"v", list_type_impl::get_instance(bytes_type, true)}}, {}, utf8_type));
auto& col = s->column_at(column_kind::regular_column, 0);
auto& ctype = *static_pointer_cast<const collection_type_impl>(col.type);
auto k = dht::global_partitioner().decorate_key(*s, partition_key::from_single_value(*s, to_bytes("key")));
mutation m1(s, k);
auto uuid = utils::UUID_gen::get_time_UUID_bytes();
collection_type_impl::mutation mcol1;
mcol1.cells.emplace_back(
bytes(reinterpret_cast<const int8_t*>(uuid.data()), uuid.size()),
atomic_cell::make_live(api::timestamp_type(1), to_bytes("element")));
m1.set_clustered_cell(clustering_key::make_empty(), col, list_type_impl::serialize_mutation_form(mcol1));
atomic_cell::make_live(*bytes_type, api::timestamp_type(1), to_bytes("element")));
m1.set_clustered_cell(clustering_key::make_empty(), col, ctype.serialize_mutation_form(mcol1));
mutation m2(s, k);
collection_type_impl::mutation mcol2;
mcol2.tomb = tombstone(api::timestamp_type(2), gc_clock::now());
m2.set_clustered_cell(clustering_key::make_empty(), col, list_type_impl::serialize_mutation_form(mcol2));
m2.set_clustered_cell(clustering_key::make_empty(), col, ctype.serialize_mutation_form(mcol2));
mutation m12 = m1;
m12.apply(m2);

20
tests/mvcc_test.cc
View File

@@ -95,7 +95,7 @@ SEASTAR_TEST_CASE(test_range_tombstone_slicing) {
m1.apply_delete(*s, rt2);
m1.apply_delete(*s, rt3);
partition_entry e(m1);
partition_entry e(mutation_partition(*s, m1));
auto snap = e.read(r, cleaner, s, no_cache_tracker);
@@ -263,7 +263,7 @@ mvcc_partition& mvcc_partition::operator+=(const mutation& m) {
void mvcc_partition::apply(const mutation_partition& mp, schema_ptr mp_s) {
with_allocator(region().allocator(), [&] {
if (_evictable) {
apply_to_evictable(partition_entry(mp), mp_s);
apply_to_evictable(partition_entry(mutation_partition(*mp_s, mp)), mp_s);
} else {
logalloc::allocating_section as;
as(region(), [&] {
@@ -473,12 +473,12 @@ SEASTAR_TEST_CASE(test_apply_to_incomplete_respects_continuity) {
auto before = e.squashed();
auto e_continuity = before.get_continuity(*s);
auto expected_to_apply_slice = to_apply.partition();
auto expected_to_apply_slice = mutation_partition(*s, to_apply.partition());
if (!before.static_row_continuous()) {
expected_to_apply_slice.static_row() = {};
}
auto expected = before;
auto expected = mutation_partition(*s, before);
expected.apply_weak(*s, std::move(expected_to_apply_slice));
e += to_apply;
@@ -565,7 +565,7 @@ SEASTAR_TEST_CASE(test_snapshot_cursor_is_consistent_with_merging_for_nonevictab
{
logalloc::reclaim_lock rl(r);
auto e = partition_entry(m3.partition());
auto e = partition_entry(mutation_partition(*s, m3.partition()));
auto snap1 = e.read(r, cleaner, s, no_cache_tracker);
e.apply(*s, m2.partition(), *s);
auto snap2 = e.read(r, cleaner, s, no_cache_tracker);
@@ -609,7 +609,7 @@ SEASTAR_TEST_CASE(test_continuity_merging_in_evictable) {
e.add_version(*s, &tracker).partition()
.clustered_row(*s, ss.make_ckey(2), is_dummy::no, is_continuous::no);
auto expected = m1.partition();
auto expected = mutation_partition(*s, m1.partition());
expected.clustered_row(*s, ss.make_ckey(1), is_dummy::no, is_continuous::no);
expected.clustered_row(*s, ss.make_ckey(2), is_dummy::no, is_continuous::no);
@@ -811,8 +811,8 @@ SEASTAR_TEST_CASE(test_apply_is_atomic) {
size_t fail_offset = 0;
while (true) {
mutation_partition m2 = second.partition();
auto e = partition_entry(target.partition());
mutation_partition m2 = mutation_partition(*second.schema(), second.partition());
auto e = partition_entry(mutation_partition(*target.schema(), target.partition()));
//auto snap1 = e.read(r, gen.schema());
alloc.fail_after(fail_offset++);
@@ -858,7 +858,7 @@ SEASTAR_TEST_CASE(test_versions_are_merged_when_snapshots_go_away) {
m3.partition().make_fully_continuous();
{
auto e = partition_entry(m1.partition());
auto e = partition_entry(mutation_partition(*s, m1.partition()));
auto snap1 = e.read(r, cleaner, s, nullptr);
{
@@ -879,7 +879,7 @@ SEASTAR_TEST_CASE(test_versions_are_merged_when_snapshots_go_away) {
}
{
auto e = partition_entry(m1.partition());
auto e = partition_entry(mutation_partition(*s, m1.partition()));
auto snap1 = e.read(r, cleaner, s, nullptr);
{

158
tests/partition_data_test.cc Normal file
View File

@@ -0,0 +1,158 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#define BOOST_TEST_MODULE partition_data
#include <boost/test/unit_test.hpp>
#include <random>
#include <boost/range/irange.hpp>
#include <boost/range/algorithm/generate.hpp>
#include "data/cell.hh"
#include "random-utils.hh"
#include "disk-error-handler.hh"
BOOST_AUTO_TEST_CASE(test_atomic_cell) {
struct test_case {
bool live;
bool fixed_size;
bytes value;
bool expiring;
bool counter_update;
};
auto cases = std::vector<test_case> {
// Live, fixed-size, empty cell
{ true, true, bytes(), false },
// Live, fixed-size cell
{ true, true, tests::random::get_bytes(data::cell::maximum_internal_storage_length / 2), false, false },
// Live, variable-size (small), cell
{ true, false, tests::random::get_bytes(data::cell::maximum_internal_storage_length / 2), false, false },
// Live, variable-size (large), cell
{ true, false, tests::random::get_bytes(data::cell::maximum_external_chunk_length * 5), false, false },
// Live, variable-size, empty cell
{ true, false, bytes(), false, false },
// Live, expiring, variable-size cell
{ true, false, tests::random::get_bytes(data::cell::maximum_internal_storage_length / 2), true, false },
// Dead cell
{ false, false, bytes(), false, false },
// Counter update cell
{ true, false, bytes(), false, true },
};
for (auto tc : cases) {
auto [live, fixed_size, value, expiring, counter_update] = tc;
auto timestamp = tests::random::get_int<api::timestamp_type>();
auto ti = [&] {
if (fixed_size) {
return data::type_info::make_fixed_size(value.size());
} else {
return data::type_info::make_variable_size();
}
}();
auto ttl = gc_clock::duration(tests::random::get_int<int32_t>(1, std::numeric_limits<int32_t>::max()));
auto expiry_time = gc_clock::time_point(gc_clock::duration(tests::random::get_int<int32_t>(1, std::numeric_limits<int32_t>::max())));
auto deletion_time = expiry_time;
auto counter_update_value = tests::random::get_int<int64_t>();
std::optional<imr::alloc::object_allocator> allocator;
allocator.emplace();
auto test_cell = [&] (auto builder) {
auto expected_size = data::cell::size_of(builder, *allocator);
if (fixed_size) {
BOOST_CHECK_GE(expected_size, value.size());
}
allocator->allocate_all();
auto buffer = std::make_unique<uint8_t[]>(expected_size);
BOOST_CHECK_EQUAL(data::cell::serialize(buffer.get(), builder, *allocator), expected_size);
auto verify_cell = [&] (auto view) {
if (!live) {
BOOST_CHECK(!view.is_live());
BOOST_CHECK(view.deletion_time() == deletion_time);
return;
}
BOOST_CHECK(view.is_live());
BOOST_CHECK_EQUAL(view.timestamp(), timestamp);
if (counter_update) {
BOOST_CHECK(view.is_counter_update());
BOOST_CHECK_EQUAL(view.counter_update_value(), counter_update_value);
} else {
BOOST_CHECK(!view.is_counter_update());
BOOST_CHECK(view.value() == value);
}
BOOST_CHECK_EQUAL(view.is_expiring(), expiring);
if (expiring) {
BOOST_CHECK(view.ttl() == ttl);
BOOST_CHECK(view.expiry() == expiry_time);
}
};
auto view = data::cell::make_atomic_cell_view(ti, buffer.get());
verify_cell(view);
allocator.emplace();
auto copier = data::cell::copy_fn(ti, buffer.get());
BOOST_CHECK_EQUAL(data::cell::size_of(copier, *allocator), expected_size);
allocator->allocate_all();
auto copied = std::make_unique<uint8_t[]>(expected_size);
BOOST_CHECK_EQUAL(data::cell::serialize(copied.get(), copier, *allocator), expected_size);
auto view2 = data::cell::make_atomic_cell_view(ti, copied.get());
verify_cell(view2);
auto ctx = data::cell::context(buffer.get(), ti);
BOOST_CHECK_EQUAL(data::cell::structure::serialized_object_size(buffer.get(), ctx), expected_size);
auto moved = std::make_unique<uint8_t[]>(expected_size);
std::copy_n(buffer.get(), expected_size, moved.get());
imr::methods::move<data::cell::structure>(moved.get());
auto view3 = data::cell::make_atomic_cell_view(ti, moved.get());
verify_cell(view3);
imr::methods::destroy<data::cell::structure>(moved.get());
imr::methods::destroy<data::cell::structure>(copied.get());
};
if (live) {
if (counter_update) {
test_cell(data::cell::make_live_counter_update(timestamp, counter_update_value));
} else if (expiring) {
test_cell(data::cell::make_live(ti, timestamp, value, expiry_time, ttl));
} else {
test_cell(data::cell::make_live(ti, timestamp, value));
}
} else {
test_cell(data::cell::make_dead(timestamp, deletion_time));
}
}
}

2
tests/perf/perf_cache_eviction.cc
View File

@@ -167,7 +167,7 @@ int main(int argc, char** argv) {
mutation m(s, pkey);
auto ck = clustering_key::from_single_value(*s, data_value(ckey_seq++).serialize());
auto&& col = *s->get_column_definition(to_bytes("v"));
m.set_clustered_cell(ck, col, atomic_cell::make_live(api::new_timestamp(), data_value(value).serialize()));
m.set_clustered_cell(ck, col, atomic_cell::make_live(*col.type, api::new_timestamp(), data_value(value).serialize()));
auto t0 = clock::now();
db.apply(s, freeze(m)).get();
writes_hist.add(std::chrono::duration_cast<std::chrono::microseconds>(clock::now() - t0).count());

6
tests/perf/perf_mutation.cc
View File

@@ -24,8 +24,8 @@
#include "perf.hh"
#include <seastar/core/app-template.hh>
static atomic_cell make_atomic_cell(bytes value) {
return atomic_cell::make_live(0, value);
static atomic_cell make_atomic_cell(data_type dt, bytes value) {
return atomic_cell::make_live(*dt, 0, value);
};
int main(int argc, char* argv[]) {
@@ -44,7 +44,7 @@ int main(int argc, char* argv[]) {
time_it([&] {
mutation m(s, key);
const column_definition& col = *s->get_column_definition("r1");
m.set_clustered_cell(c_key, col, make_atomic_cell(value));
m.set_clustered_cell(c_key, col, make_atomic_cell(col.type, value));
mt.apply(std::move(m));
});
engine().exit(0);

6
tests/perf/perf_mutation_fragment.cc
View File

@@ -97,19 +97,19 @@ PERF_TEST_F(clustering_row, make_1M)
PERF_TEST_F(clustering_row, copy_4)
{
auto mf = mutation_fragment(clustering_row_4());
auto mf = mutation_fragment(*schema(), clustering_row_4());
perf_tests::do_not_optimize(mf);
}
PERF_TEST_F(clustering_row, copy_4k)
{
auto mf = mutation_fragment(clustering_row_4k());
auto mf = mutation_fragment(*schema(), clustering_row_4k());
perf_tests::do_not_optimize(mf);
}
PERF_TEST_F(clustering_row, copy_1M)
{
auto mf = mutation_fragment(clustering_row_1M());
auto mf = mutation_fragment(*schema(), clustering_row_1M());
perf_tests::do_not_optimize(mf);
}

2
tests/perf/perf_sstable.hh
View File

@@ -113,7 +113,7 @@ public:
auto key = partition_key::from_deeply_exploded(*s, { this->random_key() });
auto mut = mutation(this->s, key);
for (auto& cdef: this->s->regular_columns()) {
mut.set_clustered_cell(clustering_key::make_empty(), cdef, atomic_cell::make_live(0, utf8_type->decompose(this->random_column())));
mut.set_clustered_cell(clustering_key::make_empty(), cdef, atomic_cell::make_live(*utf8_type, 0, utf8_type->decompose(this->random_column())));
}
this->_mt->apply(std::move(mut));
return make_ready_future<>();

64
tests/random-utils.hh Normal file
View File

@@ -0,0 +1,64 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <algorithm>
#include <random>
namespace tests::random {
static std::random_device rd;
static std::default_random_engine gen(rd());
template<typename T>
T get_int() {
std::uniform_int_distribution<T> dist;
return dist(gen);
}
template<typename T>
T get_int(T max) {
std::uniform_int_distribution<T> dist(0, max);
return dist(gen);
}
template<typename T>
T get_int(T min, T max) {
std::uniform_int_distribution<T> dist(min, max);
return dist(gen);
}
bool get_bool() {
static std::bernoulli_distribution dist;
return dist(gen);
}
bytes get_bytes(size_t n) {
bytes b(bytes::initialized_later(), n);
boost::generate(b, [] { return get_int<bytes::value_type>(); });
return b;
}
bytes get_bytes() {
return get_bytes(get_int<unsigned>(128 * 1024));
}
}

16
tests/row_cache_test.cc
View File

@@ -952,7 +952,7 @@ SEASTAR_TEST_CASE(test_update_failure) {
auto original_partitions = partitions_type(partition_key::less_compare(*s));
for (int i = 0; i < partition_count / 2; i++) {
auto m = make_new_mutation(s, i + partition_count / 2);
original_partitions.emplace(m.key(), m.partition());
original_partitions.emplace(m.key(), mutation_partition(*s, m.partition()));
cache.populate(m);
}
@@ -961,7 +961,7 @@ SEASTAR_TEST_CASE(test_update_failure) {
auto updated_partitions = partitions_type(partition_key::less_compare(*s));
for (int i = 0; i < partition_count; i++) {
auto m = make_new_large_mutation(s, i);
updated_partitions.emplace(m.key(), m.partition());
updated_partitions.emplace(m.key(), mutation_partition(*s, m.partition()));
mt->apply(m);
}
@@ -3122,7 +3122,7 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
{
auto rd = cache.make_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(!row.cells().cell_hash_for(0));
}
@@ -3131,14 +3131,14 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
slice.options.set<query::partition_slice::option::with_digest>();
auto rd = cache.make_reader(s, query::full_partition_range, slice);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
{
auto rd = cache.make_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
@@ -3149,7 +3149,7 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
{
auto rd = cache.make_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(!row.cells().cell_hash_for(0));
}
@@ -3158,14 +3158,14 @@ SEASTAR_TEST_CASE(test_hash_is_cached) {
slice.options.set<query::partition_slice::option::with_digest>();
auto rd = cache.make_reader(s, query::full_partition_range, slice);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
{
auto rd = cache.make_reader(s);
rd().get0()->as_partition_start();
clustering_row row = rd().get0()->as_clustering_row();
clustering_row row = std::move(rd().get0()->as_mutable_clustering_row());
BOOST_REQUIRE(row.cells().cell_hash_for(0));
}
});

10
tests/simple_schema.hh
View File

@@ -101,7 +101,7 @@ public:
if (t == api::missing_timestamp) {
t = new_timestamp();
}
m.set_clustered_cell(key, _v_def, atomic_cell::make_live(t, data_value(v).serialize()));
m.set_clustered_cell(key, _v_def, atomic_cell::make_live(*_v_def.type, t, data_value(v).serialize()));
return t;
}
@@ -110,23 +110,23 @@ public:
if (!cell) {
throw std::runtime_error("cell not found");
}
atomic_cell_view ac = cell->as_atomic_cell();
atomic_cell_view ac = cell->as_atomic_cell(_v_def);
if (!ac.is_live()) {
throw std::runtime_error("cell is dead");
}
return std::make_pair(value_cast<sstring>(utf8_type->deserialize(ac.value())), ac.timestamp());
return std::make_pair(value_cast<sstring>(utf8_type->deserialize(ac.value().linearize())), ac.timestamp());
}
mutation_fragment make_row(const clustering_key& key, sstring v) {
auto row = clustering_row(key);
row.cells().apply(*_s->get_column_definition(to_bytes(sstring("v"))),
atomic_cell::make_live(new_timestamp(), data_value(v).serialize()));
atomic_cell::make_live(*_v_def.type, new_timestamp(), data_value(v).serialize()));
return mutation_fragment(std::move(row));
}
mutation_fragment make_row_from_serialized_value(const clustering_key& key, bytes_view v) {
auto row = clustering_row(key);
row.cells().apply(_v_def, atomic_cell::make_live(new_timestamp(), v));
row.cells().apply(_v_def, atomic_cell::make_live(*_v_def.type, new_timestamp(), v));
return mutation_fragment(std::move(row));
}

23
tests/sstable_3_x_test.cc
View File

@@ -976,8 +976,8 @@ SEASTAR_TEST_CASE(test_write_ttled_column) {
throw std::runtime_error("no column definition found");
}
bytes value = column_def->type->decompose(data_value{1});
auto cell = atomic_cell::make_live(write_timestamp, value, write_time_point + ttl, ttl);
mut.set_clustered_cell(clustering_key::make_empty(), *column_def, cell);
auto cell = atomic_cell::make_live(*column_def->type, write_timestamp, value, write_time_point + ttl, ttl);
mut.set_clustered_cell(clustering_key::make_empty(), *column_def, std::move(cell));
mt->apply(std::move(mut));
write_and_compare_sstables(s, mt, table_name);
@@ -1051,13 +1051,10 @@ SEASTAR_TEST_CASE(test_write_collection_wide_update) {
mutation mut{s, key};
mut.partition().apply_insert(*s, clustering_key::make_empty(), write_timestamp);
set_type_impl::mutation set_values {
{write_timestamp - 1, write_time_point}, // tombstone
{
{int32_type->decompose(2), atomic_cell::make_live(write_timestamp, bytes_view{})},
{int32_type->decompose(3), atomic_cell::make_live(write_timestamp, bytes_view{})},
}
};
set_type_impl::mutation set_values;
set_values.tomb = tombstone {write_timestamp - 1, write_time_point};
set_values.cells.emplace_back(int32_type->decompose(2), atomic_cell::make_live(*bytes_type, write_timestamp, bytes_view{}));
set_values.cells.emplace_back(int32_type->decompose(3), atomic_cell::make_live(*bytes_type, write_timestamp, bytes_view{}));
mut.set_clustered_cell(clustering_key::make_empty(), *s->get_column_definition("col"), set_of_ints_type->serialize_mutation_form(set_values));
mt->apply(std::move(mut));
@@ -1083,12 +1080,8 @@ SEASTAR_TEST_CASE(test_write_collection_incremental_update) {
auto key = partition_key::from_deeply_exploded(*s, { 1 });
mutation mut{s, key};
set_type_impl::mutation set_values {
{}, // tombstone
{
{int32_type->decompose(2), atomic_cell::make_live(write_timestamp, bytes_view{})},
}
};
set_type_impl::mutation set_values;
set_values.cells.emplace_back(int32_type->decompose(2), atomic_cell::make_live(*bytes_type, write_timestamp, bytes_view{}));
mut.set_clustered_cell(clustering_key::make_empty(), *s->get_column_definition("col"), set_of_ints_type->serialize_mutation_form(set_values));
mt->apply(std::move(mut));

141
tests/sstable_datafile_test.cc
View File

@@ -76,12 +76,12 @@ static column_family::config column_family_test_config() {
return cfg;
}
atomic_cell make_atomic_cell(bytes_view value, uint32_t ttl = 0, uint32_t expiration = 0) {
atomic_cell make_atomic_cell(data_type dt, bytes_view value, uint32_t ttl = 0, uint32_t expiration = 0) {
if (ttl) {
return atomic_cell::make_live(0, value,
return atomic_cell::make_live(*dt, 0, value,
gc_clock::time_point(gc_clock::duration(expiration)), gc_clock::duration(ttl));
} else {
return atomic_cell::make_live(0, value);
return atomic_cell::make_live(*dt, 0, value);
}
}
@@ -112,7 +112,7 @@ SEASTAR_TEST_CASE(datafile_generation_01) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 1, la, big);
@@ -180,7 +180,7 @@ SEASTAR_TEST_CASE(datafile_generation_02) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 2, la, big);
@@ -250,7 +250,7 @@ SEASTAR_TEST_CASE(datafile_generation_03) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc"), to_bytes("cde")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 3, la, big);
@@ -322,8 +322,8 @@ SEASTAR_TEST_CASE(datafile_generation_04) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_static_cell(s1_col, make_atomic_cell(int32_type->decompose(10)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_static_cell(s1_col, make_atomic_cell(int32_type, int32_type->decompose(10)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 4, la, big);
@@ -395,7 +395,7 @@ SEASTAR_TEST_CASE(datafile_generation_05) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1), 3600, 3600));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1), 3600, 3600));
mt->apply(std::move(m));
auto now = to_gc_clock(db_clock::from_time_t(0));
@@ -539,14 +539,14 @@ SEASTAR_TEST_CASE(datafile_generation_07) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto key2 = partition_key::from_exploded(*s, {to_bytes("key2")});
auto c_key2 = clustering_key::from_exploded(*s, {to_bytes("cde")});
mutation m2(s, key2);
m2.set_clustered_cell(c_key2, r1_col, make_atomic_cell(int32_type->decompose(1)));
m2.set_clustered_cell(c_key2, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m2));
auto sst = make_sstable(s, tmpdir_path, 7, la, big);
@@ -601,7 +601,7 @@ SEASTAR_TEST_CASE(datafile_generation_08) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
}
@@ -661,7 +661,7 @@ SEASTAR_TEST_CASE(datafile_generation_09) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 9, la, big);
@@ -708,7 +708,7 @@ static future<> test_digest_and_checksum(sstable_version_types version) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 10, version, big);
@@ -791,11 +791,11 @@ SEASTAR_TEST_CASE(datafile_generation_11) {
mutation m(s, key);
tombstone tomb(api::new_timestamp(), gc_clock::now());
set_type_impl::mutation set_mut{{ tomb }, {
{ to_bytes("1"), make_atomic_cell({}) },
{ to_bytes("2"), make_atomic_cell({}) },
{ to_bytes("3"), make_atomic_cell({}) }
}};
set_type_impl::mutation set_mut;
set_mut.tomb = tomb;
set_mut.cells.emplace_back(to_bytes("1"), make_atomic_cell(bytes_type, {}));
set_mut.cells.emplace_back(to_bytes("2"), make_atomic_cell(bytes_type, {}));
set_mut.cells.emplace_back(to_bytes("3"), make_atomic_cell(bytes_type, {}));
auto set_type = static_pointer_cast<const set_type_impl>(set_col.type);
m.set_clustered_cell(c_key, set_col, set_type->serialize_mutation_form(set_mut));
@@ -805,7 +805,8 @@ SEASTAR_TEST_CASE(datafile_generation_11) {
auto key2 = partition_key::from_exploded(*s, {to_bytes("key2")});
mutation m2(s, key2);
set_type_impl::mutation set_mut_single{{}, {{ to_bytes("4"), make_atomic_cell({}) }}};
set_type_impl::mutation set_mut_single;
set_mut_single.cells.emplace_back(to_bytes("4"), make_atomic_cell(bytes_type, {}));
m2.set_clustered_cell(c_key, set_col, set_type->serialize_mutation_form(set_mut_single));
@@ -822,7 +823,8 @@ SEASTAR_TEST_CASE(datafile_generation_11) {
auto cell = r->find_cell(set_col.id);
BOOST_REQUIRE(cell);
auto t = static_pointer_cast<const collection_type_impl>(set_col.type);
return t->deserialize_mutation_form(cell->as_collection_mutation());
auto bv = cell->as_collection_mutation().data.linearize();
return t->deserialize_mutation_form(bv).materialize(*t);
};
auto sst = make_sstable(s, tmpdir_path, 11, la, big);
@@ -831,7 +833,7 @@ SEASTAR_TEST_CASE(datafile_generation_11) {
return do_with(make_dkey(s, "key1"), [sstp, s, verifier, tomb, &static_set_col] (auto& key) {
auto rd = make_lw_shared<flat_mutation_reader>(sstp->read_row_flat(s, key));
return read_mutation_from_flat_mutation_reader(*rd).then([sstp, s, verifier, tomb, &static_set_col, rd] (auto mutation) {
auto verify_set = [&tomb] (auto m) {
auto verify_set = [&tomb] (const collection_type_impl::mutation& m) {
BOOST_REQUIRE(bool(m.tomb) == true);
BOOST_REQUIRE(m.tomb == tomb);
BOOST_REQUIRE(m.cells.size() == 3);
@@ -848,7 +850,8 @@ SEASTAR_TEST_CASE(datafile_generation_11) {
// The static set
auto t = static_pointer_cast<const collection_type_impl>(static_set_col.type);
auto mut = t->deserialize_mutation_form(scol->as_collection_mutation());
auto bv = scol->as_collection_mutation().data.linearize();
auto mut = t->deserialize_mutation_form(bv).materialize(*t);
verify_set(mut);
// The clustered set
@@ -1056,7 +1059,7 @@ SEASTAR_TEST_CASE(compaction_manager_test) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmp->path, column_family_test::calculate_generation_for_new_table(*cf), la, big);
@@ -1148,8 +1151,10 @@ SEASTAR_TEST_CASE(compact) {
auto &row = rows.begin()->row();
BOOST_REQUIRE(!row.deleted_at());
auto &cells = row.cells();
BOOST_REQUIRE(cells.cell_at(s->get_column_definition("age")->id).as_atomic_cell().value() == bytes({0,0,0,40}));
BOOST_REQUIRE(cells.cell_at(s->get_column_definition("height")->id).as_atomic_cell().value() == bytes({0,0,0,(int8_t)170}));
auto& cdef1 = *s->get_column_definition("age");
auto& cdef2 = *s->get_column_definition("height");
BOOST_REQUIRE(cells.cell_at(cdef1.id).as_atomic_cell(cdef1).value() == bytes({0,0,0,40}));
BOOST_REQUIRE(cells.cell_at(cdef2.id).as_atomic_cell(cdef2).value() == bytes({0,0,0,(int8_t)170}));
return read_mutation_from_flat_mutation_reader(*reader);
}).then([reader, s] (mutation_opt m) {
BOOST_REQUIRE(m);
@@ -1160,8 +1165,10 @@ SEASTAR_TEST_CASE(compact) {
auto &row = rows.begin()->row();
BOOST_REQUIRE(!row.deleted_at());
auto &cells = row.cells();
BOOST_REQUIRE(cells.cell_at(s->get_column_definition("age")->id).as_atomic_cell().value() == bytes({0,0,0,20}));
BOOST_REQUIRE(cells.cell_at(s->get_column_definition("height")->id).as_atomic_cell().value() == bytes({0,0,0,(int8_t)180}));
auto& cdef1 = *s->get_column_definition("age");
auto& cdef2 = *s->get_column_definition("height");
BOOST_REQUIRE(cells.cell_at(cdef1.id).as_atomic_cell(cdef1).value() == bytes({0,0,0,20}));
BOOST_REQUIRE(cells.cell_at(cdef2.id).as_atomic_cell(cdef2).value() == bytes({0,0,0,(int8_t)180}));
return read_mutation_from_flat_mutation_reader(*reader);
}).then([reader, s] (mutation_opt m) {
BOOST_REQUIRE(m);
@@ -1172,8 +1179,10 @@ SEASTAR_TEST_CASE(compact) {
auto &row = rows.begin()->row();
BOOST_REQUIRE(!row.deleted_at());
auto &cells = row.cells();
BOOST_REQUIRE(cells.cell_at(s->get_column_definition("age")->id).as_atomic_cell().value() == bytes({0,0,0,20}));
BOOST_REQUIRE(cells.find_cell(s->get_column_definition("height")->id) == nullptr);
auto& cdef1 = *s->get_column_definition("age");
auto& cdef2 = *s->get_column_definition("height");
BOOST_REQUIRE(cells.cell_at(cdef1.id).as_atomic_cell(cdef1).value() == bytes({0,0,0,20}));
BOOST_REQUIRE(cells.find_cell(cdef2.id) == nullptr);
return read_mutation_from_flat_mutation_reader(*reader);
}).then([reader, s] (mutation_opt m) {
BOOST_REQUIRE(m);
@@ -1242,7 +1251,7 @@ static future<std::vector<unsigned long>> compact_sstables(sstring tmpdir_path,
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(bytes(min_sstable_size, 'a')));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(utf8_type, bytes(min_sstable_size, 'a')));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, generation, la, big);
@@ -1396,7 +1405,7 @@ SEASTAR_TEST_CASE(datafile_generation_37) {
auto c_key = clustering_key_prefix::from_exploded(*s, {to_bytes("cl1")});
const column_definition& cl2 = *s->get_column_definition("cl2");
m.set_clustered_cell(c_key, cl2, make_atomic_cell(bytes_type->decompose(data_value(to_bytes("cl2")))));
m.set_clustered_cell(c_key, cl2, make_atomic_cell(bytes_type, bytes_type->decompose(data_value(to_bytes("cl2")))));
mtp->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 37, la, big);
@@ -1409,7 +1418,7 @@ SEASTAR_TEST_CASE(datafile_generation_37) {
auto clustering = clustering_key_prefix::from_exploded(*s, {to_bytes("cl1")});
auto row = mp.clustered_row(*s, clustering);
auto& row = mp.clustered_row(*s, clustering);
match_live_cell(row.cells(), *s, "cl2", data_value(to_bytes("cl2")));
return make_ready_future<>();
});
@@ -1431,7 +1440,7 @@ SEASTAR_TEST_CASE(datafile_generation_38) {
auto c_key = clustering_key_prefix::from_exploded(*s, {to_bytes("cl1"), to_bytes("cl2")});
const column_definition& cl3 = *s->get_column_definition("cl3");
m.set_clustered_cell(c_key, cl3, make_atomic_cell(bytes_type->decompose(data_value(to_bytes("cl3")))));
m.set_clustered_cell(c_key, cl3, make_atomic_cell(bytes_type, bytes_type->decompose(data_value(to_bytes("cl3")))));
mtp->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 38, la, big);
@@ -1443,7 +1452,7 @@ SEASTAR_TEST_CASE(datafile_generation_38) {
auto& mp = mutation->partition();
auto clustering = clustering_key_prefix::from_exploded(*s, {to_bytes("cl1"), to_bytes("cl2")});
auto row = mp.clustered_row(*s, clustering);
auto& row = mp.clustered_row(*s, clustering);
match_live_cell(row.cells(), *s, "cl3", data_value(to_bytes("cl3")));
return make_ready_future<>();
});
@@ -1465,9 +1474,9 @@ SEASTAR_TEST_CASE(datafile_generation_39) {
auto c_key = clustering_key::make_empty();
const column_definition& cl1 = *s->get_column_definition("cl1");
m.set_clustered_cell(c_key, cl1, make_atomic_cell(bytes_type->decompose(data_value(to_bytes("cl1")))));
m.set_clustered_cell(c_key, cl1, make_atomic_cell(bytes_type, bytes_type->decompose(data_value(to_bytes("cl1")))));
const column_definition& cl2 = *s->get_column_definition("cl2");
m.set_clustered_cell(c_key, cl2, make_atomic_cell(bytes_type->decompose(data_value(to_bytes("cl2")))));
m.set_clustered_cell(c_key, cl2, make_atomic_cell(bytes_type, bytes_type->decompose(data_value(to_bytes("cl2")))));
mtp->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 39, la, big);
@@ -1477,7 +1486,7 @@ SEASTAR_TEST_CASE(datafile_generation_39) {
auto rd = make_lw_shared<flat_mutation_reader>(sstp->read_row_flat(s, key));
return read_mutation_from_flat_mutation_reader(*rd).then([sstp, s, rd] (auto mutation) {
auto& mp = mutation->partition();
auto row = mp.clustered_row(*s, clustering_key::make_empty());
auto& row = mp.clustered_row(*s, clustering_key::make_empty());
match_live_cell(row.cells(), *s, "cl1", data_value(data_value(to_bytes("cl1"))));
match_live_cell(row.cells(), *s, "cl2", data_value(data_value(to_bytes("cl2"))));
return make_ready_future<>();
@@ -1516,11 +1525,11 @@ SEASTAR_TEST_CASE(datafile_generation_40) {
const column_definition& r1_col = *s->get_column_definition("r1");
auto ca = clustering_key::from_exploded(*s, {to_bytes("a")});
m.set_clustered_cell(ca, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(ca, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto cb = clustering_key::from_exploded(*s, {to_bytes("b")});
m.set_clustered_cell(cb, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(cb, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 40, la, big);
@@ -1574,7 +1583,7 @@ SEASTAR_TEST_CASE(datafile_generation_41) {
return read_mutation_from_flat_mutation_reader(*rd).then([sstp, s, tomb, rd] (auto mutation) {
auto& mp = mutation->partition();
BOOST_REQUIRE(mp.clustered_rows().calculate_size() == 1);
auto c_row = *(mp.clustered_rows().begin());
auto& c_row = *(mp.clustered_rows().begin());
BOOST_REQUIRE(c_row.row().deleted_at().tomb() == tomb);
});
});
@@ -1622,7 +1631,7 @@ SEASTAR_TEST_CASE(datafile_generation_47) {
auto key = partition_key::from_exploded(*s, {to_bytes("key1")});
auto c_key = clustering_key::from_exploded(*s, {to_bytes("c1")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(bytes(512*1024, 'a')));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(utf8_type, bytes(512*1024, 'a')));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmpdir_path, 47, la, big);
@@ -2382,7 +2391,8 @@ SEASTAR_TEST_CASE(tombstone_purge_test) {
auto& row = rows.begin()->row();
auto& cells = row.cells();
BOOST_REQUIRE_EQUAL(cells.size(), 1);
BOOST_REQUIRE(!cells.cell_at(s->get_column_definition("value")->id).as_atomic_cell().is_live());
auto& cdef = *s->get_column_definition("value");
BOOST_REQUIRE(!cells.cell_at(cdef.id).as_atomic_cell(cdef).is_live());
return (*reader)();
}).then([reader, s] (mutation_fragment_opt m) {
BOOST_REQUIRE(!m);
@@ -2557,7 +2567,8 @@ SEASTAR_TEST_CASE(check_multi_schema) {
BOOST_REQUIRE(!row.deleted_at());
auto& cells = row.cells();
BOOST_REQUIRE_EQUAL(cells.size(), 1);
BOOST_REQUIRE_EQUAL(cells.cell_at(s->get_column_definition("e")->id).as_atomic_cell().value(), int32_type->decompose(5));
auto& cdef = *s->get_column_definition("e");
BOOST_REQUIRE_EQUAL(cells.cell_at(cdef.id).as_atomic_cell(cdef).value(), int32_type->decompose(5));
return (*reader)();
}).then([reader, s] (mutation_fragment_opt m) {
BOOST_REQUIRE(!m);
@@ -2581,7 +2592,7 @@ SEASTAR_TEST_CASE(sstable_rewrite) {
auto key = partition_key::from_exploded(*s, {to_bytes(key_to_write)});
auto c_key = clustering_key::from_exploded(*s, {to_bytes("c1")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(bytes("a")));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(utf8_type, bytes("a")));
mt->apply(std::move(m));
};
apply_key(key_for_this_shard[0].first);
@@ -2866,7 +2877,7 @@ SEASTAR_TEST_CASE(test_counter_read) {
BOOST_REQUIRE(mfopt->is_clustering_row());
const clustering_row* cr = &mfopt->as_clustering_row();
cr->cells().for_each_cell([&] (column_id id, const atomic_cell_or_collection& c) {
counter_cell_view ccv { c.as_atomic_cell() };
counter_cell_view::with_linearized(c.as_atomic_cell(s->regular_column_at(id)), [&] (counter_cell_view ccv) {
auto& col = s->column_at(column_kind::regular_column, id);
if (col.name_as_text() == "c1") {
BOOST_REQUIRE_EQUAL(ccv.total_value(), 13);
@@ -2887,6 +2898,7 @@ SEASTAR_TEST_CASE(test_counter_read) {
} else {
BOOST_FAIL(sprint("Unexpected column \'%s\'", col.name_as_text()));
}
});
});
mfopt = reader().get0();
@@ -2896,7 +2908,7 @@ SEASTAR_TEST_CASE(test_counter_read) {
cr->cells().for_each_cell([&] (column_id id, const atomic_cell_or_collection& c) {
auto& col = s->column_at(column_kind::regular_column, id);
if (col.name_as_text() == "c1") {
BOOST_REQUIRE(!c.as_atomic_cell().is_live());
BOOST_REQUIRE(!c.as_atomic_cell(col).is_live());
} else {
BOOST_FAIL(sprint("Unexpected column \'%s\'", col.name_as_text()));
}
@@ -2924,7 +2936,7 @@ SEASTAR_TEST_CASE(test_sstable_max_local_deletion_time) {
mutation m(s, key);
auto c_key = clustering_key::from_exploded(*s, {to_bytes("c1")});
last_expiry = (gc_clock::now() + gc_clock::duration(3600 + i)).time_since_epoch().count();
m.set_clustered_cell(c_key, *s->get_column_definition("r1"), make_atomic_cell(bytes("a"), 3600 + i, last_expiry));
m.set_clustered_cell(c_key, *s->get_column_definition("r1"), make_atomic_cell(utf8_type, bytes("a"), 3600 + i, last_expiry));
mt->apply(std::move(m));
}
auto sst = make_sstable(s, tmpdir_path, 53, la, big);
@@ -2953,7 +2965,7 @@ SEASTAR_TEST_CASE(test_sstable_max_local_deletion_time_2) {
auto add_row = [&now, &mt, &s, &last_expiry] (mutation& m, bytes column_name, uint32_t ttl) {
auto c_key = clustering_key::from_exploded(*s, {column_name});
last_expiry = (now + gc_clock::duration(ttl)).time_since_epoch().count();
m.set_clustered_cell(c_key, *s->get_column_definition("r1"), make_atomic_cell(bytes(""), ttl, last_expiry));
m.set_clustered_cell(c_key, *s->get_column_definition("r1"), make_atomic_cell(utf8_type, bytes(""), ttl, last_expiry));
mt->apply(std::move(m));
};
auto get_usable_sst = [s, tmpdir_path] (memtable& mt, int64_t gen) -> future<sstable_ptr> {
@@ -3417,7 +3429,7 @@ static void test_min_max_clustering_key(schema_ptr s, std::vector<bytes> explode
c_key = clustering_key::from_exploded(*s, exploded_ck);
}
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
};
auto remove_data = [&mt, &s] (std::vector<bytes>& exploded_pk, std::vector<bytes>&& exploded_ck) {
@@ -3515,7 +3527,7 @@ SEASTAR_TEST_CASE(min_max_clustering_key_test_2) {
mutation m(s, key);
for (auto i = 100; i < 150; i++) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes(to_sstring(j) + "ck" + to_sstring(i))});
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
}
mt->apply(std::move(m));
}
@@ -3529,7 +3541,7 @@ SEASTAR_TEST_CASE(min_max_clustering_key_test_2) {
mutation m(s, key);
for (auto i = 101; i < 299; i++) {
auto c_key = clustering_key::from_exploded(*s, {to_bytes(to_sstring(9) + "ck" + to_sstring(i))});
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
}
mt->apply(std::move(m));
auto sst2 = make_sstable(s, tmp->path, 2, la, big);
@@ -3583,7 +3595,7 @@ SEASTAR_TEST_CASE(sstable_tombstone_metadata_check) {
{
auto mt = make_lw_shared<memtable>(s);
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m));
auto sst = make_sstable(s, tmp->path, 3, la, big);
write_memtable_to_sstable_for_test(*mt, sst).get();
@@ -3601,7 +3613,7 @@ SEASTAR_TEST_CASE(sstable_tombstone_metadata_check) {
auto key2 = partition_key::from_exploded(*s, {to_bytes("key2")});
mutation m2(s, key2);
m2.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m2.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mt->apply(std::move(m2));
auto sst = make_sstable(s, tmp->path, 4, la, big);
@@ -4119,7 +4131,7 @@ SEASTAR_TEST_CASE(sstable_expired_data_ratio) {
auto key = partition_key::from_exploded(*s, {k});
mutation m(s, key);
auto c_key = clustering_key::from_exploded(*s, {to_bytes("c1")});
m.set_clustered_cell(c_key, *s->get_column_definition("r1"), make_atomic_cell(bytes("a"), ttl, expiration_time));
m.set_clustered_cell(c_key, *s->get_column_definition("r1"), make_atomic_cell(utf8_type, bytes("a"), ttl, expiration_time));
mt->apply(std::move(m));
};
@@ -4291,7 +4303,7 @@ SEASTAR_TEST_CASE(test_summary_entry_spanning_more_keys_than_min_interval) {
auto key = partition_key::from_exploded(*s, {int32_type->decompose(i)});
auto c_key = clustering_key::from_exploded(*s, {to_bytes("abc")});
mutation m(s, key);
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type->decompose(1)));
m.set_clustered_cell(c_key, r1_col, make_atomic_cell(int32_type, int32_type->decompose(1)));
mutations.push_back(std::move(m));
keys_written++;
}
@@ -4356,19 +4368,20 @@ SEASTAR_TEST_CASE(test_wrong_counter_shard_order) {
sst->load().get0();
auto reader = sstable_reader(sst, s);
auto verify_row = [] (mutation_fragment_opt mfopt, int64_t expected_value) {
auto verify_row = [&s] (mutation_fragment_opt mfopt, int64_t expected_value) {
BOOST_REQUIRE(bool(mfopt));
auto& mf = *mfopt;
BOOST_REQUIRE(mf.is_clustering_row());
auto& row = mf.as_clustering_row();
size_t n = 0;
row.cells().for_each_cell([&] (column_id, const atomic_cell_or_collection& ac_o_c) {
auto acv = ac_o_c.as_atomic_cell();
auto ccv = counter_cell_view(acv);
row.cells().for_each_cell([&] (column_id id, const atomic_cell_or_collection& ac_o_c) {
auto acv = ac_o_c.as_atomic_cell(s->regular_column_at(id));
counter_cell_view::with_linearized(acv, [&] (counter_cell_view ccv) {
counter_shard_view::less_compare_by_id cmp;
BOOST_REQUIRE_MESSAGE(boost::algorithm::is_sorted(ccv.shards(), cmp), ccv << " is expected to be sorted");
BOOST_REQUIRE_EQUAL(ccv.total_value(), expected_value);
n++;
});
});
BOOST_REQUIRE_EQUAL(n, 5);
};
@@ -4575,7 +4588,7 @@ SEASTAR_TEST_CASE(test_old_format_non_compound_range_tombstone_is_read) {
auto dk = dht::global_partitioner().decorate_key(*s, pk);
auto ck = clustering_key::from_exploded(*s, {int32_type->decompose(2)});
mutation m(s, dk);
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell::make_live(1511270919978349, int32_type->decompose(1), { }));
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell::make_live(*int32_type, 1511270919978349, int32_type->decompose(1), { }));
m.partition().apply_delete(*s, ck, {1511270943827278, gc_clock::from_time_t(1511270943)});
{
@@ -4599,7 +4612,7 @@ SEASTAR_TEST_CASE(summary_rebuild_sanity) {
auto make_insert = [&] (partition_key key) {
mutation m(s, key);
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(bytes(1024, 'a')));
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(utf8_type, bytes(1024, 'a')));
return m;
};
@@ -4644,13 +4657,13 @@ SEASTAR_TEST_CASE(sstable_partition_estimation_sanity_test) {
auto make_large_partition = [&] (partition_key key) {
mutation m(s, key);
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(bytes(20 * summary_byte_cost, 'a')));
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(utf8_type, bytes(20 * summary_byte_cost, 'a')));
return m;
};
auto make_small_partition = [&] (partition_key key) {
mutation m(s, key);
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(bytes(100, 'a')));
m.set_clustered_cell(clustering_key::make_empty(), col, make_atomic_cell(utf8_type, bytes(100, 'a')));
return m;
};

64
tests/sstable_mutation_test.cc
View File

@@ -142,14 +142,14 @@ future<mutation> generate_clustered(bytes&& key) {
inline auto clustered_row(mutation& mutation, const schema& s, std::vector<bytes>&& v) {
auto exploded = exploded_clustering_prefix(std::move(v));
auto clustering_pair = clustering_key::from_clustering_prefix(s, exploded);
return mutation.partition().clustered_row(s, clustering_pair);
return deletable_row(s, mutation.partition().clustered_row(s, clustering_pair));
}
SEASTAR_TEST_CASE(complex_sst1_k1) {
return generate_clustered<1>("key1").then([] (auto&& mutation) {
auto s = complex_schema();
auto sr = mutation.partition().static_row();
auto& sr = mutation.partition().static_row();
match_live_cell(sr, *s, "static_obj", data_value(to_bytes("static_value")));
auto row1 = clustered_row(mutation, *s, {"cl1.1", "cl2.1"});
@@ -178,7 +178,7 @@ SEASTAR_TEST_CASE(complex_sst1_k2) {
return generate_clustered<1>("key2").then([] (auto&& mutation) {
auto s = complex_schema();
auto sr = mutation.partition().static_row();
auto& sr = mutation.partition().static_row();
match_live_cell(sr, *s, "static_obj", data_value(to_bytes("static_value")));
auto static_set = match_collection(sr, *s, "static_collection", tombstone(deletion_time{1431451390, 1431451390225257l}));
match_collection_element<status::live>(static_set.cells[0], to_bytes("1"), bytes_opt{});
@@ -227,7 +227,7 @@ SEASTAR_TEST_CASE(complex_sst2_k2) {
return generate_clustered<2>("key2").then([] (auto&& mutation) {
auto s = complex_schema();
auto sr = mutation.partition().static_row();
auto& sr = mutation.partition().static_row();
match_dead_cell(sr, *s, "static_obj");
auto static_set = match_collection(sr, *s, "static_collection", tombstone(deletion_time{0, api::missing_timestamp}));
match_collection_element<status::dead>(static_set.cells[0], to_bytes("1"), bytes_opt{});
@@ -256,7 +256,7 @@ SEASTAR_TEST_CASE(complex_sst2_k3) {
return generate_clustered<2>("key3").then([] (auto&& mutation) {
auto s = complex_schema();
auto sr = mutation.partition().static_row();
auto& sr = mutation.partition().static_row();
match_expiring_cell(sr, *s, "static_obj", data_value(to_bytes("static_value_3")), 1431451394597062l, 1431537794);
auto row1 = clustered_row(mutation, *s, {"tcl1.1", "tcl2.1"});
@@ -294,7 +294,7 @@ SEASTAR_TEST_CASE(complex_sst3_k2) {
return generate_clustered<3>("key2").then([] (auto&& mutation) {
auto s = complex_schema();
auto sr = mutation.partition().static_row();
auto& sr = mutation.partition().static_row();
match_live_cell(sr, *s, "static_obj", data_value(to_bytes("final_static")));
auto row = clustered_row(mutation, *s, {"kcl1.1", "kcl2.1"});
@@ -451,7 +451,7 @@ SEASTAR_TEST_CASE(compact_storage_sparse_read) {
return read_mutation_from_flat_mutation_reader(*rd).then([sstp, s, &key, rd] (auto mutation) {
BOOST_REQUIRE(mutation);
auto& mp = mutation->partition();
auto row = mp.clustered_row(*s, clustering_key::make_empty());
auto& row = mp.clustered_row(*s, clustering_key::make_empty());
match_live_cell(row.cells(), *s, "cl1", data_value(to_bytes("cl1")));
match_live_cell(row.cells(), *s, "cl2", data_value(to_bytes("cl2")));
return make_ready_future<>();
@@ -471,7 +471,7 @@ SEASTAR_TEST_CASE(compact_storage_simple_dense_read) {
auto exploded = exploded_clustering_prefix({"cl1"});
auto clustering = clustering_key::from_clustering_prefix(*s, exploded);
auto row = mp.clustered_row(*s, clustering);
auto& row = mp.clustered_row(*s, clustering);
match_live_cell(row.cells(), *s, "cl2", data_value(to_bytes("cl2")));
return make_ready_future<>();
});
@@ -490,7 +490,7 @@ SEASTAR_TEST_CASE(compact_storage_dense_read) {
auto exploded = exploded_clustering_prefix({"cl1", "cl2"});
auto clustering = clustering_key::from_clustering_prefix(*s, exploded);
auto row = mp.clustered_row(*s, clustering);
auto& row = mp.clustered_row(*s, clustering);
match_live_cell(row.cells(), *s, "cl3", data_value(to_bytes("cl3")));
return make_ready_future<>();
});
@@ -515,10 +515,10 @@ SEASTAR_TEST_CASE(broken_ranges_collection) {
if (!mut) {
return stop_iteration::yes;
} else if (key_equal("127.0.0.1")) {
auto row = mut->partition().clustered_row(*s, clustering_key::make_empty());
auto& row = mut->partition().clustered_row(*s, clustering_key::make_empty());
match_absent(row.cells(), *s, "tokens");
} else if (key_equal("127.0.0.3")) {
auto row = mut->partition().clustered_row(*s, clustering_key::make_empty());
auto& row = mut->partition().clustered_row(*s, clustering_key::make_empty());
auto tokens = match_collection(row.cells(), *s, "tokens", tombstone(deletion_time{0x55E5F2D5, 0x051EB3FC99715Dl }));
match_collection_element<status::live>(tokens.cells[0], to_bytes("-8180144272884242102"), bytes_opt{});
} else {
@@ -610,7 +610,7 @@ SEASTAR_TEST_CASE(tombstone_in_tombstone) {
tombstone(1459334681228103LL, it->tomb.deletion_time))));
auto& rows = mut->partition().clustered_rows();
BOOST_REQUIRE(rows.calculate_size() == 1);
for (auto e : rows) {
for (auto& e : rows) {
BOOST_REQUIRE(e.key().equal(*s, make_ckey("aaa", "bbb")));
BOOST_REQUIRE(e.row().deleted_at().tomb().timestamp == 1459334681244989LL);
}
@@ -755,7 +755,7 @@ SEASTAR_TEST_CASE(tombstone_in_tombstone2) {
BOOST_REQUIRE(it == rts.end());
BOOST_REQUIRE(rows.calculate_size() == 1);
for (auto e : rows) {
for (auto& e : rows) {
BOOST_REQUIRE(e.key().equal(*s, make_ckey("aaa", "bbb", "ccc")));
BOOST_REQUIRE(e.row().deleted_at().tomb().timestamp == 1459438519958850LL);
}
@@ -782,7 +782,7 @@ SEASTAR_TEST_CASE(test_non_compound_table_row_is_not_marked_as_static) {
auto ck = clustering_key::from_exploded(*s, {int32_type->decompose(static_cast<int32_t>(0xffff0000))});
mutation m(s, k);
auto cell = atomic_cell::make_live(1, int32_type->decompose(17), { });
auto cell = atomic_cell::make_live(*int32_type, 1, int32_type->decompose(17), { });
m.set_clustered_cell(ck, *s->get_column_definition("v"), std::move(cell));
auto mt = make_lw_shared<memtable>(s);
@@ -818,20 +818,20 @@ SEASTAR_TEST_CASE(test_promoted_index_blocks_are_monotonic) {
auto s = builder.build();
auto k = partition_key::from_exploded(*s, {to_bytes(make_local_key(s))});
auto cell = atomic_cell::make_live(1, int32_type->decompose(88), { });
auto cell = atomic_cell::make_live(*int32_type, 1, int32_type->decompose(88), { });
mutation m(s, k);
auto ck = clustering_key::from_exploded(*s, {int32_type->decompose(1), int32_type->decompose(2)});
m.set_clustered_cell(ck, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
ck = clustering_key::from_exploded(*s, {int32_type->decompose(1), int32_type->decompose(4)});
m.set_clustered_cell(ck, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
ck = clustering_key::from_exploded(*s, {int32_type->decompose(1), int32_type->decompose(6)});
m.set_clustered_cell(ck, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
ck = clustering_key::from_exploded(*s, {int32_type->decompose(3), int32_type->decompose(9)});
m.set_clustered_cell(ck, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
m.partition().apply_row_tombstone(*s, range_tombstone(
clustering_key_prefix::from_exploded(*s, {int32_type->decompose(1)}),
@@ -870,20 +870,20 @@ SEASTAR_TEST_CASE(test_promoted_index_blocks_are_monotonic_compound_dense) {
auto s = builder.build(schema_builder::compact_storage::yes);
auto dk = dht::global_partitioner().decorate_key(*s, partition_key::from_exploded(*s, {to_bytes(make_local_key(s))}));
auto cell = atomic_cell::make_live(1, int32_type->decompose(88), { });
auto cell = atomic_cell::make_live(*int32_type, 1, int32_type->decompose(88), { });
mutation m(s, dk);
auto ck1 = clustering_key::from_exploded(*s, {int32_type->decompose(1), int32_type->decompose(2)});
m.set_clustered_cell(ck1, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck1, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
auto ck2 = clustering_key::from_exploded(*s, {int32_type->decompose(1), int32_type->decompose(4)});
m.set_clustered_cell(ck2, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck2, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
auto ck3 = clustering_key::from_exploded(*s, {int32_type->decompose(1), int32_type->decompose(6)});
m.set_clustered_cell(ck3, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck3, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
auto ck4 = clustering_key::from_exploded(*s, {int32_type->decompose(3), int32_type->decompose(9)});
m.set_clustered_cell(ck4, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck4, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
m.partition().apply_row_tombstone(*s, range_tombstone(
clustering_key_prefix::from_exploded(*s, {int32_type->decompose(1)}),
@@ -932,17 +932,17 @@ SEASTAR_TEST_CASE(test_promoted_index_blocks_are_monotonic_non_compound_dense) {
auto s = builder.build(schema_builder::compact_storage::yes);
auto dk = dht::global_partitioner().decorate_key(*s, partition_key::from_exploded(*s, {to_bytes(make_local_key(s))}));
auto cell = atomic_cell::make_live(1, int32_type->decompose(88), { });
auto cell = atomic_cell::make_live(*int32_type, 1, int32_type->decompose(88), { });
mutation m(s, dk);
auto ck1 = clustering_key::from_exploded(*s, {int32_type->decompose(1)});
m.set_clustered_cell(ck1, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck1, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
auto ck2 = clustering_key::from_exploded(*s, {int32_type->decompose(2)});
m.set_clustered_cell(ck2, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck2, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
auto ck3 = clustering_key::from_exploded(*s, {int32_type->decompose(3)});
m.set_clustered_cell(ck3, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck3, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
m.partition().apply_row_tombstone(*s, range_tombstone(
clustering_key_prefix::from_exploded(*s, {int32_type->decompose(1)}),
@@ -994,7 +994,7 @@ SEASTAR_TEST_CASE(test_promoted_index_repeats_open_tombstones) {
auto s = builder.build(compact);
auto dk = dht::global_partitioner().decorate_key(*s, partition_key::from_exploded(*s, {to_bytes(make_local_key(s))}));
auto cell = atomic_cell::make_live(1, int32_type->decompose(88), { });
auto cell = atomic_cell::make_live(*int32_type, 1, int32_type->decompose(88), { });
mutation m(s, dk);
m.partition().apply_row_tombstone(*s, range_tombstone(
@@ -1005,7 +1005,7 @@ SEASTAR_TEST_CASE(test_promoted_index_repeats_open_tombstones) {
{1, gc_clock::now()}));
auto ck = clustering_key::from_exploded(*s, {bytes_type->decompose(data_value(to_bytes("ck3")))});
m.set_clustered_cell(ck, *s->get_column_definition("v"), cell);
m.set_clustered_cell(ck, *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
auto mt = make_lw_shared<memtable>(s);
mt->apply(m);
@@ -1089,8 +1089,8 @@ SEASTAR_TEST_CASE(test_promoted_index_is_absent_for_schemas_without_clustering_k
auto dk = dht::global_partitioner().decorate_key(*s, partition_key::from_exploded(*s, {to_bytes(make_local_key(s))}));
mutation m(s, dk);
for (auto&& v : { 1, 2, 3, 4 }) {
auto cell = atomic_cell::make_live(1, int32_type->decompose(v), { });
m.set_clustered_cell(clustering_key_prefix::make_empty(), *s->get_column_definition("v"), cell);
auto cell = atomic_cell::make_live(*int32_type, 1, int32_type->decompose(v), { });
m.set_clustered_cell(clustering_key_prefix::make_empty(), *s->get_column_definition("v"), atomic_cell(*int32_type, cell));
}
auto mt = make_lw_shared<memtable>(s);
mt->apply(m);

2
tests/sstable_test.cc
View File

@@ -1246,7 +1246,7 @@ SEASTAR_TEST_CASE(promoted_index_write) {
clustering_key::from_exploded(
*s, {to_bytes(sprint("%d%c%c", k, i, j))}));
row.cells().apply(*col,
atomic_cell::make_live(2345,
atomic_cell::make_live(*col->type, 2345,
col->type->decompose(sstring(sprint("z%c",i)))));
row.apply(row_marker(1234));
}

16
tests/sstable_test.hh
View File

@@ -538,7 +538,7 @@ enum class status {
ttl,
};
inline bool check_status_and_done(const atomic_cell &c, status expected) {
inline bool check_status_and_done(atomic_cell_view c, status expected) {
if (expected == status::dead) {
BOOST_REQUIRE(c.is_live() == false);
return true;
@@ -553,13 +553,15 @@ inline void match(const row& row, const schema& s, bytes col, const data_value&
auto cdef = s.get_column_definition(col);
BOOST_CHECK_NO_THROW(row.cell_at(cdef->id));
auto c = row.cell_at(cdef->id).as_atomic_cell();
auto c = row.cell_at(cdef->id).as_atomic_cell(*cdef);
if (check_status_and_done(c, Status)) {
return;
}
auto expected = cdef->type->decompose(value);
BOOST_REQUIRE(c.value() == expected);
auto val = c.value().linearize();
assert(val == expected);
BOOST_REQUIRE(c.value().linearize() == expected);
if (timestamp) {
BOOST_REQUIRE(c.timestamp() == timestamp);
}
@@ -592,9 +594,11 @@ match_collection(const row& row, const schema& s, bytes col, const tombstone& t)
BOOST_CHECK_NO_THROW(row.cell_at(cdef->id));
auto c = row.cell_at(cdef->id).as_collection_mutation();
auto ctype = static_pointer_cast<const collection_type_impl>(cdef->type);
auto&& mut = ctype->deserialize_mutation_form(c);
return c.data.with_linearized([&] (bytes_view c_bv) {
auto&& mut = ctype->deserialize_mutation_form(c_bv);
BOOST_REQUIRE(mut.tomb == t);
return mut.materialize();
return mut.materialize(*ctype);
});
}
template <status Status>
@@ -612,7 +616,7 @@ inline void match_collection_element(const std::pair<bytes, atomic_cell>& elemen
// the schema for the set type, and is enough for the purposes of this
// test.
if (expected_serialized_value) {
BOOST_REQUIRE(element.second.value() == *expected_serialized_value);
BOOST_REQUIRE(element.second.value().linearize() == *expected_serialized_value);
}
}

2
thrift/handler.cc
View File

@@ -1795,7 +1795,7 @@ private:
}
static void add_live_cell(const schema& s, const Column& col, const column_definition& def, clustering_key_prefix ckey, mutation& m_to_apply) {
thrift_validation::validate_column(col, def);
auto cell = atomic_cell::make_live(col.timestamp, to_bytes_view(col.value), maybe_ttl(s, col));
auto cell = atomic_cell::make_live(*def.type, col.timestamp, to_bytes_view(col.value), maybe_ttl(s, col));
m_to_apply.set_clustered_cell(std::move(ckey), def, std::move(cell));
}
static void add_live_cell(const schema& s, const CounterColumn& col, const column_definition& def, clustering_key_prefix ckey, mutation& m_to_apply) {

109
types.cc
View File

@@ -110,6 +110,7 @@ static const char* empty_type_name = "org.apache.cassandra.db.marshal.EmptyT
template<typename T>
struct simple_type_traits {
static constexpr size_t serialized_size = sizeof(T);
static T read_nonempty(bytes_view v) {
return read_simple_exactly<T>(v);
}
@@ -117,6 +118,7 @@ struct simple_type_traits {
template<>
struct simple_type_traits<bool> {
static constexpr size_t serialized_size = 1;
static bool read_nonempty(bytes_view v) {
return read_simple_exactly<int8_t>(v) != 0;
}
@@ -124,6 +126,7 @@ struct simple_type_traits<bool> {
template<>
struct simple_type_traits<db_clock::time_point> {
static constexpr size_t serialized_size = sizeof(uint64_t);
static db_clock::time_point read_nonempty(bytes_view v) {
return db_clock::time_point(db_clock::duration(read_simple_exactly<int64_t>(v)));
}
@@ -132,7 +135,7 @@ struct simple_type_traits<db_clock::time_point> {
template <typename T>
struct simple_type_impl : concrete_type<T> {
simple_type_impl(sstring name, std::optional<uint32_t> value_length_if_fixed)
: concrete_type<T>(std::move(name), std::move(value_length_if_fixed)) {}
: concrete_type<T>(std::move(name), std::move(value_length_if_fixed), data::type_info::make_fixed_size(simple_type_traits<T>::serialized_size)) {}
virtual int32_t compare(bytes_view v1, bytes_view v2) const override {
if (v1.empty()) {
return v2.empty() ? 0 : -1;
@@ -299,7 +302,7 @@ struct long_type_impl : integer_type_impl<int64_t> {
struct string_type_impl : public concrete_type<sstring> {
string_type_impl(sstring name)
: concrete_type(name, { }) {}
: concrete_type(name, { }, data::type_info::make_variable_size()) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -379,7 +382,7 @@ struct utf8_type_impl final : public string_type_impl {
};
struct bytes_type_impl final : public concrete_type<bytes> {
bytes_type_impl() : concrete_type(bytes_type_name, { }) {}
bytes_type_impl() : concrete_type(bytes_type_name, { }, data::type_info::make_variable_size()) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -522,7 +525,7 @@ struct boolean_type_impl : public simple_type_impl<bool> {
class date_type_impl : public concrete_type<db_clock::time_point> {
static logging::logger _logger;
public:
date_type_impl() : concrete_type(date_type_name, 8) {}
date_type_impl() : concrete_type(date_type_name, 8, data::type_info::make_fixed_size(sizeof(uint64_t))) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -600,7 +603,7 @@ public:
logging::logger date_type_impl::_logger(date_type_name);
struct timeuuid_type_impl : public concrete_type<utils::UUID> {
timeuuid_type_impl() : concrete_type<utils::UUID>(timeuuid_type_name, 16) {}
timeuuid_type_impl() : concrete_type<utils::UUID>(timeuuid_type_name, 16, data::type_info::make_fixed_size(sizeof(uint64_t) * 2)) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -1084,7 +1087,7 @@ struct time_type_impl : public simple_type_impl<int64_t> {
};
struct uuid_type_impl : concrete_type<utils::UUID> {
uuid_type_impl() : concrete_type(uuid_type_name, 16) {}
uuid_type_impl() : concrete_type(uuid_type_name, 16, data::type_info::make_fixed_size(sizeof(uint64_t) * 2)) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -1177,7 +1180,7 @@ struct uuid_type_impl : concrete_type<utils::UUID> {
using inet_address = seastar::net::inet_address;
struct inet_addr_type_impl : concrete_type<inet_address> {
inet_addr_type_impl() : concrete_type<inet_address>(inet_addr_type_name, { }) {}
inet_addr_type_impl() : concrete_type<inet_address>(inet_addr_type_name, { }, data::type_info::make_variable_size()) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -1295,6 +1298,7 @@ template <> struct int_of_size<float> :
template <typename T>
struct float_type_traits {
static constexpr size_t serialized_size = sizeof(typename int_of_size<T>::itype);
static double read_nonempty(bytes_view v) {
union {
T d;
@@ -1447,7 +1451,7 @@ struct float_type_impl : floating_type_impl<float> {
class varint_type_impl : public concrete_type<boost::multiprecision::cpp_int> {
public:
varint_type_impl() : concrete_type{varint_type_name, { }} { }
varint_type_impl() : concrete_type{varint_type_name, { }, data::type_info::make_variable_size()} { }
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -1570,7 +1574,7 @@ public:
class decimal_type_impl : public concrete_type<big_decimal> {
public:
decimal_type_impl() : concrete_type{decimal_type_name, { }} { }
decimal_type_impl() : concrete_type{decimal_type_name, { }, data::type_info::make_variable_size()} { }
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (!value) {
return;
@@ -1676,7 +1680,7 @@ public:
class counter_type_impl : public abstract_type {
public:
counter_type_impl() : abstract_type{counter_type_name, { }} { }
counter_type_impl() : abstract_type{counter_type_name, { }, data::type_info::make_variable_size()} { }
virtual void serialize(const void* value, bytes::iterator& out) const override {
fail(unimplemented::cause::COUNTERS);
}
@@ -1761,7 +1765,7 @@ auto generate_tuple_from_index(std::index_sequence<Ts...>, Function&& f) {
class duration_type_impl : public concrete_type<cql_duration> {
public:
duration_type_impl() : concrete_type(duration_type_name, { }) {
duration_type_impl() : concrete_type(duration_type_name, { }, data::type_info::make_variable_size()) {
}
virtual void serialize(const void* value, bytes::iterator& out) const override {
if (value == nullptr) {
@@ -1912,7 +1916,7 @@ private:
};
struct empty_type_impl : abstract_type {
empty_type_impl() : abstract_type(empty_type_name, 0) {}
empty_type_impl() : abstract_type(empty_type_name, 0, data::type_info::make_fixed_size(0)) {}
virtual void serialize(const void* value, bytes::iterator& out) const override {
}
virtual size_t serialized_size(const void* value) const override {
@@ -2094,16 +2098,18 @@ collection_type_impl::as_cql3_type() const {
bytes
collection_type_impl::to_value(collection_mutation_view mut, cql_serialization_format sf) const {
return to_value(deserialize_mutation_form(mut), sf);
return mut.data.with_linearized([&] (bytes_view bv) {
return to_value(deserialize_mutation_form(bv), sf);
});
}
collection_type_impl::mutation
collection_type_impl::mutation_view::materialize() const {
collection_type_impl::mutation_view::materialize(const collection_type_impl& ctype) const {
collection_type_impl::mutation m;
m.tomb = tomb;
m.cells.reserve(cells.size());
for (auto&& e : cells) {
m.cells.emplace_back(bytes(e.first.begin(), e.first.end()), e.second);
m.cells.emplace_back(bytes(e.first.begin(), e.first.end()), atomic_cell(*ctype.value_comparator(), e.second));
}
return m;
}
@@ -2418,12 +2424,19 @@ map_type_impl::serialized_values(std::vector<atomic_cell> cells) const {
bytes
map_type_impl::to_value(mutation_view mut, cql_serialization_format sf) const {
std::vector<bytes> linearized;
std::vector<bytes_view> tmp;
tmp.reserve(mut.cells.size() * 2);
for (auto&& e : mut.cells) {
if (e.second.is_live(mut.tomb, false)) {
tmp.emplace_back(e.first);
tmp.emplace_back(e.second.value());
auto value_view = e.second.value();
if (value_view.is_fragmented()) {
auto& v = linearized.emplace_back(value_view.linearize());
tmp.emplace_back(v);
} else {
tmp.emplace_back(value_view.first_fragment());
}
}
}
return pack(tmp.begin(), tmp.end(), tmp.size() / 2, sf);
@@ -2473,8 +2486,7 @@ bool map_type_impl::references_duration() const {
return _keys->references_duration() || _values->references_duration();
}
auto collection_type_impl::deserialize_mutation_form(collection_mutation_view cm) const -> mutation_view {
auto&& in = cm.data;
auto collection_type_impl::deserialize_mutation_form(bytes_view in) const -> mutation_view {
mutation_view ret;
auto has_tomb = read_simple<bool>(in);
if (has_tomb) {
@@ -2489,7 +2501,8 @@ auto collection_type_impl::deserialize_mutation_form(collection_mutation_view cm
auto ksize = read_simple<uint32_t>(in);
auto key = read_simple_bytes(in, ksize);
auto vsize = read_simple<uint32_t>(in);
auto value = atomic_cell_view::from_bytes(read_simple_bytes(in, vsize));
// value_comparator(), ugh
auto value = atomic_cell_view::from_bytes(value_comparator()->imr_state().type_info(), read_simple_bytes(in, vsize));
ret.cells.emplace_back(key, value);
}
assert(in.empty());
@@ -2497,13 +2510,14 @@ auto collection_type_impl::deserialize_mutation_form(collection_mutation_view cm
}
bool collection_type_impl::is_empty(collection_mutation_view cm) const {
auto&& in = cm.data;
return cm.data.with_linearized([&] (bytes_view in) { // FIXME: we can guarantee that this is in the first fragment
auto has_tomb = read_simple<bool>(in);
return !has_tomb && read_simple<uint32_t>(in) == 0;
});
}
bool collection_type_impl::is_any_live(collection_mutation_view cm, tombstone tomb, gc_clock::time_point now) const {
auto&& in = cm.data;
return cm.data.with_linearized([&] (bytes_view in) {
auto has_tomb = read_simple<bool>(in);
if (has_tomb) {
auto ts = read_simple<api::timestamp_type>(in);
@@ -2515,16 +2529,17 @@ bool collection_type_impl::is_any_live(collection_mutation_view cm, tombstone to
auto ksize = read_simple<uint32_t>(in);
in.remove_prefix(ksize);
auto vsize = read_simple<uint32_t>(in);
auto value = atomic_cell_view::from_bytes(read_simple_bytes(in, vsize));
auto value = atomic_cell_view::from_bytes(value_comparator()->imr_state().type_info(), read_simple_bytes(in, vsize));
if (value.is_live(tomb, now, false)) {
return true;
}
}
return false;
});
}
api::timestamp_type collection_type_impl::last_update(collection_mutation_view cm) const {
auto&& in = cm.data;
return cm.data.with_linearized([&] (bytes_view in) {
api::timestamp_type max = api::missing_timestamp;
auto has_tomb = read_simple<bool>(in);
if (has_tomb) {
@@ -2536,15 +2551,17 @@ api::timestamp_type collection_type_impl::last_update(collection_mutation_view c
auto ksize = read_simple<uint32_t>(in);
in.remove_prefix(ksize);
auto vsize = read_simple<uint32_t>(in);
auto value = atomic_cell_view::from_bytes(read_simple_bytes(in, vsize));
auto value = atomic_cell_view::from_bytes(value_comparator()->imr_state().type_info(), read_simple_bytes(in, vsize));
max = std::max(value.timestamp(), max);
}
return max;
});
}
template <typename Iterator>
collection_mutation
do_serialize_mutation_form(
const collection_type_impl& ctype,
const tombstone& tomb,
boost::iterator_range<Iterator> cells) {
auto element_size = [] (size_t c, auto&& e) -> size_t {
@@ -2572,9 +2589,10 @@ do_serialize_mutation_form(
auto&& k = kv.first;
auto&& v = kv.second;
writeb(k);
writeb(v.serialize());
}
return collection_mutation{std::move(ret)};
return collection_mutation(ctype, ret);
}
bool collection_type_impl::mutation::compact_and_expire(row_tombstone base_tomb, gc_clock::time_point query_time,
@@ -2614,26 +2632,28 @@ bool collection_type_impl::mutation::compact_and_expire(row_tombstone base_tomb,
}
collection_mutation
collection_type_impl::serialize_mutation_form(const mutation& mut) {
return do_serialize_mutation_form(mut.tomb, boost::make_iterator_range(mut.cells.begin(), mut.cells.end()));
collection_type_impl::serialize_mutation_form(const mutation& mut) const {
return do_serialize_mutation_form(*this, mut.tomb, boost::make_iterator_range(mut.cells.begin(), mut.cells.end()));
}
collection_mutation
collection_type_impl::serialize_mutation_form(mutation_view mut) {
return do_serialize_mutation_form(mut.tomb, boost::make_iterator_range(mut.cells.begin(), mut.cells.end()));
collection_type_impl::serialize_mutation_form(mutation_view mut) const {
return do_serialize_mutation_form(*this, mut.tomb, boost::make_iterator_range(mut.cells.begin(), mut.cells.end()));
}
collection_mutation
collection_type_impl::serialize_mutation_form_only_live(mutation_view mut, gc_clock::time_point now) {
return do_serialize_mutation_form(mut.tomb, mut.cells | boost::adaptors::filtered([t = mut.tomb, now] (auto&& e) {
collection_type_impl::serialize_mutation_form_only_live(mutation_view mut, gc_clock::time_point now) const {
return do_serialize_mutation_form(*this, mut.tomb, mut.cells | boost::adaptors::filtered([t = mut.tomb, now] (auto&& e) {
return e.second.is_live(t, now, false);
}));
}
collection_mutation
collection_type_impl::merge(collection_mutation_view a, collection_mutation_view b) const {
auto aa = deserialize_mutation_form(a);
auto bb = deserialize_mutation_form(b);
return a.data.with_linearized([&] (bytes_view a_in) {
return b.data.with_linearized([&] (bytes_view b_in) {
auto aa = deserialize_mutation_form(a_in);
auto bb = deserialize_mutation_form(b_in);
mutation_view merged;
merged.cells.reserve(aa.cells.size() + bb.cells.size());
using element_type = std::pair<bytes_view, atomic_cell_view>;
@@ -2667,13 +2687,17 @@ collection_type_impl::merge(collection_mutation_view a, collection_mutation_view
merge);
merged.tomb = std::max(aa.tomb, bb.tomb);
return serialize_mutation_form(merged);
});
});
}
collection_mutation
collection_type_impl::difference(collection_mutation_view a, collection_mutation_view b) const
{
auto aa = deserialize_mutation_form(a);
auto bb = deserialize_mutation_form(b);
return a.data.with_linearized([&] (bytes_view a_in) {
return b.data.with_linearized([&] (bytes_view b_in) {
auto aa = deserialize_mutation_form(a_in);
auto bb = deserialize_mutation_form(b_in);
mutation_view diff;
diff.cells.reserve(std::max(aa.cells.size(), bb.cells.size()));
auto key_type = name_comparator();
@@ -2693,6 +2717,8 @@ collection_type_impl::difference(collection_mutation_view a, collection_mutation
diff.tomb = aa.tomb;
}
return serialize_mutation_form(diff);
});
});
}
bytes_opt
@@ -3161,11 +3187,18 @@ list_type_impl::serialized_values(std::vector<atomic_cell> cells) const {
bytes
list_type_impl::to_value(mutation_view mut, cql_serialization_format sf) const {
std::vector<bytes> linearized;
std::vector<bytes_view> tmp;
tmp.reserve(mut.cells.size());
for (auto&& e : mut.cells) {
if (e.second.is_live(mut.tomb, false)) {
tmp.emplace_back(e.second.value());
auto value_view = e.second.value();
if (value_view.is_fragmented()) {
auto& v = linearized.emplace_back(value_view.linearize());
tmp.emplace_back(v);
} else {
tmp.emplace_back(value_view.first_fragment());
}
}
}
return pack(tmp.begin(), tmp.end(), tmp.size(), sf);
@@ -3196,14 +3229,14 @@ bool list_type_impl::references_duration() const {
}
tuple_type_impl::tuple_type_impl(sstring name, std::vector<data_type> types)
: concrete_type(std::move(name), { }), _types(std::move(types)) {
: concrete_type(std::move(name), { }, data::type_info::make_variable_size()), _types(std::move(types)) {
for (auto& t : _types) {
t = t->freeze();
}
}
tuple_type_impl::tuple_type_impl(std::vector<data_type> types)
: concrete_type(make_name(types), { }), _types(std::move(types)) {
: concrete_type(make_name(types), { }, data::type_info::make_variable_size()), _types(std::move(types)) {
for (auto& t : _types) {
t = t->freeze();
}

26
types.hh
View File

@@ -24,6 +24,7 @@
#include <experimental/optional>
#include <boost/functional/hash.hpp>
#include <iosfwd>
#include "data/cell.hh"
#include <sstream>
#include "core/sstring.hh"
@@ -421,10 +422,12 @@ class user_type_impl;
class abstract_type : public enable_shared_from_this<abstract_type> {
sstring _name;
std::optional<uint32_t> _value_length_if_fixed;
data::type_imr_descriptor _imr_state;
public:
abstract_type(sstring name, std::optional<uint32_t> value_length_if_fixed)
: _name(name), _value_length_if_fixed(std::move(value_length_if_fixed)) {}
abstract_type(sstring name, std::optional<uint32_t> value_length_if_fixed, data::type_info ti)
: _name(name), _value_length_if_fixed(std::move(value_length_if_fixed)), _imr_state(ti) {}
virtual ~abstract_type() {}
const data::type_imr_descriptor& imr_state() const { return _imr_state; }
virtual void serialize(const void* value, bytes::iterator& out) const = 0;
void serialize(const data_value& value, bytes::iterator& out) const {
return serialize(get_value_ptr(value), out);
@@ -781,7 +784,7 @@ public:
protected:
explicit collection_type_impl(sstring name, const kind& k)
: abstract_type(std::move(name), {}), _kind(k) {}
: abstract_type(std::move(name), {}, data::type_info::make_collection()), _kind(k) {}
virtual sstring cql3_type_name() const = 0;
public:
// representation of a collection mutation, key/value pairs, value is a mutation itself
@@ -796,7 +799,7 @@ public:
struct mutation_view {
tombstone tomb;
std::vector<std::pair<bytes_view, atomic_cell_view>> cells;
mutation materialize() const;
mutation materialize(const collection_type_impl&) const;
};
virtual data_type name_comparator() const = 0;
virtual data_type value_comparator() const = 0;
@@ -815,26 +818,29 @@ public:
virtual shared_ptr<cql3::cql3_type> as_cql3_type() const override;
template <typename BytesViewIterator>
static bytes pack(BytesViewIterator start, BytesViewIterator finish, int elements, cql_serialization_format sf);
mutation_view deserialize_mutation_form(collection_mutation_view in) const;
// requires linearized collection_mutation_view, lifetime
mutation_view deserialize_mutation_form(bytes_view in) const;
bool is_empty(collection_mutation_view in) const;
bool is_any_live(collection_mutation_view in, tombstone tomb = tombstone(), gc_clock::time_point now = gc_clock::time_point::min()) const;
api::timestamp_type last_update(collection_mutation_view in) const;
virtual bytes to_value(mutation_view mut, cql_serialization_format sf) const = 0;
bytes to_value(collection_mutation_view mut, cql_serialization_format sf) const;
// FIXME: use iterators?
static collection_mutation serialize_mutation_form(const mutation& mut);
static collection_mutation serialize_mutation_form(mutation_view mut);
static collection_mutation serialize_mutation_form_only_live(mutation_view mut, gc_clock::time_point now);
collection_mutation serialize_mutation_form(const mutation& mut) const;
collection_mutation serialize_mutation_form(mutation_view mut) const;
collection_mutation serialize_mutation_form_only_live(mutation_view mut, gc_clock::time_point now) const;
collection_mutation merge(collection_mutation_view a, collection_mutation_view b) const;
collection_mutation difference(collection_mutation_view a, collection_mutation_view b) const;
// Calls Func(atomic_cell_view) for each cell in this collection.
// noexcept if Func doesn't throw.
template<typename Func>
void for_each_cell(collection_mutation_view c, Func&& func) const {
auto m_view = deserialize_mutation_form(std::move(c));
c.data.with_linearized([&] (bytes_view c_bv) {
auto m_view = deserialize_mutation_form(c_bv);
for (auto&& c : m_view.cells) {
func(std::move(c.second));
}
});
}
virtual void serialize(const void* value, bytes::iterator& out, cql_serialization_format sf) const = 0;
virtual data_value deserialize(bytes_view v, cql_serialization_format sf) const = 0;
@@ -911,7 +917,7 @@ class reversed_type_impl : public abstract_type {
data_type _underlying_type;
reversed_type_impl(data_type t)
: abstract_type("org.apache.cassandra.db.marshal.ReversedType(" + t->name() + ")",
t->value_length_if_fixed())
t->value_length_if_fixed(), t->imr_state().type_info())
, _underlying_type(t)
{}
protected:

105
utils/fragment_range.hh Normal file
View File

@@ -0,0 +1,105 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
#include <seastar/util/gcc6-concepts.hh>
#include "bytes.hh"
enum class mutable_view { no, yes, };
GCC6_CONCEPT(
/// Fragmented buffer
///
/// Concept `FragmentedBuffer` is satisfied by any class that is a range of
/// fragments and provides a method `size_bytes()` which returns the total
/// size of the buffer. The interfaces accepting `FragmentedBuffer` will attempt
/// to avoid unnecessary linearisation.
template<typename T>
concept bool FragmentRange = requires (T range) {
typename T::fragment_type;
requires std::is_same_v<typename T::fragment_type, bytes_view>
|| std::is_same_v<typename T::fragment_type, bytes_mutable_view>;
{ *range.begin() } -> typename T::fragment_type;
{ *range.end() } -> typename T::fragment_type;
{ range.size_bytes() } -> size_t;
{ range.empty() } -> bool; // returns true iff size_bytes() == 0.
};
)
template<typename T, typename = void>
struct is_fragment_range : std::false_type { };
template<typename T>
struct is_fragment_range<T, std::void_t<typename T::fragment_type>> : std::true_type { };
template<typename T>
static constexpr bool is_fragment_range_v = is_fragment_range<T>::value;
/// Single-element fragment range
///
/// This is a helper that allows converting a bytes_view into a FragmentRange.
template<mutable_view is_mutable>
class single_fragment_range {
public:
using fragment_type = std::conditional_t<is_mutable == mutable_view::no,
bytes_view, bytes_mutable_view>;
private:
fragment_type _view;
public:
using iterator = const fragment_type*;
using const_iterator = const fragment_type*;
explicit single_fragment_range(fragment_type f) : _view { f } { }
const_iterator begin() const { return &_view; }
const_iterator end() const { return &_view + 1; }
size_t size_bytes() const { return _view.size(); }
bool empty() const { return _view.empty(); }
};
single_fragment_range(bytes_view) -> single_fragment_range<mutable_view::no>;
single_fragment_range(bytes_mutable_view) -> single_fragment_range<mutable_view::yes>;
/// Empty fragment range.
struct empty_fragment_range {
using fragment_type = bytes_view;
using iterator = bytes_view*;
using const_iterator = bytes_view*;
iterator begin() const { return nullptr; }
iterator end() const { return nullptr; }
size_t size_bytes() const { return 0; }
bool empty() const { return true; }
};
GCC6_CONCEPT(
static_assert(FragmentRange<empty_fragment_range>);
static_assert(FragmentRange<single_fragment_range<mutable_view::no>>);
static_assert(FragmentRange<single_fragment_range<mutable_view::yes>>);
)

170
utils/meta.hh Normal file
View File

@@ -0,0 +1,170 @@
/*
* Copyright (C) 2018 ScyllaDB
*/
/*
* This file is part of Scylla.
*
* Scylla is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* Scylla is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with Scylla. If not, see <http://www.gnu.org/licenses/>.
*/
#pragma once
namespace meta {
// Wrappers that allows returning a list of types. All helpers defined in this
// file accept both unpacked and packed lists of types.
template<typename... Ts>
struct list { };
namespace internal {
template<bool... Vs>
constexpr ssize_t do_find_if_unpacked() {
ssize_t i = -1;
ssize_t j = 0;
(..., ((Vs && i == -1) ? i = j : j++));
return i;
}
template<ssize_t N>
struct negative_to_empty : std::integral_constant<size_t, N> { };
template<>
struct negative_to_empty<-1> { };
template<typename T>
struct is_same_as {
template<typename U>
using type = std::is_same<T, U>;
};
template<template<class> typename Predicate, typename... Ts>
struct do_find_if : internal::negative_to_empty<internal::do_find_if_unpacked<Predicate<Ts>::value...>()> { };
template<template<class> typename Predicate, typename... Ts>
struct do_find_if<Predicate, meta::list<Ts...>> : internal::negative_to_empty<internal::do_find_if_unpacked<Predicate<Ts>::value...>()> { };
}
// Returns the index of the first type in the list of types list of types Ts for
// which Predicate<T::value is true.
template<template<class> typename Predicate, typename... Ts>
constexpr size_t find_if = internal::do_find_if<Predicate, Ts...>::value;
// Returns the index of the first occurrence of type T in the list of types Ts.
template<typename T, typename... Ts>
constexpr size_t find = find_if<internal::is_same_as<T>::template type, Ts...>;
namespace internal {
template<size_t N, typename... Ts>
struct do_get_unpacked { };
template<size_t N, typename T, typename... Ts>
struct do_get_unpacked<N, T, Ts...> : do_get_unpacked<N - 1, Ts...> { };
template<typename T, typename... Ts>
struct do_get_unpacked<0, T, Ts...> {
using type = T;
};
template<size_t N, typename... Ts>
struct do_get : do_get_unpacked<N, Ts...> { };
template<size_t N, typename... Ts>
struct do_get<N, meta::list<Ts...>> : do_get_unpacked<N, Ts...> { };
}
// Returns the Nth type in the provided list of types.
template<size_t N, typename... Ts>
using get = typename internal::do_get<N, Ts...>::type;
namespace internal {
template<size_t N, typename Result, typename... Ts>
struct do_take_unpacked { };
template<typename... Ts>
struct do_take_unpacked<0, list<Ts...>> {
using type = list<Ts...>;
};
template<typename... Ts, typename U, typename... Us>
struct do_take_unpacked<0, list<Ts...>, U, Us...> {
using type = list<Ts...>;
};
template<size_t N, typename... Ts, typename U, typename... Us>
struct do_take_unpacked<N, list<Ts...>, U, Us...> {
using type = typename do_take_unpacked<N - 1, list<Ts..., U>, Us...>::type;
};
template<size_t N, typename Result, typename... Ts>
struct do_take : do_take_unpacked<N, Result, Ts...> { };
template<size_t N, typename Result, typename... Ts>
struct do_take<N, Result, meta::list<Ts...>> : do_take_unpacked<N, Result, Ts...> { };
}
// Returns a list containing N first elements of the provided list of types.
template<size_t N, typename... Ts>
using take = typename internal::do_take<N, list<>, Ts...>::type;
namespace internal {
template<typename... Ts>
struct do_for_each_unpacked {
template<typename Function>
static constexpr void run(Function&& fn) {
(..., fn(static_cast<Ts*>(nullptr)));
}
};
template<typename... Ts>
struct do_for_each : do_for_each_unpacked<Ts...> { };
template<typename... Ts>
struct do_for_each<meta::list<Ts...>> : do_for_each_unpacked<Ts...> { };
}
// Executes the provided function for each element in the provided list of
// types. For each type T the Function is called with an argument of type T*.
template<typename... Ts, typename Function>
constexpr void for_each(Function&& fn) {
internal::do_for_each<Ts...>::run(std::forward<Function>(fn));
};
namespace internal {
template<typename... Ts>
struct get_size : std::integral_constant<size_t, sizeof...(Ts)> { };
template<typename... Ts>
struct get_size<meta::list<Ts...>> : std::integral_constant<size_t, sizeof...(Ts)> { };
}
// Returns the size of a list of types.
template<typename... Ts>
constexpr size_t size = internal::get_size<Ts...>::value;
template<template <class> typename Predicate, typename... Ts>
static constexpr const bool all_of = std::conjunction_v<Predicate<Ts>...>;
}