bahman/scylla
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

sstables: use "me" sstable format by default

Browse Source 
in 7952200c, we changed the `selected_format` from `mc` to `me`,
but to be backward compatible the cluster starts with "md", so
when the nodes in cluster agree on the "ME_SSTABLE_FORMAT" feature,
the format selector believes that the node is already using "ME",
which is specified by `_selected_format`. even it is actually still
using "md", which is specified by `sstable_manager::_format`, as
changed by 54d49c04. as explained above, it was specified to "md"
in hope to be backward compatible when upgrading from an existign
installation which might be still using "md". but after a second
thought, since we are able to read sstables persisted with older
formats, this concern is not valid.

in other words, 7952200c introduced a regression which changed the
"default" sstable format from `me` to `md`.

to address this, we just change `sstable_manager::_format` to "me",
so that all sstables are created using "me" format.

a test is added accordingly.

Fixes #18995
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
(cherry picked from commit 5a0d30f34548fdeaf30692cebb819987f541f087)

Closes scylladb/scylladb#19422
This commit is contained in:
Kefu Chai
2024-06-14 16:54:08 +08:00
committed by Avi Kivity
parent 1f2bbf52cc
commit 1b2f10a4e7
2 changed files with 25 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
sstables/sstables_manager.hh
View File

@@ -93,7 +93,7 @@ private:
// that format. read_sstables_format() also overwrites _sstables_format
// if an sstable format was chosen earlier (and this choice was persisted
// in the system table).
sstable_version_types _format = sstable_version_types::md;
sstable_version_types _format = sstable_version_types::me;
// _active and _undergoing_close are used in scylla-gdb.py to fetch all sstables
// on current shard using "scylla sstables" command. If those fields are renamed,

24
test/cql-pytest/test_tools.py
View File

@@ -13,6 +13,7 @@ import functools
import json
import nodetool
import os
import pathlib
import pytest
import subprocess
import tempfile
@@ -1221,3 +1222,26 @@ def test_scylla_sstable_bad_scylla_yaml(cql, test_keyspace, scylla_path, scylla_
"error processing configuration item: Unknown option : foo",
"Successfully read scylla.yaml from"):
assert any(map(lambda stderr_line: expected_msg in stderr_line, stderr_lines))
def test_scylla_sstable_format_version(cql, test_keyspace, scylla_data_dir):
# Reproduces https://github.com/scylladb/scylladb/issues/16551
#
# an sstable component filename looks like:
# me-3g8w_00qf_4pbog2i7h2c7am0uoe-big-Data.db
sstable_re = re.compile(r"""(?P<version>la|m[cde])- # the sstable version
(?P<id>[^-]+)- # sstable identifier
(?P<format>\w+)- # format: 'big'
(?P<component>.*) # component: e.g., 'Data'""", re.X)
with scylla_sstable(simple_clustering_table, cql, test_keyspace, scylla_data_dir) as (_, sstables):
for fn in sstables:
stem = pathlib.Path(fn).stem
matched = sstable_re.match(stem)
assert matched is not None, f"unmatched sstable component path: {fn}"
sstable_version = matched["version"]
# "me" is specified by sstables_manager::_format, so new sstables
# created by a scylla instance are always persisted with "me" sstable
# format, unless the "sstable_format" setting persisted in
# "system.scylla_local" system tables has a different setting. but in a
# new installation of scylla, this setting does not exist.
assert sstable_version == "me", f"unexpected sstable format: {sstable_version}"