commitlog: Implement new format using CRC:ed sectors

Breaks the file into individually tagged + crc:ed pages.
Each page (sized as disk write alignment) gets a trailing
12-byte metadata, including CRC of the first page-12 bytes,
and the ID of the segment being written.

When reading, each page read is CRC:ed and checked to be part
of the expected segment by comparing ID:s. If crc is broken,
we have broken data. If crc is ok, but ID does not match, we
have a prematurely terminated segment (truncated), which, depending
on whether we use batch mode or not, implied data loss.
This commit is contained in:
Calle Wilund
2023-09-20 13:46:15 +00:00
parent 18e79d730e
commit e29bf6f9e8
5 changed files with 228 additions and 67 deletions

View File

@@ -30,7 +30,7 @@ void commitlog_entry_writer::compute_size() {
_size = ms.size();
}
void commitlog_entry_writer::write(typename seastar::memory_output_stream<std::vector<temporary_buffer<char>>::iterator>& out) const {
void commitlog_entry_writer::write(ostream& out) const {
serialize(out);
}