Building & Benchmarking TidesDB

If you want to download the source of this document, you can find it here.

---

Latest Version

Check the latest release on GitHub for the current version.

Supported Platforms

TidesDB is tested and supported on the following platforms:

Operating Systems

• Linux (x86, x64, PowerPC 32-bit, RISC-V 64-bit)
• macOS (x64, ARM64/Apple Silicon)
• Windows (x86, x64 with MSVC or MinGW)
• FreeBSD 14.0+ (x64)
• OpenBSD 7.4+ (x64)
• NetBSD (x64)
• DragonFlyBSD (x64)
• Illumos/OmniOS (x64)
• Solaris (x64)

Architectures

• x86 (32-bit Intel/AMD)
• x64 (64-bit Intel/AMD)
• ARM64 (Apple Silicon, ARM v8)
• PowerPC (32-bit, cross-compiled)
• RISC-V (64-bit, cross-compiled)

Compilers

• GCC 7.0+ (Linux, MinGW, cross-compilation)
• Clang 6.0+ (macOS, Linux, BSD)
• MSVC 2019 16.8+ (Windows with /experimental:c11atomics)

Platform-Specific Notes

• SunOS/Illumos · Snappy compression is not available; TidesDB automatically disables it
• PowerPC · Requires libatomic for 64-bit atomic operations
• 32-bit architectures · Require libatomic and multilib support
• Windows · Both MSVC and MinGW are fully supported with vcpkg for dependencies

Prerequisites

Build Tools

• CMake 3.25 or higher
• C Compiler with C11 support:
  • GCC 7.0+ (Linux/MinGW)
  • Clang 6.0+ (macOS/Linux)
  • MSVC 2019 16.8+ (Windows with /experimental:c11atomics)

Dependencies

The only hard requirements are a C11 toolchain, CMake, and threading support:

• pthreads · POSIX threads (Linux/macOS/BSD: built-in, Windows: PThreads4W via vcpkg)
• libatomic · Atomic operations library (required for 32-bit architectures like PowerPC)

Compression backends (optional, enabled by default)

Each of the three compression libraries is optional. All three are built in by default; drop any with -DTIDESDB_WITH_<NAME>=OFF. A build with all three disabled has no compression dependencies at all and supports only TDB_COMPRESS_NONE, so TidesDB can be installed with nothing beyond libc and a thread library. See Compression Backends under Build Options.

• Snappy · Fast compression/decompression (TIDESDB_WITH_SNAPPY; not packaged on SunOS/Illumos, so disabled there by default)
• LZ4 · Extremely fast compression (TIDESDB_WITH_LZ4)
• Zstandard · High compression ratio (TIDESDB_WITH_ZSTD)
• C++ standard library · Pulled in only when Snappy is enabled (Snappy’s implementation is C++); a build without Snappy carries no libstdc++ dependency

Installing Dependencies

Linux (Debian/Ubuntu)

# Install all dependencies
sudo apt update
sudo apt install -y cmake build-essential \
    libzstd-dev liblz4-dev libsnappy-dev

Other Linux Distributions

# Fedora/RHEL/CentOS
sudo dnf install cmake gcc libzstd-devel lz4-devel snappy-devel

# Arch Linux
sudo pacman -S cmake gcc zstd lz4 snappy

# 32-bit builds (e.g., x86, PowerPC)
sudo apt install -y gcc-multilib g++-multilib libatomic1

macOS

TidesDB supports multiple package managers on macOS. By default, CMake will auto-detect Homebrew, but you can use MacPorts, pkgsrc, or Fink instead.

Option 1 · Homebrew (Default)

# Install Homebrew if not already installed
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install dependencies
brew install cmake zstd lz4 snappy

# Build (Homebrew is auto-detected)
cmake -S . -B build
cmake --build build --clean-first --verbose

Option 2 · MacPorts

# Install MacPorts from https://www.macports.org/install.php

# Install dependencies
sudo port install cmake zstd lz4 snappy

# Build with MacPorts prefix
cmake -S . -B build -DMACOS_DEPENDENCY_PREFIX=/opt/local -DUSE_HOMEBREW=OFF
cmake --build build --clean-first --verbose

Option 3 · pkgsrc

# Install pkgsrc for macOS

# Install dependencies
pkgin install cmake zstd lz4 snappy

# Build with pkgsrc prefix
cmake -S . -B build -DMACOS_DEPENDENCY_PREFIX=/usr/pkg -DUSE_HOMEBREW=OFF
cmake --build build --clean-first --verbose

Option 4 · Fink

# Install Fink from https://www.finkproject.org/

# Install dependencies
fink install cmake zstd lz4 snappy

# Build with Fink prefix
cmake -S . -B build -DMACOS_DEPENDENCY_PREFIX=/sw -DUSE_HOMEBREW=OFF
cmake --build build --clean-first --verbose

Custom Paths

If you have dependencies installed in a custom location, use:

cmake -S . -B build -DMACOS_DEPENDENCY_PREFIX=/your/custom/path -DUSE_HOMEBREW=OFF

Or set CMAKE_PREFIX_PATH environment variable:

export CMAKE_PREFIX_PATH=/your/custom/path
cmake -S . -B build -DUSE_HOMEBREW=OFF

Windows

Using vcpkg (Recommended)

# Install vcpkg if not already installed
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
.\bootstrap-vcpkg.bat

# Install dependencies
.\vcpkg install zstd:x64-windows lz4:x64-windows snappy:x64-windows pthreads:x64-windows

# For 32-bit builds
.\vcpkg install zstd:x86-windows lz4:x86-windows snappy:x86-windows pthreads:x86-windows

BSD Variants

TidesDB supports FreeBSD, OpenBSD, NetBSD, and DragonFlyBSD with platform-specific package paths.

FreeBSD

# Install dependencies (packages in /usr/local)
sudo pkg install cmake pkgconf liblz4 zstd snappy

# Build
cmake -S . -B build
cmake --build build --clean-first --verbose

OpenBSD

# Install dependencies (packages in /usr/local)
sudo pkg_add cmake gmake lz4 zstd snappy pkgconf

# Build
cmake -S . -B build
cmake --build build --clean-first --verbose

NetBSD

# Install dependencies (packages in /usr/pkg)
sudo pkgin install cmake lz4 zstd snappy

# Build
cmake -S . -B build
cmake --build build --clean-first --verbose

DragonFlyBSD

# Install dependencies (packages in /usr/local)
sudo pkg install cmake lz4 zstd snappy

# Build
cmake -S . -B build
cmake --build build --clean-first --verbose

Illumos/OmniOS/Solaris

Note

Snappy is not available in OmniOS repositories. TidesDB automatically disables Snappy compression on SunOS builds.

# Install dependencies (packages in /opt/ooce)
sudo pkg install cmake lz4 zstd

# Build
cmake -S . -B build
cmake --build build --clean-first --verbose

PowerPC (32-bit)

Cross-compilation for PowerPC requires building dependencies from source.

# Install cross-compilation toolchain
sudo apt install -y crossbuild-essential-powerpc \
  gcc-powerpc-linux-gnu g++-powerpc-linux-gnu \
  qemu-user-static

# Build dependencies from source (LZ4, Zstandard, Snappy)
# See .github/workflows/build_and_test_tidesdb.yml for complete build script

# Configure with PowerPC toolchain
cmake -S . -B build \
  -DCMAKE_C_COMPILER=powerpc-linux-gnu-gcc \
  -DCMAKE_CXX_COMPILER=powerpc-linux-gnu-g++ \
  -DCMAKE_SYSTEM_NAME=Linux \
  -DCMAKE_SYSTEM_PROCESSOR=powerpc

# Build
cmake --build build --clean-first --verbose

RISC-V (64-bit)

Cross-compilation for RISC-V 64-bit requires building dependencies from source.

# Install cross-compilation toolchain
sudo apt install -y crossbuild-essential-riscv64 \
  gcc-riscv64-linux-gnu g++-riscv64-linux-gnu \
  qemu-user-static

# Build dependencies from source (LZ4, Zstandard, Snappy)
# See .github/workflows/build_and_test_tidesdb.yml for complete build script

# Configure with RISC-V toolchain
cmake -S . -B build \
  -DCMAKE_C_COMPILER=riscv64-linux-gnu-gcc \
  -DCMAKE_CXX_COMPILER=riscv64-linux-gnu-g++ \
  -DCMAKE_SYSTEM_NAME=Linux \
  -DCMAKE_SYSTEM_PROCESSOR=riscv64

# Build
cmake --build build --clean-first --verbose

Building

Once you have the dependencies installed, you can build using the commands below.

Unix (Linux/macOS)

Standard Build (Production)

A release build enables optimizations and disables debugging features:

rm -rf build && cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --clean-first --verbose
cmake --install build

# On linux run ldconfig to update the shared library cache
ldconfig

Development Build

For development, enable sanitizers and profiling:

rm -rf build && cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Debug \
  -DTIDESDB_WITH_SANITIZER=ON \
  -DENABLE_READ_PROFILING=ON
cmake --build build --clean-first --verbose

Minimal Build (No Tests)

For embedded systems or minimal deployments:

rm -rf build && cmake -S . -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DTIDESDB_BUILD_TESTS=OFF \
  -DBUILD_SHARED_LIBS=OFF
cmake --build build --clean-first --verbose
cmake --install build

Static archives are position independent

Even with BUILD_SHARED_LIBS=OFF, all object files compile with -fPIC (via CMAKE_POSITION_INDEPENDENT_CODE ON). This means the static archive can link directly into a shared module like the MariaDB engine plugin without a recompile-with-fPIC step. The flag has negligible runtime cost on modern x86_64 and ARM64 toolchains.

Windows

BUILD_SHARED_LIBS on Windows

By default, TidesDB builds as a static library on Windows (BUILD_SHARED_LIBS=OFF) to avoid DLL export/import complexity. If you need a shared library (DLL), explicitly set -DBUILD_SHARED_LIBS=ON during configuration. Note that shared builds require proper symbol exports and may need additional configuration.

MinGW-w64

MinGW-w64 provides a GCC-based toolchain with better C11 support and POSIX compatibility.

Prerequisites

• Install MinGW-w64
• Install CMake
• Install vcpkg for dependencies

Build Steps

# Clean previous build
Remove-Item -Recurse -Force build -ErrorAction SilentlyContinue

# Configure with MinGW
cmake -S . -B build -G "MinGW Makefiles" -DCMAKE_C_COMPILER=gcc -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake

# Build (Debug or Release)
cmake --build build --clean-first --verbose --config Release

# Run tests
cd build
ctest --verbose  # or use --output-on-failure to only show failures

MSVC (Visual Studio)

Prerequisites

• Install Visual Studio 2019 or later with C++ development tools
• Install CMake
• Install vcpkg for dependencies

Build Steps

# Clean previous build
Remove-Item -Recurse -Force build -ErrorAction SilentlyContinue

# Configure with MSVC
cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake

# Build (Debug or Release)
cmake --build build --clean-first --verbose --config Release

# Run tests
cd build

ctest -C Debug --verbose
# or
ctest -C Release --verbose

Note

MSVC requires Visual Studio 2019 16.8 or later for C11 atomics support (/experimental:c11atomics). Both Debug and Release builds are fully supported.

Testing

After building, run the test suite to verify everything works

cd build
ctest --output-on-failure

Or run tests directly

./build/tidesdb_tests

Running Individual Tests

All test binaries support an optional filter argument to run only tests whose names contain a given substring. This works for every component test binary (tidesdb_tests, block_manager_tests, skip_list_tests, etc.).

# Run only checkpoint-related tests
./build/tidesdb_tests checkpoint

# Run only compaction tests
./build/tidesdb_tests compaction

# Run only bloom filter serialization tests
./build/bloom_filter_tests serialize

# Run a single test by exact name
./build/tidesdb_tests test_purge_cf_basic

The filter uses substring matching - any test whose function name contains the filter string will run. Tests that don’t match are skipped. The summary output shows passed, failed, and skipped counts along with the active filter.

Consolidated Include

After building, TidesDB creates a consolidated include directory in the build tree for easy consumption. This allows you to use:

#include <tidesdb/tidesdb.h>

With the compiler flag:

-I<build_dir>/include

All public headers (from src/ and external/) plus the generated tidesdb_version.h are copied to <build_dir>/include/tidesdb/.

Build Options

TidesDB provides several CMake options to customize the build:

Option	Description	Default
TIDESDB_WITH_SANITIZER	Enable address/undefined behavior sanitizers	OFF
TIDESDB_BUILD_TESTS	Build test suite	ON
BUILD_SHARED_LIBS	Build shared libraries instead of static	ON (Unix), OFF (Windows)
ENABLE_READ_PROFILING	Enable read profiling instrumentation	OFF
TIDESDB_WARN_MAYBE_UNINIT	Enable -Wmaybe-uninitialized (GCC only; requires an optimized build)	OFF
TIDESDB_WITH_SNAPPY	Build with Snappy compression support	ON (OFF on SunOS)
TIDESDB_WITH_LZ4	Build with LZ4 compression support	ON
TIDESDB_WITH_ZSTD	Build with Zstandard compression support	ON
TIDESDB_SNAPPY_TARGET	External Snappy link target/item to use instead of autodetect	(empty)
TIDESDB_LZ4_TARGET	External LZ4 link target/item to use instead of autodetect	(empty)
TIDESDB_ZSTD_TARGET	External Zstandard link target/item to use instead of autodetect	(empty)
TIDESDB_WITH_S3	Build S3-compatible object store connector (requires libcurl only)	OFF
TIDESDB_WITH_MIMALLOC	Use mimalloc as the memory allocator	OFF
TIDESDB_WITH_TCMALLOC	Use tcmalloc as the memory allocator	OFF
TIDESDB_WITH_JEMALLOC	Use jemalloc as the memory allocator	OFF
CMAKE_BUILD_TYPE	Build type (Debug/Release/RelWithDebInfo)	(unset)

Compression Backends

The Snappy, LZ4, and Zstandard backends are independently optional and all enabled by default. Disable any you do not need:

# build without Zstandard
cmake -S . -B build -DTIDESDB_WITH_ZSTD=OFF

# zero-dependency build -- no compression libraries linked, TDB_COMPRESS_NONE only
cmake -S . -B build -DTIDESDB_WITH_SNAPPY=OFF -DTIDESDB_WITH_LZ4=OFF -DTIDESDB_WITH_ZSTD=OFF

The on-disk compression algorithm IDs are stable regardless of which backends are compiled in, so an SSTable written with an algorithm a given build lacks is reported with a clear error rather than misread. The default column family compression resolves to the best available backend at runtime (LZ4, then Zstandard, then Snappy, then none), so a default configuration always works against whatever was built.

Consumers need no compression headers

The third-party compression headers are included only inside the library, so an application that links libtidesdb and includes <tidesdb/tidesdb.h> does not need the Snappy/LZ4/Zstandard development headers itself.

Linking against a vendored compression library

By default each backend links a known CMake target if one exists in the build (for example one provided by a parent project via FetchContent or add_subdirectory), and otherwise falls back to a system -l<name>. If you vendor a compression library under a target name the autodetect cannot guess, point TidesDB at it explicitly:

# an exact CMake target from your dependency
cmake -S . -B build -DTIDESDB_ZSTD_TARGET=zstd::libzstd_static

# or any target/link item your build produces
cmake -S . -B build -DTIDESDB_ZSTD_TARGET=my_zstd_target

This avoids the forced system -l link that otherwise fails when the library is only available as an in-tree target rather than installed system-wide. The same applies to TIDESDB_SNAPPY_TARGET and TIDESDB_LZ4_TARGET.

Mimalloc Memory Allocator

TidesDB optionally supports mimalloc, Microsoft’s high-performance memory allocator. When enabled, mimalloc replaces the standard malloc/free at link time, providing improved allocation performance for memory-intensive workloads.

Enabling mimalloc

# Linux/macOS with vcpkg
cmake -S . -B build -DTIDESDB_WITH_MIMALLOC=ON

# Windows with vcpkg
cmake -S . -B build -DTIDESDB_WITH_MIMALLOC=ON -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake

Requirements

• mimalloc is installed via vcpkg as an optional feature
• On Windows, vcpkg manifest mode automatically installs mimalloc when the option is enabled
• No code changes required — mimalloc overrides standard allocators at link time

When to use mimalloc

• High-throughput workloads with frequent allocations
• Multi-threaded applications (mimalloc has excellent thread-local caching)
• Memory-constrained environments (mimalloc has lower fragmentation)

Performance impact

• Typical improvement - 5-15% reduction in allocation overhead
• Best results with many small allocations (skip list nodes, block buffers)
• Minimal overhead when disabled (default)

TCMalloc Memory Allocator

TidesDB optionally supports tcmalloc (Thread-Caching Malloc), Google’s high-performance memory allocator from gperftools. Like mimalloc, tcmalloc replaces the standard malloc/free at link time, providing improved allocation performance especially for multi-threaded workloads.

Enabling tcmalloc

# Linux/macOS
cmake -S . -B build -DTIDESDB_WITH_TCMALLOC=ON

# Windows with vcpkg
cmake -S . -B build -DTIDESDB_WITH_TCMALLOC=ON -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake

Installing tcmalloc

# Linux (Debian/Ubuntu)
sudo apt install libgoogle-perftools-dev

# macOS (Homebrew)
brew install gperftools

# Windows (vcpkg)
vcpkg install gperftools:x64-windows

When to use tcmalloc

• High-throughput multi-threaded workloads
• Applications with many small allocations
• When you need detailed heap profiling (gperftools includes profiling tools)

Performance impact

• Typical improvement - 5-20% reduction in allocation overhead for multi-threaded code
• Excellent thread-local caching reduces lock contention
• Minimal overhead when disabled (default)

Jemalloc Memory Allocator

TidesDB optionally supports jemalloc, Facebook’s high-performance memory allocator. Like mimalloc and tcmalloc, jemalloc replaces the standard malloc/free at link time, providing improved allocation performance especially for multi-threaded workloads.

Enabling jemalloc

# Linux/macOS
cmake -S . -B build -DTIDESDB_WITH_JEMALLOC=ON

# Windows with vcpkg
cmake -S . -B build -DTIDESDB_WITH_JEMALLOC=ON -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake

Installing jemalloc

# Linux (Debian/Ubuntu)
sudo apt install libjemalloc-dev

# macOS (Homebrew)
brew install jemalloc

# Windows (vcpkg)
vcpkg install jemalloc:x64-windows

When to use jemalloc

• High-throughput multi-threaded workloads
• Applications with many small allocations
• When you need detailed heap profiling (jemalloc includes profiling tools)

Performance impact

• Typical improvement - 5-25% reduction in allocation overhead for multi-threaded code
• Excellent thread-local caching reduces lock contention
• Minimal overhead when disabled (default)

Note

You cannot enable both mimalloc, tcmalloc, and jemalloc simultaneously. Choose one based on your workload characteristics.

S3 Object Store Connector

TidesDB optionally supports an S3-compatible object store connector for storing SSTables in remote storage (AWS S3, MinIO, Google Cloud Storage, or any S3-compatible endpoint). When enabled, TidesDB uses local disk as a cache and uploads SSTables to the object store for durable, replicated storage. The filesystem connector is always available without additional dependencies.

Enabling the S3 connector

# Linux
sudo apt install -y libcurl4-openssl-dev

cmake -S . -B build -DTIDESDB_WITH_S3=ON
cmake --build build --clean-first --verbose

# macOS
brew install curl

cmake -S . -B build -DTIDESDB_WITH_S3=ON
cmake --build build --clean-first --verbose

Requirements

• libcurl for HTTP requests to S3 endpoints

AWS SigV4 request signing (SHA-256 and HMAC-SHA256) is implemented inside TidesDB, so libcurl is the only external dependency — no separate crypto library is needed.

When to use object store mode

• Cloud-native deployments with separated compute and storage
• Workloads that benefit from virtually unlimited storage capacity
• Multi-node setups where cold start recovery from remote storage replaces traditional replication
• Environments where local disk is ephemeral (containers, serverless)

Filesystem connector (no extra dependencies)

The filesystem connector stores objects as files under a root directory using the same path-like key structure. It is always available and requires no build flags. Use it for testing object store mode locally or for replication to a network-mounted filesystem.

Read Profiling

TidesDB includes optional read profiling instrumentation to analyze read performance and cache effectiveness. When enabled, it tracks detailed statistics about where reads are served from (memtable, immutable, SSTable), cache hit rates, bloom filter effectiveness, and I/O patterns.

Read profiling is disabled by default to avoid overhead in production. Enable it for development or performance analysis:

# Development build with profiling
cmake -B build -DENABLE_READ_PROFILING=ON -DCMAKE_BUILD_TYPE=Debug
cmake --build build --clean-first --verbose

For production builds, profiling is automatically disabled:

cmake -B build  # ENABLE_READ_PROFILING=OFF by default

Using Read Profiling

Once enabled, use the profiling API in your code:

#include <tidesdb.h>

int main() {
    tidesdb_t *db;
    tidesdb_config_t config = {/* ... */};
    tidesdb_open(&config, &db);

    /* Perform operations */
    tidesdb_txn_t *txn;
    tidesdb_txn_begin(db, &txn);

    /* ... do reads ... */

    tidesdb_txn_commit(txn);

    /* Print profiling statistics */
    tidesdb_print_read_stats(db);

    /* Or get stats programmatically */
    tidesdb_read_stats_t stats;
    tidesdb_get_read_stats(db, &stats);
    printf("Total reads: %lu\n", stats.total_reads);
    printf("Cache hit rate: %.1f%%\n",
           100.0 * stats.cache_block_hits /
           (stats.cache_block_hits + stats.cache_block_misses));

    tidesdb_close(db);
    return 0;
}

Statistics Collected

When profiling is enabled, TidesDB tracks:

Read Hit Location

• total_reads · Total number of read operations
• memtable_hits · Reads served from active memtable (fastest)
• immutable_hits · Reads served from immutable memtables
• sstable_hits · Reads served from SSTables on disk

SSTable Search Efficiency

• levels_searched · Total LSM levels searched across all reads
• sstables_checked · Total SSTables examined (bloom filter + range checks)
• bloom_checks · Number of bloom filter checks performed
• bloom_hits · Bloom filter checks that indicated key might exist

Block-Level Cache Performance

• cache_block_hits · Blocks served from cache (zero I/O)
• cache_block_misses · Blocks read from disk
• disk_reads · Total disk read operations
• blocks_read · Total blocks read from disk

Example Output

*=== TidesDB Read Profiling Stats ===*
Total Reads:           10000

Read Hit Location:
  Memtable hits:       2500 (25.0%)
  Immutable hits:      1500 (15.0%)
  SSTable hits:        6000 (60.0%)

SSTable Search:
  Levels searched:     18000 (avg: 1.80 per read)
  SSTables checked:    24000 (avg: 2.40 per read)
  Bloom checks:        24000
  Bloom hits:          6500 (27.1%)

Block-Level Cache:
  Cache hits:          15000
  Cache misses:        3000
  Cache hit rate:      83.3%
  Blocks read:         3000 (avg: 0.30 per read)
  Disk reads:          3000

Interpreting Results

Good Performance Indicators

• High memtable/immutable hit rate (>20%) · Recent data is being accessed
• High cache hit rate (>80%) · Block cache is sized appropriately
• High bloom hit rate (>90%) · Bloom filters are effective at filtering non-existent keys
• Low avg levels per read (<2.0) · LSM tree is well-compacted
• Low avg blocks per read (<1.0) · Good data locality

Performance Issues

• Low cache hit rate (<50%) · Consider increasing block_cache_size
• High avg levels per read (>3.0) · May need more aggressive compaction
• Low bloom hit rate (<80%) · Increase bloom filter size or reduce false positive rate
• High avg SSTables per read (>5.0) · Compaction may be falling behind

Performance Impact

When enabled

• Adds atomic increment operations on read path (~1-2% overhead)
• Minimal memory overhead (~64 bytes for stats structure)
• No impact on write performance

When disabled (default)

• Zero overhead - profiling code is compiled out via #ifdef
• No runtime cost

Use Cases

Development & Testing

# Enable profiling and sanitizers for development
cmake -B build \
  -DENABLE_READ_PROFILING=ON \
  -DTIDESDB_WITH_SANITIZER=ON \
  -DCMAKE_BUILD_TYPE=Debug

Performance Tuning

# Profile with production-like config (no sanitizers)
cmake -B build \
  -DENABLE_READ_PROFILING=ON \
  -DCMAKE_BUILD_TYPE=Release

Production Deployment

# Default: profiling and sanitizers disabled for zero overhead
cmake -B build -DCMAKE_BUILD_TYPE=Release

Notes

• Statistics are cumulative from database open to close
• Thread-safe using atomic operations
• Can be queried multiple times during database lifetime
• Useful for identifying cache sizing issues and compaction tuning needs
• Benchmark suite automatically uses profiling when available

Benchmarking

TidesDB includes a comprehensive benchmark suite with fully configurable parameters.

Build benchmarks in Release

Configure with -DCMAKE_BUILD_TYPE=Release. CMake sets no build type by default, which compiles at -O0, and an unoptimized binary produces numbers that do not represent real performance.

Quick Start

# Build and run with defaults
cmake -B build
cmake --build build --clean-first --verbose
./build/tidesdb_bench

Benchmark Operations

The benchmark tests the following operations

• Concurrent writes across multiple threads
• Concurrent reads across multiple threads
• Concurrent deletes across multiple threads
• Full scan from first to last key
• Full scan from last to first key
• Random seeks to specific keys

Configuration Options

All benchmark parameters can be customized at build time using CMake variables

Operation Parameters

Variable	Description	Default
BENCH_NUM_OPERATIONS	Number of put/get/delete operations	10000000
BENCH_NUM_SEEK_OPS	Number of iterator seek operations	100
BENCH_KEY_SIZE	Key size in bytes	16
BENCH_VALUE_SIZE	Value size in bytes	100
BENCH_NUM_THREADS	Number of concurrent threads	8
BENCH_KEY_PATTERN	Key distribution pattern	”random”
BENCH_CF_NAME	Column family name	”benchmark_cf”
BENCH_DB_PATH	Directory path	”benchmark_db”
BENCH_DB_LOG_LEVEL	Database log level	TDB_LOG_DEBUG
BENCH_DB_MAX_MEMORY	Global memory limit in bytes (0 = auto, 75% of system RAM)	0
BENCH_DB_FLUSH_POOL_THREADS	Flush thread pool size	2
BENCH_DB_COMPACTION_POOL_THREADS	Compaction thread pool size	2

Column Family Configuration

Variable	Description	Default
BENCH_WRITE_BUFFER_SIZE	Memtable flush threshold (bytes)	67108864 (64MB)
BENCH_LEVEL_RATIO	Level size multiplier	10
BENCH_DIVIDING_LEVEL_OFFSET	Compaction dividing level offset	2
BENCH_MIN_LEVELS	Minimum LSM levels, TidesDB can scale up levels if need be.	5
BENCH_SKIP_LIST_MAX_LEVEL	Skip list max level	12
BENCH_SKIP_LIST_PROBABILITY	Skip list probability	0.25
BENCH_ENABLE_COMPRESSION	Enable compression (0=off, 1=on)	1
BENCH_COMPRESSION_ALGORITHM	Compression algorithm	TDB_COMPRESS_LZ4
BENCH_ENABLE_BLOOM_FILTER	Enable bloom filter (0=off, 1=on)	1
BENCH_BLOOM_FILTER_FP_RATE	Bloom filter false positive rate	0.01
BENCH_ENABLE_BLOCK_INDEXES	Enable block indexes	1
BENCH_BLOCK_INDEX_SAMPLING_COUNT	Block index sampling ratio (every Nth block)	1
BENCH_BLOCK_INDEX_PREFIX_LEN	Prefix length for block index keys (bytes)	16
BENCH_SYNC_MODE	Sync mode	TDB_SYNC_NONE
BENCH_SYNC_INTERVAL_US	Sync interval in microseconds (for TDB_SYNC_INTERVAL)	128000 (128ms)
BENCH_COMPARATOR_NAME	Key comparator	”memcmp”
BENCH_BLOCK_CACHE_SIZE	Global block cache size (bytes)	67108864 (64MB)
BENCH_ISOLATION_LEVEL	Transaction isolation level	TDB_ISOLATION_READ_COMMITTED
BENCH_KLOG_VALUE_THRESHOLD	Key-log value separation threshold (bytes)	512
BENCH_MIN_DISK_SPACE	Minimum free disk space required (bytes)	104857600 (100MB)
BENCH_L1_FILE_COUNT_TRIGGER	L1 file count trigger for compaction	4
BENCH_L0_QUEUE_STALL_THRESHOLD	L0 queue stall threshold for backpressure	20
BENCH_MAX_OPEN_SSTABLES	Maximum number of open SSTables	256
BENCH_USE_BTREE	Use B+tree format for klog instead of block-based (0=off, 1=on)	0

Key Distribution Patterns

The benchmark supports three key distribution patterns to simulate different workloads:

Sequential

Keys are generated in order: key_0000000000000000, key_0000000000000001, etc.

• Use case · Testing sequential write/read performance
• Best for · Measuring raw throughput with optimal cache behavior

cmake -B build -DBENCH_KEY_PATTERN=sequential

Random (Default)

Keys are randomly generated alphanumeric strings.

• Use case · Testing random access patterns
• Best for · Simulating unpredictable workloads with poor cache locality

cmake -B build -DBENCH_KEY_PATTERN=random

Zipfian

Keys follow a Zipfian distribution (80/20 rule) where 80% of accesses go to 20% of keys.

• Use case · Simulating real-world workloads with “hot” keys
• Best for · Testing cache effectiveness and realistic access patterns

cmake -B build -DBENCH_KEY_PATTERN=zipfian

Example Configurations

Small Workload (Default)

cmake -B build
cmake --build build --clean-first --verbose
./build/tidesdb_bench

Medium Workload

cmake -B build \
  -DBENCH_NUM_OPERATIONS=1000 \
  -DBENCH_NUM_SEEK_OPS=100 \
  -DBENCH_NUM_THREADS=4 \
  -DBENCH_KEY_PATTERN=random

cmake --build build --clean-first --verbose
./build/tidesdb_bench

Large Workload with Zipfian Distribution

cmake -B build \
  -DBENCH_NUM_OPERATIONS=100000 \
  -DBENCH_NUM_SEEK_OPS=10000 \
  -DBENCH_KEY_SIZE=32 \
  -DBENCH_VALUE_SIZE=512 \
  -DBENCH_NUM_THREADS=8 \
  -DBENCH_KEY_PATTERN=zipfian

cmake --build build --clean-first --verbose
./build/tidesdb_bench

Stress Test with No Compression

cmake -B build \
  -DBENCH_NUM_OPERATIONS=10000000 \
  -DBENCH_KEY_SIZE=128 \
  -DBENCH_VALUE_SIZE=4096 \
  -DBENCH_NUM_THREADS=16 \
  -DBENCH_ENABLE_COMPRESSION=0 \
  -DBENCH_ENABLE_BLOOM_FILTER=0 \
  -DBENCH_KEY_PATTERN=sequential

cmake --build build --clean-first --verbose
./build/tidesdb_bench

B+tree Format Benchmark

# Enable B+tree format for klog (faster point lookups, O(log N) tree traversal)
cmake -B build \
  -DBENCH_USE_BTREE=1 \
  -DBENCH_NUM_OPERATIONS=100000 \
  -DBENCH_KEY_PATTERN=random

cmake --build build --clean-first --verbose
./build/tidesdb_bench

Testing Different Compression Algorithms

# LZ4 (default, good balance)
cmake -B build -DBENCH_COMPRESSION_ALGORITHM=TDB_COMPRESS_LZ4

# LZ4 Fast (fastest compression)
cmake -B build -DBENCH_COMPRESSION_ALGORITHM=TDB_COMPRESS_LZ4_FAST

# Snappy (balanced)
cmake -B build -DBENCH_COMPRESSION_ALGORITHM=TDB_COMPRESS_SNAPPY

# Zstandard (best compression ratio)
cmake -B build -DBENCH_COMPRESSION_ALGORITHM=TDB_COMPRESS_ZSTD

Benchmark Output

The benchmark displays

• Configuration summary (operations, threads, key pattern, features enabled)
• Operations per second for each operation type
• Total time taken for each operation
• Debug logs (if enable_debug_logging is enabled in config)

Notes

• The benchmark creates a temporary storage directory that is cleaned up before each run
• All operations use transactions for consistency
• Results may vary based on system resources, disk speed, and configuration
• For accurate results, run multiple times and average the results
• Disable debug logging for production-like performance measurements

Getting Help

If you encounter issues not covered here:

1. Check the GitHub Issues for similar problems
2. Review the CI build logs for working configurations
3. Open a new issue with
   • Your OS and version
   • Compiler and version
   • Full build output
   • CMake configuration command used