package Data::HashMap::Shared;
use strict;
use warnings;
our $VERSION = '0.18';

require XSLoader;
XSLoader::load('Data::HashMap::Shared', $VERSION);

# ithreads: blessed shared-memory handles must never be cloned into a
# child thread -- the clone would double-free the handle on thread exit.
{ no strict 'refs'; *{"${_}::CLONE_SKIP"} = sub { 1 } for qw(
  Data::HashMap::Shared::I16
  Data::HashMap::Shared::I16::Cursor
  Data::HashMap::Shared::I16S
  Data::HashMap::Shared::I16S::Cursor
  Data::HashMap::Shared::I32
  Data::HashMap::Shared::I32::Cursor
  Data::HashMap::Shared::I32S
  Data::HashMap::Shared::I32S::Cursor
  Data::HashMap::Shared::II
  Data::HashMap::Shared::II::Cursor
  Data::HashMap::Shared::IS
  Data::HashMap::Shared::IS::Cursor
  Data::HashMap::Shared::SI
  Data::HashMap::Shared::SI::Cursor
  Data::HashMap::Shared::SI16
  Data::HashMap::Shared::SI16::Cursor
  Data::HashMap::Shared::SI32
  Data::HashMap::Shared::SI32::Cursor
  Data::HashMap::Shared::SS
  Data::HashMap::Shared::SS::Cursor
); }

1;

__END__

=encoding utf-8

=head1 NAME

Data::HashMap::Shared - Multiprocess shared-memory hash maps with LRU eviction
and per-key TTL

=head1 SYNOPSIS

    use Data::HashMap::Shared::II;

    # Create or open a shared map (file-backed mmap)
    my $map = Data::HashMap::Shared::II->new('/tmp/mymap.shm', 100000);

    # Keyword API (fastest)
    shm_ii_put $map, 42, 100;
    my $val = shm_ii_get $map, 42;

    # Method API
    $map->put(42, 100);
    my $v = $map->get(42);

    # Atomic counters (lock-free fast path)
    shm_ii_incr $map, 1;
    shm_ii_incr_by $map, 1, 10;
    shm_ii_max $map, 1, 50;         # monotonic: store max(current, 50)

    # Compare-and-swap (all variants; byte-compare for string values)
    shm_ii_cas $map, 1, 11, 42;     # swap to 42 only if current == 11

    # LRU cache (evicts least-recently-used when full)
    my $cache = Data::HashMap::Shared::II->new('/tmp/cache.shm', 100000, 1000);
    shm_ii_put $cache, 42, 100;    # auto-evicts LRU entry if size > 1000

    # TTL (entries expire after N seconds)
    my $ttl_map = Data::HashMap::Shared::II->new('/tmp/ttl.shm', 100000, 0, 60);
    shm_ii_put $ttl_map, 1, 10;          # expires in 60s
    shm_ii_put_ttl $ttl_map, 2, 20, 5;   # per-key: expires in 5s

    # Multiprocess
    if (fork() == 0) {
        my $child = Data::HashMap::Shared::II->new('/tmp/mymap.shm', 100000);
        shm_ii_incr $child, 1;   # atomic increment visible to parent
        exit;
    }
    wait;

=head1 DESCRIPTION

Data::HashMap::Shared provides type-specialized hash maps stored in
file-backed shared memory (C<mmap(MAP_SHARED)>), enabling efficient
multiprocess data sharing on Linux. With opt-in B<LRU eviction> and
B<per-key TTL> it doubles as a fast cross-process B<cache>; lookups take a
lock-free seqlock fast path.

B<Linux-only>. Requires 64-bit Perl.

=head2 Features

=over

=item * File-backed mmap for cross-process sharing

=item * Futex-based read-write lock (fast userspace path)

=item * Lock-free atomic counters (incr/decr under read lock)

=item * Elastic capacity (starts small, grows/shrinks automatically)

=item * Arena allocator for string storage in shared memory

=item * Keyword API via XS::Parse::Keyword for maximum speed

=item * Opt-in B<LRU eviction> -- clock/second-chance algorithm; reads stay lock-free

=item * Opt-in B<per-key TTL> expiry -- lazy removal on access; monotonic clock

=item * Stale lock recovery for both writers and readers (dead PIDs detected and drained automatically)

=back

=head2 Variants

=over

=item L<Data::HashMap::Shared::I16> - int16 to int16

=item L<Data::HashMap::Shared::I32> - int32 to int32

=item L<Data::HashMap::Shared::II> - int64 to int64

=item L<Data::HashMap::Shared::I16S> - int16 to string

=item L<Data::HashMap::Shared::I32S> - int32 to string

=item L<Data::HashMap::Shared::IS> - int64 to string

=item L<Data::HashMap::Shared::SI16> - string to int16

=item L<Data::HashMap::Shared::SI32> - string to int32

=item L<Data::HashMap::Shared::SI> - string to int64

=item L<Data::HashMap::Shared::SS> - string to string

=back

=head2 Integer Range and Wrapping

Integer keys and values are stored as fixed-width two's-complement
integers: C<I16>/C<SI16>/C<I16S> use a signed 16-bit range
(-32768 .. 32767), C<I32>/C<SI32>/C<I32S> a signed 32-bit range, and
C<II>/C<IS>/C<SI> a signed 64-bit range. A key or value outside the
variant's range is B<silently truncated> to the low bits (two's
complement), with no warning: on an C<I16> map, C<< $map->put(70000, ...) >>
stores under key C<4464> (C<70000 & 0xFFFF>), so C<get(70000)> and
C<get(4464)> address the same entry. C<incr>/C<decr> wrap the same way
(C<32767 + 1> becomes C<-32768>). Pick a variant wide enough for your data.

=head2 Constructor

    my $map = Data::HashMap::Shared::II->new($path, $max_entries);
    my $map = Data::HashMap::Shared::II->new(undef, $max_entries);    # anonymous
    my $map = Data::HashMap::Shared::II->new($path, $max_entries, $max_size);
    my $map = Data::HashMap::Shared::II->new($path, $max_entries, $max_size, $ttl);
    my $map = Data::HashMap::Shared::II->new($path, $max_entries, $max_size, $ttl, $lru_skip);
    my $map = Data::HashMap::Shared::SS->new($path, $max_entries, 0, 0, 0, $arena_cap); # explicit arena bytes
    my $map = Data::HashMap::Shared::II->new($path, $max_entries, $max_size, $ttl, $lru_skip, $arena_cap, $file_mode);
    my $map = Data::HashMap::Shared::II->new_sharded($prefix, $shards, $max_entries, $max_size, $ttl, $lru_skip, $arena_cap, $file_mode);
    my $map = Data::HashMap::Shared::II->new_memfd($name, $max_entries, ...); # memfd-backed
    my $map = Data::HashMap::Shared::II->new_from_fd($fd);            # reopen memfd
    my $fd  = $map->memfd;                                            # -1 if not memfd

Creates or opens a shared hash map backed by file C<$path>. Passing C<undef>
as the path creates an anonymous C<MAP_SHARED|MAP_ANONYMOUS> mapping that is
inherited across C<fork> but has no filesystem presence.

C<new_memfd> creates an unlinked memfd-backed map whose file descriptor can be
passed to another process (via C<SCM_RIGHTS>, C<fork>+C<exec>, or duped+open).
C<new_from_fd> reopens such a descriptor. The descriptor you pass is
duplicated (C<F_DUPFD_CLOEXEC>), so it stays yours to close and closing it
does not disturb the handle. Both require a 64-bit Perl on Linux
(C<memfd_create(2)>).

C<$max_entries>, C<$max_size>, C<$ttl>, C<$lru_skip>, C<$arena_cap>, and
C<$file_mode> are used only when creating a new file; when opening an
existing one, all parameters are read from the stored header and the
constructor arguments are ignored.
Multiple processes can open the same file simultaneously.
Dies if the file exists but was created by a different variant or is corrupt.

Optional C<$max_size> enables LRU eviction: when the map reaches C<$max_size>
entries, the least-recently-used entry is evicted on insert. Set to 0 (default)
to disable. LRU uses a clock/second-chance algorithm: C<get> sets an accessed
bit (lock-free, no write lock), and eviction gives a second chance to recently
accessed entries before evicting.

Optional C<$ttl> sets a default time-to-live in seconds for all entries.
Expired entries are lazily removed on access. Set to 0 (default) to disable.
When TTL is active, C<get> and C<exists> check expiry. Expiry is measured
against a monotonic clock (C<CLOCK_MONOTONIC_COARSE>): TTLs track elapsed
running time and do not advance while the system is suspended or hibernating.

That clock is also B<local to the current boot>: it restarts at zero on reboot
and is unrelated between machines, while the expiry timestamps live in the
backing file. A file that outlives the boot which wrote it -- or is copied to
another host, including a frozen map shipped elsewhere (see
L</"Frozen (Read-Only) Mode">) -- therefore carries expiries on a timeline that
no longer exists. The error is always in the safe direction: C<now> has not yet
reached the stored values, so entries live B<longer> than their TTL, never
shorter, and it corrects itself once uptime passes them. Nothing expires early
and nothing crashes, but do not rely on TTL across a reboot or a host move --
call C<flush_expired> or rebuild the map.

Optional C<$lru_skip> (0-99, default 0) reduces how often LRU promotion
reorders the recency list: higher values skip more promotions. Promotion runs
only when an operation updates an existing entry under the write lock
(C<put>/C<incr>/C<get_or_set> on a hit, the update family); plain C<get> and
C<get_with_ttl> never promote -- they set a lock-free accessed bit that the clock
eviction consumes. Skipping promotions cuts write-lock churn for Zipfian
(power-law) patterns where a few hot keys dominate. The LRU tail (eviction
victim) is never skipped, preserving eviction correctness. Set to 0 for strict
LRU ordering.

Optional C<$arena_cap> (bytes) sizes the string-storage arena explicitly,
decoupling it from C<$max_entries>. By default the arena holds roughly 128
bytes per entry (4096 minimum), so a map with a few large strings -- keys or
values -- can exhaust it while the table is nearly empty; pass an C<$arena_cap>
sized for your strings instead. Clamped to C<[4096, 0xFFFFFFFF]> (arena offsets are
32-bit). Integer-only variants (C<II>/C<I16>/C<I32>) have no arena and ignore
it. For sharded maps it is the per-shard cap, like C<$max_entries>.

Optional C<$file_mode> (octal, default C<0600>) sets the permission bits used
when the backing file is created; the exact mode is applied via C<fchmod>, so
the process umask does not narrow it. It is ignored when attaching an existing
file and for anonymous or memfd-backed maps. The default is owner-only; pass a
wider mode such as C<0666> to opt in to cross-user sharing. Before version
0.14 the default was C<0666>.

B<Zero-cost when disabled>: with both C<$max_size=0> and C<$ttl=0>, the fast
lock-free read path is used. The only overhead is a branch (predicted away).

=head2 String Keys/Values and UTF-8

String-key variants (C<SS>, C<SI>, C<SI16>, C<SI32>) compare keys as raw
bytes: two keys are the same entry if and only if they contain the same
byte sequence. The SV UTF-8 flag is stored alongside the key so retrieval
round-trips it to the returned SV, but it is B<not> part of key identity.
Consequences:

=over

=item *

ASCII keys with a toggled UTF-8 flag hash and match the same entry
(C<use utf8>, C<utf8::upgrade>, and C<utf8::downgrade> on ASCII are all
equivalent from the map's point of view).

=item *

Non-ASCII keys with different byte encodings are B<distinct>. C<"caf\xe9">
(latin-1, 4 bytes) and the same character sequence under C<use utf8>
(C<"caf\xc3\xa9">, 5 UTF-8 bytes) are two different keys. If your input
comes in mixed encodings, normalize with
C<Encode::encode_utf8> before use.

=back

String-value variants (C<SS>, C<IS>, C<I16S>, C<I32S>) store the SV UTF-8
flag alongside each value and round-trip it on retrieval. The C<cas>
comparison of C<$expected> against the stored value is byte-only -- the
UTF-8 flag on C<$expected> is ignored (same rationale as string-key
equality).

=head2 Sharding

    my $map = Data::HashMap::Shared::II->new_sharded($path_prefix, $shards, $max_entries, ...);

Creates C<$shards> independent maps (files C<$path_prefix.0>, C<$path_prefix.1>,
...) behind a single handle, each with up to C<$max_entries> entries
(total capacity is C<$shards * $max_entries>). Per-key operations automatically
route to the correct shard via hash dispatch. Writes to different shards
proceed in parallel with independent locks. C<new_sharded> requires a
filesystem C<$path_prefix>; anonymous (C<undef>-path) sharded maps are not
supported.

The batch ops (C<set_multi>, C<get_multi>, C<remove_multi>) dispatch each key
to its shard independently rather than holding one lock for the whole call, so
on a sharded map a batch is B<not> atomic across shards (the "single lock"
note in the API below applies to non-sharded maps).

All operations work transparently on sharded maps: C<put>, C<get>, C<remove>,
C<exists>, C<add>, C<update>, C<swap>, C<take>, C<incr>, C<max>, C<min>,
C<cas>, C<cas_take>, C<get_or_set>, C<put_ttl>, C<add_ttl>, C<update_ttl>,
C<touch>, C<persist>, C<set_ttl>, C<keys>, C<values>, C<items>, C<to_hash>,
C<set_multi> (method only), C<remove_multi> (method only), C<get_multi>
(method only), C<get_with_ttl> (method only), C<each>, C<pop>, C<shift>,
C<drain>, C<clear>, C<flush_expired>, C<flush_expired_partial>, C<size>,
C<stats> (method only), C<reserve>, and all diagnostic keywords.

Diagnostic counters and capacities reported for a sharded handle are
aggregate totals across all shards: C<size>, C<capacity>, C<max_entries>,
C<max_size>, C<tombstones>, C<mmap_size>, C<arena_used>, C<arena_cap>, and the
C<stats> eviction/expiry/recovery counts all sum over the shards. (C<ttl> is
the shared per-entry default, so it reports a single shard's value.)
C<reserve $n> pre-grows B<each> shard to C<$n> entries (not C<$n> in total).

Cursors chain across shards automatically. C<cursor_seek> routes to the
correct shard based on key hash. C<$shards> is rounded up to the next
power of 2.

=head2 API

Replace C<xx> with variant prefix: C<i16>, C<i32>, C<ii>, C<i16s>,
C<i32s>, C<is>, C<si16>, C<si32>, C<si>, C<ss>.

    my $ok = shm_xx_put $map, $key, $value;   # insert or overwrite
    my $ok = shm_xx_add $map, $key, $value;   # insert only if key absent
    my $ok = shm_xx_update $map, $key, $value; # overwrite only if key exists
    my $old = shm_xx_swap $map, $key, $value; # put + return old value (undef if new)
    my $ok = shm_xx_cas $map, $key, $expected, $desired; # compare-and-swap
    my $v  = shm_xx_cas_take $map, $key, $expected; # compare-and-remove; returns value on match, undef otherwise
    my $n  = $map->set_multi($k, $v, ...);   # batch put under single lock, returns count
    my $n  = $map->remove_multi(@keys);      # batch remove under single lock, returns count
    my @v  = $map->get_multi($k1, $k2, ...); # batch get under single lock with prefetch pipeline
    my ($v, $ttl) = $map->get_with_ttl($key); # atomic snapshot; () if missing, $ttl is undef on non-TTL map, 0 = permanent; sets LRU clock bit
    my $v  = shm_xx_get $map, $key;           # returns undef if not found
    my $ok = shm_xx_remove $map, $key;        # returns false if not found
    my $ok = shm_xx_exists $map, $key;        # returns boolean
    my $s  = shm_xx_size $map;
    my $m  = shm_xx_max_entries $map;
    my @k  = shm_xx_keys $map;
    my @v  = shm_xx_values $map;
    my @items = shm_xx_items $map;            # flat (k, v, k, v, ...)
    while (my ($k, $v) = shm_xx_each $map) { ... }  # auto-resets at end
    shm_xx_iter_reset $map;
    shm_xx_clear $map;
    my $href = shm_xx_to_hash $map;
    my $v  = shm_xx_get_or_set $map, $key, $default;  # returns value

C<get_or_set> returns the existing value, or stores and returns C<$default>
when the key is absent. It returns C<undef> only if the key is absent and the
value cannot be stored -- the map is at C<max_entries>, or (string-value
variants) the arena is full.

C<cas> is available for all variants. Returns true when the stored value
matched C<$expected> and was atomically replaced with C<$desired>; false
if the key is missing or expired, the value did not match, or (string-value
variants) the arena is full. See L</"String Keys/Values and UTF-8"> for
the byte-only comparison rule.

C<swap> returns the previous value, or C<undef> when the key did not exist
(a fresh insert). C<undef> is B<also> returned when the new value cannot be
stored -- the map is already at C<max_entries> capacity, or (string-value
variants) the arena is full -- in which case an existing key keeps its old
value. C<swap> by itself therefore cannot distinguish a fresh insert from a
full-map failure; check C<exists> or C<size> first if that matters. On TTL
maps, C<swap> refreshes an existing entry's TTL to the default and assigns the
default TTL on insert, but leaves a permanent entry (TTL 0) permanent.

Integer-value variants also have:

    my $n = shm_xx_incr $map, $key;           # returns new value
    my $n = shm_xx_decr $map, $key;           # returns new value
    my $n = shm_xx_incr_by $map, $key, $delta;
    my $n = shm_xx_max $map, $key, $desired;  # store max(current, desired), return it
    my $n = shm_xx_min $map, $key, $desired;  # store min(current, desired), return it

A missing key is created starting from zero (Redis-style): the first
C<incr> returns 1, C<decr> returns -1, and C<incr_by> returns C<$delta>.
These die only when the key is new and the map is already at
C<max_entries> (no room to insert). The result wraps at the variant's
integer width (see L</"Integer Range and Wrapping">).

C<max>/C<min> atomically store C<max($current, $desired)> /
C<min($current, $desired)> and return the resulting value; a missing key is
inserted as C<$desired>. The compare-and-set runs under a single lock
acquisition, so there is no read-modify-write gap for a concurrent
C<incr_by>/C<cas>/C<max>/C<min> on the same key: the result is monotonic
(C<max> never lowers, C<min> never raises) and never clobbers a concurrent
increment. The common "value already past the bound" case completes under a
shared read lock, writing nothing. Like C<incr_by>, they die only when the
key is new and the map is at C<max_entries>, and the result wraps at the
variant's integer width.

LRU/TTL operations (C<put_ttl>, C<add_ttl>, and C<update_ttl> require a TTL-enabled map):

    my $ok = shm_xx_put_ttl $map, $key, $value, $ttl_sec;  # per-key TTL (0 = permanent); requires TTL-enabled map
    my $ok = shm_xx_add_ttl $map, $key, $value, $ttl_sec;  # insert-if-absent with per-key TTL (0 = permanent)
    my $ok = shm_xx_update_ttl $map, $key, $value, $ttl_sec; # overwrite-only with per-key TTL (0 = permanent)
    my $ms = shm_xx_max_size $map;            # LRU capacity (0 = disabled)
    my $t  = shm_xx_ttl $map;                 # default TTL in seconds
    my $r  = shm_xx_ttl_remaining $map, $key; # seconds left (0 = permanent, undef if missing/expired/no TTL)
    my $ok = shm_xx_touch $map, $key;         # refresh TTL to default (permanent entries stay permanent); promotes in LRU; false if no TTL/LRU
    my $ok = shm_xx_persist $map, $key;       # remove TTL, make key permanent; false on non-TTL maps
    my $ok = shm_xx_set_ttl $map, $key, $sec; # change TTL without changing value (0 = permanent); false on non-TTL maps
    my $n  = shm_xx_flush_expired $map;       # proactively expire all stale entries, returns count
    my ($n, $done) = shm_xx_flush_expired_partial $map, $limit;  # gradual: scan $limit slots ($limit per shard on sharded maps; $done true once every shard completes a cycle)

Atomic remove-and-return:

    my $v = shm_xx_take $map, $key;           # remove key and return value (undef if missing)
    my ($k, $v) = shm_xx_pop $map;            # remove+return from LRU tail / scan forward
    my ($k, $v) = shm_xx_shift $map;          # remove+return from LRU head / scan backward
    my @kv = shm_xx_drain $map, $n;           # remove+return up to N entries as flat (k,v,...) list

C<pop> and C<shift> remove from opposite ends: C<pop> takes the LRU tail
(oldest / least recently used) while C<shift> takes the LRU head (newest /
most recently used). On non-LRU maps, C<pop> scans forward and C<shift>
scans backward. C<drain> removes in C<pop> order (tail-first).
Useful for work-queue patterns and batch processing.

Cursors (independent iterators, allow nesting and removal during iteration):

    my $cur = shm_xx_cursor $map;             # create cursor
    while (my ($k, $v) = shm_xx_cursor_next $cur) { ... }
    shm_xx_cursor_reset $cur;                 # restart from beginning
    my $ok = shm_xx_cursor_seek $cur, $key;   # position at key (best-effort across resize); true if found, false if missing/expired
    # cursor auto-destroyed when out of scope

C<shm_xx_each> is also safe to use with C<remove> during iteration.
Tombstone compaction and shrink are deferred until iteration ends; growth is
not -- a load-driven insert still resizes, and both iterators restart on the
resulting table-generation bump.

Diagnostics:

    my $cap = shm_xx_capacity $map;           # current table capacity (slots)
    my $tb  = shm_xx_tombstones $map;         # tombstone count
    my $au  = shm_xx_arena_used $map;         # arena bytes used (0 for int-only)
    my $ac  = shm_xx_arena_cap $map;          # arena total capacity (0 for int-only)
    my $sz  = shm_xx_mmap_size $map;          # backing file size in bytes
    my $ok  = shm_xx_reserve $map, $n;        # pre-grow (false if exceeds max)
    my $ev  = shm_xx_stat_evictions $map;     # cumulative LRU eviction count
    my $ex  = shm_xx_stat_expired $map;       # cumulative TTL expiration count
    my $rc  = shm_xx_stat_recoveries $map;    # cumulative stale lock recovery count
    my $p   = $map->path;                    # backing file path (method only)
    my $s   = $map->stats;                   # hashref with all diagnostics in one call
    # stats keys: size, capacity, max_entries, tombstones, mmap_size,
    #   arena_used, arena_cap, evictions, expired, recoveries, max_size, ttl,
    #   frozen, readonly

C<set_multi>, C<get_multi>, C<remove_multi>, C<get_with_ttl>, C<stats>,
C<path>, C<sync>, and C<unlink> are method-only (no keyword form).

File management:

    $map->sync;                               # flush the mmap to the backing file (msync MS_SYNC)
    $map->unlink;                             # remove backing file (mmap stays valid)
    Data::HashMap::Shared::II->unlink($path); # class method form

C<sync> issues a synchronous C<msync(2)> over the whole mapping (every
shard, for sharded maps) and dies on error. Use it to force durability of
a file-backed map; it is a no-op for anonymous mappings, which have no
backing file. Changes are visible to other processes sharing the mapping
without C<sync> -- it only affects on-disk persistence.

C<unlink> reports through its return value rather than by dying: it returns
true when the file (every shard, for sharded maps) was removed and false
otherwise, including when the file was already gone and when removal was
refused -- a read-only directory, for instance. Check it if the removal
mattered; an ignored false is how a file survives a cleanup unnoticed.

=head2 Frozen (Read-Only) Mode

    $map->freeze;                                       # seal the file immutable (durable)
    my $ro = Data::HashMap::Shared::II->new_readonly($path);
    my $v  = $ro->get($key);                            # lock-free query; writes nothing
    my $is_frozen   = $map->frozen;                     # true once sealed
    my $is_readonly = $ro->readonly;                    # true for a read-only handle

C<freeze> seals a file-backed map permanently immutable so it can be shipped and
served read-only. It takes the write lock (so no mutation is in flight), sets a
seal flag in the header, and flushes it to disk (C<msync>). Afterwards every
mutator on that handle croaks and the handle itself becomes read-only. A sharded
map seals every shard file. Freezing is one-way; there is no unfreeze.

C<new_readonly> opens an already-frozen file with C<O_RDONLY> and maps it
C<PROT_READ>. Because a sealed map is immutable and can have no writers, queries
take B<no lock at all> -- no reader-slot bookkeeping, no LRU clock bit, no lazy
TTL cleanup -- and never write the mapping. A read-only view therefore works
from a read-only file or filesystem, and any number of processes can share one
frozen file at once. All queries and full iteration are supported: C<get>,
C<exists>, C<get_with_ttl>, C<get_multi>, C<keys>, C<values>, C<items>,
C<to_hash>, C<each>, and cursors (C<cursor> and C<cursor_next>). Every mutator
croaks (including the integer counters C<incr>/C<decr>/C<max>/C<min>, which on a
writable map update in place under the read lock). C<sync> is a silent no-op.
C<frozen> and C<readonly> report the state, and C<stats> gains matching
C<frozen> and C<readonly> keys.

The seal reuses a previously reserved header byte, so the on-disk format and
version are unchanged: a file written by an older release is simply not frozen
(the byte reads 0) and opens read-write exactly as before.

The two modes never mix: opening a frozen file read-write (C<new>,
C<new_from_fd>) is refused -- open it with C<new_readonly> instead -- and
C<new_readonly> refuses a file that has not been frozen (its lock-free readers
must never race a live writer). C<new_readonly> is for a single backing file,
not a sharded set.

B<Portability>: a frozen file is a raw memory image. Read it back on the B<same
architecture> that wrote it (same word size and endianness; the native magic and
variant id reject a mismatched or wrong-variant file at attach time). Ship it by
B<copying> the file; do not serve it over NFS or another network filesystem
while another host has it mapped.

=head2 Crash Safety

If a process dies (e.g., SIGKILL, OOM kill) while holding the write lock,
other processes detect the stale lock within 2 seconds and automatically
recover. The writer's PID is encoded in the rwlock word itself (single
atomic CAS, no crash window). On C<FUTEX_WAIT> timeout, waiters
C<kill($pid, 0)> the holder and CAS-release the lock if it's dead.

Reader-side recovery uses a 1024-slot table in the shared mmap (one slot
per process, claimed lazily on first lock; fork()'d children claim a
fresh slot via C<pthread_atfork>).  A reader's B<entire> contribution to
the lock is the C<rdepth> word in its own slot -- there is no shared
reader counter -- so a dead reader is neutralised by clearing that one
slot's PID, and a draining writer does so unconditionally as it scans.
Because no orphaned count can exist, there is no quiescent force-reset:
a worker killed mid-C<incr_by> cannot pin the lock at all.  An occupancy
bitmap (one bit per slot, published before the slot can hold a lock) lets
the writer visit only occupied slots rather than all 1024.  Beyond 1024
simultaneous handles per map, a handle that cannot claim a slot proceeds
"slotless"; see L</"Reader-slot exhaustion"> for the one case that
recovery cannot cover.

The same path validates and rebuilds the LRU doubly-linked list if a
dead writer left it inconsistent.  C<stat_recoveries> in C<stats> counts
every recovery event.

Recovery uses C<kill($pid, 0)> for liveness, which cannot distinguish a
reused PID from the original. Hitting a false "alive" requires a process to
die in the brief window it holds a read lock B<and> the kernel to cycle
through the entire PID space back to that exact number within the ~2-second
recovery window B<and> hand it to a long-lived process -- i.e. a runaway fork
storm. Even then the effect is bounded: writers stall until the recycled
process exits; reads are unaffected and no data is corrupted. Writer-crash
recovery is immune (the writer PID lives in the lock word and is reclaimed
independently of the slot table).

B<Limitation>: PID-based recovery assumes all processes share the same
PID namespace. Cross-container sharing (different PID namespaces) is not
supported.

After recovery from a mid-mutation crash, the map data may be partially
inconsistent (e.g., one entry was being updated when the writer died).
Locks, the LRU chain and the entry counters are restored. The arena free
lists are not rebuilt, so blocks in flight at the crash may leak; the specific
entry being mutated may have stale or partial bytes; and a crash part-way
through a table resize permanently drops the entries that had not yet been
re-inserted, because they existed only in the dead writer's private buffer.
Calling
C<clear> after detecting a stale lock recovery is recommended for
safety-critical applications.

=head2 Reader-slot exhaustion

Dead-process recovery attributes a
crashed lock holder's contribution through its reader-slot. The slot table holds
1024 entries (one per concurrent reader process). If more than that many reader
processes share one mapping at once, a reader that cannot claim a slot proceeds
"slotless" -- it still takes the read lock but leaves no per-process record. If
such a slotless reader is then killed while holding the read lock, its share of
the lock cannot be attributed to a dead process, so writer recovery cannot
reclaim it and writers may block until the mapping is recreated. Reaching this
needs more than 1024 concurrent reader processes on one mapping plus a crash in
the brief read-lock window; the dead-process slot reclaim keeps the table from
filling with stale entries, so in practice it is very unlikely.

=head1 BENCHMARKS

Throughput versus other shared-memory / on-disk solutions, 25K entries,
single process, Linux x86_64.  All values in M ops/s (higher is better).
Run C<perl -Mblib bench/vs.pl 25000> to reproduce.

B<Integer key -> integer value> (Shared::II):

              BerkeleyDB   LMDB   Shared::II
    INSERT          31       46         184
    LOOKUP          35       40         383
    INCREMENT       16       18         165

B<String key -> string value, short> (inline <= 7B, Shared::SS):

              FastMmap   BerkeleyDB   LMDB   SharedMem   Shared::SS
    INSERT        11          26       40        62          130
    LOOKUP        10          32       34       146          213
    DELETE        14          18       --        32           68

B<String key -> string value, long> (~50-100B, Shared::SS):

              BerkeleyDB   LMDB   SharedMem   Shared::SS
    INSERT        25         37        61          133
    LOOKUP        30         33       125          229

B<LRU cache lookup> (25K entries, lock-free clock eviction):

              plain   LRU
    II         350    373   (lock-free, ~6% faster via clock)
    SS         159    159

B<Cross-process> (25K SS entries, 2 processes, ops/s):

                  Shared::SS   SharedMem       LMDB
    READS        3,250,000    1,986,000     728,000
    WRITES       2,801,000      826,000      95,000
    MIXED 50/50  3,691,000    1,963,000     211,000

LMDB benchmarked with MDB_WRITEMAP|MDB_NOSYNC|MDB_NOMETASYNC|MDB_NORDAHEAD.
BerkeleyDB with DB_PRIVATE|128MB cache.

Key takeaways:

=over

=item * B<10x> faster lookups than LMDB for integer keys (lock-free seqlock path)

=item * B<1.5x> faster than Hash::SharedMem for short string lookups (inline strings, no arena overhead)

=item * B<1.8x> faster than Hash::SharedMem for long string lookups

=item * B<4.5x> faster cross-process reads than LMDB; B<3.4x> faster writes than SharedMem

=item * LRU reads are lock-free (clock eviction) -- no overhead vs plain maps

=item * Atomic C<incr> is B<9x> faster than get+put on competitors

=item * Strings <= 7 bytes stored inline in node (zero arena overhead)

=back

=head1 CRASH SAFETY

An interrupted create is recovered too. A creator killed after the backing
file is sized but before its header is committed leaves a full-size, all-zero
file. C<new> re-initializes such a file automatically, but only when it is
exactly the size the requested geometry needs, is owned by your effective uid,
and is still entirely zero -- a file holding data is never re-initialized. If
the creator got as far as writing part of the header, the file cannot be told
apart from a corrupt one and C<new> croaks with C<incomplete map file left by
an interrupted create; remove it and retry>. A file left behind by an
interrupted create never held data, so removing it is safe -- but a file whose
header was corrupted after the fact reaches the same croak, so confirm it is
an abandoned create before deleting anything you care about.

=head1 SEE ALSO

L<Data::Buffer::Shared> - typed shared array

L<Data::Queue::Shared> - FIFO queue

L<Data::PubSub::Shared> - publish-subscribe ring

L<Data::ReqRep::Shared> - request-reply

L<Data::Sync::Shared> - synchronization primitives

L<Data::Pool::Shared> - fixed-size object pool

L<Data::Stack::Shared> - LIFO stack

L<Data::Deque::Shared> - double-ended queue

L<Data::Log::Shared> - append-only log (WAL)

L<Data::Heap::Shared> - priority queue

L<Data::Graph::Shared> - directed weighted graph

L<Data::BitSet::Shared> - shared bitset (lock-free per-bit ops)

L<Data::RingBuffer::Shared> - fixed-size overwriting ring buffer

=head1 SECURITY

Backing files are created with mode C<0600> (owner-only) by default, so only
the creating user can open and attach them. To share a backing file across
users, pass an explicit octal file mode such as C<0660> as the last argument
to C<new>; the mode is applied when the file is created, and when a file left
behind by an interrupted create is re-initialized (see L</CRASH SAFETY>); a
file already in use keeps its own permissions. The file is opened with
C<O_NOFOLLOW>, so a symlink planted at the path is refused, and created with
C<O_EXCL>; the on-disk header is validated when the file is attached. Any
process you grant write access to a shared mapping is trusted not to corrupt
its contents while other processes are using it.

=head1 UPGRADING

The on-disk format changed in 0.16 (v9 to v10; a reader-slot occupancy
bitmap was added). A file-backed map written by an older release is rejected with
a version-mismatch error when attached. Migrate existing files in place -- with
no process attached -- using the bundled tool:

    hashmap-shared-upgrade FILE...

or from code:

    my $r = Data::HashMap::Shared->upgrade_file($path);
    # 1 = upgraded, 0 = already current; croaks on a bad, locked or unknown file

The migration is a structural upcast (no re-hashing): it is written to a sibling
temporary file and atomically renamed over the original, preserving its mode, and
refuses to run if a live writer holds the file.

A sharded map (C<new_sharded>) keeps each shard in its own backing file named
C<PREFIX.0>, C<PREFIX.1>, and so on; upgrade them all, for example:

    hashmap-shared-upgrade PREFIX.*

Anonymous and memfd maps are process-local and need no migration.

=head1 AUTHOR

vividsnow

=head1 LICENSE

This is free software; you can redistribute it and/or modify it under
the same terms as Perl itself.

=cut
