Act nowcan affect a running system
Tunnel and certificate locks settle at build 16672
This batch repairs deadlocks and races in tunnel relays, certificate chains, time-zone lookup and POSIX thread-pool shutdown. It also replaces the public TWebSocketProtocol.SetSubprotocol hook, so custom WebSocket protocol subclasses require a source change. Tunnel and Zstandard users should take the final build rather than an intermediate revision.
TL;DR
- Land on 2.4.16672: tunnel synchronization continues through the final commit, after BSD and Delphi follow-up fixes.
- Custom TWebSocketProtocol subclasses must replace SetSubprotocol overrides with IsSubprotocol and, when setup is required, SetName.
- TEccCertificateChain.ValidateItems no longer locks itself; callers must serialize it, while SaveToBinary now takes a write lock.
- POSIX TSynThreadPool now uses an O(1) circular queue and a shutdown path that rejects or aborts contexts safely.
- Concurrent TAlgoZstd calls create transient contexts instead of blocking behind one shared context; the unit remains explicitly unfinished.
Upgrade advice
Use build 2.4.16672 for tunnel, POSIX thread-pool, ECC certificate-chain or Zstandard workloads. Do not stop on the intermediate tunnel sequence. Before rebuilding, grep custom WebSocket protocol classes for SetSubprotocol and certificate workflows for concurrent ValidateItems calls.
20 of 20 entries · 20 commits upstream
1
High
Tunnel callbacks are released outside relay and session locks
In mormot.net.tunnel, TTunnelList now derives from TSynPersistent and owns a dedicated TRWLightLock named fSessionSafe. New Extract removes an ITunnelTransmit while retaining a local reference; Delete and TTunnelRelay.ConsoleDelete release that reference after leaving the lock. ConsoleTunnelSend likewise invokes the remote callback outside fConsoleSafe.
Breaking changes
mormot.net.tunnel
aef05e96+70-41
net: some Tunnel logic fixes
Impact
Tunnel closure and frame delivery no longer run interface finalizers or remote code while shared relay locks are held, reducing deadlock and re-entrancy risk. Code that subclasses TTunnelList or accesses its inherited Safe property will no longer compile because the base class changed.
Worth a look
Land on build 2.4.16672, then grep for TTunnelList subclasses and .Safe access before rebuilding tunnel services.
Public API
TTunnelListsignature-changed — Its base class changes from TObjectRWLightLock to TSynPersistent; the inherited Safe surface disappears.TTunnelList.Extractadded — Removes a session and returns its ITunnelTransmit so release can happen outside the lock.
Units
2
High
WebSocket subprotocol matching splits recognition from instance setup
TWebSocketProtocol.SetSubprotocol is removed. Public virtual IsSubprotocol now performs read-only recognition, while protected virtual SetName configures a cloned or selected instance through the Name property. Binary, relay, client and Engine.IO protocol classes adopt the split so the shared prototype is not mutated during matching.
Breaking changes
mormot.net.relay+2
17f74f81+31-25
net: new TWebSocketProtocol.IsSubprotocol/SetName methods
Impact
Subprotocol negotiation becomes thread-safe because matching no longer mutates a shared protocol object. Any custom subclass that overrides or calls SetSubprotocol will fail to compile and must separate recognition from per-instance setup.
Migration required
Replace SetSubprotocol overrides with IsSubprotocol; move state changes into SetName when the selected protocol name configures the clone.
Public API
TWebSocketProtocol.SetSubprotocolremoved — Replaced by IsSubprotocol for matching and SetName for instance setup.TWebSocketProtocol.IsSubprotocoladded — Public virtual read-only subprotocol recognition method.TWebSocketProtocol.Namebehaviour-changed — Writing Name now calls virtual SetName, allowing per-instance setup.
3
High
Certificate-chain validation avoids deadlock but becomes caller-serialized
TEccCertificateChain.ValidateItems removes its surrounding fSafe.ReadLock because each nested IsValid call obtains the chain lock itself and could deadlock. Its public documentation now explicitly states that ValidateItems is not thread-safe and should run after loading.
Breaking changes
mormot.crypt.ecc
fbb6f6dc+7-10
crypt: fixed potential deadlock in TEccCertificateChain.ValidateItems
Impact
Validation no longer self-deadlocks, but concurrent mutation or another ValidateItems call can race the unprotected fItems traversal. Backends that expose chain reload and validation concurrently must serialize this method externally.
Migration required
Grep for ValidateItems and ensure it cannot overlap chain loading, revocation updates or another validation call.
Public API
TEccCertificateChain.ValidateItemsbehaviour-changed — The method is now explicitly not thread-safe and requires caller-side serialization.
Units
4
High
POSIX thread-pool shutdown closes its enqueue race
TSynThreadPool renames its POSIX queue lock to fPendingSafe and sets fTerminated while holding it. Enqueue checks termination under the same lock, workers abort a captured context when stopping, and DoTaskAbort makes TaskAbort exception-safe. TSynParallelProcess also documents and performs the BSD-required TOSLightLock initialization.
Fixes
mormot.core.threads
76f15a86+60-44
core: another pass on TSynThreadPool shuhtdown logic
Impact
A producer can no longer enqueue work into a pool that is already entering destruction, and queued contexts are less likely to leak or deadlock shutdown. This matters to POSIX servers built on TSynThreadPool; Windows IOCP keeps its separate path.
Upgrade recommended
Upgrade POSIX services that destroy active thread pools and exercise shutdown under load.
Units
5
High
Tunnel lists no longer invoke remote code while read-locked
TTunnelList adds Get to copy an ITunnelTransmit under its read lock. TunnelSend, GetInfo and GetAllInfo then call interface methods after unlocking; GetAllInfo snapshots both interfaces and session IDs. DeleteFrom rejects self-deletion, and later follow-ups further refine this sequence.
Fixes
mormot.net.tunnel
4be92b7a+44-39
net: fixed race/deadlock in TTunnelList
Impact
A callback that re-enters the list or releases another interface can no longer deadlock the list's reader-writer lock. Because this area keeps changing through build 16672, test relay creation, frame delivery and teardown only on the final revision.
Worth a look
Use this fix only as part of the complete sequence ending at build 16672 and run concurrent tunnel lifecycle tests.
Public API
TTunnelList.Getadded — Returns a thread-safe local ITunnelTransmit reference for a session.
Units
6
High
Follow-up closes three lock initialization and nesting defects
TAlgoZstd now creates its shared compression and decompression contexts lazily while holding their locks. THttpPeerCrypt explicitly initializes and finalizes its new TOSLightLock for BSD. TTunnelList.DeleteFrom copies session IDs under the source read lock and performs deletions after unlocking, avoiding nested list locks.
Fixes
mormot.lib.zstd+2
0a0f36a4+23-13
all: fixes several locking issues
Impact
This repairs regressions introduced earlier in the same batch: premature Zstandard context creation, an uninitialized BSD peer-client lock, and a possible tunnel list lock cycle. Builds 16660-16667 are not suitable landing points for those components.
Upgrade recommended
Skip the affected intermediate builds and use 2.4.16672 for Zstandard, peer-cache or tunnel deployments.
7
High
Time-zone lookup protects its shared last-result cache
TSynTimeZone adds fLastZoneSafe, a TLightLock dedicated to fLastZone and fLastIndex. LockedSearch checks and updates that cache under the new lock while leaving the hashed zone lookup under the existing outer read lock.
Fixes
mormot.core.search
55fa224e+17-12
core: fixed TSynTimeZone.LockedSearch thread safety
Impact
Concurrent conversions using different zone IDs can no longer race the shared last-zone text and index, which could return a mismatched cached index. Multi-threaded REST backends using TSynTimeZone.Default should take this fix.
Upgrade recommended
Upgrade services that perform concurrent TSynTimeZone conversions and rerun cross-zone date tests.
Units
8
High
Delphi builds catch up with the evolving tunnel types
mormot.net.tunnel introduces the public ITunnelTransmits dynamic-array alias, uses it for TTunnelList storage and local copies, changes Get's interface parameter to out, and replaces a non-inlineable TRAIL_SIZE reference inside FrameSession with SizeOf(TTunnelSession).
Compiler & platform
mormot.net.tunnel
804bc970+6-5
net: fixed Delphi compilation
Impact
The preceding tunnel revision did not compile with Delphi. This commit restores Delphi compatibility, but tunnel logic changes again in build 16672, so consumers should use the edition head.
Upgrade recommended
Do not stop on build 16668; compile Delphi tunnel projects against build 16672.
Public API
ITunnelTransmitsadded — Public alias for array of ITunnelTransmit.TTunnelList.Getsignature-changed — The ITunnelTransmit output parameter changes from var to out.
Units
9
High
BSD tunnel threads now initialize their OS lock
TTunnelLocalThread.Create calls fSafe.Init and its destructor calls fSafe.Done. On BSD, TOSLightLock maps to TRtlCriticalSection and cannot rely on zero-filled class storage alone. The ORM diff only clarifies the same existing requirement around TOrmPropertiesAbstract.fSafe.
Compiler & platform
mormot.net.tunnel+1
428ee221+6-4
net: fixed TTunnelLocalThread.Create on BSD
Impact
BSD tunnel creation could otherwise operate on an uninitialized critical section. Later commits continue changing tunnel code, so this fix should be consumed through build 16672.
10
Medium
Binary certificate export now excludes concurrent writers
TEccCertificateChain.SaveToBinary changes fSafe.ReadLock to fSafe.WriteLock because serialization temporarily toggles each TEccCertificate.fStoreOnlyPublicKey flag before and after SaveToStream.
Fixes
mormot.crypt.ecc
6c82cf93+4-4
crypt: fixed TEccCertificateChain.SaveToBinary thread safety
Impact
Parallel SaveToBinary calls can no longer race on the temporary certificate flag and produce inconsistent private/public serialization. The stronger lock may serialize rare concurrent exports but does not change the binary format.
Upgrade recommended
Upgrade backends that export one shared TEccCertificateChain from multiple threads.
Public API
TEccCertificateChain.SaveToBinarybehaviour-changed — Now takes the chain write lock during serialization.
Units
11
Medium
Zstandard calls stop serializing behind one shared context
TAlgoZstd preallocates shared compression and decompression contexts and attempts TryLock for each operation. A contending thread creates and frees a transient ZSTD_CCtx or ZSTD_DCtx instead of blocking until the shared context is available.
Performance
mormot.lib.zstd
83c4df20+44-36
lib: fixed TAlgoZstd context usage
Impact
Parallel Zstandard operations can run concurrently instead of forming a single lock queue. The unit is still marked unfinished and FPC-only, and build 16668 changes shared-context creation to lazy initialization, so this commit is not a safe standalone target.
Worth a look
If mormot.lib.zstd is enabled, benchmark and stress it only on build 16668 or later; prefer the edition head.
Units
12
Medium
POSIX pending work now leaves the queue in constant time
The POSIX TSynThreadPool pending array becomes a fixed circular FIFO tracked by fPendingFirst and fPendingLast. PopPendingContext advances an index instead of moving every remaining pointer, and the queue is allocated once from QueueLength.
Performance
mormot.core.threads
6f064610+43-30
core: ensure TSynThreadPool POSIX queue is O(1)
Impact
Dequeuing no longer scales with backlog size, shortening the internal lock hold time during bursts. The change is POSIX-only and does not alter the TSynThreadPool public API.
Units
13
Medium
Tunnel accept waits stop burning CPU under contention
TTunnelLocalThread.fSafe changes from TLightLock to TOSLightLock. OnReceived can therefore wait through the Linux or modern Windows futex path while the background thread is blocked in accept.
Performance
mormot.net.tunnel
75bf7660+4-4
net: optimized TTunnelLocalThread accept() step
Impact
Early incoming tunnel frames no longer spin on a CPU while waiting for the local socket to be accepted. This change required the BSD initialization fix in the next build.
Units
14
Medium
Peer-cache client contention moves to an OS-assisted lock
THttpPeerCrypt.fClientSafe changes from TLightLock to TOSLightLock, allowing contended waits to use the platform futex-style implementation instead of pure spinning.
Performance
mormot.net.server
a5e68484+3-3
net: refined THttpPeerCrypt HTTP client lock
Impact
Background direct downloads should consume less CPU when several threads contend for the shared HTTP client. This revision lacked the explicit BSD Init and Done calls added in build 16668.
Units
15
Low
POSIX thread-pool coordination returns to a short spin lock
The internal POSIX TSynThreadPool.fSafe field changes from TOSLightLock to the four-byte TLightLock, removing the associated Init and Done calls. The protected critical sections only assign pointers and queue metadata.
Performance
mormot.core.threads
844e8029+5-8
core: refined TSynThreadPool internal POSIX lock
Impact
Very short POSIX pool operations avoid OS-assisted lock overhead. The queue and shutdown logic is reworked again in builds 16670 and 16671, so this is not a standalone landing point.
Units
16
Low
SQL proxy transaction locking adopts the lighter OS primitive
TSqlDBProxyConnectionProtocol.fSafe changes from TOSLock to TOSLightLock, which maps to the lighter futex-backed mutex on Linux and Windows 8 or later. No transaction algorithm or public method changes.
Performance
mormot.db.proxy
907c92f5+3-3
db: refined TSqlDBProxyConnectionProtocol lock
Impact
Remote SQL proxy transactions may pay less lock overhead under light contention. There is no source migration or behavior change for application code.
Units
17
Low
Lock cleanup keeps runtime behavior unchanged
Comments and lock scopes are clarified across mormot.core.os, mormot.net.async, mormot.rest.http.server and mormot.soa.server. The async TLS path repeats its Enabled check while holding fGC1.Safe, and the SOA session lookup releases its read lock before returning.
Under the hood
mormot.core.os+3
ccde07c6+8-7
core: some code refactoring with no functional change
Impact
There is no public API or intended application behavior change. The double-check prevents duplicate TLS setup if several connections enter initialization together.
18
Low
A multithreaded harness exercises every lightweight lock family
The new test.core.threads unit adds 1,166 lines of stress coverage for TLightLock, TOSLightLock, TOSRWLightLock, TRWLightLock and related lock behavior across multiple worker threads.
Tests
719cd877+1168-2
tests: extended T*Lock coverage
Impact
Application code is unchanged. The suite raises confidence in the recent lock rewrites but does not by itself prove every higher-level tunnel or shutdown sequence is race-free.
19
Low
LDAP tests cover custom group-member attributes
TNetworkProtocols adds assertions for custom LDAP group-member parsing, including normalized distinguished names and the configured member attribute path. Only test/test.net.proto.pas changes.
Tests
4e783875+8-4
tests: cover LDAP custom members parsing
Impact
There is no application or framework runtime change. The tests protect the LDAP custom-member behavior introduced in the previous edition.
20
Low
The main test runner now executes the new lock suite
test/mormot2tests.dpr adds test.core.threads to its uses list, making the newly introduced lock tests part of the regular mormot2tests executable.
Tests
62bd0e82+5-2
tests: include test.core.threads in mormot2tests project
Impact
There is no runtime or application-code impact. Continuous test runs now exercise the lock harness that was added in the preceding commit.