mORMot2 DailyDaily read of upstream commits

WebSocket deadlocks are closed while TLS and exception logging settle

Async WebSocket broadcast and RTSP forwarding no longer send while holding a connection read lock, closing a deadlock when a failed write removes the peer. SChannel now treats orderly send-side shutdown as a closed connection and completes TLS 1.3 renegotiation coverage. Exception interception is also repaired across concurrent threads, Delphi Win32 and Delphi POSIX, while AVX-512 detection now verifies that the OS actually saves ZMM state.

TL;DR

  • Land on build 2.4.17324 at 05e40cd692d391efb62ed97f78f264867477e8e5.
  • TWebSocketAsyncServer.WebSocketBroadcast and TPostConnection.OnRead no longer deadlock when a failed send deletes a connection.
  • SChannel sends return nrClosed for graceful shutdown and its receive path handles TLS 1.3 SEC_I_RENEGOTIATE.
  • Concurrent exceptions are no longer lost, Delphi Win32 duplicate records are suppressed, and Delphi POSIX gains interception.
  • IntelAvx512 is now a startup-computed variable filtered through XCR0, with cpuAVX512 identifying OS-enabled x86-64-v4 support.
  • The Windows http.sys WebSocket regression test for issue #418 is disabled because it remains unreliable on Windows 11.

Upgrade advice

Land on build 2.4.17324. Upgrade promptly if production uses TWebSocketAsyncServer.WebSocketBroadcast, RTSP-over-HTTP forwarding, Windows SChannel, TTunnelLocal or relies on complete exception logs under concurrency. Stress broadcasts while clients reset, TLS 1.3 key updates and peer shutdown, and tunnel close during WaitForHandshake. The raw payload has no superseded_by markers, so no formal avoid window is asserted; take each ordered sequence whole. Keep a separate eye on issue #418 because its Windows http.sys WebSocket test is disabled.

Build2.4.17293 → 2.4.17324
Commits30
Lines+992 -386
Read6 min

27 of 27 entries · 30 commits upstream

Stories
# Entry Category Units
1 High WebSocket broadcasts use safe snapshots for both timeout modes GetConnectionHandles and GetConnectionInstances iterate the actual class-reference array. WebSocketBroadcast uses instance snapshots only for timeout zero, handle lookup for blocking sends, and checks ConnectionFind for nil before SendDirect. Fixes mormot.net.async+1 f253c603+53-40

net: properly fixed #629

Impact

Broadcasting to peers that close concurrently no longer risks deadlock or a nil dereference, while non-blocking broadcasts avoid repeated searches.

Public API

  • TWebSocketAsyncServer.WebSocketBroadcastbehaviour-changed

Arnaud Bouchez · 14:12 UTC · PR #629 · view the commit on GitHub →

2 High SChannel continues the handshake on TLS 1.3 key updates TSChannelNetTls.Receive recognizes SEC_I_RENEGOTIATE and feeds remaining input back into HandshakeStep. The test client can request SSL_key_update mid-transfer, and TBuffer32K is added for the expanded TLS test paths. Fixes mormot.core.base+3 8e6133ef+56-26

net: SChannel fixes and new tests about TLS 1.3 renegotiation

Impact

Long-lived SChannel TLS 1.3 connections survive peer key updates instead of failing after DecryptMessage requests renegotiation.

Public API

  • TBuffer32Kadded

Arnaud Bouchez · 16:27 UTC · view the commit on GitHub →

3 High SChannel socket closure is reported without shutdown exceptions NetErrorFromSystem maps WSAENOTCONN and WSAESHUTDOWN to nrClosed. TSChannelNetTls.SendRawAll returns TNetResult, callers stop when the socket closes, and receive errors include the normalized result and Windows message. Fixes mormot.net.sock+1 2d800eaa+46-26

net: several fixes to SChannel socket closure

Impact

Normal Windows TLS teardown no longer appears as a fatal ESChannel path, including during handshake-token or encrypted-output sending.

Public API

  • NetErrorFromSystembehaviour-changed

Arnaud Bouchez · 17:01 UTC · view the commit on GitHub →

4 High WebSocket broadcast sends leave the connection read lock before writing WebSocketBroadcast copies destination handles while holding ConnectionLock.ReadOnlyLock, releases it, then encodes and sends frames via ConnectionFind. Documentation warns that Write may call ConnectionDelete and must not run under a read lock. Fixes mormot.net.async+1 fd8b176f+25-15

net: fixed TWebSocketAsyncServer.WebSocketBroadcast deadlock when a send fails

Impact

A failed broadcast send can delete its connection without trying to upgrade a read lock and deadlocking the async server. Later commits complete nil and lifetime handling.

je · 09:21 UTC · view the commit on GitHub →

5 High Concurrent exceptions no longer disappear from TSynLog Exception handlers stop clearing the global _RawLogException callback around each log operation. Recursion remains suppressed per thread through PerThreadInfo.ExceptionIgnore, so other threads keep their handler active. Fixes mormot.core.os+1 6b486e00+18-17

core: fixed Exception diseappearing in logs on multi-thread process :(

Impact

Exceptions raised simultaneously on different threads are no longer silently omitted from logs. Delphi and FPC services with concurrent workloads are affected.

Arnaud Bouchez · 10:05 UTC · view the commit on GitHub →

6 High WebSocket clients and local tunnels publish shutdown state safely TTunnelLocal.ClosePort sets fClosed while fSendSafe is still locked, THttpClientWebSockets.Destroy uses FreeAndNilSafe, and callback requests use fProtocol.ConnectionFlags rather than reaching through the owner thread. Fixes mormot.net.tunnel+1 b60779e1+5-5

net: hardened THttpClientWebSockets shutdown

Impact

Tunnel senders and WaitForHandshake observers see closure in a consistent order, and WebSocket client destruction is safer under concurrent callbacks.

Arnaud Bouchez · 08:56 UTC · view the commit on GitHub →

7 High A graceful SChannel send-side close now returns nrClosed TSChannelNetTls.SendRawAll initializes the result as nrClosed before examining a zero-byte send, so an orderly peer shutdown exits instead of raising ESChannel. Fixes mormot.net.sock.windows 1a03af2e+5-4

net: fixed graceful disconnection during SChannel sending

Impact

Windows TLS clients and servers can close during a pending send without converting normal shutdown into an exception.

Arnaud Bouchez · 17:52 UTC · view the commit on GitHub →

8 High RTSP-over-HTTP forwarding no longer writes under ConnectionLock TPostConnection.OnRead replaces ConnectionFindAndLock(cReadOnly) with ConnectionFind before WriteString, because a failed write may call ConnectionDelete and require the same lock in write mode. Fixes mormot.net.rtsphttp a1c8844a+3-4

net: fixed TPostConnection.OnRead holding ConnectionLock during WriteString()

Impact

RTSP-over-HTTP POST forwarding no longer self-deadlocks when the paired RTSP connection disappears during a write.

je · 09:21 UTC · view the commit on GitHub →

9 Medium Modern Delphi uses one exception hook without duplicate records SynRaiseProc is generalized for RaiseExceptObjProc, Windows vectored and unwind handlers skip Delphi exceptions already handled by that hook, fake RTL exception codes are filtered, and original handlers remain chained. Fixes mormot.core.os b6de95bf+56-40

core: refined Delphi exception logging

Impact

Delphi exception logs are emitted once while third-party and RTL hooks keep running. Windows and POSIX routing is more consistent.

Arnaud Bouchez · 12:08 UTC · view the commit on GitHub →

10 Medium Delphi Win32 exceptions stop being logged twice Delphi 2009+ Win32 installs RaiseExceptObjProc for language exceptions and makes RTLUnwindProc skip cRtlException records already handled by the cleaner hook. Fixes mormot.core.os 0a512f58+51-23

core: fixed duplicate Exception logging on Delphi Win32

Impact

TSynLog no longer emits duplicate records for ordinary Delphi Win32 exceptions; older Delphi and FPC retain their existing hooks.

Arnaud Bouchez · 09:24 UTC · view the commit on GitHub →

11 Medium Exception logs tolerate an unknown instruction address SynLogException omits the 'at' pointer field when TSynLogExceptionContext.EAddr is zero, using safe writer calls before appending any available stack frames. Fixes mormot.core.log 128e7134+11-6

core: fixed SynLogException() with Ctxt.EAddr = 0

Impact

Delphi LLVM exception records with no reliable address remain well-formed and retain their available stack trace.

Arnaud Bouchez · 12:06 UTC · view the commit on GitHub →

12 Medium Default RSA generation timeouts scale with key size TRsa.Generate multiplies RSA_DEFAULT_GENERATION_TIMEOUTMS by 4 for 3072-bit keys, 16 for 4096-bit keys and 64 for 7680/8192-bit keys, while preserving explicit custom timeouts. Fixes mormot.crypt.rsa ad2f17e3+10-3

crypt: RSA generation timeout auto adaptation

Impact

Weak CPUs are less likely to report rgrTimeout when generating large RSA keys with defaults. Callers supplying their own TimeOutMS keep the same limit.

Worth a look

Review latency budgets for 3072-bit and larger key generation; set TimeOutMS explicitly if the longer default is unsuitable.

Public API

  • TRsa.Generatebehaviour-changed

Arnaud Bouchez · 06:41 UTC · view the commit on GitHub →

13 Medium TTunnelLocal marks closure before waking handshake waiters TTunnelLocal.ClosePort and the remote-close frame path set fClosed before signaling fHandshakeEvent, so WaitForHandshake cannot wake and observe the previous open state. Fixes mormot.net.tunnel e7c2a1fb+6-6

net: hardened TTunnelLocal.ClosePort

Impact

Handshake waiters receive a consistent closed state. The following commit moves the assignment inside the send lock to complete synchronization.

Arnaud Bouchez · 08:49 UTC · view the commit on GitHub →

14 Medium Async connection owners can snapshot live handles or instances TAsyncConnections adds public GetConnectionHandles and GetConnectionInstances methods that copy non-closed connections while holding ConnectionLock.ReadOnlyLock. New features mormot.net.async 40766691+61-2

net: new TAsyncConnections.GetConnectionInstances/GetConnectionHandles methods

Impact

Custom async code gains safer snapshot APIs. Instance snapshots are only protected from recycling for KeepConnectionInstanceMS and must not be used for blocking work.

Worth a look

Prefer handles for delayed or blocking work; use instance snapshots only for immediate non-blocking operations.

Public API

  • TAsyncConnections.GetConnectionHandlesadded
  • TAsyncConnections.GetConnectionInstancesadded

Arnaud Bouchez · 13:50 UTC · view the commit on GitHub →

15 Medium AVX-512 detection now verifies OS-managed ZMM state mormot.core.base adds IsZmmOSEnabled assembly, AVX-512 feature-set constants, IntelAvx512 filtering through XCR0 and cpuAVX512 in TX64CpuFeatures for the x86-64-v4 subset. New features mormot.core.base+1 6941a143+56-2

core: introducing IntelAvx512 to detect AVX-512 support enabled by the OS

Impact

CPU dispatch can avoid AVX-512 instructions when CPUID advertises them but an old OS or VM does not preserve opmask and ZMM registers, preventing #UD faults.

Worth a look

Use IntelAvx512 or cpuAVX512 rather than raw CpuFeatures before custom EVEX dispatch, especially inside VMs.

Public API

  • CPUAVX512FEATURESadded
  • CPUAVX512X64V4added
  • IntelAvx512added
  • cpuAVX512added

Dimitris Kounalakis · 04:03 UTC · view the commit on GitHub →

16 Medium The OpenSSL wrapper exposes TLS 1.3 key updates mormot.lib.openssl11 adds SSL_KEY_UPDATE_NOT_REQUESTED, SSL_KEY_UPDATE_REQUESTED and the optional SSL_key_update function to dynamic and static OpenSSL bindings. New features mormot.lib.openssl11 8f4fd5a1+28-14

lib: new SSL_key_update() OpenSSL function wrapper

Impact

Low-level OpenSSL users can request a TLS 1.3 traffic-key rotation. Older OpenSSL libraries keep loading because the symbol is optional.

Public API

  • SSL_key_updateadded
  • SSL_KEY_UPDATE_NOT_REQUESTEDadded
  • SSL_KEY_UPDATE_REQUESTEDadded

Arnaud Bouchez · 15:09 UTC · view the commit on GitHub →

17 Medium Delphi POSIX gains preliminary automatic exception logging mormot.defines.inc enables exception interception on Delphi POSIX, and mormot.core.os.posix selects RaiseExceptObjProc instead of the FPC RaiseProc hook. New features mormot.core.os+1 e2798199+11-6

core: preliminary exception logging on Delphi POSIX

Impact

Delphi Linux and mobile-style POSIX builds can emit automatic TSynLog exception records where they previously had no interceptor. FPC behavior is unchanged.

Worth a look

Check log volume and exception stack formatting on each Delphi POSIX target before production rollout.

Public API

  • RawExceptionInterceptbehaviour-changed

Arnaud Bouchez · 12:16 UTC · view the commit on GitHub →

18 Low AVX-512 availability is computed once during CPU startup detection The public no-argument IntelAvx512 function becomes a TIntelCpuFeatures global populated by TestCpuFeatures, avoiding repeated XGETBV calls while retaining the XCR0 filter. Performance mormot.core.base fb734823+14-18

core: IntelAvx512 is now a global variable pre-computed at startup

Impact

Feature checks become a variable read. Ordinary Pascal expressions using IntelAvx512 remain natural, but code explicitly taking the function address must change.

Worth a look

Search only low-level code that stores or passes the IntelAvx512 function pointer; normal value checks need no migration.

Public API

  • IntelAvx512signature-changed — The same-edition function is finalized as a precomputed global variable.

Dimitris Kounalakis · 13:41 UTC · view the commit on GitHub →

19 Low Delphi exception interception is guarded by actual RTL availability The POSIX include defines WITH_RAISEEXCEPTOBJPROC only when HASRAISEEXCEPTOBJPROC is available; the remaining changes clarify stack filtering and comments. Compiler & platform mormot.core.log+3 3327833c+17-15

core: some minor fixes

Impact

Older Delphi targets avoid compiling against an unavailable interception hook. FPC and supported modern Delphi behavior is unchanged.

Arnaud Bouchez · 13:00 UTC · view the commit on GitHub →

20 Low mTLS tests stream large request bodies across many TLS records test.net.proto adds a large POST route, chunked client sends, CRC32C verification and TLS 1.2/TLS 1.3 cases for SChannel and OpenSSL servers. Tests test.net.proto 9a5d0e77+105-28

test: cover huge content with mTLS

Impact

Regression coverage now spans record boundaries and large mTLS payloads; no application API changes.

Arnaud Bouchez · 15:48 UTC · view the commit on GitHub →

21 Low A reset-peer regression test reproduces the WebSocket broadcast deadlock TNetworkProtocols adds a server, broadcast worker and 50 short-lived clients that close with unread frames and RST; failure to accept the next client exposes a wedged server. Tests test.net.proto c0f6defb+83-0

test: new TNetworkProtocols._TWebSocketAsyncServer regression test

Impact

The test captures issue #629 without changing production behavior.

je · 09:21 UTC · view the commit on GitHub →

22 Low mTLS test workers catch and classify network exceptions reliably The SChannel/OpenSSL test runner wraps each worker execution, logs the raised class, accepts ENetSock as the expected network failure and gives each async server a distinct logged process name. Tests mormot.core.log+1 cc95a9eb+28-14

test: hardened error catching for SChannel/OpenSSL mTLS

Impact

TLS regression failures become diagnosable without changing application runtime behavior.

Arnaud Bouchez · 08:08 UTC · view the commit on GitHub →

23 Low The WebSocket deadlock regression joins the multithread suite The issue #629 regression is moved into SlowTests, uses TLoggedThread and runs with the other background network tests when /multithread is enabled. Tests test.net.proto bb3be0ae+23-16

tests: fixed #629 coverage

Impact

Only test scheduling and diagnostics change; production code is unaffected.

Arnaud Bouchez · 14:13 UTC · PR #629 · view the commit on GitHub →

24 Low WebSocket broadcast stress testing starts only after the first client connects The broadcast regression thread starts after the first successful client connection, alternates zero and 50 ms sends, records its local sent count and avoids dereferencing that count if shutdown wedges. Tests test.net.proto 9a417d31+21-15

test: fixed WS broadcast thread synchronization

Impact

Only the regression harness changes; production Delphi/FPC code and APIs are unaffected.

Arnaud Bouchez · 17:52 UTC · view the commit on GitHub →

25 Low The broadcast regression alternates blocking and non-blocking sends The WebSocket regression test moves the sent counter out of the worker object and alternates aTimeOut between zero and 50 ms while repeatedly connecting and resetting clients. Tests test.net.proto 0eadadb7+22-7

test: enhanced WS broadcast process

Impact

Coverage now exercises both WebSocketBroadcast connection-snapshot paths; application behavior is unchanged.

Arnaud Bouchez · 17:13 UTC · view the commit on GitHub →

26 Low The Windows http.sys WebSocket regression test is temporarily disabled TNetworkProtocols._HttpApiWebSocketServer now exits immediately, disabling issue #418 coverage because the test is reported unreliable, especially on Windows 11. Tests test.net.proto 05e40cd6+3-2

tests: disable issue #418 coverage

Impact

Application behavior is unchanged, but CI no longer detects regressions in this Windows http.sys WebSocket scenario.

Arnaud Bouchez · 18:10 UTC · PR #418 · view the commit on GitHub →

27 Low LDAP and WebSocket comments are clarified without runtime changes Comments around CldapBroadcast, AVX-512 globals and WebSocket client-ID sorting are corrected or clarified; executable statements remain unchanged. Docs mormot.core.base+2 d3366efd+6-6

net: minor fixes

Impact

No impact on Delphi/FPC application code or runtime behavior.

Arnaud Bouchez · 15:09 UTC · view the commit on GitHub →

Upstream activity commits per day · last 60 days
Commits977
Active days44/60
Busiest2026-09-30
Peak60
2026-08-15: 0 commits2026-08-16: 0 commits2026-08-17: 0 commits2026-08-18: 0 commits2026-08-19: 0 commits2026-08-20: 0 commits2026-08-21: 0 commits2026-08-22: 0 commits2026-08-23: 0 commits2026-08-24: 0 commits2026-08-25: 0 commits2026-08-26: 0 commits2026-08-27: 26 commits2026-08-28: 11 commits2026-08-29: 10 commits2026-08-30: 1 commit2026-08-31: 23 commits2026-09-01: 0 commits2026-09-02: 20 commits2026-09-03: 29 commits2026-09-04: 35 commits2026-09-05: 13 commits2026-09-06: 0 commits2026-09-07: 0 commits2026-09-08: 23 commits2026-09-09: 10 commits2026-09-10: 17 commits2026-09-11: 23 commits2026-09-12: 14 commits2026-09-13: 9 commits2026-09-14: 7 commits2026-09-15: 4 commits2026-09-16: 24 commits2026-09-17: 26 commits2026-09-18: 21 commits2026-09-19: 59 commits2026-09-20: 13 commits2026-09-21: 5 commits2026-09-22: 34 commits2026-09-23: 17 commits2026-09-24: 57 commits2026-09-25: 22 commits2026-09-26: 26 commits2026-09-27: 0 commits2026-09-28: 2 commits2026-09-29: 19 commits2026-09-30: 60 commits2026-10-01: 26 commits2026-10-02: 30 commits2026-10-03: 32 commits2026-10-04: 15 commits2026-10-05: 1 commit2026-10-06: 26 commits2026-10-07: 37 commits2026-10-08: 27 commits2026-10-09: 35 commits2026-10-10: 33 commits2026-10-11: 9 commits2026-10-12: 11 commits2026-10-13: 35 commits
2026-08-152026-10-13