Act nowcan affect a running system
Lock and process fixes make build 16652 the safe landing point
This batch fixes two production hazards: a missed wake-up race in TOSLightLock and several long-standing POSIX RunRedirect failures, including orphaned child processes and an EOF busy loop. It also adds a futex-backed read-write lock, corrects ODBC type inference, expands LDAP and stack-trace APIs, and closes Delphi and BSD compilation regressions before the final build.
TL;DR
- TOSLightLock.UnLock now uses a mandatory CAS, preventing a waiter from being forgotten during unlock.
- POSIX RunRedirect no longer leaks child process groups or spins at 100% CPU after EOF, and now honors environment and working-directory arguments.
- ODBC column binding now uses the real precision and maps SQL_FLOAT separately from DECIMAL and NUMERIC.
- New public APIs include TOSRWLightLock, TDebugFile.StackTrace, LDAP formatting and DN filtering helpers, and TAsnObjects.
- Land on build 2.4.16652; earlier snapshots in this batch contain Delphi, BSD, or WaitForSafe source-compatibility regressions.
Upgrade advice
Pull the final 2.4.16652 revision for POSIX services using RunRedirect, workloads using TOSLightLock, ODBC consumers, or any Delphi/BSD build. Do not stop on an intermediate revision: later commits in the batch repair compilation and source compatibility introduced earlier in the same sequence.
21 of 21 entries · 23 commits upstream
1
High
POSIX RunRedirect stops leaking children and spinning after EOF
The FPC POSIX implementation replaces popen with a prepared fork/execve path. The child creates its own session, stdout is piped explicitly, timeouts signal the whole process group, EOF uses n<=0, environment and working-directory parameters are honored, simple commands use PATH from the supplied environment, and exitcode receives WaitProcess output unless aborted.
Fixes
mormot.core.os.posix
667509df+231-77
core: rewrite POSIX RunRedirect() from popen() to fork/execve - fix #463
Impact
INFINITE waits no longer enter a 100% CPU loop when the child closes stdout, aborts no longer leave shell-spawned workers orphaned under PID 1, wrkdir no longer changes the parent process, and POSIX env/options are no longer silently ignored. Command execution semantics are materially corrected without changing the public signature.
Upgrade recommended
Upgrade POSIX services using RunRedirect, then test successful exit, EOF, timeout, callback abort, custom env/PATH, working directory, and descendant termination.
Public API
RunRedirectbehaviour-changed — POSIX execution now uses fork/execve, honors existing parameters, returns the actual exit code, and terminates process groups.
Units
2
High
TOSLightLock could miss a waiter during unlock
TOSLightLock.UnLock now always performs LockedExc32 from EV_LOCKED to EV_NONE before choosing the wake-up path. The former Intel fast path read fMutex and then stored EV_NONE non-atomically, allowing a waiter to publish EV_WAITER between those operations and be overwritten.
Fixes
mormot.core.os
ad8f8d38+6-16
core: fixed race in TOSLightLock.UnLock
Impact
Under contention, the old unlock sequence could forget to wake a waiter and leave a thread blocked indefinitely. Services using TOSLightLock directly or through framework internals should take this fix despite the small uncontended CAS cost.
Upgrade recommended
Rebuild concurrency-heavy services on build 2.4.16652 or later and rerun lock-contention tests.
Public API
TOSLightLock.UnLockbehaviour-changed — Unlock now uses an atomic transition before deciding whether a waiter must be woken.
Units
3
High
Delphi compilation is restored for new lock and LDAP APIs
TOSRWLightLock removes Delphi inline directives from methods that could not be inlined there. In mormot.net.ldap, the anonymous set declaration becomes a TNormalizeDN enum plus a TNormalizeDNs set, and DNToCN, DNsToCN and NormalizeDN take TNormalizeDNs.
Compiler & platform
mormot.core.os+1
63b43ba0+12-12
net: fixed Delphi compilation
Impact
Delphi can compile the APIs introduced earlier in the batch. Code written against transient build 16630 must replace set-typed TNormalizeDN variables with TNormalizeDNs; code using the final API has no migration.
Upgrade recommended
Skip intermediate builds and compile Delphi applications against build 2.4.16652; grep for TNormalizeDN if build 16630 was already adopted.
Public API
TNormalizeDNsadded — The set type used by DN filtering is now explicitly named TNormalizeDNs.DNToCN / DNsToCN / NormalizeDNsignature-changed — Their extend parameter uses TNormalizeDNs instead of the transient set-typed TNormalizeDN.
4
High
Final revision restores BSD builds and the WaitForSafe default
In mormot.core.os, SPIN_FUTEX was moved outside the OSFUTEX-only block so the fallback TOSRWLightLock code can reference it on BSD. TSynEvent.WaitForSafe also regains DisableSafe=false as its public default argument.
Compiler & platform
mormot.core.os
951b4afd+6-5
core: fixed BSD compilation
Impact
BSD targets can compile the new read-write lock, and existing source that calls WaitForSafe with only a timeout compiles again. This is the revision to use instead of intermediate builds 16645-16651.
Upgrade recommended
Land on this revision before compiling BSD targets or adopting any other change from this edition.
Public API
TSynEvent.WaitForSafesignature-changed — DisableSafe once again defaults to false after being made mandatory in build 16645.
Units
5
Medium
ODBC binding now respects declared numeric precision
TSqlDBOdbcStatement.BindColumns passes the actual ColumnSize and DecimalDigits from SQLDescribeColW to ODBCColumnToFieldType instead of a hard-coded precision of 10. SQL_FLOAT is also moved out of the DECIMAL/NUMERIC integer and currency heuristic and maps directly to ftDouble.
Fixes
mormot.db.sql.odbc
5c4a0328+7-7
db: fixed TSqlDBOdbcStatement.BindColumns
Impact
ODBC result columns may now expose the correct ftInt64, ftCurrency or ftDouble type instead of a type inferred from precision 10. Applications relying on previously incorrect metadata or conversion behavior should be regression-tested.
Upgrade recommended
Retest DECIMAL, NUMERIC and FLOAT result columns through every ODBC driver used in production.
Units
6
Medium
WaitForSafe gives synchronized main-thread work priority
On the main thread, TSynEvent.WaitForSafe now calls CheckSynchronize(10) and then polls the event with WaitFor(0), instead of checking synchronization for 1 ms and blocking on the event for 10 ms. A zero timeout bypasses the safe loop.
Fixes
mormot.core.os
46404091+6-5
core: fixe TSynEvent.WaitForSafe Synchronize() call
Impact
Main-thread waits process queued Synchronize calls more promptly and check the event without an additional blocking interval. Code sensitive to synchronization ordering or timeout boundaries should be retested.
Worth a look
Retest main-thread RunWait or direct WaitForSafe usage that depends on precise timeout or Synchronize ordering.
Public API
TSynEvent.WaitForSafebehaviour-changed — Safe main-thread waits now prioritize CheckSynchronize and use a nonblocking event check.
Units
7
Medium
Stack traces can be captured without raising an exception
TDebugFile gains StackTrace overloads returning RawUtf8 or appending to TTextWriter, with skip, depth and TSynLogStackTraceUse controls. FPC uses CaptureBacktrace, Delphi Windows uses RtlCaptureStackBackTrace with a Win32 manual fallback, and TSynLog shares the implementation.
New features
mormot.core.log
702c8d0c+208-47
core: add TDebugFile.StackTrace() methods - request #401
Impact
Diagnostic endpoints and error reports can capture the current thread stack without manufacturing an exception. On FPC x86_64, useful frames still depend on stack-frame generation; Delphi POSIX currently appends nothing.
Public API
TDebugFile.StackTraceadded — Two overloads capture the current thread stack as text without an exception.
Units
8
Medium
New OS-assisted read-write lock favors readers
mormot.core.os adds TOSRWLightLock, a 32-bit reader counter and writer flag using WaitOnAddress/WakeByAddress on Windows 8+, futex on Linux, and spinning fallbacks elsewhere. It exposes Init, Done, read/write lock, try-lock, unlock and IsLocked methods.
New features
mormot.core.os
583fe89c+196-7
core: new TOSRWLightLock efficient object
Impact
Applications gain a low-overhead reader-preference alternative to writer-preference TRWLightLock. Continuous readers may delay a writer, and attempting a write lock inside a read or write lock still deadlocks by contract.
Worth a look
Use TOSRWLightLock only where reader preference is acceptable; keep TRWLightLock when queued writers must block new readers.
Public API
TOSRWLightLockadded — New reader-preference, OS-assisted read-write lock record/object.
Units
9
Medium
LDAP DN conversion can select CN, OU and DC components
DNToCN, DNsToCN and NormalizeDN gain an optional extend set controlling whether CN, OU and DC parts appear in the result. Defaults preserve the former all-components output; build 16639 later gives the set its final TNormalizeDNs name.
New features
mormot.net.ldap
9411aea0+52-24
net: introducing TNormalizeDN extend e.g. for DNToCN()
Impact
LDAP applications can derive canonical or normalized names containing only selected DN levels without parsing the string again. Existing calls keep their output because DN_ALL remains the default.
Public API
DNToCN / DNsToCN / NormalizeDNsignature-changed — An optional component-selection parameter was appended with DN_ALL as default.
Units
10
Medium
Cross-platform futex layer can wake every waiter
A new OsWakeAllOnValue callback is resolved to WakeByAddressAll on Windows 8+ and to FUTEX_WAKE_PRIVATE with MaxInt on Linux. It complements OsWaitOnValue and the single-waiter OsWakeOnValue callback.
New features
mormot.core.os+2
c849d860+25-12
core: implemented WakeByAddressAll() on Win8+ and Linux
Impact
Framework primitives such as TOSRWLightLock can wake all readers after a writer releases the lock. Existing application behavior is unchanged unless code chooses the new callback directly.
Public API
OsWakeAllOnValueadded — New optional callback for waking every waiter on a 32-bit value.
11
Medium
TLoggedWorker waits on an event instead of polling
TLoggedWorker now owns a TSynEvent named fIdle. RunOne resets it when work starts, RunDone signals it when fRunning reaches zero, and RunWait delegates its timeout and optional main-thread synchronization to WaitForSafe.
Performance
mormot.core.threads
a22faf70+31-20
core: let TLoggedWorker use TSynEvent for WaitFor()
Impact
Waiting no longer wakes every 10 ms, reducing idle CPU use and reacting immediately when the worker becomes idle. The contract now explicitly permits only one concurrent RunWait caller.
Worth a look
Check that application code does not call RunWait concurrently on the same TLoggedWorker instance.
Public API
TLoggedWorker.RunWaitbehaviour-changed — The implementation is event-driven and supports one waiting thread at a time.
Units
12
Low
LDAP attribute lists gain formatted value insertion
TLdapAttributeList.AddFmt formats a RawUtf8 value with FormatUtf8 and forwards it to the existing Add overload with the requested TLdapAddOption. The method returns the affected TLdapAttribute.
New features
mormot.net.ldap
c7ba1bb3+17-3
net: new TLdapAttributeList.AddFmt() method
Impact
LDAP client code can build formatted attribute values without a temporary string. Existing Add calls and behavior are unchanged.
Public API
TLdapAttributeList.AddFmtadded — New formatted-value overload-like helper using FormatUtf8 arguments.
Units
13
Low
ASN.1 objects gain a typed dynamic-array helper
mormot.core.os.security adds TAsnObjects as an array of TAsnObject and AsnAddItem to append one encoded object while returning its zero-based index. LDAP modifier construction is the stated use case.
New features
mormot.core.os.security
446a5be1+15-2
core: new TAsnObjects type and its AsnAddItem() wrapper
Impact
Code assembling multiple ASN.1 values can use a named array type and avoid repeated SetLength boilerplate. Existing ASN.1 encoding APIs are unchanged.
Public API
TAsnObjectsadded — Dynamic array type for multiple TAsnObject values.AsnAddItemadded — Appends an ASN.1 object and returns its index.
14
Low
Windows socket polling takes the pending-event fast path
TPollSockets.PollForPendingEvents no longer limits its fPendingSafe.TryLock fast path to POSIX. Windows now attempts the same nonblocking lock and moves pending events without entering the slower fallback when contention is absent.
Performance
mormot.core.os+1
79dd6c23+2-5
core: some TPollSockets optimization on Windows
Impact
Windows servers with frequent socket subscription changes may spend less time merging pending events. The change is internal and requires no application code modification.
15
Low
Pending socket events switch to the cheaper low-contention lock
TPollSockets.fPendingSafe changes from TOSLightLock to TLightLock. The commit records that pending-event contention is small and that the mandatory CAS added to TOSLightLock.UnLock makes the simpler lock faster here.
Performance
mormot.net.sock
60cb8bc7+3-3
net: let TPollSockets use TLightLock for the Pending envents
Impact
Socket polling keeps the race fix while avoiding its extra uncontended unlock cost for this internal queue. No application code change is required.
Units
16
Low
TRWLightLock gains a no-op finalizer for lock interchangeability
TRWLightLock now exposes Done as an empty method, matching the lifecycle shape of TOSLock and related primitives. Its documentation also states writer preference explicitly.
Compiler & platform
mormot.core.os
d5844b6a+12-2
core: new TRWLightLock.Done void method
Impact
Generic code can call Init and Done for either lock type without conditional cleanup. Done performs no work, so existing runtime behavior is unchanged.
Public API
TRWLightLock.Doneadded — No-op lifecycle method compatible with other lock records.
Units
17
Low
Database and REST class registries use pointer arrays
Internal class-definition registries in TSqlDBConnectionProperties and TRest now call PtrArrayAddOnce instead of storing TClass values through TObject casts. The ODBC helper name and formatting were normalized without changing its scope.
Under the hood
mormot.db.sql+2
9fbd6c32+13-13
all: minor code refactoring
Impact
There is no functional change and no application migration. The registry representation now matches the pointer values it actually stores.
18
Low
Tunnel relay test drops an unused random-generator variable
TNetworkProtocols.TunnelRelay no longer declares and initializes an unused PLecuyer variable. No test assertion, control flow, or framework unit changed.
Tests
b1f1302d+2-4
tests: minor unused variable cleanup
Impact
There is no impact on application code or runtime behavior; the change only removes a compiler warning in the test suite.
19
Low
Text unit asserts its formatter still fits the stack buffer
The mormot.core.text initialization adds an Assert that SizeOf(TFormatUtf8) does not exceed SizeOf(TTextWriterStackBuffer). The current expected sizes are 4 KB and 8 KB respectively.
Tests
mormot.core.text
e171e5aa+3-2
core: added paranoid Assert() about TFormatUtf8
Impact
Normal builds see no behavior change. Assertion-enabled builds will fail early if a future formatter growth would invalidate the stack-buffer assumption.
Units
20
Low
Lock preference and pending-socket behavior are documented
Comments now distinguish TOSRWLightLock reader preference from the Windows SRW locking pattern and describe TPollSockets pending-event overrides more precisely. EnsurePending also names its starting index explicitly without changing the search.
Docs
mormot.core.os+1
181c2d7b+13-9
core: mostly comments/documentation precisions
Impact
There is no runtime or source impact. The documentation makes lock-selection tradeoffs and socket subclass responsibilities clearer.
21
Low
TUri documentation confirms bare server names are accepted
The TUri.From comment now explicitly lists a bare server name, without scheme or path, as an input interpreted as HTTP. No parser code changed.
Docs
mormot.net.sock
ff62d966+4-4
net: TUri.From() documentation precision
Impact
There is no runtime change. Developers can rely on already-existing bare-host parsing with clearer documentation.