Act nowcan affect a running systemReviewed all 14 non-merge commits and excluded one merge. The total is 569 additions and 493 deletions after excluding mormot.commit.inc and mormot.commit-num.inc. Final source at 913a072a26c0305bc468a79b658137de5f7d124b was inspected. No commit carries superseded_by metadata.
PostgreSQL array binding and concurrent waits are repaired
PostgreSQL batches no longer serialize unused array capacity or lose trailing quotes, closing both a hard failure and a silent value change. TPipeStream now returns EOF at its declared size, while the final POSIX poll path preserves timeouts across signals and temporary resource exhaustion. Thread-pool accounting is made atomic, and the rest of the batch builds shared task-pool helpers and broader concurrency tests.
TL;DR
- Land on build 2.4.17369 at 913a072a26c0305bc468a79b658137de5f7d124b.
- PostgreSQL bound arrays now honor ValuesCount and preserve values ending in one or more quotes.
- TPipeStream.Read no longer waits forever when the final block ends exactly at the declared stream size.
- POSIX poll retries now preserve the wall-clock timeout and handle EINTR, EAGAIN and POLLNVAL consistently.
- TSynThreadPool pending-task accounting is atomic, including failed IOCP queue submissions and waiter wake-ups.
- New TSynThreadTasks and TSynTestCase helpers make persistent shared test workers easier to use.
Upgrade advice
Take the complete head at build 2.4.17369, including merge 913a072a. Upgrade promptly if production uses PostgreSQL array parameters, TPipeStream, POSIX sockets or TSynThreadPool.WaitFor. Retest mixed external PostgreSQL batches, quoted array values, exact-size pipe reads, interrupted socket waits and concurrent pool completion. No raw commit carries superseded_by metadata, so no formal avoid range is asserted; do not stop on the intermediate POSIX build 2.4.17366.
14 of 14 entries · 15 commits upstream
1
High
POSIX poll retries now honor real timeouts and transient errors
PollNoEintr now returns the captured errno separately, retries both EINTR and EAGAIN, sleeps briefly for EAGAIN, and recomputes finite timeouts from GetTickCount64. TNetSocketWrap.WaitFor, TPollSocketPoll.WaitForModified and WaitForSeveral use this helper. The temporary retry loop added to RecvWait in the preceding commit is removed.
Fixes
mormot.net.sock+1
e2ef363c+48-56
net: properly fixed poll() usage in POSIX
Impact
POSIX Delphi and FPC servers avoid extending waits indefinitely after repeated signals, avoid a busy loop on EAGAIN and receive consistent timeout or fatal-error results across single and multi-socket polling. Windows paths are unchanged.
Upgrade recommended
Take build 2.4.17367 or later and retest interrupted reads, zero-time polls, infinite waits and shutdown cancellation on POSIX.
Public API
TNetSocketWrap.WaitForbehaviour-changed — Transient poll errors consume the original wall-clock timeout instead of restarting it.
2
High
Thread-pool completion counters no longer race dispatch or waiter wake-up
TSynThreadPool increments fPendingTasks atomically before exposing work to IOCP or worker events. A failed IocpPostQueuedStatus rolls both counters back. The new TaskDone performs the single atomic decrement, wakes WaitFor waiters when zero is reached and then invokes TaskAllDone from both normal and aborted completion paths.
Fixes
mormot.core.threads
8b5a68ed+29-25
core: fixed some potential races in TSynThreadPool
Impact
Concurrent workers can no longer lose pending-task updates or let WaitFor observe premature completion because enqueue increments raced task decrements. Failed Windows IOCP submissions no longer leave pending work counted forever. The shared Pascal counter fix applies to Delphi and FPC non-IOCP pools as well.
Upgrade recommended
Upgrade to build 2.4.17369 and stress concurrent Push, WaitFor, pool termination and failed enqueue paths on every backend you deploy.
Units
3
High
PostgreSQL array batches stop emitting invalid literals or losing final quotes
TSqlDBPostgresStatement.BindParams truncates its local VArray copy to VInt64 before BoundArrayToJsonArray serializes it. BoundArrayToJsonArray now treats two quotes as an escaped pair only while more than the SQL literal terminator remains, and advances over both characters in its length and copy passes.
Fixes
mormot.db.sql+1
d91c3a57+20-9
db: fixed PostgreSQL array binding of over-allocated arrays and of trailing quotes
Impact
External PostgreSQL batches with several operation groups no longer send unused array slots as malformed comma runs. Text values ending in a quote or consecutive quotes are no longer silently shortened. Delphi and FPC applications using PostgreSQL array parameters share this corrected path.
Upgrade recommended
Upgrade to head build 2.4.17369 and retest mixed Update/Add batches plus values ending in one and two quotes.
Public API
BoundArrayToJsonArraybehaviour-changed — Doubled quotes are consumed as pairs without mistaking the closing SQL quote for data.
4
High
Exact-size pipe reads now return EOF instead of waiting again
TPipeStream.Read checks fPending, fSize and fPosition before resetting fCanRead. When no bytes remain and the declared size has been reached, it exits the loop with result 0 instead of arming another wait.
Fixes
mormot.core.threads
82960cd5+6-2
core: fixed TPipeStream.Read
Impact
Consumers that read a sized pipe until EOF no longer hang when the last delivered block ends exactly on fSize. The fix applies to Delphi and FPC because it is in the shared Pascal implementation.
Upgrade recommended
Upgrade to build 2.4.17369 and add a regression read whose final write lands exactly on the declared size.
Public API
TPipeStream.Readbehaviour-changed — Returns 0 immediately after the declared size is fully consumed.
Units
5
Medium
POSIX receive waits recognize invalid descriptors and retry early wake-ups
TNetSocketWrap.RecvWait temporarily loops on nrRetry while recomputing its remaining POSIX timeout. POSIX event translation also maps POLLNVAL to neError and pseError. The following commit moves retry control into PollNoEintr and removes the RecvWait loop.
Fixes
mormot.net.sock+1
f3ea20ed+35-19
net: fixed TNetSocketWrap.RecvWait on POSIX
Impact
Build 2.4.17366 improved POSIX receive waits but was an intermediate implementation immediately replaced by the broader poll fix. POLLNVAL classification remains in the final code; applications should take the later head rather than this commit alone.
Public API
TNetSocketWrap.RecvWaitbehaviour-changed — This intermediate retry loop is replaced by the next commit.
6
Medium
Thread-task pools can dispatch TNotifyEvent callbacks directly
The new TSynThreadEventTask stores Sender, Worker and Tag and invokes Worker(Sender) from DoExecute. TSynThreadTasks.Add gains an overload that allocates and queues Count such tasks and returns how many were accepted. TSynThreadTask.DoExecute moves from public to protected visibility.
New features
mormot.core.threads
6fb9233f+50-4
core: TSynThreadTasks.Add() overload redirecting to a TNotifyEvent
Impact
Delphi and FPC code can submit object-method callbacks without defining a task subclass. Existing subclasses still override DoExecute, but external code that called DoExecute directly will no longer compile because the method is protected.
Worth a look
If application code calls TSynThreadTask.DoExecute directly, remove that call and submit the task through TSynThreadTasks.Add. New callback jobs can use the Sender/Worker overload.
Public API
TSynThreadEventTaskaddedTSynThreadTasks.Add(Sender, Worker, Count, Tag)addedTSynThreadTask.DoExecutesignature-changed — Visibility changes from public to protected.
Units
7
Low
Test cases gain shared task submission and completion helpers
TSynTestCase publishes RunTask, RunTasks and WaitTasks around a lazily created TSynTests.Tasks pool. TSynTests.Destroy frees that pool after inherited destruction, and the test runner waits for queued tasks before calling each case's CleanUp. Additional trace logging records synchronous Run calls, waits and progress.
New features
mormot.core.test
a7ca5495+81-18
core: new TSynTestCase.RunTask/RunTasks/WaitTasks methods
Impact
Custom mormot.core.test suites can schedule repeated TNotifyEvent workers without maintaining their own pool. Existing application backends are unaffected unless they build custom test infrastructure on these classes.
Worth a look
For custom concurrency tests, consider replacing per-case pools with RunTask, RunTasks and WaitTasks, then verify CleanUp does not free state before the shared tasks finish.
Public API
TSynTestCase.RunTaskaddedTSynTestCase.RunTasksaddedTSynTestCase.WaitTasksaddedTSynTests.Tasksadded
Units
8
Low
Shared test tasks receive a default timeout and platform-adjusted worker count
TSynTestCase.WaitTasks gains a 5000 ms default, task log messages become more explicit, and TSynTests.Tasks passes its initial count of 16 through ThreadCountAdjust before constructing TSynThreadTasks. The TSynThreadPool.WaitFor documentation now states that concurrent waiters are supported.
Under the hood
mormot.core.test+1
ba3f2135+16-9
core: refined tests thread pool
Impact
Custom test suites can omit the timeout when five seconds is suitable, and their shared pool respects constrained Windows configurations such as PRISM. Production server behavior is unchanged.
Public API
TSynTestCase.WaitTaskssignature-changed — TimeOutMS now defaults to 5000.
9
Low
Queue stress tests reuse the suite-wide task pool
TTestCoreThreads.TSynQueueSlow2 removes its private TSynThreadTasks instance. Push and waiter tasks are submitted through Owner.Tasks, and completion uses TSynTestCase.WaitTasks while the queue scenarios are split into separate try/finally lifetimes.
Tests
f8583177+179-191
tests: refined TTestCoreThreads.TSynQueueSlow2
Impact
Tests exercise queue wake, finalize and reset operations on the same persistent worker infrastructure as other cross-thread cases. There is no production backend behavior change.
10
Low
Cross-thread tests move onto one persistent task pool
TTestCoreThreads creates one TSynThreadTasks in Setup, frees it before event state in CleanUp and replaces TLoggedWorker dispatch with Add and WaitFor. Lock, read-write lock, TSynQueue and TSynEvent tests submit work to that pool, with ThreadCountAdjust applied to WorkerCount.
Tests
feb9bf67+52-58
test: persistent task pool shared by all cross-thread tests
Impact
The test suite validates queue reuse and completion with stable workers instead of repeatedly creating background threads. Production backend code and public runtime behavior are unchanged.
11
Low
Queue wake-up tests use pool completion instead of manual counters
TTestCoreThreads.TSynQueueSlow2 removes Done pointers, atomic done counters and its polling WaitForTasks loop. TSynQueuePushTask and TSynQueueWaitTask now only perform their queue operations, while TSynThreadTasks.WaitFor provides completion and trace messages expose task scheduling.
Tests
e474d85c+42-68
test: rewritten TTestCoreThreads.TSynQueueSlow2
Impact
The regression test now exercises the thread pool's own pending-task accounting while checking TSynQueue wake, finalize and reset behavior. Production application code is unchanged.
12
Low
Queue concurrency tests scale to the worker count available on PRISM
TTestCoreThreads.TSynQueueSlow2 derives waiters from Owner.Tasks.WorkThreadCount and caps the fixed result arrays at eight. All loops and masks use that runtime count. TSynTests.Tasks also logs the pool class and actual worker count.
Tests
mormot.core.test
1fc7cd6e+26-24
test: fixed threads adaptation on PRISM
Impact
The thread test suite no longer assumes eight workers on Windows ARM PRISM, where the pool may expose four. Production Delphi and FPC application code is unchanged.
Units
13
Low
Core thread tests switch from a private pool to the shared test pool
TTestCoreThreads removes its fTasks field, setup allocation, cleanup free and local RunTask, RunTasks and WaitTasks wrappers. Existing test calls now resolve to the helpers inherited from TSynTestCase and therefore use TSynTests.Tasks.
Tests
ac230852+2-28
test: let TTestCoreThreads use new sharead thread pool
Impact
The concurrency tests reuse one persistent pool for the suite and avoid repeated worker creation. No Delphi or FPC production application API is changed by this test migration.
14
Low
Peer-cache tests wait until their HTTP listener is bound
TNetworkProtocols.RunPeerCacheDirect calls localserver.WaitStarted after assigning OnRequest and before issuing direct-resource requests. The remaining edits only fix indentation in the huge-content test and a comment in mormot.net.sock.posix.
Tests
mormot.net.sock.posix
aa663674+9-8
test: fixed TNetworkProtocols.RunPeerCacheDirect HTTP server startup
Impact
The test no longer races port 8889 startup on slower hosts. There is no runtime application-code change for Delphi or FPC backends.