[PHP-DEV] PHP True Async RFC

Good day, everyone. I hope you’re doing well.

I’d like to introduce a draft version of the RFC for the True Async component.

https://wiki.php.net/rfc/true_async

I believe this version is not perfect and requires analysis. And I strongly believe that things like this shouldn’t be developed in isolation. So, if you think any important (or even minor) aspects have been overlooked, please bring them to attention.

The draft status also highlights the fact that it includes doubts about the implementation and criticism. The main global issue I see is the lack of “future experience” regarding how this API will be used—another reason to bring it up for public discussion.

Wishing you all a great day, and thank you for your feedback!

On Sat, Mar 1, 2025, at 10:11, Edmond Dantes wrote:

Good day, everyone. I hope you’re doing well.

I’d like to introduce a draft version of the RFC for the True Async component.

https://wiki.php.net/rfc/true_async

I believe this version is not perfect and requires analysis. And I strongly believe that things like this shouldn’t be developed in isolation. So, if you think any important (or even minor) aspects have been overlooked, please bring them to attention.

The draft status also highlights the fact that it includes doubts about the implementation and criticism. The main global issue I see is the lack of “future experience” regarding how this API will be used—another reason to bring it up for public discussion.

Wishing you all a great day, and thank you for your feedback!

FYI: once you introduce a draft RFC for discussion, the RFC should change status to “under discussion” per (4): https://wiki.php.net/rfc/howto

— Rob

FYI: once you introduce a draft RFC for discussion, the RFC should change status to “under discussion” per (4):

It’s done. Thank you.
Ed.

On 01/03/2025 09:11, Edmond Dantes wrote:

Good day, everyone. I hope you're doing well.

I’d like to introduce a draft version of the RFC for the True Async component.

PHP: rfc:true_async

My reaction to this can be summed up as "this is huge!" By that I mean multiple things...

First: PHP having native async support would be a huge step forward for the language. It's really exciting to see how this proposal develops.

Second: it's clear you've put a huge amount of work into this, so a huge thank you for that, and I hope it is rewarded.

Third: this is a huge proposal to digest. I wonder if there are ways it can be split into smaller pieces, so that we don't overlook details in one part because our focus is drawn to another. That might mean releasing a partial implementation this year, and more features next year; or it might just mean discussing and merging some core pieces first, then immediately following up with a series of feature RFCs, all targeting the same release.

Fourth: design decisions here will have a huge impact on the language for years to come. We should spend plenty of time looking at experience from elsewhere - other languages, and existing third-party async implementations for PHP. This is closely related to the previous point, since expanding the current RFC with comparisons for every decision would make it impractically long.

Fifth: this is a huge amount of new code - GitHub says 24 thousand lines of added code, although some of that is tests and documentation (which is great to see included!) We need to make sure there are enough people who understand the implementation to maintain that. Maybe we can try to tempt some of the core contributors to existing third-party libraries to spend some of their time on php-src instead.

I realise I haven't actually given any concrete feedback on the proposal - I don't have any experience with other async implementations, and don't fully understand the concepts involved, so don't feel qualified to comment on the high-level design questions. I might have opinions on smaller design details (random example: RESOLVE, CANCEL, and TIMEOUT should be cases on an enum, not int constants) but see point 4: there's just too much here to discuss in that level of detail, and there are top-level decisions which should be our focus first.

To re-iterate: this is really exciting, and thanks for getting it to this stage!

--
Rowan Tommins
[IMSoP]

First: PHP having native async support would be a huge step forward for
the language. It’s really exciting to see how this proposal develops.

Thank you for the kind words, it was awesome to read.

I wonder if there are ways it can be split into smaller pieces, so that we don’t overlook details in one part because our focus is drawn to another.

I can suggest the following workflow:

  1. Approval of the core concept: Changes affecting the language core.

  2. Decision on the low-level API: Async\wait + Resume + microtask. Should it be exposed to PHP developers or not? (I don’t have a definitive answer). This is a crucial point that impacts 30-40% of the code. If the decision is made to hide this API, the code will need to be adjusted.

Next, the RFC can be split into two parts:

  • Low-level: Basic PHP primitive functions + C API
  • High-level: Future, await, Channel, and maybe Pool.

So the process would be:
3. Approval of the Low-level RFC
4. Approval of the High-level RFC. Step 4 depends on Step 3 in terms of implementation but is almost independent in terms of semantics. This means it can be discussed separately and more freely.

Additionally, the Low-level API can be released independently, allowing PHP extensions to adopt concurrency earlier.

As for function names, I really hope for your support in this matter because it’s far from trivial.

Thanks, Ed.

On Sat, Mar 1, 2025, at 18:20, Rowan Tommins [IMSoP] wrote:

On 01/03/2025 09:11, Edmond Dantes wrote:

Good day, everyone. I hope you’re doing well.

I’d like to introduce a draft version of the RFC for the True Async

component.

https://wiki.php.net/rfc/true_async

My reaction to this can be summed up as “this is huge!” By that I mean

multiple things…

First: PHP having native async support would be a huge step forward for

the language. It’s really exciting to see how this proposal develops.

Second: it’s clear you’ve put a huge amount of work into this, so a huge

thank you for that, and I hope it is rewarded.

Third: this is a huge proposal to digest. I wonder if there are ways it

can be split into smaller pieces, so that we don’t overlook details in

one part because our focus is drawn to another. That might mean

releasing a partial implementation this year, and more features next

year; or it might just mean discussing and merging some core pieces

first, then immediately following up with a series of feature RFCs, all

targeting the same release.

Fourth: design decisions here will have a huge impact on the language

for years to come. We should spend plenty of time looking at experience

from elsewhere - other languages, and existing third-party async

implementations for PHP. This is closely related to the previous point,

since expanding the current RFC with comparisons for every decision

would make it impractically long.

Fifth: this is a huge amount of new code - GitHub says 24 thousand lines

of added code, although some of that is tests and documentation (which

is great to see included!) We need to make sure there are enough people

who understand the implementation to maintain that. Maybe we can try to

tempt some of the core contributors to existing third-party libraries to

spend some of their time on php-src instead.

I realise I haven’t actually given any concrete feedback on the proposal

  • I don’t have any experience with other async implementations, and

don’t fully understand the concepts involved, so don’t feel qualified to

comment on the high-level design questions. I might have opinions on

smaller design details (random example: RESOLVE, CANCEL, and TIMEOUT

should be cases on an enum, not int constants) but see point 4: there’s

just too much here to discuss in that level of detail, and there are

top-level decisions which should be our focus first.

To re-iterate: this is really exciting, and thanks for getting it to

this stage!

Rowan Tommins

[IMSoP]

I second this, and as a long time user of amphp, go, and C#, I’d be coming into it with a specific mindset.

My only thing so far is that it appears the scheduler cannot be replaced; at least, easily. I don’t know if we would do so over on FrankenPHP, but it would be interesting to replace the scheduler with something that utilized go-routines for true multi-threading. Whether that works or not, is a whole different can of worms.

I’m compiling a deeper review, but that speaks more to the implementation than the spec.

— Rob

but it would be interesting to replace the scheduler with something that utilized go-routines for true multi-threading. Whether that works or not, is a whole different can of worms.

— Rob

If the question is whether it is possible to interact with a PHP thread from another thread by sending an event to the Reactor, the answer is yes, it is possible. Moreover, from the PHP-land side, this could be a Channel.

If the question is deeper — replacing the Scheduler with a Scheduler in another language or from a different ecosystem — then it is more likely possible than not, considering that the module itself is separated from the rest of the implementation.
If you know a situation where this would be useful, then why not.

For example, in cases of integration with a web server, we can just send a message through a channel from “server-thread” to “php-thread”, and in a microtask written in C, for example, create Fibers to handle the request. This approach is used in Swoole.

And this solution should be even slightly faster than in Swoole because the interaction will occur through memory copying within a single process.

If memory copying is to be avoided, then the web server can be integrated directly into the Reactor, making the web server itself run as a microtask. Since the memory will be allocated immediately in the correct thread, there won’t even be a need to copy it, which in some cases might provide a performance boost. Or maybe not…

Ed.

On Sat, Mar 1, 2025, at 10:11, Edmond Dantes wrote:

Good day, everyone. I hope you’re doing well.

I’d like to introduce a draft version of the RFC for the True Async component.

https://wiki.php.net/rfc/true_async

I believe this version is not perfect and requires analysis. And I strongly believe that things like this shouldn’t be developed in isolation. So, if you think any important (or even minor) aspects have been overlooked, please bring them to attention.

The draft status also highlights the fact that it includes doubts about the implementation and criticism. The main global issue I see is the lack of “future experience” regarding how this API will be used—another reason to bring it up for public discussion.

Wishing you all a great day, and thank you for your feedback!

Hey Edmond:

I find this feature quite exciting! I’ve got some feedback so far, though most of it is for clarification or potential optimizations:

A PHP developer SHOULD NOT make any assumptions about the order in which Fibers will be executed, as this order may change or be too complex to predict.

There should be a defined ordering (or at least, some guarantees). Being able to understand what things run in what order can help with understanding a complex system. Even if it is just a vague notion (user tasks are processed before events, or vice versa), it would still give developers more confidence in the code they write. You actually mention a bit of the order later (microtasks happen before fibers/events), so this sentence maybe doesn’t make complete sense.

Personally, I feel as though an async task should run as though it were a function call until it hits a suspension. This is mostly an optimization though (C# does this), but it could potentially reduce overhead of queueing a function that may never suspend (which you mention as a potential problem much later on):

Async\run(*function*() {
 
   $fiber = Async\async(*function*() {
       [sleep](http://www.php.net/sleep)(1); // this gets enqueued now
       return "Fiber completed!";
   });
 
   *// Execution is paused until the fiber completes*
   $result = Async\await($fiber); // immediately enter $fiber without queuing
 
   echo $result . "**\n**";
 
   echo "Done!**\n**";
});

Until it is activated, PHP code behaves as before: calls to blocking functions will block the execution thread and will not switch the Fiber context. Thus, code written without the Scheduler component will function exactly the same way, without side effects. This ensures backward compatibility.

I’m not sure I understand this. Won’t php code behave exactly the same as it did before once enabling the scheduler? Will libraries written before this feature existed suddenly behave differently? Do we need to worry about the color of functions because it changes the behavior?

True Async prohibits initializing the Scheduler twice.

How will a library take advantage of this feature if it cannot be certain the scheduler is running or not? Do I need to write a library for async and another version for non-async? Or do all the async functions with this feature work without the scheduler running, or do they throw a catchable error?

This is crucial because the process may handle an OS signal that imposes a time limit on execution (for example, as Windows does).

Will this change the way os signals are handled then? Will it break compatibility if a library uses pcntl traps and I’m using true async traps too? Note there are several different ways (timeout) signals are handled in PHP – so if (per-chance) the scheduler could always be running, maybe we can unify the way signals are handled in php.

Code that uses Resume cannot rely on when exactly the Fiber will resume execution.

What if it never resumes at all? Will it call a finally block if it is try/catched or will execution just be abandoned? Is there some way to ensure cleanup of resources? It should probably mention this case and how abandoning execution works.

If an exception is thrown inside a fiber and not handled, it will stop the Scheduler and be thrown at the point where Async\launchScheduler() is called.

The RFC doesn’t mention the stack trace. Will it throw away any information about the inner exception?

The Graceful Shutdown mode can also be triggered using the function:

What will calling exit or die do?

A concurrent runtime allows handling requests using Fibers, where each Fiber can process its own request. In this case, storing request-associated data in global variables is no longer an option.

Why is this the case? Furthermore, if it inherits from the fiber that started its current fiber, won’t using Resume/Notifier potentially cause problems when used manually? There are examples over the RFC using global variables in closures; so do these examples not actually work? Will sharing instances of objects in scope of the functions break things? For example:

Async\run($obj->method1(…));

Async\run($obj->method2(…));

This is technically sharing global variables (well, global to that scope – global is just a scope after all) – so what happens here? Would it make sense to delegate this fiber-local storage to user-land libraries instead?

Objects of the Future class are high-level patterns for handling deferred results.

By this point we have covered FiberHandle, Resume, and Contexts. Now we have Futures? Can we simplify this to just Futures? Why do we need all these different ways to handle execution?

A channel is a primitive for message exchange between Fibers.

Why is there an isEmpty and isNotEmpty function? Wouldn’t !$channel->isEmpty() suffice?

It’s also not clear what the value of most of these function is. For example:

if ($chan->isFull()) {
  doSomething(); // suspends at some point inside? We may not know when we write the code.
  // chan is no longer full, or maybe it is -- who knows, but the original assumption entering this branch is no longer true.
  ...
}

Whether a channel is full or not is not really important, and if you rely on that information, this is usually an architectural smell (at least in other languages). Same thing with empty or writable, or many others of these functions. You basically just write to a channel and eventually (or not, which is a bug and causes a deadlock) something will read it. The entire point is to use channels to decouple async code, but most of the functions here allow for code to become strongly coupled.

As for the single producer method, I am not sure why you would use this. I can see some upside for the built-in constraints (potentially in a dev-mode environment) but in a production system, single-producer bottlenecks are a real thing that can cause serious performance issues. This is usually something you explicitly want to avoid.

In addition to the send/receive methods, which suspend the execution of a Fiber, the channel also provides non-blocking methods: trySend, tryReceive, and auxiliary explicit blocking methods: waitUntilWritable and waitUntilReadable.

It isn’t clear what happens when trySend fails. Is this an error or does nothing?

Thinking through it, there may be cases where trySend is valid, but more often than not, it is probably an antipattern. I cannot think of a valid reason for tryReceive and it’s usage is most likely guaranteed to cause a deadlock in real code. For true multi-threaded applications, it makes more sense, but not for single-threaded concurrency like this.

In other words, the following code is likely to be more robust, and not depend on execution order (which we are told at the beginning not to do):

Async\run(*function*() {
    $channel = *new* Async\Channel();
 
    $reader = Async\async(*function*() *use*($channel) {
        while ($data = $channel->read() && $data !== NULL) {
            echo "receive: **$data****\n**";
        }
    });
 
    for ($i = 0; $i < 4; $i++) {
        echo "send: event data **$i****\n**";
        $data = $channel->send("event data **$i**");
    }
    
    $reader->cancel(); // clean up our reader
    // or
    $channel->close(); // will receive NULL I believe?
});

A trySend is still useful when you want to send a message but don’t want to block if it is full. However, this is going to largely depend on how long is has been since the developer last suspended the current fiber, and nothing else – thus it is probably an antipattern since it totally depends on the literal structure of the code, not the structure of the program – if that makes sense.

This means that trapSignal is not intended for “regular code” and should not be used “anywhere”.

Can you expand on what this means in the RFC? Why expose it if it shouldn’t be used?


I didn’t go into the low level api details yet – this email is already pretty long. But I would suggest maybe thinking about how to unify Notifiers/Resume/FiberHandle/Future into a single thing. These things are pretty similar to one another (from a developer’s standpoint) – a way to continue execution, and they all offer a slightly different api.

I also noticed that you seem to be relying heavily on the current implementation to define behavior. Ideally, the RFC should define behavior and the implementation implement that behavior as described in the RFC. In other words, the RFC is used as a reference point as to whether something is a bug or an enhancement in the future. There has been more than once where the list looks back at an old RFC to try and determine the intent for discovering if something is working as intended or a bug. RFCs are also used to write documentation, so the more detailed the RFC, the better the documentation will be for new users of PHP.

— Rob

There should be a defined ordering (or at least, some guarantees).

The execution order, which is part of the contract, is as follows:

  1. Microtasks are executed first.
  2. Then I/O events and OS signals are processed.
  3. Then timer events are executed.
  4. Only after that are fibers scheduled for execution.

In the current implementation, fibers are stored in a queue without priorities (this is not a random choice). During one cycle period, only one fiber is taken from the queue.

This results in the following code (I’ve removed unnecessary details):

do {

execute_microtasks_handler();

has_handles = execute_callbacks_handler(circular_buffer_is_not_empty(&ASYNC_G(deferred_resumes)));

execute_microtasks_handler();

bool was_executed = execute_next_fiber_handler();

if (UNEXPECTED(
false == has_handles
&& false == was_executed
&& zend_hash_num_elements(&ASYNC_G(fibers_state)) > 0
&& circular_buffer_is_empty(&ASYNC_G(deferred_resumes))
&& circular_buffer_is_empty(&ASYNC_G(microtasks))
&& resolve_deadlocks()
)) {
break;
}

} while (zend_hash_num_elements(&ASYNC_G(fibers_state)) > 0
|| circular_buffer_is_not_empty(&ASYNC_G(microtasks))
|| reactor_loop_alive_fn()
);

If we go into details, it is also noticeable that microtasks are executed twice - before and after event processing - because an event handler might enqueue a microtask, and the loop ensures that this code executes as early as possible.

The contract for the execution order of microtasks and events is important because it must be considered when developing event handlers. The concurrent iterator relies on this rule.
However, making assumptions about when a fiber will be executed is not part of the contract, if only because this algorithm can be changed at any moment.

// Execution is paused until the fiber completes $result = Async\await($fiber); // immediately enter $fiber without queuing

So is it possible to change the execution order and optimize context switches? Yes, there are ways to do this. However, it would require modifying the Fiber code, possibly in a significant way (I haven’t explored this aspect in depth).

But… let’s consider whether this would be a good idea.

We have a web server. A single thread is handling five requests. They all compete with each other because this is a typical application interacting with MySQL.
In each Fiber, you send a query and wait for the result as quickly as possible.

In what case should we create a new coroutine within a request handler?
The answer: usually, we do this when we want to run something in the background while continuing to process the request and return a response as soon as possible.

In this paradigm, it is beneficial to execute coroutines in the order they were enqueued.

For other scenarios, it might be a better approach for a child coroutine to execute immediately. In that case, these scenarios should be considered, and it may be worth introducing specific semantics for such cases.

Won’t php code behave exactly the same as it did before once enabling the scheduler?

Suppose we have a sleep() function. Normally, it calls php_sleep((unsigned int)num).
The php_sleep function blocks the execution of the thread.

But we need to add an alternative path:

if (IN_ASYNC_CONTEXT) {
async_wait_timeout((unsigned int) num * 1000, NULL);
RETURN_LONG(0);
}

The IN_ASYNC_CONTEXT condition consists of two points:

  • The current execution context is inside a Fiber.
  • The Scheduler is active.

What’s the difference?

If the Scheduler is not active, calling sleep() will block the entire Thread because, without an event loop, it simply cannot correctly handle concurrency.
However, if the Scheduler is active, the code will set up handlers and return control to the “main loop”, which will pick the next Fiber from the queue, and so on.

This means that without a Scheduler and Reactor, concurrent execution is impossible (without additional effort).

From the perspective of a PHP developer, if they are working with AMPHP/Swoole, nothing changes, because the code inside the if condition will never execute in their case.

Does this change the execution order inside a Fiber? No.

If you had code working with RabbitMQ sockets, and you copied this code into a Fiber, then enabled concurrency, it would work exactly the same way. If the code used blocking sockets, the Fiber would yield control to the Scheduler. And if two such Fibers are running, they will start working with RabbitMQ sequentially. Of course, each Fiber should use a different socket.

The same applies to CURL. Do you have an existing module that sends requests to a service using CURL in a synchronous style? Just copy the code into a coroutine.

This means almost 98% transparency. Why almost? Because there might be nuances in helper functions and internal states. There may also be differences in OS state management or file system, which could affect the final result.

How will a library take advantage of this feature if it cannot be certain the scheduler is
running or not? Do I need to write a library for async and another version for non-async?
Or do all the async functions with this feature work without the scheduler running, or do
they throw a catchable error?

This means that the launchScheduler() function should be called only once during the entire lifecycle of the application. If an error occurs and is not handled, the application should terminate. This is not a technical limitation but rather a logical constraint.

If launchScheduler() were replaced with a CLI option, such as php --enable-scheduler, where the Scheduler is implicitly activated, then it would be like the last line of code it must exist only once.

Will this change the way os signals are handled then? Will it break compatibility if a
library uses pcntl traps and I'm using true async traps too? Note there are several
different ways (timeout) signals are handled in PHP -- so if (per-chance) the scheduler
could always be running, maybe we can unify the way signals are handled in php.

Regarding this phrase in the RFC: it refers to the window close event in Windows, which provides a few seconds before the process is forcibly terminated.

There are signals intended for application termination, such as SIGBREAK or CTRL-C, which should typically be handled in only one place in the application. Developers are often tempted to insert signal handlers in multiple locations, making the code dependent on the environment. But more importantly, this should not happen at all.

True Async explicitly defines a Flow for emergency or unexpected application termination. Attempting to disrupt this Flow by adding a custom termination signal handler introduces ambiguity.

There should be only one termination handler. And at the end of its execution, it must call gracefulShutdown.

As for pcntl, this will need to be tested.

What if it never resumes at all?

If a Fiber is never resumed, it means the application has completely crashed with no way to recover :slight_smile:

The RFC has two sections dedicated to this issue:
Cancellation Operation + Graceful Shutdown.

If the application terminates due to an unhandled exception, all Fibers must be executed.

Any Fiber can be canceled at any time, and there is no need to use explicit Cancellation, which I personally find an inconvenient pattern.

The RFC doesn’t mention the stack trace. Will it throw away any information about the inner exception?

This is literally “exception transfer”. The stack trace will be exactly the same as if the exception were thrown at the call site.

To be honest, I haven’t had enough time to thoroughly test this. Let’s try it:

<?php Async\async(function() { echo "async function 1\n"; Async\async(function() { echo "2\n"; throw new Error("Error"); }); }); echo "start\n"; try { Async\launchScheduler(); } catch (\Throwable $exception) { print_r($exception); } echo "end\n"; ?>

004+ Error Object
005+ (
006+ [message:protected] => Error
007+ [string:Error:private] =>
008+ [code:protected] => 0
009+ [file:protected] => async.php
010+ [line:protected] => 8
011+ [trace:Error:private] => Array
012+ (
013+ [0] => Array
014+ (
015+ [function] => {closure:{closure:async.php:3}:6}
016+ [args] => Array
017+ (
018+ )
019+ )
020+ [1] => Array
021+ (
022+ [file] => async.php
023+ [line] => 14
024+ [function] => Async\launchScheduler
025+ [args] => Array
026+ (
027+ )
028+ )
029+ )
030+ [previous:Error:private] =>
031+ )

Seems perfectly correct.

What will calling exit or die do?

I completely forgot about them! Well, of course, Swoole override them. This needs to be added to the TODO.

Why is this the case?

For example, consider a long-running application where a service is a class that remains in memory continuously. The web server receives an HTTP request and starts a Fiber for each request. Each request has its own User Session ID.

You want to call a service function, but you don’t want to pass the Session ID every time, because there are also 5-10 other request-related variables. However, you cannot simply store the Session ID in a class property, because context switching is unpredictable. At one moment, you’re handling Request #1, and a second later, you’re already processing Request #2.

When a Fiber creates another Fiber, it copies a reference to the context object, which has minimal performance impact while maintaining execution environment consistency.

Closure variables work as expected they are pure closures with no modifications.
I didn’t mean that True Async breaks anything at the language level. The issue is logical:

You cannot use a global variable in two Fibers, modify it, read it, and expect its state to remain consistent.

By this point we have covered FiberHandle, Resume, and Contexts. Now we have Futures? Can we simplify this to just Futures? Why do we need all these different ways to handle execution?

Futures and Notifiers are two different patterns.

  • A Future changes its state only once.
  • A Notifier generates one or more events.
  • Internally, Future uses Notifier.

In the RFC, I mention that these are essentially two APIs:

  • High-level API
  • Low-level API

One of the open questions is whether both APIs should remain in PHP-land.

The low-level API allows for close interaction with the event loop, which might be useful if someone wants to write a service in PHP that requires this level of control.

Additionally, this API helps minimize Fiber context switches, since its callbacks execute without switching.
This is both an advantage and a disadvantage.

It's also not clear what the value of most of these function is. For example:

Your comment made me think, especially in the context of anti-patterns. And I agree that it’s better to remove unnecessary methods than to let programmers shoot themselves in the foot.

As for the single producer method, I am not sure why you would use this.

Yes, in other languages there are no explicit restrictions. If the single producer approach is indeed rarely used, then it’s not such an important feature to include. However, I lack certainty on whether it’s truly a rare case. On the other hand, these functions are inexpensive to implement and do not affect performance. Moreover, they have another drawback: they increase the number of behavioral variants in a single class, which seems a more significant disadvantage than the frequency of use.

It isn't clear what happens when `trySend` fails. Is this an error or does nothing? 

Yes, this is a documentation oversight. I’ll add it to the TODO.

Thinking through it, there may be cases where trySend is valid,

Code using tryReceive could be useful in cases where a channel is used to implement a pool. Suppose you need to retrieve an object from the pool, but if it’s not available, you’d prefer to do something else (like throw an exception) rather than block the fiber.
Overall, though, you’re right — it’s an antipattern. It’s better to implement the pool as an explicit class and reserve channels for their classic use.

Can you expand on what this means in the RFC? Why expose it if it shouldn’t be used?

I answered a similar question above.

I also noticed that you seem to be relying heavily on the current implementation to define

behavior.

I love an iterative approach: prototype => RFC => prototype => RFC.

Thank you for the excellent remarks and analysis!

Ed.

Hi,

Any Fiber can be canceled at any time, and there is no need to use explicit Cancellation, which I personally find an inconvenient pattern.

As a heavy use of both amphp and go, cancellations (contexts in go) are absolutely needed, as a fiber may spawn further background fibers in order to execute some operation, just cancelling that specific fiber will not cancel the spawned fibers, unless a bunch of boilerplate try-catch blocks are added to propagate CancellationExceptions.

A nicer API should use only explicit cancellation objects, as this pattern of preemptive implicit cancellations (i.e. a fiber may be cancelled at any point via cancel()) is super dangerous IMO, as it can lead to all sorts of nasty behaviour: what if we cancel execution of a fiber in the middle of a critical section (i.e. between a lock() and an unlock() of a file or a database? What if unlocking() in the catch (CancelledException) block requires spawning a new fiber as part of the interaction with the database?).
Consider also the huge amount of CancelledException blocks that would have to be added to handle state cleanup in case of premature implicit cancellations, as opposed to explicit cancellations that only throw when we ask them to: there’s a reason why golang, amphp & others use explicit cancellations.

Another thing I’m not happy with is how unless the scheduler is launched, all code executes in blocking mode: this seems like a super bad idea, as it will hold back the ecosystem again, and create a split in the project similar to JIT (i.e. a separate “execution mode” with its own bugs, that get fixed slowly because few people are using it, and few people are using it because of its bugs).
The main reason given in the RFC (Code written without using the Scheduler should not experience any side effects) makes no sense, because legacy code not spawning fibers will not experience concurrency side effects anyway, regardless of whether the scheduler is started or not.

A thing I would love to see, on the other hand, is for Context to become a “provider” for superglobals such as $_REQUEST, $_POST, $_GET, and all globals in general (and perhaps all other global state such as static properties): this would allow to very easily to turn i.e. php-fpm into a fully asynchronous application server, where each request is started in the same thread (or in N threads in an M-N M>N execution model) but its global state is entirely isolated between fibers.

Regards,
Daniil Gentili - Senior software engineer

Portfolio: https://daniil.it
Telegram: https://t.me/danogentili

As a heavy use of both amphp and go, cancellations (contexts in go) are absolutely needed, as a fiber may spawn further background fibers in order to execute some operation, just cancelling that specific fiber will not cancel the spawned fibers, unless a bunch of boilerplate try-catch blocks are added to propagate CancellationExceptions.

I didn’t mean that Cancellation isn’t needed at all. I meant that canceling a Fiber is sufficient in most scenarios and leads to clean, understandable code.

Other languages have child coroutines (Swoole supports them too), but I’m not sure if that’s the right approach.

I like context.WithCancel from Go, but it can essentially be implemented directly in PHP land since all the necessary tools are available.

A nicer API should use only explicit cancellation objects, as this pattern of preemptive implicit cancellations

The exception mechanism is the standard way to alter the execution flow in PHP. If a programmer writes code with lock and unlock outside of a try-finally block but calls functions between these methods, they are potentially creating a bad solution—at the very least because someone else might later introduce an exception in one of those functions. This is a classic case for languages with exceptions.

So far, I haven’t found a better way to ensure the logical consistency and integrity of the execution flow. Maybe someone has a suggestion?

The main reason given in the RFC

The main reason is that PHP has been around for many years and didn’t just appear yesterday.

If you have an idea on how to start the Scheduler implicitly, let’s implement it. So far, I have a few ideas:

  1. Using an option in php.ini (downside: if PHP is used for multiple projects).
  2. Using a CLI option – so far, I like this the most.

A thing I would love to see, on the other hand, is for Context to become a

“provider”

It’s hard for me to evaluate this idea. Intuitively, it doesn’t seem ideal. In general, I’m not very fond of $_GET/$_POST. But on the other hand, why not? This needs some consideration.

allow to very easily to turn i.e. php-fpm into a fully asynchronous application server,
where each request is started in the same thread (or in N threads in an M-N M>N
execution model) but its global state is entirely isolated between fibers.

I haven’t thought about this possibility. But wouldn’t this break the FCGI contract?

Thanks! Ed.

On 3 Mar 2025, at 13:05, Edmond Dantes edmond.ht@gmail.com wrote:

As a heavy use of both amphp and go, cancellations (contexts in go) are absolutely needed, as a fiber may spawn further background fibers in order to execute some operation, just cancelling that specific fiber will not cancel the spawned fibers, unless a bunch of boilerplate try-catch blocks are added to propagate CancellationExceptions.

I didn’t mean that Cancellation isn’t needed at all. I meant that canceling a Fiber is sufficient in most scenarios and leads to clean, understandable code.

Other languages have child coroutines (Swoole supports them too), but I’m not sure if that’s the right approach.

I like context.WithCancel from Go, but it can essentially be implemented directly in PHP land since all the necessary tools are available.

Note, this is precisely the problem, implement cancellation propagation to child fibers in userland PHP requires writing a bunch of boilerplate try-catch blocks to propagate CancellationExceptions to child FutureHandle::cancel()s (spawning multiple fibers to execute subtasks concurrently during an async method call is pretty common, and the current implicit cancellation mode requires writing a bunch of try-catch blocks to propagate cancellation, instead of just passing a cancellation object, or a flag to inherit the cancellation of the current fiber when spawning a new one).

A nicer API should use only explicit cancellation objects, as this pattern of preemptive implicit cancellations

The exception mechanism is the standard way to alter the execution flow in PHP. If a programmer writes code with lock and unlock outside of a try-finally block but calls functions between these methods, they are potentially creating a bad solution—at the very least because someone else might later introduce an exception in one of those functions. This is a classic case for languages with exceptions.

Note the explicit use case I listed is that of an unlock() in a finally block that requires spawning a new fiber in order to execute the actual unlock() RPC call: this is explicitly in contrast with the RFC, which specifies that

ATTENTION: A programmer must never attempt to create a new fiber while handling a CancellationException, as this behavior may trigger an exception during Graceful Shutdown mode.

While this is somewhat understandable in the context of graceful shutdown, it still means that unlocking in a finally block (the only way of properly handling cancellations with the current model) isn’t always possible…

So far, I haven’t found a better way to ensure the logical consistency and integrity of the execution flow. Maybe someone has a suggestion?

The main reason given in the RFC

The main reason is that PHP has been around for many years and didn’t just appear yesterday.

If you have an idea on how to start the Scheduler implicitly, let’s implement it. So far, I have a few ideas:

  1. Using an option in php.ini (downside: if PHP is used for multiple projects).
  2. Using a CLI option – so far, I like this the most.

I would really prefer it to be always enabled, no fallback at all, because as I said, it will make absolutely no difference to legacy, non-async projects that do not use fibers, but it will avoid a split ecosystem scenario.

A thing I would love to see, on the other hand, is for Context to become a

“provider”

It’s hard for me to evaluate this idea. Intuitively, it doesn’t seem ideal. In general, I’m not very fond of $_GET/$_POST. But on the other hand, why not? This needs some consideration.

allow to very easily to turn i.e. php-fpm into a fully asynchronous application server,
where each request is started in the same thread (or in N threads in an M-N M>N
execution model) but its global state is entirely isolated between fibers.

I haven’t thought about this possibility. But wouldn’t this break the FCGI contract?

I see no reason why it should break the contract, if implemented by isolating the global state of each fiber, it can be treated as a mere implementation detail of the (eventually new) SAPI.

Regards,
Daniil Gentili

Daniil Gentili - Senior software engineer

Portfolio: https://daniil.it
Telegram: https://t.me/danogentili

I like context.WithCancel from Go, but it can essentially be implemented directly in PHP land since all the necessary tools are available.

Note, this is precisely the problem, implement cancellation propagation to child fibers in userland PHP requires writing a bunch of boilerplate try-catch blocks to propagate CancellationExceptions to child FutureHandle::cancel()s (spawning multiple fibers to execute subtasks concurrently during an async method call is pretty common, and the current implicit cancellation mode requires writing a bunch of try-catch blocks to propagate cancellation, instead of just passing a cancellation object, or a flag to inherit the cancellation of the current fiber when spawning a new one).

Catching CancellationException is only necessary if there is some defer code.
If there isn’t, then there’s no need to catch it. Try-catch blocks are not mandatory.

We can create a Cancellation object, pass it via use or as a parameter to all child fibers, and check it in await(). This is the most explicit approach. In this case, try-catch would only be needed if we want to clean up some resources. Otherwise, we can omit it.

According to the RFC, if a fiber does not catch CancellationException, it will be handled by the Scheduler. Therefore, catching this exception is not strictly necessary.

If this solution also seems too verbose, there is another one that can be implemented without modifying this RFC. For example, implementing a cancellation operation for a Context. All coroutines associated with this context would be canceled. From an implementation perspective, this is essentially iterating over all coroutines and checking which context they belong to.

Note the explicit use case I listed is that of an unlock() in a finally block that requires spawning a new fiber in order to execute the actual unlock() RPC call: this is explicitly in contrast with the RFC, which specifies that

So, if I understand correctly, the code in question looks like this:

try { lock(); … } finally { unlock(); }

function unlock() {
async\run();
}

If I got it right, then the following happens:

  • The code inside try {} allocates resources.
  • The code inside finally {} also allocates resources.

So, what do we get? We’re trying to terminate the execution of a fiber, and instead, it creates a new one. It seems like there’s a logical error here.

Instead of creating a new fiber, it would be better to use microtasks.

I would really prefer it to be always enabled, no fallback at all, because as I said, it will make absolutely no difference to legacy, non-async projects that do not use fibers, but it will avoid a split ecosystem scenario.

I’m not arguing at all that avoiding the call to this function is a good solution. I’m on your side. The only question is how to achieve this technically.

Could you describe an example of “ecosystem split” in the context of this function? What exactly is the danger?

I see no reason why it should break the contract, if implemented by isolating the global state of each fiber, it can be treated as a mere implementation detail of the (eventually new) SAPI.

So, I can take NGINX and FCGI, and without changing the FCGI interface itself, but modifying its internal implementation, get a working application. Yes, but… that means all global variables, including static ones, need to be tied to the context. It’s not that it can’t be done, but what about memory consumption.

I’m afraid that if the code wasn’t designed for a LongRunning APP, it’s unlikely to handle this task correctly.


Ed.

Lock/Unlock issue

It seems that this is actually about a database query that puts the Fiber into a waiting state specifically, query("UNLOCK").

In that case, everything should work correctly.

Although there are some dangerous edge cases. The database might be under high load, causing the query("UNLOCK") request to wait for too long, leading to a timeout. This would trigger another exception, which could then be interpreted as a complete failure.

Putting a Fiber into a waiting state inside a finally block does not contradict the shutdown mode. However, the programmer must be careful inside finally section because if a second exception occurs, it means the code cannot properly complete execution.


Ed.

Hi Edmond,

Thanks for sharing the huge amount of work that went into this!

I would really prefer it to be always enabled, no fallback at all, because as I said, it will make absolutely no difference to legacy, non-async projects that do not use fibers, but it will avoid a split ecosystem scenario.

I’m not arguing at all that avoiding the call to this function is a good solution. I’m on your side. The only question is how to achieve this technically.

Could you describe an example of “ecosystem split” in the context of this function? What exactly is the danger?

Not sure it’s an answer to this question but in Symfony’s HttpClient, we have an amphp-based implementation that’s working both outside and inside an event loop:

  • inside means amphp’s scheduler already started, and then each request is scheduled thanks to amphp’s http client
  • outside means Symfony’s code is going to trigger amphp’s event loop internally.

The target DX is that when outside any event loop, we’re still able to leverage fibers to provide concurrency, for requests only, and when inside an event loop, requests run concurrently to any other things that the loop monitors.

Is that something that could be achieved with your proposal?
If not, maybe that’s the split we’re wondering about?

Nicolas

Note the explicit use case I listed is that of an unlock() in a finally block that requires spawning a new fiber in order to execute the actual unlock() RPC call: this is explicitly in contrast with the RFC, which specifies that

ATTENTION: A programmer must never attempt to create a new fiber while handling a CancellationException, as this behavior may trigger an exception during Graceful Shutdown mode.

I think you are right. This restriction increases complexity without providing significant benefits. I will remove this condition from the RFC entirely and simply state that the programmer should handle such situations carefully.
Thank you!

Hi, Nicolas.

Hi Edmond,

The target DX is that when outside any event loop, we’re still able to leverage fibers to provide concurrency, for requests only, and when inside an event loop, requests run concurrently to any other things that the loop monitors.

Is that something that could be achieved with your proposal?
If not, maybe that’s the split we’re wondering about?

This RFC leads to PHP operating in two modes:

  1. Blocking mode: The Event Loop needs to be implemented manually, AMPHP works. This is how PHP currently operates.
  2. Concurrent mode: Code runs in coroutines. The Event Loop works under the hood. AMPHP does not work.

If we try to imagine a way to keep PHP in a single mode, it would likely require implementing coroutines separately from Fiber and leaving Fiber as legacy.

This solution has both advantages and disadvantages.

Advantages:

  1. Switching can be optimized considering the new architecture.
  2. The Event Loop will start automatically when needed.
  3. Code using Fiber will work as before, and most likely, AMPHP will be able to create an event loop in user-land.

Disadvantages:

  • More work is required.
  • There is a risk of ending up with a Frankenstein-like result. :slight_smile:

A relative advantage of the current implementation is that it changes only about 100-500 lines in the PHP core (probably even fewer, since part of the changes are in extensions like CURL and Socket).

The downside is that it cannot change the rules that were previously established.


Ed.

On Sat, Mar 1, 2025, at 3:11 AM, Edmond Dantes wrote:

Good day, everyone. I hope you're doing well.

I’d like to introduce a draft version of the RFC for the True Async component.

PHP: rfc:true_async

I believe this version is not perfect and requires analysis. And I
strongly believe that things like this shouldn't be developed in
isolation. So, if you think any important (or even minor) aspects have
been overlooked, please bring them to attention.

The draft status also highlights the fact that it includes doubts about
the implementation and criticism. The main global issue I see is the
lack of "future experience" regarding how this API will be used—another
reason to bring it up for public discussion.

Wishing you all a great day, and thank you for your feedback!

I finally managed to read through enough of the RFC to say something intelligent. :slight_smile:

First off, as others have said, thank you for a thorough and detailed proposal. It's clear you've thought through a lot of details. I also especially like that it's transparent for most IO operations, which is mandatory for adoption. It's clear to me that async in PHP will never be more than niche until there is a built-in dev-facing API that is easy to use on its own without any 3rd party libraries.

Unfortunately, at this point I cannot support this proposal, because I disagree with the fundamental design primitives.

Let's look at the core design primitives:

* A series of free-standing functions.
* That only work if the scheduler is active.
* The scheduler being active is a run-once global flag.
* So code that uses those functions is only useful based on a global state not present in that function.
* And a host of other seemingly low-level objects that have a myriad of methods on them that do, um, stuff.
* Oh, and a lot of static methods, too, instead of free-standing functions.

The number of ways for this to go wrong and confuse the heck out of a developer is disturbingly high.

In the Low-Level API section, the RFC notes:

I came to the conclusion that, in the long run, sacrificing flexibility in favor of code safety is a reasonable trade-off.

I completely agree with this statement! And feel the RFC doesn't go even remotely far enough in that direction.

In particular, I commend to your attention this post about a Python async library that very deliberately works at a much higher level of abstraction, and is therefore vastly safer:

I won't repeat the post, but suffice to say I agree with it almost entirely. (I dislike the name "nursery," but otherwise...) That is the direction we should be looking at for PHP, from the get-go.

PHP doesn't have Python-style context managers (though I would like them), so a PHP version of that might look something like this (just spitballing):

async $context {
  // $context is an object of AsyncContext, and can be passed around as such.
  // It is the *only* way to span anything async, or interact with the async controls.
  // If a function doesn't take an AsyncContext param, it cannot control async. This is good.

  $context->run(some_function(...));
  $result = $context->run(function(AsyncContext $ctx) use ($someObj) {
    // This queues a thunk to run at the end of the closest async {} block.
    $ctx->defer($someObj->shutdown(...));
  });
}
catch (SomeException $e) {
  // Exception thrown by one of the fibers.
}

// This is an unwrapped value.
print $result;

Naturally there would be more to the API, but I'm just showing the basics.

Importantly:

* There is no global modal (schedulerStarted) to think about.
* When the async {} block ends, you know with 100% certainty that there are no dangling background tasks.
* It's explicitly obvious what functions are going to try and mess with the async context, and therefore cannot be called except within an async context.
* An application can have sync portions and async portions very easily, without worrying about which "mode" it's in at a given time.

It also means that writing a number of the utilities mentioned in the RFC do not require any engine code. Eg:

function parallel_map(iterable $it, Closure $fn) {
  $result = ;
  async $ctx {
    foreach ($it as $k => $v) {
      $result[$k] = $ctx->run($fn($v));
    }
  }
  return $result;
}

Now I know that's safe to call anywhere, whether I'm current in an active async mode or not.

I'm not convinced that sticking arbitrary key/value pairs into the Context object is wise; that's global state by another name. But if we must, the above would handle all the inheritance and override stuff quite naturally. Possibly with:

async $ctx from $parentCtx {
  // ...
}

Similarly, the two different modes for channels strike me as quite unnecessary. I also would tend to favor how Rust does channels (via a library, I don't think it's a built-in): have separate variables for the in-side and out-side. Again, just spitballing:

[$in, $out] = Channel::create($buffer_size);

$in->send($val);

$out->receive($val);

(Give or take variations of those methods.)

Now you don't need to worry about fibers owning things. You just have a ChannelIn object and a ChannelOut object, and can pass either one to as many or as few functions as you want. And those functions could be spawning new fibers if you'd like, or not. (There's likely some complications here I'm not thinking of, but I've not dug into it in depth yet.) You can now close either side, or just let the objects go out of scope.

In short, I am fully in favor of better async logic in PHP. I am very against an API that even allows me to do something stupid or deadlock-creating, or that relies on hidden global state. That would be worse than the status quo, and there are better models than what is shown here that offer much stronger "correct by construction" guarantees.

--Larry Garfield

Hi there,

I would also like to highlight some interesting ideas that I find being useful to consider.

Recently Bend programming language has been released, and it incorporates a completely different view on the conception of “code”, in the definition of “what it is” and “how it should be interpreted”.

While we interpret it as a sequence of instructions, the proper way of seeing it is the graph of instructions. On every step we reduce that graph, by running the code of the nodes current node depends on.

Therefore, basically everything could paralleled w/o the need to have fancy management of threads and other low-level things.

For example, having this code:

$foo = foo();
$bar = bar();
$baz = $foo + $bar;

If it was run in Bend, it would be interpreted so that foo() and bar() functions are executed in parallel, and $baz = $foo + $bar is executed afterwards, since this computation depends on the other two.

The key, most excellently beautiful feature
here is that all async management is under the hood, exposing nothing for the developers to be bothered with.

That being said, I also want to mention that Bend has a primitive for concurrent loops. Actually, they used another solution, different from loops, since loops are sequential by their essense (iterative one by one). They introduced a concurrent alternative for loops with “bend” keyword, allowing data structures to be traversed in parallel.

I think this is actually “the right way” of doing parallel processing in general and async programming in particular, and this is greatly to be considered for having at least some principles applied in PHP.

What I think it could be.

async function baz(): int {
$foo = foo();
$bar = bar();

return $foo + $bar;
}

// value is returned just like from any other ordinary function
$val = baz();

Function above could run foo() in one fiber, and bar() in another, both of them being awaited at the return statement (at the first statement where the value is actually used / referenced, if we put it more generally) so that actual values could be taken.

In other words, async function is not promise-based as in other languages that suffer from red blue function problem, but rather it is function with coroutine flow of execution, so that foo() is executed as the first coroutine, and when it blocks, then bar() is executed until it also blocks. Then, at plus operator being evaluated, $foo is awaited, and $bar is awaited, since they are necessary parts for + operation to complete.

Best regards

On Tue, Mar 4, 2025, at 23:54, Eugene Sidelnyk wrote:

Hi there,

I would also like to highlight some interesting ideas that I find being useful to consider.

Recently Bend programming language has been released, and it incorporates a completely different view on the conception of “code”, in the definition of “what it is” and “how it should be interpreted”.

While we interpret it as a sequence of instructions, the proper way of seeing it is the graph of instructions. On every step we reduce that graph, by running the code of the nodes current node depends on.

Therefore, basically everything could paralleled w/o the need to have fancy management of threads and other low-level things.

For example, having this code:

$foo = foo();

$bar = bar();

$baz = $foo + $bar;

If it was run in Bend, it would be interpreted so that foo() and bar() functions are executed in parallel, and $baz = $foo + $bar is executed afterwards, since this computation depends on the other two.

The key, most excellently beautiful feature

here is that all async management is under the hood, exposing nothing for the developers to be bothered with.

That being said, I also want to mention that Bend has a primitive for concurrent loops. Actually, they used another solution, different from loops, since loops are sequential by their essense (iterative one by one). They introduced a concurrent alternative for loops with “bend” keyword, allowing data structures to be traversed in parallel.

I think this is actually “the right way” of doing parallel processing in general and async programming in particular, and this is greatly to be considered for having at least some principles applied in PHP.

What I think it could be.

async function baz(): int {

$foo = foo();

$bar = bar();

return $foo + $bar;

}

// value is returned just like from any other ordinary function

$val = baz();

Function above could run foo() in one fiber, and bar() in another, both of them being awaited at the return statement (at the first statement where the value is actually used / referenced, if we put it more generally) so that actual values could be taken.

In other words, async function is not promise-based as in other languages that suffer from red blue function problem, but rather it is function with coroutine flow of execution, so that foo() is executed as the first coroutine, and when it blocks, then bar() is executed until it also blocks. Then, at plus operator being evaluated, $foo is awaited, and $bar is awaited, since they are necessary parts for + operation to complete.

Best regards

Huh. Reminds me of SSA, which can identify independent computations like that. It’s used by go, and many other compiled languages, but not in the same way this bend language does it. So, that’s interesting.

I don’t know if php could implement SSA (maybe opcache could), but with how dynamic php is, I’m not sure it would be helpful.

An interesting application nonetheless, thanks for sharing!

— Rob