[PHP-DEV] [Initial Feedback] Typed Arrays

Richard_Miles · June 26, 2024, 7:59pm

Howdy people,

The pattern-matching RFC inspired me to write this. I have not done any work for this yet; just looking for initial feedback. I think we should have typed arrays in PHP. I propose adding the class SplTypeDefinedArray. A few considerations: A new syntax allowing Array interfaces will be needed. Functions should be allowed to return array interface types. How do we pass the interfaces to the constructor? Do we stick to traditional syntax, creating the object with new, or do we support a new array definition syntax?

interface iArrayA [‘a’ => string ]
interface iArrayB implements iArrayA [‘b’ => string, ‘c’ => ?string ]

$a = new SplTypeDefinedArray(iArrayB, [ // iArrayB::class is invalid
‘a’ => ‘hello’,
‘b’ => ‘world’
]);

// or

$a : iArrayB = [ // this would implicitly initialize SplTypeDefinedArray
‘a’ => ‘hello’,
‘b’ => ‘world’
];

If a typed array tries to define an index that does not exist it will throw a RuntimeException: Index invalid, which is consistant with the current implentation of SplFixedArray; If a type is nullable then it is not required to exist during construction.

Standard-type operators should be available.

$a : iArrayA & iArrayB = [ // throws a RuntimeException since B is required in iArrayB
‘a’ => ‘fail’
];

$a : iArrayA | iArrayB = [
‘a’ => ‘profit’
];

$a : iArrayA &| iArrayB = [ // throws a RuntimeException since c does not exist in iArrayA and b is required in iArrayB
‘a’ => ‘fail’
‘c’ => ‘’
];

I’m not sure what the best data structure would be. Rob Landers suggested a circular buffer in the pattern-matching email thread, but I’m open to anything.

Best,

Richard Miles

Bilge · June 26, 2024, 8:08pm

On 26/06/2024 20:59, Richard Miles wrote:

I think we should have typed arrays in PHP.

Generics or bust.

Cheers,
Bilge

Casper_Langemeijer · June 26, 2024, 8:56pm

On Wed, Jun 26, 2024, at 21:59, Richard Miles wrote:

I think we should have typed arrays in PHP.

Yes! I cannot stand sitting through conference talks on ‘generics’ that only talk about ‘collections’. This could be solved if we had typed arrays. If anything we would get better talks on Generics.

Arrays of a type is one of the last cases where I need docblocks to tell my editor about the types.

In my opinion, even if we would have some implementation of generics, having typed arrays with a simple syntax would be awesome.

A syntax suggestion:

$array = stdClass;

class A {
public stdClass $array;
}

Adding an invalid array member should throw TypeError.

I know there are way more edge-case situations to think of (for example: if class B extends A, $b is of type B, but holds only A’s, can $b be assigned as value of public A $a ?)

Generics or bust.

I do not understand the reasoning behind that. Is it because we really want generics, but when the 95% use-case is solved we fear that there would not be enough momentum to get that? I’d love to have generics too, but a very simple array syntax would in my opinion still add a lot of value, even if we already had generics.

Bilge · June 26, 2024, 9:04pm

On 26/06/2024 21:56, Casper Langemeijer wrote:

Generics or bust.

I do not understand the reasoning behind that. Is it because we really want generics, but when the 95% use-case is solved we fear that there would not be enough momentum to get that? I'd love to have generics too, but a very simple array syntax would in my opinion still add a lot of value, even if we already had generics.

It's not that there wouldn't be enough momentum; if anything, a successful typed arrays PR could give rise to such momentum. If it were possible to have typed arrays that were fully compatible with an (imagined) generics specification then I would be all for it. The problem is, how can we ever ensure compatibility with something that doesn't exist?

It seems to me we either solve is problem properly (with generics) or not at all.

Cheers,
Bilge

Derick_Rethans · June 26, 2024, 9:49pm

On 26 June 2024 20:59:00 BST, Richard Miles <richard@miles.systems> wrote:

Howdy people,

The pattern-matching RFC inspired me to write this. I have not done any work for this yet; just looking for initial feedback. I think we should have typed arrays in PHP. I propose adding the class SplTypeDefinedArray.

We've done some initial work related to this as part of PHP Foundation work:
<https://github.com/derickr/php-src/tree/collections/Zend/tests/collection>

A summary on the state of this, and actual genetics will be available soon too.

cheers
Derick

Richard_Miles · June 26, 2024, 8:53pm

I looked online, and I see two RFC’s relating to generics. One talks about arrays specifically.

[

rfc:generics
wiki.php.net

](https://wiki.php.net/rfc/generics)

[

rfc:generic-arrays
wiki.php.net

](PHP: rfc:generic-arrays)

I don’t think that it gives the full power we’re looking for, and also not sure where the work at on these either.
Would love to get involved with generic classes.
How would you pose the syntax?

Best,
Richard Miles

On Jun 26, 2024, at 2:08 PM, Bilge bilge@scriptfusion.com wrote:

On 26/06/2024 20:59, Richard Miles wrote:

I think we should have typed arrays in PHP.

Generics or bust.

Cheers,
Bilge

(Attachment favicon.png is missing)

Rob_Landers · June 26, 2024, 9:23pm

On Wed, Jun 26, 2024, at 22:56, Casper Langemeijer wrote:

On Wed, Jun 26, 2024, at 21:59, Richard Miles wrote:

I think we should have typed arrays in PHP.

Yes! I cannot stand sitting through conference talks on ‘generics’ that only talk about ‘collections’. This could be solved if we had typed arrays. If anything we would get better talks on Generics.

Arrays of a type is one of the last cases where I need docblocks to tell my editor about the types.

In my opinion, even if we would have some implementation of generics, having typed arrays with a simple syntax would be awesome.

A syntax suggestion:

$array = stdClass;

class A {

public stdClass $array;

}

Adding an invalid array member should throw TypeError.

I know there are way more edge-case situations to think of (for example: if class B extends A, $b is of type B, but holds only A’s, can $b be assigned as value of public A $a ?)

Generics or bust.

I do not understand the reasoning behind that. Is it because we really want generics, but when the 95% use-case is solved we fear that there would not be enough momentum to get that? I’d love to have generics too, but a very simple array syntax would in my opinion still add a lot of value, even if we already had generics.

You actually just gave me an evil idea on how to get generics in userland… unfortunately it is bed time as I have an early train to catch in the morning. I will stew on it, but here is the gist:

Create a class that generates a typed collection when accessed via array, such that (new InternalTypedArray)[‘int’] produces a class that acts like an array but only accepts ints.

Using the composer file loading key, load a file that contains:

define(‘TypedArray’, new InternalTypedArray);

You can now use $arr = TypedArray[‘MyType’]

Could be interesting and I wouldn’t be surprised if it hasn’t been done before.

— Rob

Lanre · June 26, 2024, 9:15pm

On Wed, Jun 26, 2024 at 3:10 PM Bilge <bilge@scriptfusion.com> wrote:

On 26/06/2024 21:56, Casper Langemeijer wrote:

Generics or bust.

I do not understand the reasoning behind that. Is it because we really want generics, but when the 95% use-case is solved we fear that there would not be enough momentum to get that? I’d love to have generics too, but a very simple array syntax would in my opinion still add a lot of value, even if we already had generics.

It’s not that there wouldn’t be enough momentum; if anything, a successful typed arrays PR could give rise to such momentum. If it were possible to have typed arrays that were fully compatible with an (imagined) generics specification then I would be all for it. The problem is, how can we ever ensure compatibility with something that doesn’t exist?

It seems to me we either solve is problem properly (with generics) or not at all.

Cheers,
Bilge

I disagree, in fact it would be super easy to make T translate to array if generics ever become a thing in php, but it doesn’t make any sense to keep holding off on implementing this just because it might not be compatible with an implementation of generics that may never come.

Cheers,
Lanre

Richard_Miles · June 26, 2024, 9:46pm

I actually do like the idea of Generics, but the more I think about it, it’s not the problem here. Everyone should absolutely catch up on Larry Garfield’s Pattern Matching RFC. It could be the key for generics (the is operator). The problem is making “array interfaces” a thing. Larry said he probably wouldn’t add support in this version for array types, but I think that’s a mistake. We should define a direction for what we would pass to new Array<T> . I love Casper Lanemeijer’s syntax suggestion with some changes:

interface iArrayA [‘a’ => string ]
interface iArrayB implements iArrayA [‘b’ => string, ‘c’ => ?string ]

$array = iArrayA [
‘a’ => ‘hello’
];

// reads the same as a typecast
$array = (iArrayA) [
‘a’ => ‘hello’
];

// it’s essentially like a typecast, which should probably be allowed. If the set of possible values needs to increase a typecast would do it.

class A {
public iArrayB $array = [
‘a’ => ‘hello’,
‘b’ => ‘world’
];
}

Best,
Richard Miles

···

On 26/06/2024 21:56, Casper Langemeijer wrote:

Generics or bust.

I do not understand the reasoning behind that. Is it because we really want generics, but when the 95% use-case is solved we fear that there would not be enough momentum to get that? I’d love to have generics too, but a very simple array syntax would in my opinion still add a lot of value, even if we already had generics.

It’s not that there wouldn’t be enough momentum; if anything, a successful typed arrays PR could give rise to such momentum. If it were possible to have typed arrays that were fully compatible with an (imagined) generics specification then I would be all for it. The problem is, how can we ever ensure compatibility with something that doesn’t exist?

It seems to me we either solve is problem properly (with generics) or not at all.

Cheers,
Bilge

Richard_Miles · June 26, 2024, 10:01pm

Hey Derick,

A summary on the state of this, and actual genetics will be available soon too.

My mouth is watering already I have tons of free time at the moment if you’d like any help!

Best,
Richard Miles

Richard_Miles · June 26, 2024, 11:26pm

We've done some initial work related to this as part of PHP Foundation work:
<https://github.com/derickr/php-src/tree/collections/Zend/tests/collection>

After reviewing the PR I don't think this accurately captures what we're attempting to do/discuss.
Are there other branches you could share? You’ve posed a lot of new syntax. I have questions.

collection(Dict) Articles<string => Article> {}

The code above is limiting compared to the posed syntax in this thread. Since your just working on providing a specific datatype with a custom syntax. What am I supposed to be able todo inside the {}? Why not?

collection(Dict<Article>) Articles {}

or just

Dict<Article> Articles {}
Seq<Article> Articles {}

and then there's the completely new syntax? Am I supposed to be able to add methods in this block?

class Articles extends Dict<Article> {}

If I'm not then it should read more like the following:

$a = (Dict<Article>) ;

____________________________________________________________

But this all feels off-topic. Because, we need to get Typed Array syntax!

interface iArrayA ['a' => string ]
interface iArrayB implements iArrayA ['b' => string, 'c' => ?string ]

$array = iArrayA [
‘a’ => ‘hello'
];
// reads the same as a typecast
$array = (iArrayA &| iArrayB) [
‘a’ => ‘hello'
];

// It’s essentially like a typecast, which should probably be allowed. If the set of possible values needs to increase, a typecast would do it.

class A {
  public iArrayB $array = [
    ‘a’ => ‘hello’,
    ‘b’ => ‘world'
  ];
}

If generics and the is operator get passed then one could in theory do.

class A <T is iArrayA>{
        public T $array = [
    ‘a’ => ‘hello’
  ];
}

$a = new A<iArrayB>;

Best,
Richard Miles

radar3301 · June 26, 2024, 11:08pm

Several people said:

generics

Is there a reason why traits can't be used to introduce generics?
i.e. extending the code that composes the trait into the class to
replace the generic type? eg. in `zend_add_trait_method`?
(I couldn't quickly find where trait properties get added, even
though my example is for properties.)

trait GenericTrait<T>
{
private ?T $object = null;
}

class ComposableClass
{
use GenericTrait<Foo>;
}

// Result:
class ComposedClass
{
private ?Foo $object = null;
}

Just a random thought...

Regards,
radar3301

Levi_Morrison · June 26, 2024, 11:09pm

On Wed, Jun 26, 2024 at 3:27 PM Rob Landers <rob@bottled.codes> wrote:

On Wed, Jun 26, 2024, at 22:56, Casper Langemeijer wrote:

On Wed, Jun 26, 2024, at 21:59, Richard Miles wrote:

I think we should have typed arrays in PHP.

Yes! I cannot stand sitting through conference talks on 'generics' that only talk about 'collections'. This could be solved if we had typed arrays. If anything we would get better talks on Generics.

Arrays of a type is one of the last cases where I need docblocks to tell my editor about the types.

In my opinion, even if we would have some implementation of generics, having typed arrays with a simple syntax would be awesome.

A syntax suggestion:

$array = stdClass;
class A {
public stdClass $array;
}

Adding an invalid array member should throw TypeError.

I know there are way more edge-case situations to think of (for example: if class B extends A, $b is of type B, but holds only A's, can $b be assigned as value of public A $a ?)

Generics or bust.

I do not understand the reasoning behind that. Is it because we really want generics, but when the 95% use-case is solved we fear that there would not be enough momentum to get that? I'd love to have generics too, but a very simple array syntax would in my opinion still add a lot of value, even if we already had generics.

You actually just gave me an evil idea on how to get generics in userland… unfortunately it is bed time as I have an early train to catch in the morning. I will stew on it, but here is the gist:

Create a class that generates a typed collection when accessed via array, such that (new InternalTypedArray)['int'] produces a class that acts like an array but only accepts ints.

Using the composer file loading key, load a file that contains:

define('TypedArray', new InternalTypedArray);

You can now use $arr = TypedArray['MyType']

Could be interesting and I wouldn’t be surprised if it hasn’t been done before.

— Rob

I have seen variants of this several times. Many will hook into the
autoloader and generate a class/interface definition on-the-fly to
make it work.

Richard_Miles · June 26, 2024, 11:30pm

I suppose you don’t actually need the is op; you can use the implements &| extends keywords

Best,
Richard Miles

On Jun 26, 2024, at 5:26 PM, Richard Miles richard@miles.systems wrote:

We’ve done some initial work related to this as part of PHP Foundation work:
https://github.com/derickr/php-src/tree/collections/Zend/tests/collection

After reviewing the PR I don’t think this accurately captures what we’re attempting to do/discuss.
Are there other branches you could share? You’ve posed a lot of new syntax. I have questions.

collection(Dict) Articles<string => Article> {}

The code above is limiting compared to the posed syntax in this thread. Since your just working on providing a specific datatype with a custom syntax. What am I supposed to be able todo inside the {}? Why not?

collection(Dict) Articles {}

or just

Dict Articles {}
Seq Articles {}

and then there’s the completely new syntax? Am I supposed to be able to add methods in this block?

class Articles extends Dict {}

If I’m not then it should read more like the following:

$a = (Dict) ;

But this all feels off-topic. Because, we need to get Typed Array syntax!

interface iArrayA [‘a’ => string ]
interface iArrayB implements iArrayA [‘b’ => string, ‘c’ => ?string ]

$array = iArrayA [
‘a’ => ‘hello’
];
// reads the same as a typecast
$array = (iArrayA &| iArrayB) [
‘a’ => ‘hello’
];

// It’s essentially like a typecast, which should probably be allowed. If the set of possible values needs to increase, a typecast would do it.

class A {
public iArrayB $array = [
‘a’ => ‘hello’,
‘b’ => ‘world’
];
}

If generics and the is operator get passed then one could in theory do.

class A {
public T $array = [
‘a’ => ‘hello’
];
}

$a = new A;

Best,
Richard Miles

Levi_Morrison · June 27, 2024, 8:15pm

Is there a reason why traits can't be used to introduce generics?
i.e. extending the code that composes the trait into the class to
replace the generic type? eg. in `zend_add_trait_method`?
(I couldn't quickly find where trait properties get added, even
though my example is for properties.)

trait GenericTrait<T>
{
private ?T $object = null;
}

class ComposableClass
{
use GenericTrait<Foo>;
}

// Result:
class ComposedClass
{
private ?Foo $object = null;
}

Just a random thought...

Regards,
radar3301

I worked with Joe Watkins to do a proof-of-concept for generic traits.
It's a bit old since it's from 2017, but could be a useful starting
point if you are serious about pursuing this idea:

Richard_Miles · June 27, 2024, 7:41pm

Could someone please spare a moment to give me karma so I can open an RFC?
I greatly appreciate your time. My username is: wookieetyler

Derick Rethans commit inspired me to start looking at the source code in more depth.
I got started on this branch <https://github.com/php/php-src/compare/master…RichardTMiles:php-src:arrayTypes>
last night, I focused on the zend_language_parser.y. I think I got that figured out, and not I need to adjust the functions it touches (zend_ast_create_decl and zend_ast_create).
I might call myself a very seasoned n00b here, so my main goal is to get it working.


```
interface_declaration_statement:
               T_INTERFACE { $<num>$ = CG(zend_lineno); }
               T_STRING interface_extends_list backup_doc_comment '{' class_statement_list '}'
                      { $$ = zend_ast_create_decl(ZEND_AST_CLASS, ZEND_ACC_INTERFACE, $<num>2, $5, zend_ast_get_str($3), NULL, $4, $7, NULL, NULL); }
               | T_INTERFACE { $<num>$ = CG(zend_lineno); }
               T_STRING interface_extends_list backup_doc_comment '[' array_statement_list ']'
                      { $$ = zend_ast_create_decl(ZEND_AST_CLASS, ZEND_ACC_INTERFACE, $<num>2, $4, zend_ast_get_str($2), NULL, $3, $6, NULL, NULL); }
;
```

Then for casting:


expr:

...

|       type_without_static expr
                       { $$ = zend_ast_create(ZEND_AST_CAST, $2, $4); }
|       '(' type_expr_without_static_or_nullable ')' expr
               { $$ = zend_ast_create(ZEND_AST_CAST, $2, $4); }
|       '(' expr ')' {
               $$ = $2;
               if ($$->kind == ZEND_AST_CONDITIONAL) $$->attr = ZEND_PARENTHESIZED_CONDITIONAL;
        }
|       new_expr { $$ = $1; }

...

I should also point out that the syntax I posed yesterday was a bit incorrect as I extended an interface using the implements keyword.

interface iArrayA [‘a’ => string ]
interface iArrayB implements iArrayA [‘b’ => string, ‘c’ => ?string ]

It should actually read:

interface iArrayA [ ‘a’ => string ]
interface iArrayB extends iArrayA [ ‘b’ => string, ‘c’ => ?string]

P.s. I also started a draft for the addition of the apache_connection_stream function https://github.com/php/php-src/pull/14047
and would like to create an RFC for that as well.

Best,
Richard Miles

Richard_Miles · June 27, 2024, 8:30pm

I worked with Joe Watkins to do a proof-of-concept for generic traits.
It's a bit old since it's from 2017, but could be a useful starting
point if you are serious about pursuing this idea:

Comparing php:master...morrisonlevi:parameterized_traits · php/php-src · GitHub

I’m also interested in this; it will help see branches like these.
Did you ever get the POC working? What did you feel like was the biggest hurdle?

Best,
Richard Miles

Michal_Marcin_Brzuch · June 30, 2024, 4:13pm

Hi Richard,

czw., 27 cze 2024, 22:33 użytkownik Richard Miles richard@miles.systems napisał:

I worked with Joe Watkins to do a proof-of-concept for generic traits.
It’s a bit old since it’s from 2017, but could be a useful starting
point if you are serious about pursuing this idea:

https://github.com/php/php-src/compare/master…morrisonlevi:php-src:parameterized_traits

I’m also interested in this; it will help see branches like these.
Did you ever get the POC working? What did you feel like was the biggest hurdle?

There even was an RFC in voting which Joe implemented and it addresses nearly what is discussed it this thread https://wiki.php.net/rfc/arrayof

I must admit that the collection proposal is bit too complex to address such tiny but powerfully in my opinion functionality which is typed array. What Derick showed looks more like generic collection and such require declaring it’s type. In my opinion this is way too mush hassle to declare as many collection types as many usages in application. Also two collection types with the same collection item type will not be compatible from type perspective. While typed array seems much more clear and compatible in all places where typed array is needed without declaring separate type for each usage.

If I were to choose between typed-array and collection like Derick showed a little bit I’d choose typed arrays definitely as a first feature to be merged into PHP.

Cheers,
Michał Marcin Brzuchalski

Richard_Miles · June 30, 2024, 11:10pm

If I were to choose between typed-array and collection like Derick showed a little bit I’d choose typed arrays definitely as a first feature to be merged into PHP.

Cheers indeed. I’m still looking for a karma sponsor

Tuples are nice because they are bounded, they can probably act like arrays …

I’ve been leaning toward this working under the hood like SplObjectStorage and SplFixedArray.
For those who may not be super familiar, I’ve added the PHP documentation links:

[

SplObjectStorage - Manual
php.net

](PHP: SplObjectStorage - Manual)

[

SplFixedArray - Manual
php.net

](PHP: SplFixedArray - Manual)

A typed array would/may not be a hash table, but a fixed list of some kind. I’ve had a few suggestions
and one individual in the is operator conversation said hash tables may be appropriate anyway.
Time will tell on this front and its implementation is worth the open discussion. What should be absolute
is that arrays with a ‘fixed’ type maynot assign members whoes types are not strictly defined. Thus, typed
arrays are stictly bound.

SplFixedArray acted like numeric Seq<> whereas SplObjectStorge adds functionionallity to our normal
arrays and acts our Dict<>.

I think a big issue this proposal solves is how to syntactically approach saying a member is present or null
or if member is present it should only be a string. The initial proposed syntax in the early feedback of the
pattern matching rfc poses:

So yes, [?'foo' => string] and ['foo' => ?string] are indeed
different.

A lot of discussion around this, but I think with interfaces this becomes trivail. Note my example code sovles
this using operators we already have: &, |, &|

interface iArrayA [‘a’ => string ]
interface iArrayB implements iArrayA [‘b’ => string, ‘c’ => ?string ]

// reads the same as a typecast
$array = (iArrayA &| iArrayB) [
‘a’ => ‘hello’
];

… (possibly even cast to them and vice-versa).

The way I implemented declaring types is actually just a typecast, so perhaps that should change or
we just extend the current array implementation to hold another metadata field.?

Best,
Richard Miles

(Attachment favicon-196x196.png is missing)

Crell · July 1, 2024, 12:59am

On Sun, Jun 30, 2024, at 11:13 AM, Michał Marcin Brzuchalski wrote:

Hi Richard,

czw., 27 cze 2024, 22:33 użytkownik Richard Miles
<richard@miles.systems> napisał:

> I worked with Joe Watkins to do a proof-of-concept for generic traits.
> It's a bit old since it's from 2017, but could be a useful starting
> point if you are serious about pursuing this idea:
>
> Comparing php:master...morrisonlevi:parameterized_traits · php/php-src · GitHub

I’m also interested in this; it will help see branches like these.
Did you ever get the POC working? What did you feel like was the biggest hurdle?

There even was an RFC in voting which Joe implemented and it addresses
nearly what is discussed it this thread PHP: rfc:arrayof

I must admit that the collection<Dict> proposal is bit too complex to
address such tiny but powerfully in my opinion functionality which is
typed array. What Derick showed looks more like generic collection and
such require declaring it's type. In my opinion this is way too mush
hassle to declare as many collection types as many usages in
application. Also two collection types with the same collection item
type will not be compatible from type perspective. While typed array
seems much more clear and compatible in all places where typed array is
needed without declaring separate type for each usage.

If I were to choose between typed-array and collection like Derick
showed a little bit I'd choose typed arrays definitely as a first
feature to be merged into PHP.

Cheers,
Michał Marcin Brzuchalski

Contextual point: Nearly every other major language at this point that has a meaningful standard library has gone with a three-separate-object approach (Seq, Set, Dict, called various things). At least Python, Javascript, Rust, Kotlin, and Swift, in my research. (Go doesn't, but Go avoids having features by design.) And AFAIK *every* language except PHP and Lua separates sequences from dictionaries. Typed arrays will not full resolve the seq vs dict problem, which is arguably PHP's original sin. And there's a lot of weird issues to resolve around what the syntax could even be, since PHP has untyped variables.

The custom collection syntax Derick has been working on is a second-best alternative to generics, essentially. It is less ergonomic, no question, but can also be implemented without generics, and so is about 5x easier to do. If we could get native generics, then I think everyone involved agrees building collections off of that -- in essentially the same way as every language I mentioned above --- would be preferable to a custom one-off syntax.

--Larry Garfield