Folder structure for Expressive modules

expressive

#1

I would like to open a discussion about a possible folder structure for an Expressive module.
I’m working with some prototypes and I’ve got some different ideas. The goal is to design a simple folder structure that is reusable, general purpose, and can facilitate people to start working, without wondering where to put code.

At the moment, I’m using something like this:

/src/Foo
/src/Foo/config
/src/Foo/src
/src/Foo/templates
/src/Foo/ConfigProvider.php

where Foo is the name of the module under the /src folder. The module configuration is stored in the config folder (consumed by ConfigProvider.php). The source code of the module is stored in src folder. If present, the templates are stored in a template folder.

The source code of the module is stored in the Foo/src folder with the following sub-dirs:

/src/Foo/src/Middleware
/src/Foo/src/Action
/src/Foo/src/Service

The middleware classes are stored in Middleware or Action folder, depending if the class delegates processing in one or more paths through its logic (Middlware) or the class never delegate processing, but always generate and return a response on their own (Action). Typically, a routed middleware is stored in Action, and pretty much everything else in Middleware.

The Service folder contains classes related to the business logic (like a model) or classes that facilitate the business logic. These classes are stored in a service container (PSR-11).

Thoughts?


#2

I guess the intent is to emulate a ZF3 module structure? Sounds about right. I have a src folder that is very similar. The only suggestion would be to not have something simply titled “Middleware” as the classes in Action are middleware too. I went with PipedMiddleware, RoutableAction, Service, but in retrospect that could still be a touch confusing. Perhaps something like:

PipedMiddleware
RoutedMiddleware

would be a little more explicit without having to be explained in documentation.

Having what amounts to nested /src directories seems a touch redundant and might make the namespacing more difficult to follow. Do we really need the Foo folder? Most PSR-4 compliant packages would just have the PipedMiddleware, RoutedMiddleware, and Service folders at the top level /src folder for a structure like:

composer.json
/config
/src
---/src/ConfigProvider.php
---/src/PipedMiddleware
---/src/RoutedMiddleware
---/src/Service
/templates
/tests

This keeps the src folder limited to namespaced classes that can be called in the application.

(Edited: Added a cleaned up structure idea)


#3

@moderndeveloperllc yes, the idea is to provide a common way to design reusable module against different Expressive projects, like we have in ZF2/3 with MVC.

I like your idea to simplify the namespaces. In case of a module as external vendor you right, we omit the Foo folder, but if we have modules in the same Expressive skeleton we need to use a name (and a folder).

I also like the idea to use only suffix for specific src classes, like src/RoutedMiddleware. My only concern is about some module that contains a lot of actions, middleware and services. Maybe this should be a good signal to split it in seperate modules?


#4

I had been thinking of third-party modules when suggesting to remove Foo. I’m currently refactoring some API middleware and services out of a project so I can reuse it elsewhere.

Sorry, I realized I might not have been the most specific about PipedMiddleware and RoutedMiddleware. I was suggesting that they were folders that could contain however many classes inside. I guess having lots or few middleware is up to the dev. Some of us are “lumpers” and some “splitters”. For instance, I have

PipedMiddleware/
  AuthenticationFactory.php
  AuthenticationMiddleware.php
  AuthorizationFactory.php
  AuthorizationMiddleware.php
  Helper/
    LoggingSetUserMiddleware.php
    QueryAttributeParamsMiddleware.php
  LoggingFactory.php
  LoggingMiddleware.php
  RouteNotFoundHandler.php

I kept my factories in the same namespace because there were so few. I know many would create a Factory directory for those.


#5

I think the first “/src” directory is the module root. All other directories inside “/src/Foo/src” are PSR-4 compliant.

To avoid such a missunstanding the directory structure should look like this:

/project_root
—/module
------/Foo
---------/src
------------/ConfigProvider.php
------------/PipedMiddleware
---------------/Service
------------------/AuthenticationFactory.php
---------------/AuthenticationMiddleware.php
------------/RoutedMiddleware
---------------/Service
------------------/MyRoutedMiddlewareFactory.php
---------------/MyRoutedMiddleware.php
------------/Service
------/Bar
---------/src
------------/ConfigProvider.php
------------/PipedMiddleware
------------/RoutedMiddleware
------------/Service

Additionally i would suggest to put all factories inside a separate directories to avaoid cluttering than a module have many factories.


#6

I personally dislike this practice, for two reasons:

  • It makes discovery of the factories more difficult. Where do I look — in a Factory directory under the current one? in a Service directory under the current one? Or are those up a level, i.e., in a peer namespace? If so, they then might contain factories for multiple namespaces, which can also be confusing.
  • It makes writing the factories harder. When the class the factory creates is in the same namespace, you don’t need to import it. When it’s in a sibling namespace, you need to import it. When it’s in a subnamespace, you have redundant information in the class name: Factory\SomeMiddlewareFactory or Service\SomeMiddlewareFactory.

Which brings me to a couple more notes.

First, in your example, you have things like the following:

|-- AuthenticationMiddleware.php
|-- Service
    |-- AuthenticationFactory.php

This is confusing, as it’s unclear from the name whether AuthenticationFactory will create an instance of the AuthenticationMiddleware, or some nebulous Authentication class. The factories should clearly indicate what they create.

Second, the name Service is nebulous, particularly for somebody not familiar with Zend Framework in particular. Does it refer to something accessing a web service? Or is it something that provides a particular web service? or is it some other type of service entirely? To answer the question, it’s the latter. In your example, it’s referring to a “service” registered in a “service manager”, which is what ZF calls its dependency injection container. In Enrico’s original post, it’s actually referring to domain models and similar.

If we consider your example, where the Service directory and subnamespace is for seeding the DI container, if we consider the fact that Expressive is a PSR-11 consumer, which defines a ContainerInterface, and the classes in these directories provide factories by which the container can retrieve dependencies, Factory is a more clear namespace if you really, really need to separate such classes. As noted earlier, I think it’s far better to keep them together in the same namespace.

In Enrico’s example, I’d argue that the directory/namespace should be Model or Domain, if it needs to exist at all. Many of these classes are often the core of a module. As a quick example, let’s say I’ve created a module for a web service that will list books in an inventory, and allow manipulating the inventory. My basic entities might be a Book class, a Collection (for sets of books), a Repository (for retrieving books from my persistent storage), and an InputFilter (for validating new and/or updated book entries submitted to the service). These could be in the root of the module. I might then have a Middleware subdirectory/subnamespace that contains the various middleware for the web service itself, and which uses these classes:

Books
|-- src
    |-- Book.php
    |-- Collection.php
    |-- ConfigProvider.php
    |-- InputFilter.php
    |-- Repository.php
    |-- Middleware
        |-- AddBookMiddleware.php
        |-- BookMiddleware.php
        |-- SearchMiddleware.php
        |-- UpdateBookMiddleware.php
|-- templates
|-- test

All that said, though, I think what Enrico is trying to get at is having a suggested, recommended directory structure for modules that is predictable. This makes it easier for developers working on an existing module for the first time to understand the structure: they know where to find classes that provide specific areas of functionality, and where to put new classes, based on their functionality. Additionally, it allows us to have our tooling that creates modules create these standard subdirectories, eliminating the need for developers to create their own structure. If this is what we want to provide, then a Domain or Model subnamespace would make sense.

Of course, any of these subnamespaces may be omitted if the module does not provide classes that fill those roles.

Finally, I’ll throw this out there, as it’s a discussion I’ve had with @enrico and others a few times the past month:

The term middleware should refer to any middleware that delegates processing of the request in one or more branches of its logic. The terms action or handler should refer to middleware that never delegates processing, and, instead, always generates and returns a response on its own.

With that in mind, we’d have Middleware and/or Action (or Handler) subnamespaces, depending on what functionality the module exposes.


#7

Thank you for this interessting ponts. The class name “AuthenticationFactory” was just a misspelling, sorry for confusion, I totally agree with you.

What is you optinon to “/module” vs “/src” as the name of the module root directory?


#8

I prefer src/, as it allows either modules or libraries, thus keeping all source code for the application in one place.


#9

I created a very similar structure as Enrico show with little modifications.
@enrico are you sure about /src/Foo/ConfigProvider.php shouldn’t that be /src/Foo/src/ConfigProvider.php ?

  1. First of all I prefer to name top level dir “packages” since “src” is little bit confusing for me.

    /src/Foo/src/
    /src/Bar/src/
    would be

    /packages/Foo/src/
    /packages/Bar/src/
    “modules” would be also fine.

  2. In small package I will put big array in ConfigProvider.php and return it.
    If package is a bigger separated configs would be better.

  3. Also I prefer to hold all Factories in the same dir where classes are.
    But sometimes if package grow for eg. 70 separated Actions (that would be 140 files with factories)
    it is little bit easier to brows the files if they are in separate folders like

    src/Action/Factory/*
    src/Action/*

  4. I like the idea to hold all domain classes in src dir like matthew show,
    but don’t know how that would look in just a little bigger package/module…

This would be one typical module/package from my app

├── Page
    ├── src
    │   ├── ConfigProvider.php
    │   ├── Action
    │   │   ├── IndexActionFactory.php
    │   │   └── IndexAction.php
    │   ├── Entity
    │   │   └── Page.php
    │   ├── Filter
    │   │   └── PageFilter.php
    │   ├── Mapper
    │   │   ├── PageMapperFactory.php
    │   │   └── PageMapper.php
    |   ├── Middleware
    │   └── Service
    │       ├── PageServiceFactory.php
    │       └── PageService.php
    ├── test
    └── templates
        ├── page
        │   └── index.phtml
        └── partial
            └── pagination.php

I guess it’s all depend from one question “How big is package/module?”.

It’s a really god topic! Thanks Enrico.


#10

Very interesting and important topic!

First, I also do not like to top /src/ folder as well. Whenever I use the skeleton installer, the very first thing I do is to rename it to /module/. I know that all files in a module are somehow source code (templates, config files, classes). But the duplication of /src/ in the same pass just doesn’t feel right. I had the same discussion with @matthew a while ago without a result. There are different opinions.

Anyway I really do not like a folder named /service/. It always raises the question: What is a service? If you are doing the DDD approach it is well defined, but a lot of programmers just name their classes WhateverService if they have no proper name. I really like the layer approach with the following subdirs:

  • /module/Foo/src/Application/ for actions and other application services
  • /module/Foo/src/Domain/ for all domain classes like entities, repositories, storages, input filters and domain services
  • /module/Foo/src/Infrastructure/ for all infrastructure stuff like table gateway classes and infrastructure services like emailing, logging and stuff
  • /module/Foo/src/Presentation/ for forms, view helper or other presentation classes

Not every module has all of these subdirs but every subdir is divided again.

So my point here is it will be very hard to find a good recommendation for the /src/ subdir folder. I would always only set some common rules like place factory in the same folder than the class it creates and stuff like that.

Thanks, Ralf


#11

This is kinda controversial though: all modules are sources, not all
sources are modules :wink:

I currently don’t even use modules anymore: just put everything under
config/autoload.php.

Marco Pivetta

http://twitter.com/Ocramius

http://ocramius.github.com/


#12

Put your sources that are not a module in the /src/ folder and the sources that are a module in a /module/ folder. Problem fixed :wink:


#13

Hi all,

I like the proposal of brain storming the modules. When I hear the term module, I always think about re-usability. So a module should be

1 ) Re-usable
2 ) Flexibile enough to change templates / assets
3 ) Easy to configure

I have been thinking about this idea for sometime. And tried to come up with some prototype. My idea was not just for expressive but for any PSR-7, PSR-15, PSR-17 frameworks.

Probably some of you may have came across the post : http://harikt.com/blog/2017/03/24/building-modular-applications-with-psr-7/

The structure of the module was as

src
    Container
    Middleware
    ConfigProvider.php
templates
   action1.twig
   // or
   action2.php
public
    css
    images
    js

What I actually struggled was there was no way in ConfigProvider.php to load an object from di container and modify something on the particular service.

You may not be sure why I want to do the same, and probably @ocramius would say that is actually a bad idea changing the state :wink: .

What I was trying to do was get the AssetLocator and set the path where the assets resides for each module.

And once all ConfigProviders are merged you can alter where the application specific asset paths.

The assets are actually served by a separate library : https://github.com/harikt/psr7-asset which is actually a fork of https://github.com/friendsofaura/Aura.Asset_Bundle .

You can see a complete example on expressive : https://github.com/harikt/psr7-asset-example-zendexpressive

As I mentioned if you created a module user then it is possible to change the js , css, images and with the help of namespace template with zend-expressive you can alter the template also.

Thank you for your time. If there is anything unclear I am happy to discuss about the same.


#14

We do have a way to do this already: delegator factories. These are supported by the v2.X skeleton and later for all supported containers, and allow you to modify or replace a service after initial creation.

Additionally, all ConfigProvider classes are merged during bootstrapping of the config service, which happens even before a container is created, which fulfills the other requirement you specify.


#15

Thank you @matthew . I have looked at the delegator factories doc. One thing I am confused is the callback. How can we make sure what is the callback it is receiving. But that looks a separate topic to be discussed.


#16

Indeed, but the short answer is: do an instanceof check, and if it’s not of a type you recognize, just return it verbatim without operating on it.


#17

Ok, thanks for the feedback! I think we can summarize the idea and simplify a Module structure with the following folders:

  • config;
  • src;
  • templates;

The src folder will contain all the source code of the module. Here we can have also subfolder, such as:

  • Action (for the routed middleware, returning a response);
  • Middleware (for generic middleware, usually consuming $delegate/$next);
  • Model (for the business logic).

All the source code can be also stored in src without any subfolder. This really depends on the use case and developer’s preference. The class files should be named using a significant suffix, e.g. Action, Middleware, etc.

The only requirement of src is that it MUST contain a ConfigProvider.php file to serve the configuration.

The only open question, related with the @harikt reply, it’s the public folder. IMHO I think it’s ok, we need to find a “standard” way to serve this folder in the web server, using some asset library/idea.


#18

Recently I came across a CMS framework : https://github.com/dms-org/ which made me install laravel for it has some integration for the same.

I noticed a few good things which I liked.

Laravel pushes the templates to a resources/views/vendor folder. Now you have all the flexibility to change the views, css, etc for the app. That looks interesting to me. I guess it has some similarities to that of https://github.com/zfcampus/zf-asset-manager .

I may be also wrong for I don’t have a very deep understanding of the laravel eco system. But their way of packing docs look interesting. May be we should borrow some of the good things.

I have been playing with it to integrate to expressive, but due to limited time wasn’t able to make it work as expected.