Contao news

Each year, the Contao Core development team meets twice for a short code sprint of three days. As there are currently four developers from Switzerland, the meeting takes place a bit outside of Zurich. The meetings are supported by the Contao Association and are very important to us. Consider becoming a member and support us too!

I have decided to gather the most important information after every meeting and share it with the entire community. This time, the following topics have kept us on our toes:

Contao 4.9 LTS and Symfony 4.4 LTS

The next Contao version will be Contao 4.9, our next LTS version. Therefore, stability is a top priority and in all the decisions we have made over the past few days, we have tried to make it in the spirit of the next LTS version.

One thing we will definitely change is that we are going to increase the minimum required versions of all Symfony components used to version 4.4. This will be the upcoming LTS version of Symfony and was also the reason for adjusting our release cycle (otherwise 4.8 would have been the next LTS version).

We do not expect to make any changes to the minimum requirements for the PHP version. An increase to PHP 7.4 and possibly a conversion to "typed properties" may be up for discussion with Contao 4.10.

Ecosystem

There is relatively little to report about the ecosystem this time. The Contao Manager and the Cloud run very stable and contao.org got a makeover of the homepage.

The two biggest construction sites of the Contao Manager are the installation of private packages and the configuration of extensions via UI and thus also the replacement for the separate install tool. Private packages for the Manager should become a fact at the Contao Conference and all further optimizations have been postponed. This is due to the upcoming LTS version and the focus for this meeting on Contao as software.

Two notable exceptions are the developer documentation and the "Maker Bundle". The documentation was also worked on during this meeting. Especially Fritz invested a lot of time here. Jim's new baby is the upcoming "Maker Bundle". Similar to the version of Symfony it will make the creation of front end modules, content elements, DCA's etc. easier.

The documentation will be published for the Contao Conference and will hopefully be improved and extended from then on.

Webpack/Symfony Encore Integration

In Contao 4.5 we have introduced the {{asset::*}} insert tag, which allows you to dynamically output the path to an asset via manifest.json. This makes it very easy to generate changing paths (with versioning/hashing for cache busting) and include them in the page layout.

We discussed whether we would like to have a deeper integration which would for example allow the selection of assets via checkbox wizard. The Symfony Encore specific Entrypoints which can group several assets may be interesting as well.

After a long discussion we came to the conclusion that a deeper integration doesn't really make sense at the moment. This is mainly due to the Symfony Encore Entrypoints that do not seem really practicable for us because we don't want to decide which assets should be loaded in the page layout but rather the respective elements or modules should decide themselves. Otherwise, we'd need a lot of page layouts, which doesn't seem very handy. There are also some third-party bundles that allow the integration of Webpack already.

The topic is certainly not off the table, but in the end there were far more important topics to which we wanted to devote ourselves.

Migration Framework

More important topics such as a migration framework! I personally can't remember when the concept of the runonce.php was introduced. In any case, it feels as if it has always existed. A file that is simply included in the install tool, executed and then deleted. The easiest way to offer migrations in your own extensions.

But of course it also has some disadvantages:

• Deleting a file after its execution so that it is not executed again is not really state of the art.
• Error messages are simply ignored.
• They cannot be executed on the command line.
• You can't see which migrations will be executed before you run them effectively, let alone know if anything has happened at all.
• and more…

The Contao Core itself, on the other hand, uses its own migration classes for its migrations and other extensions have invented their own interfaces as well. For the 4.9 LTS it was therefore high time that we sat down and worked on a solution for everyone.

Martin has taken on the task and worked on a solution where migrations of all extensions (including the core itself) can be easily tagged as a service and then executed either via install tool or command line. This allows us to ensure a uniform execution of all migrations for both, the install tool and the command line.

We have discussed the requirements for quite some time and are convinced that we can offer a well-crafted solution for everyone, but if you would like to participate in the discussion, then you can of course do this in the corresponding pull request.

Cron Job Framework

In view of our next LTS version, we would like to unify even more things. These include the way you can both, run and register cron jobs in Contao. In particular, we would like to be able to disable the Poor Man's cron job and replace it with a real cron job.

Furthermore, developers must be able to tag those cron jobs again so they can be discovered in the DI container and they should also be able to make a difference on whether their job should be run for PoorMan's cron jobs as well or only for real cron jobs. That way, easy tasks could still be executed as a Poor Man's cron job but extensions could also provide jobs for more complex tasks to be executed (e.g. rebuilding the search index daily), if a real cron job was set up.

Fritz has been working on an implementation and if you want to participate in the discussion, feel free to do this in Pull Request #708.

Enhancing the two-factor authentication

Dave's been back to two-factor authentication. After Contao 4.8 introduced 2FA for front end members, the integration of "Trusted Devices" and "Recovery Codes" remains to be done. A trusted device does not need to re-enter the second factor and the recovery codes allow you to log in even if the second factor is lost.

These two features complete our long road to full two-factor authentication for the next LTS!

Routing

Andy has continued his work on Contao routing and "Folder URLs" are now no longer system settings, but can be configured differently per root page and thus within the system. The "URL suffix" and the "Add language to URL" settings are still system settings. It remains to be seen whether these can also be defined per root page or whether it is not possible for technical or backward compatibility reasons.

robots.txt and favicon.ico in the root page

My humble self worked on the dynamic output of favicons and the "robots.txt". You will be able to select a different favicon per root page and the content of the "robots.txt" can be maintained in the root page settings as well. Thus, different favicons and "robots.txt" contents can be maintained for multi-domain installations. Additionally, there is an event for the "robots.txt", which allows every developer to dynamically change the content of the "robots.txt".

The core itself now uses this to automatically output all sitemaps links of all root pages, so that manual registration of all these URLs with search engines is no longer necessary in the future.

Permissions for content elements

Leo has been working on the adoption of the popular extension "ce_access" into the core, so that in Contao 4.9 certain content elements may be restricted for certain user groups or users.

Crawler/Search Indexer

My major project for Contao 4.9 will be a complete makeover of the search indexer. At the moment pages are indexed when they are visited. If the search index is rebuilt in the back end, Contao searches for a list of all "searchable pages" via the hook getSearchablePages and simply requests them, which in turn indexes these pages.

This procedure has several disadvantages:

• The search index cannot be rebuilt via command line (e.g. via cron job).
• URLs that are forgotten in the hook are not found and therefore not indexed.
• Possibly fails with large pages, because the number of links is simply too large.
• Parts of the website (speaking of ESI) may be ignored.

Some time ago I therefore started to write a crawler library called "Escargot" which crawls web pages with the help of the latest Symfony components. The goal is to search and index Contao installations via a real crawler in the front end. This should provide much better results. In addition I plan a small abstraction of the search indexer so that developers can connect arbitrary indexers to e.g. index the data in ElasticSearch or wherever they prefer.

The crawler itself is not limited to indexing search results. Further integrations such as a "broken link checker" would be conceivable.

Universal Picker

Andy has tried to do all developers a big favor by building a universal picker widget. The picker is currently mainly known from the file picker or news picker but building the correct widget and picker providers, you can have your users pick anything in the back end.

The new implementation makes it easy for developers to add new pickers and also adds support for selecting any content element and article. As an example: This makes the selection of the alias content element (the content element that can output the content of another content element) much easier.

Lazy Loading for images

Martin has added support for the new loading="lazy" attribute. At the moment this is only supported by Chrome but we will closely follow the development and adjust the implementation if necessary so that you can assume that lazy loading is only one click away for all your images in the future.

Bugfixes

In addition to the new features mentioned above, we have also fixed some more complex bugs as we do every time we meet. We had to fix a few caching related bugs and they tend to be rather challenging so fixing them can be very time-consuming.

Failure is (not always) part of the process

In the last blog post I mentioned that I would like dedicate a small section to talk about whatever we were working on but did not succeed. But fortunately, I cannot think of anything special I would need to mention here for this meeting :-)

That's it, folks! :-)

I'm very happy about the outcome of this developer meeting! Once again we have achieved quite a lot and in my opinion we have assigned the right priority to the right tasks. It's important that we can give the developers the right tools for the next LTS and I think we will succeed in doing so. But also for our users there are already a lot of improvements in sight!

Contao 4.9 LTS will be the best Contao version ever and the best time for a toast would be at the Contao Conference 2019 on October 10th and 11th!

The next developer meeting is expected to take place in February or early March 2020.

– Yanick