Contao news

Read the official Contao announcements.

Recap of the first Contao Core Developers Meeting 2019

by

Each year, the Contao Core development team meets twice for a short code sprint of three days. As there are currently four developers from Switzerland, the meeting takes place a bit outside of Zurich. The meetings are supported by the Contao Association and are very important to us. Consider becoming a member and support us too!

I have decided to gather the most important information after every meeting and share it with the entire community. This time, the following topics have kept us on our toes:

Ecosystem

As always when we meet, we are not only talking about Contao as the software in itself, but also about its ecosystem.

English

An important decision we have made is that we want to devote ourselves more to the English language again. Popular blog series such as this summary of developer meetings as well as the two-month Contao review will thus be published in English as well from now on.

A new repository «contao/conflicts»

A new repository called «contao/conflicts» finds its way into the Contao universe. For the users of the Contao Manager, it will automatically be loaded in the background as of the next release. All users who would like to work without the Manager on command line can install the package as documented on GitHub.

Background: In contrast to previous versions of Contao, the page administrator no longer simply manages Contao but a Symfony application. Contao itself is a dependency of this application like any other. This has the immense advantage that we can reuse libraries but at the same time we suffer from the fact that we are not in control of all the versions of our all our dependencies that are being released. Recently we had some issues with e.g. doctrine/dbal because their releases caused some issues and could destroy a regular Contao bug fix update. So when you ran an update, Contao itself was actually fine but as you are also updating doctrine/dbal (maybe without even knowing it because it's a dependency) that caused issues. We then had to ask the users to enter the corresponding "conflicts" in their composer.json, so that these packages would not be installed. With the introduction of a central «contao/conflicts» package, we now have the ability to intervene in the dependency resolution process as necessary by managing these conflicts ourselves, so ideally the vast majority of users do not even notice such issues anymore.

Documentation

Ever since I can remember, the documentation was an everlasting topic in the Contao environment. Everybody wants to read it but nobody wants to write it. Something is different this year though, as during the last General Assembly of the Contao Association the members voted to financially support the development of a documentation. That's why we invited Fritz and Bjarke over to join us at our meeting, who, together with Jim and my humble self, were intensively working on creating a documentation framework based on Hugo and starting to write content. We want to say thank you to Firtz and Bjarke for taking the time to visit us in Zurich!

The current plan includes a user manual in English and German language. In addition to the user manual, a developer documentation that is written exclusively in English is on schedule.

First results looked promising! The developer documentation is already at a level we could've only dreamt of before our meeting.

The goal is to open source the documentation at the Contao Camp in Munich in May so we can start to accept contributions from the community. At the moment it's not ready for this yet as important chapters such as how to write documentation and contribute are still missing.

At this point I would like to add two personal comments:

  1. The said funds are relatively low and assuming that documentation will all of a sudden write itself without any help would be utopian, to say the least. The funds maybe cover to ensure the infrastructure is being managed and our docs follow a golden thread but the majority of content still has to be provided by the community. We are a community-driven project and we will remain one.
  2. For 12 years the documentation has been an omnipresent topic at conferences, camps etc. and I think here's finally a great opportunity to put an end to these discussions. We finally have a team of motivated people who are guaranteed to be on the lookout for fellow campaigners. So, if you're being approached by someone and asked for support, you know what to do, right? The core team is also aware of the importance of documentation for the project and will play their part in the future.

Autoupdates

Based on inputs from the community, we discussed the feasibility of automatic updates again. However, the topic was kicked into the long grass pretty quickly as no team member considers automatic updates to turn out to be advantageous. The potential pitfalls and our safety concerns are simply too big. Thanks to Composer (and where applicable the Contao Manager) the effort for updates has already been greatly reduced.

Two-factor authentication for the frontend

Dave has dealt with two-factor authentication for the front end throughout the entire meeting. The difficulty here seems to be the great flexibility of Contao which allows us to place login modules on arbitrary pages. A good mix between modules and possibly page types has to be evaluated and thus poses a nice challenge. Moreover, preparatory measures such as pull requests to third-party bundles had to be contributed first so that an integration in Contao can even be considered. He showcased a first version just before the end of the meeting which had some rough edges but overall looked very promising and was working just fine!

In other words, sooner or later 2FA will become a fact for the front end in Contao!

Deferred generation of thumbnails

The thumbnails in Contao have always been generated during the first page request. This has the serious disadvantage that if the image cache is not built yet, the page load time may be very bad. Nothing a visitor or any search engine likes to see. In worst case, the request even runs into a timeout, so that the user receives the famous «white page» as a response. This became especially famous in the file manager in the back end when the user opens a directory with too many images in it. The introduction of responsive images support also worsened performance especially in the front end.

Over the past 1.5 years, we've been working on a concept of how we can potentially solve this issue and Martin has now taken on this task and it actually seems to work! During the request, the image is not generated but only its hash is calculated which is then stored in a cache together with the respective image resize settings. If the image does not exist in assets/images yet, the server will automatically redirect the request to Contao where we can extract the hash, fetch the respective resize settings from the cache and generate the image only now. Resizing the images then doesn't happen in the initial page request anymore but instead during the request of the image. Sounds like a pretty straight-forward and simple task but there's a lot more to consider such as locking mechanisms as it's possible that two clients are requesting the same image at the same time and much more.

But overall the implementation has already advanced to a point where I feel comfortable to say that you can almost certainly look forward to a maximum performance boost in Contao 4.8!

Completely new concept for entry points and the front end preview

Our current Contao Managed Edition provides two entry points: the app.php and the app_dev.php. The production environment then runs on e.g. https://www.domain.com/page.html and to enter the debug mode, you would call https://www.domain.com/app_dev.php/page.html. This situation is unsatisfactory for the following reasons:

  • To enter the debug mode, you have to first create a password via command line or via Contao Manager for security reasons and then manually type app_dev.php into the address bar. This is a bit too complex.

  • Due to reverse proxy cache we have the issue that https://www.domain.com/page.html is possibly already cached. But during front end preview and debug mode we should not get the cached response as otherwise we'd never get the desired content. Now imagine that you do not use the provided PHP reverse proxy but an external one, like e.g. Varnish: That proxy has no way to know if a user is currently logged in because it is not running within the same application. The current solution to work around this issue is technically extremely complicated in that before each actual request, an additional request is dispatched. The application is asked if a user is currently logged in and if so the cache is circumvented (for developers: we do this using the header-replay-bundle). In theory, any reverse proxy could dispatch such an additional request so we do provide a general solution. But it's just too complicated, which means we have to get rid of the header-replay-bundle which in turn means we need another solution for our entry points and we cannot support the mobile page layout in its current form anymore.

Andy and I have had a concept in mind for a long time and have now started to make it a reality. The idea is that once implemented we'll have something like this:

  • We will now have an index.php and a preview.php. Any requests to preview.php will never be cached. In our example, we would thus access the front end preview via https://www.domain.com/preview.php/page.html. Of course, all this happens automatically when clicking on the front end preview button in the back end.

  • The index.php checks if the request contains a Cookie or Authorization header and in this case bypasses the reverse proxy cache completely. For external proxies like Varnish this means a significant simplification of the configuration. No additional requests whatsoever. For Contao, this means that we get rid of the header-replay-bundle and thus a get rid of big complexity and hurdle when debugging.

  • The debug mode can be enabled by a simple click in the back end. In the event that Contao is not running properly it must still be activatable in the Contao Manager. The implementation involves the use of a JSON web token.

  • The mobile page layout has already been moved to a separate bundle (likely going to be called contao/mobile-page-layout-bundle) which removes the functionality from the core in Contao 4.8. It can be re-enabled by installing the bundle for those who still depend on this feature. However, once this bundle is installed and a mobile layout is assigned to a page, any page cache settings will be ignored and the cache will be deactivated implicitly.

Routing settings in the root page

In Contao 4.7 Andy has ported our own routing logic to the Symfony CMF router. That was a huge pull request but we have not gained any real benefits yet, except for perhaps performance gains (we have no concrete benchmarks though) and automatic redirection to https:// in case the corresponding checkbox in the page settings is checked. Time to change that! In the future there will probably be no more prepend_locale parameter configuration because the following settings will likely find their way into the root page settings and can therefore be managed per domain instead of the whole setup only:

  • Use folder URLs
  • Add language code to the URL
  • URL suffix
  • Do not redirect empty URLs

Here too, we were able to get a taste of Andy's first results and it seemed to work just fine!

Failure is part of the process

It is also important to mention that we don't just meet, sit down and build something great. Sometimes we fail and I wanted to introduce a new section in this blog series to cover the failures as well:

  • Leo, who's normally very occupied with reviewing and merging the pull requests of the other devs, did manage to some time to work on a feature: He once again tried to allow us to write our DCAs in YAML. Unfortunately that seems like a never-ending story especially because of the references to superglobals. So unfortunately, there is no solution in sight right now.

  • Martin has tried to bring drag n' drop functionality to our page tree. Unfortunately, that also seems to be very hard to achieve especially because of permissions. Pages cannot simply be moved within any other page. Permissions will only be checked after submission and as if that was not hard enough there are all the button_callbacks in which developers can do arbitrary things etc.

End of the novel :-)

This time I am extremely euphoric! The list of features is super long and really great! As of today, we don't know which features will make it into Contao 4.8 and if they will make it as described in this blog post but I would really like to have Contao 4.8 today already! Well, they say it's healthy to have something to look forward to, don't they? :-)

The next developer meeting is expected to take place in August 2019.

But before that, we'll see each other at the Contao Camp on 18th and 19th May in Munich!

– Yanick

Back to the news overview.

Comments

Add a comment

Please calculate 6 plus 7.