Read the official Contao announcements.
Recap of the second Contao Core Developers Meeting 2022
by Yanick Witschi – Current issues
Every year, the Contao Core development team meets twice for a short code sprint of three days. Those who read my review of the last meeting know that last time I was I could not be there. Unfortunately, the scenario repeated itself similarly this year. Only this time it hit Martin and therefore only six of us met in the beautiful Black Forest. What? Colmar is not in the Black Forest after all and one less core developer makes seven and not six?
Very well observed! Yes, we turned our backs on Colmar. The town is definitely worth a visit and should certainly not be missing in the itinerary when exploring Alsace! But the premises there were just overall a bit unsuitable for us. So we decided to try something new and after Switzerland and France we found something in Germany again. In the very Black Forest, which I mentioned at the beginning. Small tip for those who are already planning their next vacation: Germany has incredibly much to offer, you just have to look - here you go:
And then there was the discrepancy in the number of core developers. Those who read the blog regularly already know the answer, of course. For everyone else: We were one extra person less after Jim left the core team at his own request. At this point, I would also like to emphasize that Jim has considerably helped to shape and improve Contao over the years. Please leave him a thank you on Twitter, because without him the Contao project would not be where it is today. Thank you very much, also from me, dear Jim. We missed you and will continue to miss you. <3
Obligatory note: The developer meetings are crucial for the further development of Contao and therefore, as always, I call attention to the fact that the Contao Association funds these meetings. As Supporter you can do your part.
The last developer meeting was marked by Contao 5. Marked by hundreds of pull requests, which removed the often mentioned "BC layers" from the core one after the other. Or in other words, the last meeting consisted of one main task for all of us: A lot of grunt work. This time, the situation was a bit different, as Contao 5 is now available and running very stable. So we could use the time again to do the things that you can't really do well at home or at the office on your own: Discuss, experiment and conceptualize.
As always, a lot of things are not ready to be said. We also discuss long-term visions so that everyone is in agreement, where the path should lead. But in the end, we can't rebuild everything in a three-day meeting, but we just eat the elephant the way you eat an elephant: piece by piece. The long-term goal of a general API is still a central point of discussion. Since Contao 5 we have a new way to check permissions independently of the Contao back end and thus also in a future API, but there are still more building blocks missing. The biggest problem is and remains the lack of a central repository. Records are stored in the back end via the DC_ drivers, especially in the front end we use models and then there are also a lot of places that write directly to the database. In some places hooks are called, in others not etc. These inconsistencies complicate the implementation of a project like the central API considerably and we will therefore have to rethink the way we store data sooner or later. Here we have questioned the use of Doctrine ORM. Doctrine entities are not extensible and for a content management system like Contao, which thrives on extensibility, it may not be the right approach. However, we may have found a new approach based on CQRS. We will see where these discussions will lead us.
After each developer meeting there are usually a lot of smaller pull requests that are more or less ready for a merge. But especially the bigger nuggets can't be finished in three days. Rather, the developer meeting is a kind of kick-off event for proof of concepts that will be completed over the coming months in the free time of all core team members and finally find their way into the core.
So let's take a look at some of the POCs we worked on this time.
It has already become a running gag that Dave deals with Symfony Security at the developer meetings.
But this time was no different. With the release of version 5.1 (experimental), Symfony has switched to a completely
revised version of Symfony Security. The switch to this much easier to understand
version has taken place in Contao 5.0. So we can now benefit from the simplifications and
new features. Dave has been working on an authenticator for temporary JSON Web Tokens. These
are the foundation for using
contao:crawl for protected pages in the future. Furthermore,
Contao 5.1 will most likely also come with additional rate limiter protection, which compared to the current
solution also works for non-existing user accounts. This way, password spraying attacks.
will be prevented even better in the future.
But not only Dave has moved in the area of security. Fritz also worked on his concept for the integration of OAuth support in Contao Core and presented us a great POC. I think this could be something - exciting times!
Speaking of Fritz: He has also looked into the possibility of temporary switching the language in our
Translator. An example of use would be to send an English e-mail from the Contao
back end that is set to German. Because of our
this is unfortunately not very easy. So a tip to the developers at this point: To access language files
in "contao/languages" just use the Translator as shown in the documentation and for your
own, newer bundles it is recommended to use Symfony Translations anyway.
Virtual Filesystem (VFS)
Moritz is our Mister Twig. But he is not only responsible for Twig, but also for the virtual filesystem, which we introduced in Contao 4.13. Since my humble self has been involved in the design of this component, the two of us have been talking about its future. Moritz noticed a few inconsistencies, which he quickly fixed. In addition, we discussed in the plenum, to what extent it is worthwhile to convert the existing file manager to our new VFS. As always, the question of backwards compatibility or the "BC Layer" comes up again. Now that Contao 5.0 is stable it means, for the whole 5 series we are not allowed to introduce breaks again. So what else can you do? Exactly. Come up with a completely new file manager that would work in parallel to the current one. Of course, this is another Herculean task. In any case, Moritz has cast his net wide and looked at various existing solutions and their integration possibilities in Contao.
After Contao 5.0 was mostly a "clean up" project, we also discussed the introduction of new user features at this meeting. Of course, we always take a look at the extensions that are being installed most frequently. These are always a good indicator of what might be missing in the core of Contao. However, a popular extension alone is no reason to include the functionality in the core. After all there is a great extension for it. But there are other cases, where the extension is very old or the use case could be generalized. For example with "easy_themes". In fact this is one of the oldest of my extensions. I think I might have published it in 2009/2010 in a first version, probably because of discussions at a Contao event. The fact that the extension still exists in 2022 and that lines of code from 2010 still work in Contao 4.13 also shows how well Contao's BC layers have worked over all these years. For Contao 5 there will most likely be no more easy_themes, because I myself don't need the extension much. Moreover, the use case of "reach your target with less clicks" could be generalized and there is no reason to solve this problem only for themes. In the process we developed a concept that would allow both the administration of personal favorites and an automated third level in the back end. Leo has taken the lead here and I am looking forward to the first proposal. By the way, the concept is roughly outlined on GitHub and the discussion is open to all.
Another very popular extension is "URL Rewrite". The fact that this one is also developed by us (terminal42 gmbh) is a coincidence. With this extension you can redirect URLs to other pages or generate "410 Gone" responses which is a frequently requested feature, especially in the SEO area. In the core of Contao we would like to have support for automatic redirects or "410 Gone" pages when a user changes the page alias of a page or deletes it. But since Contao does not only consist of pages, but also news, calendar entries etc., you have to be able to maintain redirects or rewrites outside the page tree. And when I say outside the page tree, I still mean somehow related to the root page. So not quite as the extension does at the moment, but ideally a bit smarter and with better UX. This initiative is led by Andy and input is also welcome here.
One of my personal longer term goals would be to have a search function in the back end. We already have a search in the front end, which has gotten better and better over the years. Only unfortunately it can't be used for arbitrary indexes. Or in non-technical terms: we can't use the same logic at the moment, because otherwise we would have to display the front end search results in the back end and vice versa. Besides, there would be several approaches to make our search much, much better (keywords: stemming, normalization etc.). Therefore, I would like to work on a reusable search component. Martin has also expressed interest here and if someone reads this and thinks "Oh cool!" and wants to work on it or even co-finance it and thereby speed up the process (always possible with everything everywhere mentioned in this blogpost!), please feel free to get in touch with the respective people. You can find us all on Slack.
Unfortunately, since Martin wasn't on site, I wasn't able to work on the search component itself yet. However, we already know that - should we be able to improve the search further - the indexing of the data may take of the data will take more time than it does today. An even better search means usually even more analysis of the data during indexing, so that it can be searched efficiently afterwards. At the moment, this happens by default on every page load (when the cache is disabled), which puts a load on the web server and in the worst case even makes Contao's response times slower. It would be nice if indexing would happen in the background, which is also true for many other tasks. Here are a few examples:
- Sending emails
- Compressing many files
- Generate reports
- Querying slow APIs
I was looking into providing Contao with a background worker based on Symfony Messenger, which would be used automatically when a real CLI cronjob was set up for Contao (always recommended anyway, go here for documentation). This would provide the developers with a new, very powerful tool and might improve the performance of Contao under certain circumstances. Again, feel free to help on GitHub.
Apart from that, there are, as always, a lot of things I can't really tell you much about, but of course they are important and mean a lot of work in some cases. For example, Leo cleaned up dozens of tickets from our old, Contao 3 GitHub repository and moved some of them that are still relevant to the current project. Moritz added X-Sendfile support to the download element in Contao 5, and thanks to Andy, every user will be able to control the expanded or collapsed display of content elements in the back end themselves starting with Contao 5.1. And I, well, spent a few extra hours as usual writing these ~2200 words here and translating them into English.
That's almost all Folks!
I still have a little spoiler for you: I'm looking forward to seeing you in real life at a Contao conference next year!