Contao news

Every year, the Contao Core development team meets twice for a short code sprint of three days. Unfortunately, we had to miss one person for the third time in a row at this meeting as well due to Covid. This time, it was Leo who was affected, and we hope that this series will break in fall, and we can finally meet again in corpore. At least Leo's health has allowed him to attend the meeting via remote video calls.

Obligatory note: The developer meetings are crucial for the further development of Contao and therefore, as always, I call attention to the fact that the Contao Association funds these meetings. As Supporter you can do your part.

Future vision of the Contao back end

Contao gets better and tidier with every release. Those who have been with us for a while know that we are still getting rid of legacy issues version by version and replacing them with new, better alternatives. Contao 5 gave us the opportunity to get rid of a lot of this legacy stuff. And we continue to follow this plan. We continue to build better ways, mark the old ones as deprecated and will eventually say goodbye to them with Contao 6. Whenever that will be.

But slowly but surely we are approaching the heart of Contao: the back end.

The Contao back end has received several facelifts over the years and has been adapted from time to time. From a mobile variant to new icons to - just recently - a new dark mode and improved display of content elements.

But bigger, more fundamental changes failed to materialize. Don't misunderstand: We love the consistent user experience of Contao and we are convinced that it was and still is one of the key success factors for our CMS. But there are areas where we would like to see improvements - such as a third column to make use of the available space on large screens.

Why this has never been done before is explained relatively easily: it would be incredibly costly and complex. This is because of the way Contao hardwires the presentation and storage of data within the DC_* drivers and the associated DCA. For example, we cannot reuse the list view but have the data loaded from a different source. We cannot simply insert a new column without reworking all existing views, from the list view to the tree view. If you want to have the default representation of data in the back end, you also have to adhere to the way the data is stored, as it is in the core. Again: We love the simplicity with which you can quickly define your own fields and have them displayed in the back end, and this - we are also convinced of - is also a big success factor. Of course, we will try to keep the positive aspects of the current solution. But we need to slowly but surely approach this core and think about how we can reuse elements like the filter panel, edit masks, tree displays, etc. more flexibly and as independent components.

Besides the more flexible reuse of presentation elements, we also want to make data storage more flexible and say goodbye to the various save_callbacks, onsubmit_callbacks and our own models in the longer term. We lack a central "storage engine" through which all content in Contao runs. Developers will know very well what I am talking about.

The good news is that we were actually in relative agreement about the future of the back end, and for the first time in all the developer meetings, we were able to establish a shared vision or idea of how the back end should work in the future, at least technically, so that it can provide us with the flexibility we need for future customizations.

Most likely we will be working on a new content repository. Doctrine ORM is simply not the right solution for an extensible Content Management System. We will probably rely on concepts like CQRS and (possibly optional) Event Sourcing to additionally provide Contao with better versioning and auditing capabilities.

For more flexibility of the graphical user interface, we will probably rely on the Hotwire stack, as this is most likely to meet our requirements. We want to give our users the feeling of a "live app" without forcing developers to write thousands of lines of JavaScript.

All decisions are of course not set in stone and the main goal must and will be that the Developer Experience (DX) does not suffer, but ideally, even improves. We want to provide better and easier to understand interfaces and move more complexity into the background.

Let me put it this way: It is, of course, a mammoth project. But we also have to face reality: At some point, we will no longer be able to avoid a rebuild. And it simply makes much more sense to invest our time in this project rather than to spend weeks and months trying to fit new ideas into the old structure only to discover at some point that we can't avoid a rebuild after all.

We are also talking about a vision, as mentioned. Visions are long-term by definition. We're talking about a 5.7 LTS, or maybe 6.3 LTS by then. Furthermore, it is inconceivable to build these concepts into the existing DC_* drivers in a backwards compatible way. It is therefore much more likely that we will introduce this vision as a kind of new, beautiful parallel world to the current environment. The current plan is to do this possibly by means of a new blog-bundle, which could exist in parallel to the current news-bundle. So either one or the other bundle could be used, depending on which functions and extensions are needed. Also there are already drafts for a new file manager, which could likewise exist parallel to the current file manager. Thanks to all those new "realtime" Turbo streams, it just feels like a modern application should feel like 🔥

Of course, we will continue to build new features and do our best to bring Contao forward as usual. So let's get right on with it.

Nested content elements

Martin, Leo and Moritz have been working on a new design of nested content elements. We already know the concept of "wrappers", which we use for example in the slider or accordion elements. However, the opening and closing wrappers do not know each other and it is easily possible to forget or delete the closing wrapper, for example. The new concept provides that content elements themselves can define that they expect child content elements, which means that they themselves contain the opening and closing part and corresponding operating errors would no longer be possible.
Furthermore, it should be possible to define which "slots" such a content element makes available. Child elements could then only be assigned to the designated slots. It should be possible to apply this pattern recursively and support nesting content elements to any desired depth.

Saving templates - combinations of nested elements - for faster reuse was also a topic.

In the long term, the current concept of "articles" could then also be dropped, as these are ultimately nothing more than a content element that is output in a specific slot and can in turn consist of nested child elements. It remains to be seen whether this will continue to be called "article" in the back end and only use the same technical logic or whether we will find a different designation.

JSON data type in DCAs

I tried to teach the DCA and the DC_* drivers to store fields in a central JSON data field. The big advantage would be that you would no longer have to create a matching column in the database table for each field. But I had to give up this project at some point, because - and here we are back to the already described problem - too many functions in the drivers simply rely on the fact that the data structure exists in a certain schema and that there is one database field per field. From the toggle operation, filter and search attributes, various callbacks, the diff view to subpalettes and submitOnChange and much more.

We finally agreed that the introduction of such a feature would possibly cause more problems than it would solve.

File system permissions

Since Contao 4.13 we have a virtual file system, which is still in "experimental" state, but grows and gets better with each new version. One of the last (I dare to say it) hurdles for a new file management is a usable permission system. And I don't even refer to the way permissions are assigned today, but to their technical implementation. Here we were able to find a solution that should also give us the flexibility to expand our current authorization system in the future. Today we are only able to define allowed file mounts and operations. But we can't set permissions differently per directory. So something like giving delete permissions in directory A but not in directory B is not possible today and would be very difficult to implement. The new solution will give us the necessary flexibility. Moritz has been working on the implementation of this solution.

Crawler for protected pages

David has - like at every developer meeting - once again dealt with security. There seems to be always something to do in this area. Maybe it's just the fact that a good Content Management System has a lot to do with security, authentication and authorization. In any case, he was busy with a token-based authentication option that should eventually allow our contao:crawl command to crawl and thus also index protected pages.

Content Security Policy

The Content Security Policy (CSP) is an additional layer of security that helps detect and mitigate certain types of attacks, such as cross-site scripting (XSS) and data injection attacks. These attacks are used for everything from stealing data to defacing websites to spreading malware. If you've already done a bit of research on this, you may have noticed that integrating it into a CMS like Contao is not that easy. In our constant effort to make Contao as secure as possible, we have discussed possible solutions to this problem and found an approach. In the future, it will be possible to activate the automated creation of a Content Security Policy in the root page settings. In principle, it should work analogously to the solution for Canonical links introduced in 4.13: As a website administrator, the basic policy can be defined by oneself. Contao then provides a framework that allows content elements and their templates, and thus all extensions, to extend this policy themselves with dynamically included JavaScript files, styles, iFrames, etc. This includes automatic nonce management. Ideally, this would be another feature that would require Contao users to only activate a single checkbox. The rest is done by Contao 🚀

Bugfixes and miscellaneous

The developer meetings are also always an opportunity to discuss more complex bugs and find appropriate bug fixes. Again, there were too many to mention individually, but you may have noticed that the user session can get quite large. We found the cause and also improved the logic. In addition you can now use contao:crawl with a job ID and a long-standing problem related to permissions and the picker has been fixed.

Besides bug fixes, we cleaned up the source a bit in general, no longer use the Combiner in the back end and Andy spent hours rebuilding old back end permissions to our new security voter system. One of the many, many rather boring but time-consuming and no less important cleanups I can't really talk much about, but which are of course just as valuable for Contao.

Organizational matters

Besides all the technical challenges, we also talked about our release planning. The current release schedule provides for the release candidates of new minor versions to be published in mid-January and mid-July. Conversely, this means that all new functions must be completed and reviewed by the core team in the weeks beforehand. This extremely time-consuming work therefore falls exactly in the vacation and Christmas season and is thus somewhat unsuitable. However, the dates are also deliberately placed so that our LTS versions coincide with Symfony's LTS periods. We have gone through a few alternatives, but have not yet been able to agree on a better solution. As soon as there is news about this, we would of course announce it via this channel.

That's all Folks! Maybe one or the other topic mentioned here will be part of a talk at the Contao Conference from 19th to 20th of October 2023 in Kiel - see you there?!

– Yanick