infiniterecursion.ca

Stuff and Rants by Stephen DeGrace

• Site Map
• Home
• Blogs
  • Latest Article
  • Recent Articles
  • Topics
• Galleries
• Links
• Guestbook
• Contact
• About

A Plugin System for Django?

Posted in Django on November 6th, 2010 by Stephen DeGrace
Topics: Popular Django CMS

If you're designing a corporate intranet, news website, or dating service, platforms like Django and Rails have PHP beat. They allow for much cleaner design by nature, and hence they scale better, at least conceptually. Facebook, written in PHP, is the big exception, but we as constantly see Facebook struggling to keep up with its volume of traffic, and as we see various security holes come to light in the news, it's apparent that this may be a misfortune for Facebook rather than an advantage for PHP. Of course, Java has a commanding domination of the corporate market, for reasons which are far from obvious. But PHP has a powerful domination in the world of prepackaged web applications, like CMS's such as Joomla, blogs such as Wordpress and forums such as phpBB. I don't think this is entirely due to the fact that PHP is simply so widely supported - I think PHP actually has some significant advantages which make it better for these kinds of applications, and I would like to explore why Django specifically is inherently deficient in this area and how those deficiencies might be circumvented.

Fundamentally, what makes PHP such a hot mess is also its greatest strength. A PHP application is basically a collection of files with a .php extension, which triggers a properly configured web server to parse the files using the PHP engine. The files consist of raw HTML, which is passed straight through, and PHP code, in <?php ?> processing instruction blocks, which can do basically anything, from alter entries in a database to outputting text. Logic and presentation are just mashed together. It's an elegant use of the concept of the processing instruction from SGML, and really easy and convenient for small effects. You just edit a file on the filesystem which the web server then directly parses in response to requests. Nothing could be easier.

Of course, this leads to monstrous spaghetti code. While you can set up fairly clean architectures with PHP, it has to be by dint of considerable effort, because the platform itself actively encourages an easy-breezy, ad hoc style of programming. It takes enormous discipline not to fall into this kind of trap in PHP.

Django (and Rails) work differently, and by and large actively encourage you to write code that is very structured and organized along the MVC (what Django calls "MVT") programming pattern. The framework handles incoming requests, possibly with the aid of a web server, interprets the request in any way it sees fit (the URL may look like a filesystem path, but it isn't - the framework parses it in some fashion to decide what view (controller) to hand the request to, and the view examines this and other aspects of the request to decide what kind of response to send back), provides some kind of templating system to strongly encourage separation of presentation logic from business logic, and some kind of object relational mapper (ORM) module to encourage you to "model" your data and keep your data a separate concern from your business logic and presentation. The framework is therefore not an ephemeral interpretation of files in response to a request but a constantly running application which provides numerous hooks to the programmer to define its behavior.

(Let's not even talk about Python alternatives Zope and Plone - these have their uses but are extremely heavy-weight and lend themselves to a certain kind of architecture.)

The Django approach is conceptually much more pleasing and far cleaner and easier to scale from the point of view of the human ability to conceive of the design of the system as a whole, but the fact that a PHP application consists of a collection of "dead" files gives it an important advantage. That is, it is trivially easy to install PHP applications, because installation consists of uploading, plus some additional activities which can be activated by requesting a page. It is also trivially easy to extend PHP applications - all you do is upload some more files. Sometimes there needs to be more to it than that, but applications like Joomla have achieved an impressive ability to be fully managed from within their administrative interfaces, without editing files or restarting web servers.

In contrast, installing a Django application, even on an interface like Webfaction which makes it easy, is a much less trivial operation. At minimum, once the empty installation has been created, you need to upload the files and manually edit the settings.py file to reflect the new environment, then from the command line run python manage.py syncdb to create your database tables. When you want to add a "plug in," be it a whole new type of content or something "widgety" as something cute to put in a sidebar, like polls, or a package with elements of both, you need to upload the files, edit settings.py at minimum to add the new application, add/edit any other settings pertinent to the app, run syncdb, and restart the web server. You also at minimum usually need to manually edit some templates to display the new functionality.

While not prohibitive in any way to a competent Python programmer, all of the above requires you to understand what the Django site does and what the application does on a not insignificant level. This is prohibitive to the average end user who would know enough to be able to install something like Wordpress but would not know the first thing about programming it. Wordpress is accessible to such a person - no Django application is. Your basic Django installation is the province of professionals and unusually competent hobbyists only. And I believe the same certainly holds true of Rails. This limits the market for Python web applications and in turn limits the scope to which it is supported by common hosting plans.

However, I don't think the task of creating a Joomla-like CMS in Django is necessarily hopeless, only that it would require creating some very specialized applications which to some extent alter the basic way that Django works. I believe that the qualities of a plausible competitor for Joomla are approximately as follows:

• Must have one-off pages, some kind of blogging and hopefully photo galleries as core applications.
• Must have an extremely flexible admin interface with hooks for applications to add their administrative functions
• Should have an application to create navigation structures that can be output as ULs and to which other applications can export items to link.
• Should have a front page app, because the home page of a site is usually a one-off, and potentially some way for applications to export content for it.
• Needs to be possible to download and install themes, hopefully without editing files and without too many clicks.
• Needs to have a mechanism to download and install applications, again without editing files and without too many clicks.
• Should be possible for applications to export "widgets," which can be selected for certain predefined locations on the page.

To achieve this, first of all I think the Django admin site needs to be ditched. It's enormously useful to a programmer, or even to end users for certain kinds of applications, but for this purpose it is simply necessary to have more flexibility than an application which is essentially in the business of exposing models. A whole new admin site is needed that operates on a similar concept, that applications have an admin.py file that can be parsed to find the administrative interface exposed by an application (better yet, an admin folder with a suite of admin files used by the application), but it needs to be something at once more rudimentary and more powerful, giving applications a huge degree of flexibility to define their admin interface and also a much larger responsibility to define their interface, which much less done for you.

Stuff like blogs, galleries, and pages are easily handled by Django with little modification from the usual pattern except that they need to expose interfaces to the admin, homepage, and navigation system, and possibly also "widget" content.

Every application should be able to define a links.py which contains instructions to find all the links that the application exports which should be available to the navigation manager. The navigation manager should graphically allow the construction of trees from these exposed links which can be output at predefined locations as navigation widgets.

There should be a homepage manager which can select content exposed by applications for front page display to design the front page of the site.

The heart of the project is a plugin application. This would allow uploading of entire apps to a certain directory, and executing a shell script to restart the web server. Code would need to be added to settings.py allowing the plugin app's app loader to dynamically extend the list of applications to load on start up with the ones in the upload directory (from which it could also delete apps and similarly restart the server). It would define a "Widget" model which apps could create instances of that defines how to create chunks of content on the page and links to another model a Location model, which allows locations on the page to be defined. The locations are entered on the template using a special template tag, which checks the database for widgets at that location and then builds and displays them. The user would place and order widgets at each location graphically through the admin.

Dynamic apps should move their media to the public static directory upon first use and should know where their media is and be able to remove it upon uninstall. An App model in the plugins app would be responsible for registering this knowledge and allowing apps to be uninstalled.

The site should have a themes app which implements a custom template loader, allowing templates to be loaded from the same location where the web server serves static media in the recommended configuration (the packages would have .htaccess files to prevent casual access to the templates). It would have a Template model for which every installed template would have an instance, with a mechanism to ensure only one can be active. This template loader would be used for the base.html and other non-app content. It would provide template tags which would allow the path to media for the particular theme to automatically be selected from the database.

It would also be preferable if many if not most of the settings moved to the database through a settings app, which may require the app to update the settings after loading.

None of this solves the easy-install problem, although a package for web hosting providers to allow easy one-click installation of Django plus the core apps would be easy. None of the above I think is insurmountable in Django as it stands, but particularly the plugin and theme apps are challenging and essential glue to make the rest of it work and to make altering and extending the site something that can be "pre-packaged."

Comments:

There are 4 comments on this item.

Post a Comment

Copyright © 2009 Stephen DeGrace | Site Map | RSS