infiniterecursion.ca

Stuff and Rants by Stephen DeGrace

• Site Map
• Home
• Blogs
  • Latest Article
  • Recent Articles
  • Topics
• Galleries
• Links
• Guestbook
• Contact
• About

Django and jQuery Ajax Patterns

Posted in Django on January 30th, 2011 by Stephen DeGrace
Topics: jQuery, AJAX

In a previous article, I talked about the increasing role of Ajax libraries in the development of websites in Django's main target genre (news sites) in light of Django's policy not to "bless" any given popular Javascript library. This is in contrast with Rails's tight integration with the Prototype Javascript library, Rails being the framework Django is most often compared to. My experience with Javascript frameworks is with jQuery, apparently the rising star and most prominent example at the moment. I have found that Django and jQuery are actually able to interoperate quite well, and in this article I will summarize some basic development patterns in using Django together with jQuery.

I love jQuery. jQuery is conceptually very clean, with a feature set that is easy to hold in mind. Unlike many libraries, jQuery is content to let Javascript be Javascript, not attempting to use the language's surprisingly powerful features to twist it into behaving like some different language. jQuery leaves built-in objects alone and has the tiniest possible footprint in the Javascript global namespace, making it play nice with other libraries. Yet it is feature-rich and has an incredibly powerful syntax.

I used to hate Javascript, like any right-minded person, but I have to say, since I've been introduced to jQuery I have really changed my tune. I think it's not so much because of jQuery's special features as the way it abstracts away the cross-browser headaches which I think were the real cause of so much Javascript hate. When you get past that, Javascript actually has some surprisingly nice features. The first time I ever programmed in Python after doing some heavy Javascript coding and actually missed Javascript for some feature was a disorienting and eye-opening experience. Big example: the ease with which anonymous functions can be created with arbitrary complexity and passed around as first-class objects makes Python's lambda forms look truly weak and pathetic. And I say this even though I'm a pretty committed fan of Python.

Anyway, let's get down to it. I've developed two applications with tight Django-AJAX integration, the shift calendar that I wrote for my staff at work, and the administrative section of the Linkbacks application for this site. Executive summary:

• In jQuery 1.4, the serializer changed the encoding format for JSON data to suit current features of Rails and PHP, which messes up Python. There is an easy workaround, but it is not obvious.
• Forms can be serialized and submitted via AJAX very cleanly with jQuery in a way that can be fed directly into Django Form objects without modification. This is the good news.
• The bad news is that without built-in integration, there is no really simple way to handle things like validation errors. You have to devise your own solution. I'll show a couple of proposals.

OK, let's start on the Python side. You're going to want to be including a fair amount of Javascript files in your site, and you might as well leverage the power of Django templates so that you can make dynamic alterations directly to the files if you want. The single biggest place you will want this is for the URL's for your AJAX views, which you will probably want to generate with {% url %} tags as per the Django accepted best practice. So here's how I've been doing that: I add the following URL to my main urls.py file:

# Javascript    
url(r'^js/([\w\.\-]+)/([\w\.\-]+)/$', 'views.javascript', name='javascript'),

A views.py in the main project directory contains the following view:

from django.shortcuts import render_to_response
from django.template import RequestContext

def javascript(request, app, js):
    """
    Used to request javascript templates.
    """
    return render_to_response("%s/%s.js" % (app, js),
        context_instance=RequestContext(request))

Javascript templates are then rendered with the following code within the html Django template:

<script type="text/javascript" src="{% url javascript 'schedule' 'calendar' %}"></script>

This example is taken from my shift calendar application and results in the application's calendar.js template being linked and rendered when requested from the browser. This very simple paradigm allows you to make use of Django template language features in your Javascript files, although in this case does not push a lot of information into the template about the original request.

Now let's circle around to the jQuery side. I include the following elements in each of my pages which is using jQuery somewhere in the templates. This is the basic pattern I recommend. First of all, my base.html template contains the following code in the <head></head> - note that only the first line is really necessary:

<script type="text/javascript" src="{{ MEDIA_URL }}js/jquery.js"></script>
{% block js %}{% endblock %}{# Override for view-specific js libraries to be loaded #}
<script type="text/javascript">
//<![CDATA[
{# Override this block to add functions to bind on document ready. #}
$(document).ready(function() {
    {% block docreadyjs %}{% endblock %}
});
{# Override this block to add inline js #}
{% block inlinejs %}{% endblock %}
//]]>
</script>

The first line loads the jQuery library. I allow jQuery to make $ equal to the jQuery object. For those who don't want this for practical or aesthetic reasons, consult the jQuery documentation for how to get around this. I have included a $(document).ready() block with a hook for child templates to insert snippets of jQuery, as well as a couple of other hooks for children to add Javascript.

Then in the main child template for the app, I have the following block that includes my Javascript file:

{% block js %}
    <script type="text/javascript" src="{{ MEDIA_URL }}js/jquery-ui.js"></script>
    <script type="text/javascript" src="{{ MEDIA_URL }}js/jquery.confirm_dialog.js"></script>
    <script type="text/javascript" src="{{ MEDIA_URL }}js/jquery.message_box.js"></script>
    <script type="text/javascript" src="{{ MEDIA_URL }}js/jquery.blockUI.js"></script>
    <script type="text/javascript" src="{% url javascript 'schedule' 'calendar' %}"></script>
{% endblock %}

This example comes from my shift calendar app. The last script tag causes the calendar.js template to be requested and rendered to the client. The first tag is the jQuery UI library, which I used in this app. The other two are jQuery plugins I wrote that make specific functionality available.

Finally, this is the basic pattern for the calendar.js template, the one that will contain all of the subsequent Javascript examples:

$(document).ready(function () {
    /* Setup */
    $.blockUI.defaults.applyPlatformOpacityRules = false;
    $.ajaxSetup({traditional: true});
    
    /* All jQuery and AJAX go below this comment */
    
    /* All jQuery and AJAX go above this comment */
});

The $.blockUI line pertains to a plugin I happened to use to show a spinner and grey out the user interface while a long-running AJAX request is in process, and is completely optional. However, the $.ajaxSetup({traditional: true}); line is highly recommended. Without this, jQuery will encode certain JSON data in a format that Django will misinterpret.

I will demonstrate the rest of the basic pattern with a simplified example of a form which corresponds to an instance of a Django model, which is submitted, validated, saved, and confirmation or errors returned all via AJAX. You can take this basic idea and do variations, like for example embedding the form in a jQuery UI dialog. The toy app is called "articles." Our simple model will be an Article, which has a body, which may be any length or empty, and a title which may not be empty and must not be more than 140 characters. The model.py looks like this:

from django.db import models
        
class Article(models.Model):
    title = models.CharField("Title", max_length=140, blank=False)
    body = models.TextField("Body", blank=True, default="")

Corresponding to this will be a Form class in a forms.py file which we'll use to work with the AJAX data:

from django.forms import ModelForm
from articles.models import Article
        
class ArticleForm(ModelForm):
    class Meta:
        model = Article

This ModelForm has the required behavior and is simple, but other forms will mesh with jQuery in basically the same way. Here's the template snippet that outputs the form on the client side:

<form id="article_form">
    <table>
        {{ form.as_table }}
    </table>
    <div id="feedback">Article not saved.</div>
    <div><input type="button" name="save_button" id="save_button" value="Save" /></div>
    <input type="hidden" name="article_id" id="article_id" value="{{ article.id }}" />
</form>

The article will be submitted, validated, saved if it checks out, and feedback returned completely through a single Ajax method. For simplicity, we're not worrying about how the article was created. The user has loaded an existing article to edit.

Back to the Python side... we need a URL pattern and a view in Django to handle the Ajax interaction with the client. There is nothing special about this. Just like a "normal" view, it receives input from a request object plus possibly snippets from the URL pattern and returns a HttpResponse object. While it is true that there is no convenient shortcuts like render_to_response that are ideal for typical Ajax communications, the HttpResponse object is extremely flexible and easy-to-use and can return any kind of information you might need, including HTML or JSON. And Django does have built-in utilities for handling JSON. The simplejson package comes bundled with Python from version 2.6 on, but the django.utils package contains simplejson for your convenience in case you are using Python 2.5.

There are two ways I could identify the Article object to work on to the view... either through a hidden form field whose value gets submitted to the view via the HttpRequest object, or directly via function parameters that get populated from a segment of the URL. However, I have chosen to make my Javascript reside in a separate file rather than inline in my template, and I haven't gone to any trouble to pass information like the object ID in to the Javascript template via its URL, so my decision is made for me and I will handle everything through the request. So here is the URL pattern from the urls.py file for the articles app:

### AJAX URLS ###
url(r'^update/$', 'update_article', name='update_article'),

I keep my Ajax URL's in a separate section of the urlpatterns just for organizational purposes.

Now, there are two basic ways we could handle the form validation feedback from the server on the client side. We can have the server render a small template with just the form and send it back to us, and use jQuery to replace the form on the page with the form sent back to us. Or, we can have the server send us more fine-grained information about what happened in the form of JSON and then individually manipulate parts of the form with jQuery to display the feedback.

The second approach is more elegant, but Django really doesn't help us much here, so this approach is also a lot more work. The first approach has kind of a sledge-hammer-y feel to it. Especially if the form is quite large, it might not be much better than just submitting the form and rendering the response as a new page, the old fashioned way. But for things like dialog boxes containing short forms it can be a good solution and I used it in my shift calendar app. Anyway, I have tried both ways, and will give an example of both.

Method 1: Loading HTML

Let's start on the client side. The client knows that if it serializes the form and sends it to the server, it should get back a new rendering of the form. To facilitate this, the inner part of the template above (inside the form tags) is actually rendered as a small included template so that it can be rendered again for the Ajax response. We add the following to the Javascript template in the document ready block:

$('#save_button').click(function () {
    $('#feedback').text("Saving...");
    $('#article_form').load('{% url update_article %}', $('#article_form').serializeArray());
});

Note that the URL to submit to is specified by a Django template tag. We use our feedback element to let the user know that the system is trying to save the document, and we rely on Django to indicate the success or failure of the action in rendering the template snippet back to the user. We haven't done anything for the contingency of the Ajax request failing entirely, although we could easily. Because we are supplying the data parameter as an array, load will automatically use POST for us. The load method takes care of encoding as JSON. The following modifications are made to the template. The form looks like this in the main template:

<form id="article_form">
{% include 'articles/article_form.html' %}
</form>

And article_form.html looks like this:

<table>
    {{ form.as_table }}
</table>
<div id="feedback">{% if success %}Article saved {% now "r" %}{% else %}Article not saved.{% endif %}</div>
<div><input type="button" name="save_button" id="save_button" value="Save" /></div>
<input type="hidden" name="article_id" id="article_id"
    value="{{ article.id }}" />

We can now indicate whether or not the article was successfully saved and when by passing a boolean success parameter to the template, and we can optionally pass an article object if we're working on a particular article as opposed to creating a new one.

Lastly we need to write our view function:

from django.shortcuts import render_to_response, get_object_or_404
from django.http import Http404
from django.contrib.auth.decorators import login_required
from django.template import RequestContext
from articles.models import Article
from articles.forms import ArticleForm

@login_required
def update_article(request):
    """
    Must be called by POST.
    
    Should be called via AJAX and returns HTML.
    """
    # Get the article from the article_id hidden field
    try:
        article_id = int(request.POST['article_id'])
    except (KeyError):
        raise Http404
    article = get_object_or_404(Article, id=article_id)
    success = False
    form = ArticleForm(request.POST, instance=article)
    if form.is_valid():
        form.save()
        success = True
    return render_to_response('schedule/article_form.html',
        {'success': success, 'article':article, 'form':form},
        context_instance = RequestContext(request))

There is nothing all that remarkable here, aside from the refusal to treat with GET, which is not a necessary condition, either - we're just returning HTML. If you didn't explicitly state in the docstring that the function was to be called by Ajax you would never know the difference from a "normal" Django view. Note that we validate our input just as we would with any normal user input, and I have used the built-in auth package to control access.

That's basically it. If the form validates, the user may see the screen jump as the form is redrawn, and the message above the button will indicate save was successful and the time. If the form does not validate, the form object in the template will output the error lists so that the user sees the errors which occurred and the Article Not Saved message will appear above the button.

Method 2: Feedback with JSON

For this method, the modified templates described in Method 1 are not needed. We use the simpler template given earlier in the article. We are going to return a JSON object with information about the results of the request to save and modify individual elements of the page accordingly. We will not modify actual content like the title or body text.

The client will expect either a JSON serialized Article model as per the Django serializer, or else the error list from the failed form validation and will determine which has been sent by testing for the existence of data[0]. The Django serializer returns an array of objects, whereas the error list will be returned as a single object. There are a number of other ways this problem could be attacked - there is really no need, for example, to return a serialized Django object to the client, other than just to show that it can be done.

The following Javascript goes into the document ready callback in the Javascript template:

$('#save_button').click(function () {
    // Remove any error messages that may be present from previous attempts
    $('#article_form ul.errorlist').remove();
    
    // Let the user know that the system is working
    $('#feedback').text('Saving...');

    // Make the attempt to save
    $.post('{% url update_article %}', $('#article_form').serialize(), function (data) {
        if (data[0]) {
            var date = new Date();
            $('#feedback').text('Article saved ' + date.toLocaleString());
        }
        else {
            // Set the feedback string
            $('#feedback').text('Article not saved');
            // Put in the error messages
            $.each(data, function(key, value) {
                // The edit was unsuccessful, put in the error messages.
                var UL = $('#article_form input[name=' + key + ']').closest('td')
                    .prepend('<ul class="errorlist"></ul>')
                    .find('ul');
                $.each(value, function() {
                    UL.append("<li>" + this + "</li>");
                });
            });
        }
    }, "json");
});

You can see that the Javascript is correspondingly much more complicated in this solution, but here we are basically manipulating only the individual elements which need it and not hacking out and replacing the entire chunk of the DOM containing the form. The corresponding Django view function is as follows:

from django.shortcuts import get_object_or_404
from django.core.serializers import serialize
from django.contrib.auth.decorators import login_required
from django.http import HttpResponse, Http404
from django.utils import simplejson as json
from articles.models import Article
from articles.forms import ArticleForm

@login_required
def update_article(request):
    """
    AJAX client sends a serialized form by POST. ID comes from the article_id key.
    
    The function returns JSON, either the updated Article object, or the form's
    error dict.
    """
    try:
        article_id = int(request.POST['article_id'])
    except KeyError:
        raise Http404
    article = get_object_or_404(Article, id=article_id)
    form = ArticleForm(request.POST, instance=article)
    if form.is_valid():
        form.save()
        return HttpResponse(serialize('json', (article,)), mimetype="application/json")
    else:
        return HttpResponse(json.dumps(form.errors), mimetype="application/json")

You can see that the Python portion is not terribly complex. Django provides functions that allow you to easily serialize either lists of models or arbitrary Python objects such as the error dictionary as JSON and send back to the client, which jQuery readily understands and processes.

Special Bonus

Notice that we have not dealt with any actual Ajax errors which might occur in the code above. You can do this directly with any of the Ajax functions by checking the status passed to a callback function, but various Ajax events also generate actual jQuery events which can be listened for and responded to just like UI events like mouse clicks. The following code places an event listener on the feedback div waiting for an Ajax error of some kind to be generated. In the event that an error occurs, the user is alerted by changing the text of the feedback block.

$('#feedback').ajaxError(function () {
    $(this).text("Server Error - Save Attempt Failed");
});

This code would be placed near the top of the document ready event listener block in the Javascript template file.

Conclusion

You can see that Django is able to interoperate readily with Javascript frameworks like jQuery to create the type of dynamic web applications that users are coming to expect and programmers are coming to demand.

The increasing invasion of Javascript into the Django admin interface shows that the Django developers themselves are not immune to this push - in particular, the Javascript components in the Admin are actually provided by jQuery as of the current Admin site documentation!

Using Ajax with Django is practically no harder than with any other framework once you know how to do it. With useful serializers present and good interoperability between form data serialized by jQuery and the Django Form objects, there are actually some very useful conveniences available.

My complaint, however, would be the lack of emphasis on Javascript integration that continues within the project as a whole, which to my present knowledge continues to be Javascript agnostic despite the inroads of jQuery. People who are interested in doing this type of development - which is increasingly becoming "everyone" - have to do their homework from the ground up.

Say what you like about the growth of modern dynamic web sites - love em or hate em, they represent the future. Clients want them and programmers want tools to develop them. Trying to go against this current, or ignoring it, is a path straight to irrelevance.

I think this is potentially a missed opportunity for Django so far. Django already provides an excellent grounding for writing AJAX applications, but the documentation is written in such a way that the automatic assumption is that you are always loading whole pages at a time. There is nothing in the examples to excite the imagination as to the possibilities. Furthermore, in the design of the Form objects, which happen to provide a useful foundation for Ajax form validation and submission, it seems readily apparent that zero thought was given to this functionality of the forms. I'm not sure what else really could or should be done, but the task of picking out and displaying form errors via Ajax using Forms seems like it could perhaps be more friendly.

A useful side project of Django could possibly be a set of helpers for jQuery similar to what Rails has with Prototype - possibly several side projects for several different frameworks. But regardless, I think that the design of dynamic web sites needs to become more central to Django's philosophy and focus in order to remain competitive and provide programmers with the tools they need for rapid web site development.

Comments:

There are 5 comments on this item.

Post a Comment

Copyright © 2009 Stephen DeGrace | Site Map | RSS