I was honored to be invited to participate in a recent DjangoDose panel covering Reusable Apps in Django which featured Brian Rosner, Eric Florenzano, Alex Gaynor, James Bennett, James Tauber and Jannis Leidel. After listening to the episode I realized I offered up a "brain fart" when responding to @TheSeanOC's question "How should schema migrations be handled in reusable apps?".

Let's review the brain fart

I jumped in to answer the question quickly but my answer to this question was essentially "use South." Wow, great. Very enlightening. But I forgot to elaborate on the concept I've had in my head for a long time but have yet documented. As the callcast discussion related to this question continued the panel ended up agreeing that the best solution was making the use of South a convention. But in hindsight I wanted to offer something more. I had what I believe is an answer, or at least a suggestion for a solution. Remember in "A Christmas Story" when Ralphie went to the mall to ask Santa for his Christmas present wish? (No? Watch it.). He froze up when he was on Santa's lap and asked for a football. After he was removed from Santa's lap, and pushed down the slide, he quickly realized that's not what he wanted at all. He wanted an official Red Rider BB Gun. This is my Red Rider BB gun request.

South and reusable apps

First, South is an amazing migration solution for Django. I can't imagine developing without it. I include it in Django-Mingus only because I want to use South but Mingus itself, as a "project", defines no models and therefor South doesn't care about it :( Working on Mingus and the 28+ apps it includes I came to wanting this feature some time ago. So here's the gist of it all...

Mingus uses a collection of reusable Django apps to build a blog engine. I noted this in my DjangoCon 09 how-to, but this concept isn't new. In fact, in researching my DjangoCon presentation I came across a James Bennett post where he answers "Where is the Django blog application" and he suggested exactly what Mingus set out to do - leverage the awesome reusable apps feature to piece together a blog engine. That was posted 2 years ago! I thought only @gregnewman could see in into the future. Anyway, I'd love to offer migrations along with Mingus but sadly we're not there yet.

Today if you were to run a local version of Django-Mingus and test the app locally using runserver after you followed the quickstart documentation you would notice some of the Django reusable apps required in the pip requirements.txt file included with Mingus found a home in the [Root]/src directory, while others found their home in /lib/python2.5/site-packges (that's all related to how they were packaged and referenced in the requirements file). Anyway, here's a screenshot below.

Default Django-Mingus project directoy structure.

In the above screenshot the directory django-mingus is my git repository root, not the above. Now what I want is the ability to easily include migrations from my initial project start, and ongoing into the future (@gregnewman knows all about the future.) so I can evolve my project along with these reusable apps code base. So what I would like to do here is (you need to have some familiarity with South to recognize these commands) the below pseudo code:

./manage.py startmigration django-mingus

South would then create a migrations directory in my project's root (or wherever I specify it via a settings variable) and then iterate through all the apps in INSTALLED_APPS and create directories for each reusable app and initial migration scripts. Then moving forward any time I update the reusable apps, I could easily create migrations, evolve my project database, and have it safely encapsulated in my project's repository. Right now this is not possible. This is what would happen now if I tried to create an initial migration for django-flatblocks.

./manage.py startmigration flatblocks --initial

The below is the resulting directory structure.

Djang-flatblocks initial migration.

You'll see the migration directory was created in the root directory for the django-flatblocks project which in the Django-Mingus project is located here /testproj/src/django-flatblocks/flatblocks/migrations. My issue with this is it's not encapsulated in my project root, which means it's outside my repository (therefor no version control) and depending on how the developer is installing the project, the migration script could end up anywhere on the filesystem he/she feels best fit to install the related reusable apps for the Mingus project. The last point is a bit weak, because we're using virtualenv and pip which helps us out in that regard. But I do see the need for South migrations related to reusable apps being included in the projects root versus off the applications root, and I think this may be the solution.

Updated: I should have initially added a visual to detail what I'm suggesting. It's now below.

Alternative project view with South included reusable app migrations on the project level.


Whether this is the best idea or not I'm not 100% sure. But I've been meaning to throw it out there for quite some time. So it's my Redrider BB-Gun, or my South pony. Whatever you want to call it. I'd love to hear your thoughts or how you handle this in your own projects - leave a comment below. Thanks.