thefoundation

QuickTime Reference Movies in Python

2010-07-01T22:53:07Z

HTML5 video is on its way, and Apple likes to use reference movies to send different versions of movies to different clients. So, how do you generate these if you don't have OSX handy (for example on a server)?

This is pretty much the premise of a little tool I wrote to generate QuickTime reference movies. It is written in Python, because it is to be used in a Django app that handles video uploads and encodes. Well, that and because I love Python! There isn't much else to say, so here is the code if you are interested:

hg clone http://bitbucket.org/dmurmann/root pyqtref

This is very much alpha software, it may fail horribly for you!

The simplest possible example, generating a reference movie to 'content.mov' looks like this:

from pyqtref import * ref_mov = Movie(ReferenceMovie(Descriptor( DataReference('content.mov')))).to_bytes()

The boilerplate is necessary, because the API is pretty close to the Quicktime File Format Specification.

It appears I was wrong

2009-04-08T23:18:11Z

Yesterday, Google announced the availability of Java as the new programming language for the App Engine, refuting my guess from last year that it might be JavaScript — though of course, not entirely.

If you take the demand of the users into account, Java is of course the right choice as the next language. It might just alienate lots of Java-Developers if a niche language of the server zoo such as JavaScript was to emerge first. Additionally, Java is one Step short of full (albeit sandboxed) JVM support.

To allow for sandboxing, Google wraps some of its own API's into Java SE or JSR–standardized services such as javax.mail and java.net.URL. Also, there is a white-list currently containing 1323 of the 3700+ Java SE 6 classes. Most of the classes that are not available are from the Swing and AWT suites which a web developer will not need anyway. Instead, Google provides the homebrewn GWT.

Thanks to the free and open source Rhino JavaScript Interpreter written in Java, server side JavaScript on the App Engine is rather easy to achieve now. I guess I might have to check it out and report back about it later, so I just signed up for the Java technology preview on my App Engine account.

There are some videos from the Google Campfire event over at Youtube. Most of the time they are rather interesting, plus Kevin Gibbs does a pretty decent imitation of Steve Jobs during the presentation (voluntary or not).

Streaming from iTunes to the iPhone: A Sketch

2008-10-20T01:18:12Z

Wireless playback of your music collection from iTunes to your stereo equipment, using the iPhone instead of cables or an AirPort Express. Requires a Mac, an iPhone, the Apple Developer tools and Fink, and the power to use them all.

I own an AirPort Express wireless access point that I used to stream my iTunes music to, so that it would play on the stereo equipment. In my current home though, my stereo sits in a corner, where an airport unit would cover my room but never the whole apartment. Also, I would have to pull an ethernet cable into my room for wireless internet access. So I put the airport unit into the hall instead.

Before the App Store launched, I used to run a simple web service on my notebook. Written in Python, it would report the currently playing iTunes track using Python — AppleScript bindings. The iPhone would load a page containing an iframe for the actual audio resource (the HTML 5 audio support is not yet available on the iPhone) and an poll for song updates via XMLHttpRequest. This solution worked so-so, as there was a pause between tracks because the iPhone browser had to bring up the Quicktime plugin in a slow animation for each track, before even preloading the actual song file.

Fortunately, Julien-Pierre Avérous offers his web radio streaming client FStream at the App Store for free! What's left to do now, is turning iTunes into a web radio station using free components:

Jack Audio Connection Kit to rewire the iTunes output to a stream encoder
DarkIce to create an MP3 stream that can be fed to a streaming web server. You will also need to get lame using Fink.
IceCast 2 as the actual streaming server. I had to switch to Fink unstable to get this version.

Next thing is to configure these tools to play along, which I shall describe in a follow-up article for the patient.

Project »INFINITY« [update]

2008-10-18T23:43:40Z

Bringing movement to lights.

Together with the German light designer Stefan Hofmann we created the animations for his light installation INFINITY for the Luminale 2008 in Frankfurt.
The installation was constructed in Frankfurt's Senckenberg Museum in April. The construction consists of 50, 3m high columns with integrated LED Pixels. The columns stand in a circle of a about 7m radius.
The idea was to entirely surround the spectators with light, creating an infinite light room. Our abstract 360° animations supported the infinite surrounding we wanted to create.

Project »INFINITY«

The installation will be presented again at GLOW 2008 in Eindhoven, Netherlands in November 2008.

Is JavaScript The Next App Engine Language?

2008-09-21T04:00:28Z

As the language of the browser, JavaScript has become a kind of common denominator among web programmers. On the server it is rare compared to Java, PHP, Ruby or Python. Could this change, since Google has built an implementation of JS and an affordable yet scalable web application infrastructure?

Scripting Language of The Open Web

JavaScript was created as a scripting language for the Netscape 2.0 browser. Other than its namesake Java, it has remained in the browser since then. Web Developers know this language because they have to use it, no matter if they like it. Throughout the late 90s and well into this decade, JavaScript was mostly regarded as a toy language for silly effects and simple form checks. Due to the more recent experience with Ajax applications and the resulting thorough coverage in technical literature, the users of JavaScript have become more professional.

The Server Domain

The development on the server side has — with some notable exceptions — ignored the language JavaScript for many years. Perl was a dominant language in the early days. Then, PHP gained momentum in the fast growing developer community during the internet bubble due to its gentle learning curve, the simple integration with MySQL and the overwhelming availability of inexpensive shared hosting. You could say that in its ubiquity, PHP has mimicked JavaScript, only on the server.
Other languages and tool chains have gained momentum since then, most notably Ruby, Java and Python. They are not as affordable as PHP (except Python on the App Engine), and seem to be used mostly because of their modern MVC frameworks.

Why JavaScript Is Not Popular on The Server

JavaScript has some fundamental weaknesses that have inhibited its inception as a server side programming language:

JavaScript, — or rather ECMAScript — has no real standard library, it is just a language. There are no tools for file or process manipulation. There are no database bindings and there is no shared memory. There is no way to open a socket.
In the past, JavaScript Interpreters have not been known for their performance. Anyone who has seen the message A script on this page may cause Firefox to run slow. would think twice before writing an application in JavaScript.

The Helma framework tries to address the first issue by using the Rhino Interpreter, making the Java SDK available to the JavaScript programmer. The second issue is at least partially resolved as some performance intensive tasks are moved to Java. In Java 6, Rhino is even bundled with Java. Nevertheless Helma and similar integration efforts are not very popular, compared to Rails or Struts. A reason for this might be that Java for the web (the enterprise edition) is a very serious business, while the perception of JavaScript has long been ...well, the opposite of serious. The two cultures do not seem to mix well yet, as good as the technologies might blend.

Why The App Engine Loves JavaScript

When Guido van Rossum adopted Python for the Google App Engine, he had to remove lots of features because they would not work with the sandboxed, replicating environment which is the App Engine. It turns out that the limitations of JavaScript really make it an ideal candidate for the App Engine.

JavaScript is just a language: It cannot create sockets, files or processes. App Engine does not allow for that anyway, so this is a plus!
The language is easily sandboxed. Web Browsers have to do this all the time. This is ideal for a shared hosting environment.
It has a no database bindings included, but a natural serialization format, JSON. This is great for the Google Data Store, because it stores objects, not table rows.
Recently, JavaScript Interpreters (or, lets say JavaScript Machines) have become fast. There is V8 and it is fast.

It is known (at least since Guido van Rossums talk at DjangoCon 2008) that Google plans to adopt more languages for the app engine. Also, there are four primary languages that Google uses for development: Python, Java, C++ and JavaScript. Python is already available. Java is a good candidate for the App Engine, but it is not exactly lightweight certainly not shared nothing. C++ is probably the opposite of any sandboxable language.

In combination with a template engine, a free offer on the App Engine might give the already ubiquitous JavaScript language a final boost. The prospect of one language from model definition to client side computation is certainly compelling!

To encourage further speculations: With Google's V8 machine comes an embedding, process.cc, that contains code necessary to extend a hypothetical HTTP request processing application - which could be part of a web server, for example (...) (V8 Documentation).

Nobody Expects The Production Problems

2008-09-21T00:08:05Z

Sometimes you need to track down problems in a production setup. How the combination of Django and Flup make this difficult, and why I think that both should provide more than log by mail.

Our Setup

Our chief weapon is development! — Cardinal Ximinez of Spain

To the developer, the Django web development Framework offers a set of debugging helpers such as the verbose error page with interactive traceback and the management commands. The automatic reloading of python modules allows you to instantaneously see changes to your application, without even losing running sessions. If that is not enough for you, you might be interested in the powerful Django Command Extensions. The Django Debug Toolbar also looks promising yet rather unfinished. I might come back to that another time.

The Other Setup

... and production! Our two weapons are development and production!

Web applications written in Django are also highly portable: Development can take place on a different OS than production will, using a different version of Python and different database and HTTP systems. With respect to debugging, these alternatives are equalized by the abstraction layers of the Python and Django stack — and fairly well. Once you have sorted out the do's and dont's of your RDBMS, there is the decision on how to handle HTTP requests. Django recommends using the Apache HTTP Server and mod_python. Another performant setup that seems to be quite common is using a modern FastCGI-capable Web Server in combination with the flup FCGI/WSGI bridge. This is what I did.

Production: Turn Off the Debug Switch!

When preparing your production setup, you will want to look into your settings.py. Actually, you will probably have multiple settings files of different names in revision control. Then, you will create a symlink to the settings matching the current setup. In your production settings, you will turn off debugging. This replaces the verbose traceback pages with neat-looking 500 views which will — hopefully — never see the light of day! And otherwise, you will receive a nice error report somehow, right?

Of course you will probably encounter a production error in your site, be it a bunch of downloaded third party applications or your own magnificent creation. In my case, I soon had problems with an index on a text column.

The Third Setup

... and ruthless stage testing! Our three weapons are development, production, and ruthless stage testing.

To catch such problems before they occur with your production setup, it is highly desirable to use a stage setup. This setup has its own database, usually a copy of your production database, updated on demand. The other settings should mirror your production setup. Run your staging application on the production machine or — if you can afford it — on a separate identical system and make it accessible to selected testers only. For example, you might make it listen to the local IP only and ssh-tunnel from your development machine there. On this machine, you can turn on debugging whenever needed.

Problem One: Logging by E-mail

Of course, there might still be problems that are discovered in your production setup. Unfortunately, out of the box Django reports errors only via e-mail. To me, this has some critical drawbacks:

You do not know if the error reporting works unless a problem occurs. In this case, you will only know that error reporting works if it works. Avoid this problem by enabling 404 reporting and then testing an invalid URL!
It is not secure! To my knowledge there is no PGP-support for the error mails sent by Django. The messages might contain sensitive data that should not be sent unencrypted.
If your production setup does not work right away, you will need debugging information in fast iterations. Obtaining such information by mail is inconvenient as it is tiresome to link a browser-click to an e-mail that you might receive minutes later.
Your machine might not be allowed to send mails! Many sysops put firewall rules in place to prevent machines in their domain from becoming spam robots. This is what I did on my production machine, so fortunately I knew in advance that debugging by mail would not work. I am not sure if every django admin is aware of this potential problem.
If you host a high traffic site and an error occurs (for example if you database does not respond due to slashdotting effects or denial of service attacks), the volumes of e-mail it generates might be so high that the error reporting competes with your application on IP-connections, worsening your problems.

Solution: Exception-Logging Middleware

For my basic needs, I have written a simple middleware that should handle any exceptions raised by views in your production setup. Please note that this middleware relies on the Exception Helpers module written by Dmitry Dvoinikov to format tracebacks. The traceback information is written into the logfile specified by the (custom) LOG_FILE setting. If the LOG_LEVEL is at least as verbose as DEBUG, the request object is dumped as well.

Problem Two: Exceptions Outside of Views

We are now prepared for errors that might occur when running your views production. But there is a different class of exceptions that cannot be handled by our middleware because of their nature:

On your Site, you probably want caching, ETags and transfer compression. So you enable the corresponding middleware, hopefully in the right order, as explained by Fredrik Lundh. And you might just add some middleware from djangosnippets or some of your own. Perhaps you want to prettyprint or to simplify your output, or to yield custom error pages for Ajax-Requests.

Solution: Enable Flup Traceback Pages

Any exceptions raised during middleware processing are not caught and handled by Django, but instead propagated up to the flup FastCGI handler. By default, Flup would present an error page like Django does in debugging mode, which is quite helpful (if not as beautiful). But Django disables this error page explicitly during startup. I filed a patch that allows to specify a debug flag to your FastCGI process to remedy this issue.

Summary

Amongst our weaponry are such diverse elements as development, production, ruthless stage testing, almost fanatical verboseness to the log and nice red flup tracebacks, my oh my oh my.

To anticipate and tackle production bugs, the solution consists of these parts:

separate staging and production setups
use exception logging middleware
enable flup error pages in the stage setup

This is still not perfect. Ideally, flup would also log errors in production mode. Weirdly, flup also knows log by e-mail, and in this case it is not even configurable from your Django app! I will probably complement this article when I find an elegant way to fix this.

Using ffmpeg in Python

2008-09-11T16:18:32Z

Or: how to effectively use other peoples work...

First some background: A while ago I wrote a simple Python script that processed a sequence of images in some specific way. I used the PIL to open the images, then do the transformation and write a sequence of images again. It quickly became apparent that there was significant overhead involved in dealing with larger sequences (>10000 files), especially for some OS/filesystem combinations. The slowest of these was probably the OSX Finder trying to access such a directory over SMB, switching to AFP helped a lot there.

To get rid of the need to use (at least one of) these image sequences at all, I decided to redo the reading part of the script to use ffmpeg to read the frames from a quicktime file. As it stands there are some attempts to wrap ffmpeg in Python (see PyMedia, PyFFmpeg), but they seemed to be outdated or weren't compiling on my 64bit machine. So, I went ahead to try a ctypes-based approach on my own, which, as it turns out, is really easy. I don't want to bore you with the details of what I tried, here just comes the summary of what worked well:

First some prerequisites that you might not have: sudo apt-get install gccxml libavcodec-dev \ libavformat-dev libswscale-dev libavfilter-dev \ libavdevice-dev libpostproc-dev Next is the tool that makes everything incredibly simple: ctypeslib. svn co http://svn.python.org/projects/ctypes/trunk/ctypeslib Note that the trunk version of ctypeslib only works well with older versions of gccxml (like 0.7), if you have a newer version, get the following branch: svn co http://svn.python.org/projects/ctypes/branches/ctypeslib-gccxml-0.9 After that a simple sudo ./setup.py install should give you a working h2xml.py and xml2py.py. With these you can generate ctypes wrappers: h2xml.py -c -I/usr/include/ffmpeg libavcodec/avcodec.h \ libavformat/avformat.h libswscale/swscale.h \ libavdevice/avdevice.h libavfilter/avfilter.h \ libpostproc/postprocess.h -o av.xml xml2py.py av.xml -o av.py -l libavcodec.so -l libavformat.so \ -l libswscale.so -l libavdevice.so -l libavfilter.so \ -l libpostproc.so If that worked you can use ffmpeg in Python like this: from av import * av_register_all() pFormatCtx = POINTER(AVFormatContext)() if av_open_input_file(pFormatCtx, filename, None, 0, None): handle_error("could not open file") ... (insert your usual ffmpeg code, examples here and here) ...

Up to now I only found two things that I needed that weren't converted correctly by ctypeslib. The first were some defines, which could probably be done right but I didn't need many so I just added them by hand. The second was this inline function static inline void av_free_packet(AVPacket *pkt) { if (pkt && pkt->destruct) { pkt->destruct(pkt); } } which I replaced by this definition (to be called with an AVPacket object instead of a pointer to one): def av_free_packet(pkt): if pkt and pkt.destruct: pkt.destruct(byref(pkt)) With a little bit of scripting to convert the ffmpeg decoded frame to a PIL image the script can now read anything ffmpeg can read. All thats left to do is encode the output with ffmpeg, once that is done I might post this as a module that encapsules the conversion from ffmpeg to PIL image sequence and back.

Last update on: Thursday, 9th October, 2008

Animating Frankfurt #1

2008-09-10T14:56:53Z

Creating images for a media facade. 2km of cabels and a UMTS internet card in a small three bed hotel room in the middle of Frankfurt, Germany.

Yesterday, Daniel, David and me moved all our stuff to Frankfurt, Germany to proceed with a project we began in 2007. A bank in Frankfurt redesigned their front facade and integrated a grid of LEDs into the glass facade.
So now we are here to animate a video for the facade. The tricky thing about this is the arrangement of the pixels. We have a grid of approx.1100 Pixels vertically and only 49 horizontally on a building that is 6 stories high and 250 meters long. So we have 49 high resolution LED lines on the building. For our animations we have to consider this lack in horizontal resolution.
Daniel and me animate the themes while David is in charge of the correct pixel conversion of the animation and all the connection to the servers we need.

Now we really have to get something to eat! I will add some images soon...