flup

With the advent of flup 1.0 and the disappearance of the middleware modules (I don't think anyone really cares about Publisher ), I've been getting a few queries here and there about alternatives.

Rather than continue to answer people individually, here are some alternatives to flup's middleware and publisher modules. However, they represent alternatives that I've personally chosen. (And considering how far removed I am nowadays from the world of Python web development, take my choices with a grain of salt.) Suggestions for other alternatives (alternative alternatives?!?) are welcome.

For users of flup's SessionMiddleware, I recommend Beaker. Switching over is a simple matter of changing the environ key from 'com.saddi.service.session' to 'beaker.session' and ensuring that session.save() is called before the WSGI application returns. (After you've appropriately swapped out flup's SessionMiddleware with Beaker's in the WSGI stack, of course.)

GzipMiddleware and ErrorMiddleware have analogues in Python Paste.

flup's Publisher is probably most like Colubrid, though I haven't personally tried it. (Also remember that flup's Publisher was inspired by mod_python's Publisher, which was apparently inspired by Zope's ZPublisher ).

Anyway, I updated the old flup 0.5 branch and uploaded the source distribution to the Cheese Shop. I really don't want to maintain two branches, so consider this 0.5 release final.

I resolved a few more outstanding issues with flup and decided to make a release. I went ahead and called it 1.0. And as I previously mentioned, I went ahead and removed the publisher and middleware modules.

If you've been following the Subversion repository for flup (which has been quiet due to my switching to Mercurial), there have only been 1 or 2 new fixes.

I also uploaded the tarball and 2.5 egg to the Cheese Shop. Further updates will also go to the Cheese Shop.

And on the subject of further updates, consider flup to be in maintenance mode - there won't be any more new development. Bug fixes and bug reports are still welcome, of course. (Though this has been the way I've run flup for quite a while now, so I'm just making a formal announcement. )

And as a bit of editorializing, did I ever mention how much I hate FastCGI? Around the time I first released flup, I had transitioned my apps from FastCGI to AJP, and I never missed FastCGI since. But much to my disappointment, the FastCGI module was the most popular... most likely due to ubiquitous & cheap shared hosting services (e.g. Dreamhost).

Oh well, I don't even use flup myself anymore. I've switched to ajp-wsgi...

In an effort to quell a warning from wsgiref's validator, QUERY_STRING will now default to an empty string if it doesn't exist in the environ. Despite not being required to always be present by the WSGI spec, it looks like the cgi module will assume sys.argv[1] is the query string if QUERY_STRING isn't present in the environ.

Also, I changed the keyword parameters in GzipMiddleware a bit:

• mimeTypes -> mime_types
• compresslevel -> compress_level

Lastly, unless I hear from people otherwise, I will be removing flup.publisher and flup.resolver from flup.

And as far as flup.middleware goes, ErrorMiddleware will be disappearing as well. There are better options from Paste, especially paste.evalexception for development. GzipMiddleware and SessionMiddleware are now currently looking for a new home, as I would like to remove them from flup eventually too.

I added FCGIApp and SCGIApp to the (new) flup.client package. I wrote FCGIApp a few months ago for Ian Bicking's WPHP. While attempting to debug a problem with flup's fcgi server on Dreamhost (vs. my old fcgi.py module), I thought of an interesting solution.

Since the problem seemed to stem from the app's long startup time, and since Dreamhost only allows dynamic (i.e. web server-launched) FastCGI apps, why not have the web server-started app be as simple as possible and have it forward requests to a static, manually-launched app server?

In other words:

httpd (mod_fastcgi) -> fcgi.WSGIServer -> FCGIApp -> fcgi.WSGIServer -> app

Yeah, it ends up doing some extra work... which is why I wrote SCGIApp (SCGI is a far simpler protocol):

httpd (mod_fastcgi) -> fcgi.WSGIServer -> SCGIApp -> scgi.WSGIServer -> app

Anyway, it's a trade off and I really don't know how significant it is. But I was never really a fan of web server-launched FastCGI apps. Unless you configure mod_fastcgi explicitly, it has a tendency to launch an application multiple times. And if your application isn't multi-process safe and aware, it can lead to problems.

Consider these two modules experimental for now. I'm not really sure where I want them to live. But they're there for the time being to play around with.

I briefly toyed with Trac some time ago. Though I had to patch it to use WSGI, I found it pretty easy to use and maintain. My efforts eventually stalled, however, probably due to the way my public repository was derived from my real repository (i.e. lots of empty changesets).

Anyway, yesterday I updated to Trac 0.10.2 and was pleasantly surprised to find that it supported WSGI and flup's ajp and scgi servers out-of-the-box. The empty changeset issue is still there (which is my problem to fix). I just might have to split my repository up in the future.

Anyhow, flup has its own Trac site now:

http://trac.saddi.com/flup

The old flup site, http://www.saddi.com/software/flup/ simply redirects there now. (And will probably do so for the foreseeable future.) Please continue to use this URL when referencing flup.

I locked down all the default pages, but all the flup-related pages (save for the front page) are editable by anyone. So feel free to contribute examples, FAQs, etc.

For whatever reason, I had set up the AJP, FastCGI and SCGI servers to ignore EPIPE exceptions. (The exception that's thrown when writing to a disconnected socket, i.e. a web client that cancels a web page load or a download.) I think I had written the original version of fcgi (which I had written a millennia ago, it feels like), to do this... and since it was the basis for all the other servers... I never really questioned why EPIPE was being ignored. (To save the application from having to handle it?) I guess I just never tried downloading large files and then subsequently cancelling.

Anyhow, that behavior's been changed. It's still being ignored, but at a higher level. What this means is that WSGI applications may possibly see EPIPE socket.error exceptions, especially if they use the WSGI write callable. Also, the servers will stop iterating over the application's iterator should an EPIPE exception be thrown. (Which seems to be the correct thing to do anyway.)

Thanks to Ivan Sagalaev for bringing the problem to my attention!

I added a umask keyword parameter to both fcgi.WSGIServer and fcgi_fork.WSGIServer. If given and binding to a UNIX socket, the umask will be changed right before the socket is bound. Afterwards, the umask will be restored.

This should allow one to change the file permissions of the socket without altering the umask of the entire application. (If you need it to be, say, group-writable or world-writable.)

I've added paste.server_factory-compliant factories as well as respective egg entry points for each server within flup. Thanks to Luis Bruno for providing the code.

Additionally, each server now accepts a 'debug' bool keyword, which is True by default. It affects the output of the server-level error handler (which really shouldn't be used, but it's there if you're lazy like me ).

Though I just noticed that PasteScript already includes server runners for the flup servers...

Apparently, the use of signal.SIGHUP was the only thing preventing the use of the (threaded) servers on Windows. I made a minor change to conditionally install the SIGHUP handler. Thanks to Brad Miller for pointing this out.

Now whether the forked servers work on Windows, I really really doubt it.

Though I'm curious, how functional is flup under a CygWin-built Python?

I mentioned the new error404 feature in the last entry - after some review, I decided to change this a bit. There was no reason for it to be at the Resolver level, setting at Publisher made more sense. So I changed things around to reflect this new design. However, if you previously subclassed Publisher to override the _error404 method, that feature is now broken. You'll have to rewrite your custom _error404, which should be trivial.

Resolvers will now take a keyword argument, "error404", which allows you to specify a callable which will be called to render the 404 error page. Previously, you had to subclass Publisher to customize the 404 page (and it was rather low-level - basically a WSGI mini-app). The callable is called using the normal Publisher conventions. Thanks to Scot Doyle for the idea & code!

Also, I decided to look at the Python Egg thing more and converted flup's setup.py to use setuptools. (It does not require setuptools, however. It will fall back to distutils, if needed.)

From now on, the dist directory will contain an egg of the latest version of flup. (I haven't decided on a version numbering scheme yet, so it will always be 0.5. )

I just fixed a rather insiduous 2.3-only problem in fcgi: Apparently, the 2.3 socket module does not define the SHUT_* constants for the shutdown() syscall. This caused an attribute error that was silently being ignored. The end result was that the FastCGI socket connection with the server was not being closed - this caused output to hang in certain situations.

I didn't see the problem because as I said previously, I run 2.4 everywhere - even on my Macs (which normally come with 2.3 out-of-the-box).

Anyhow, big thanks to Eugene Lazutkin for bringing the problem to my attention, helping to isolate the problem (since it was unreproducible on my end, I had a hard time believing it was fcgi's fault ), and finally, providing access to an environment where it was reproducible 100% of the time (which allowed me to figure it out).

Namely, the modules that were repackaged into flup. What this means is that the py-lib versions will not be updated anymore. I plan to eventually to delete the affected files from py-lib, but not for some time.

HOWEVER, I will continue to update fcgi.py and session.py in py-lib. I realize the non-packaged versions of these modules are somewhat popular and convenient to have around. (e.g. No need to install the entire flup package if all you're interested in is the FastCGI WSGI module.)

Regardless, I encourage all new code to use flup instead, and all old code to be ported to flup. (C'mon! After installing flup, it just requires a single change to the import statement. )

Session's generateIdentifier() method was returning non-ASCII strings for international locales. This has been fixed. Thanks to Ksenia Marasanova for the problem report and patch.

A minor change was made to how fcgi closes its connection with the web server. In certain cases, fcgi may close the connection early, causing the connection to be reset at the web server's end. This usually results in a shortened HTTP response. Thanks to Dima Barsky for pointing this out and suggesting a fix!