Allan Saddi's projects blog - flup

new flup snapshot

nospam@example.com (Allan Saddi) — Wed, 21 Oct 2009 09:48:10 -0700

I just released a new snapshot, it includes the following changes:

fix fork bomb and check number of children against minSpare not maxSpare
Restore check of the absolute number of children against maxSpare and document rationale.
Add configurable timeout (default: no timeout) to be used when the WSGI application is called. Only applies to forked servers!

Here are links to the Hg repository and the PyPI page.

If there are any issues, please open a ticket at the flup Trac and/or submit a patch.

Submitting Patches

nospam@example.com (Allan Saddi) — Mon, 03 Aug 2009 11:34:18 -0700

I've written up brief guidelines on submitting patches for flup. (Though it also pertains to any of my other public projects.)

And I'd like to thank everyone who has recently contributed! Don't worry, there's nothing in the guidelines that you weren't already doing.

flup 1.0.2 released

nospam@example.com (Allan Saddi) — Tue, 26 May 2009 08:02:23 -0700

See this entry for details.

flup 1.0.2 on the horizon...

nospam@example.com (Allan Saddi) — Mon, 18 May 2009 16:06:49 -0700

I guess I've sat on these changes long enough:

Import Paste factories (and dependencies...) from PasteScript
Be tolerant of EAGAIN when sending messages to parent process.
Add forceCGI keyword argument to FastCGI servers to programmatically force CGI behavior.
Merge Tommi Virtanen's "single server" (sequential server) patch.
Update ez_setup.py.
Re-seed random module after each fork.
Add an indication as to which header fails assertion when passing in non-string header names and/or values.

The only major change are the Paste factories — they should be a lot closer to the (deprecated) factories for flup servers found in PasteScript. In fact, they are the same factories found in PasteScript. Defaults will have changed, with respect to the previous factory implementations.

I'm looking to release in a week or so. Testing/comments are welcome:

http://hg.saddi.com/flup-server/

And, as you probably have noticed, I don't cut releases that often (not a big fan of "release early, release often"...) But my public repository is always up-to-date. So it's almost the same thing... without the overhead.

Dabbling in Python 3.0

nospam@example.com (Allan Saddi) — Fri, 05 Dec 2008 16:50:30 -0800

Well, Python 3.0 was released a few days ago. I admit, I really haven't been keeping up much with Python development. Up until yesterday, Python 2.5 was all that I've used.

Anyway, I was curious. I was curious about all the backwards-incompatible changes to the language/libraries. I was curious about how WSGI dealt with Python 3.0. With the list of proposed WSGI 1.0 amendments for Python 3.0 (and the Web-SIG posts it referenced), I attempted to port both flup and ajp-wsgi.

It was interesting, at least, to see the issues in two different lights. Though flup and ajp-wsgi do the same thing conceptually, flup is pure-Python and ajp-wsgi is pure-C. The most pain for both, unsurprisingly, came from the new str/bytes distinction. While it was an annoying issue to deal with (and I realize as far as the spec is concerned, all details about it still haven't been worked out), I do welcome the change.

I've published both repositories, for the curious. They work for simple "Hello World" type applications. I kind of doubt that any frameworks have been ported to Python 3.0 yet. I probably won't touch them again until the spec has been clarified more. (Encoding of environ key/values? file_wrapper support with e.g. text-mode files?)

Interestingly, I didn't actually test the FastCGI and SCGI adapters with "real" web servers. (I don't have anything set up.) I used the flup client implementations to cross-test clients/servers on 2.x/3.0. So those flup WSGI servers might actually be broken.

The repositories:

flup 1.0.1 released

nospam@example.com (Allan Saddi) — Tue, 22 Jul 2008 11:50:55 -0700

A very minor update to flup. If you aren't having any problems now, then there's probably no need to upgrade.

This update concerns certain web servers & configurations that don't correctly supply PATH_INFO and/or QUERY_STRING to flup. When missing these environment entries, the flup servers will now attempt to deduce PATH_INFO and/or QUERY_STRING from REQUEST_URI (if it is present) before defaulting to the empty string (as it did before).

Thanks to Richard Davies for the patch.

Middleware, publisher alternatives

nospam@example.com (Allan Saddi) — Tue, 25 Sep 2007 17:06:12 -0700

With the advent of flup 1.0 and the disappearance of the middleware modules (I don't think anyone really cares about Publisher ), I've been getting a few queries here and there about alternatives.

Rather than continue to answer people individually, here are some alternatives to flup's middleware and publisher modules. However, they represent alternatives that I've personally chosen. (And considering how far removed I am nowadays from the world of Python web development, take my choices with a grain of salt.) Suggestions for other alternatives (alternative alternatives?!?) are welcome.

For users of flup's SessionMiddleware, I recommend Beaker. Switching over is a simple matter of changing the environ key from 'com.saddi.service.session' to 'beaker.session' and ensuring that session.save() is called before the WSGI application returns. (After you've appropriately swapped out flup's SessionMiddleware with Beaker's in the WSGI stack, of course.)

GzipMiddleware and ErrorMiddleware have analogues in Python Paste.

flup's Publisher is probably most like Colubrid, though I haven't personally tried it. (Also remember that flup's Publisher was inspired by mod_python's Publisher, which was apparently inspired by Zope's ZPublisher ).

Anyway, I updated the old flup 0.5 branch and uploaded the source distribution to the Cheese Shop. I really don't want to maintain two branches, so consider this 0.5 release final.

flup 1.0 released

nospam@example.com (Allan Saddi) — Mon, 10 Sep 2007 16:59:33 -0700

I resolved a few more outstanding issues with flup and decided to make a release. I went ahead and called it 1.0. And as I previously mentioned, I went ahead and removed the publisher and middleware modules.

If you've been following the Subversion repository for flup (which has been quiet due to my switching to Mercurial), there have only been 1 or 2 new fixes.

I also uploaded the tarball and 2.5 egg to the Cheese Shop. Further updates will also go to the Cheese Shop.

And on the subject of further updates, consider flup to be in maintenance mode - there won't be any more new development. Bug fixes and bug reports are still welcome, of course. (Though this has been the way I've run flup for quite a while now, so I'm just making a formal announcement. )

And as a bit of editorializing, did I ever mention how much I hate FastCGI? Around the time I first released flup, I had transitioned my apps from FastCGI to AJP, and I never missed FastCGI since. But much to my disappointment, the FastCGI module was the most popular... most likely due to ubiquitous & cheap shared hosting services (e.g. Dreamhost).

Oh well, I don't even use flup myself anymore. I've switched to ajp-wsgi...

QUERY_STRING changes to servers, API changes to GzipMiddleware, future changes

nospam@example.com (Allan Saddi) — Tue, 05 Dec 2006 14:15:51 -0800

In an effort to quell a warning from wsgiref's validator, QUERY_STRING will now default to an empty string if it doesn't exist in the environ. Despite not being required to always be present by the WSGI spec, it looks like the cgi module will assume sys.argv[1] is the query string if QUERY_STRING isn't present in the environ.

Also, I changed the keyword parameters in GzipMiddleware a bit:

mimeTypes -> mime_types
compresslevel -> compress_level

Lastly, unless I hear from people otherwise, I will be removing flup.publisher and flup.resolver from flup.

And as far as flup.middleware goes, ErrorMiddleware will be disappearing as well. There are better options from Paste, especially paste.evalexception for development. GzipMiddleware and SessionMiddleware are now currently looking for a new home, as I would like to remove them from flup eventually too.

Added FCGIApp and SCGIApp

nospam@example.com (Allan Saddi) — Fri, 24 Nov 2006 15:12:21 -0800

I added FCGIApp and SCGIApp to the (new) flup.client package. I wrote FCGIApp a few months ago for Ian Bicking's WPHP. While attempting to debug a problem with flup's fcgi server on Dreamhost (vs. my old fcgi.py module), I thought of an interesting solution.

Since the problem seemed to stem from the app's long startup time, and since Dreamhost only allows dynamic (i.e. web server-launched) FastCGI apps, why not have the web server-started app be as simple as possible and have it forward requests to a static, manually-launched app server?

In other words:

httpd (mod_fastcgi) -> fcgi.WSGIServer -> FCGIApp -> fcgi.WSGIServer -> app

Yeah, it ends up doing some extra work... which is why I wrote SCGIApp (SCGI is a far simpler protocol):

httpd (mod_fastcgi) -> fcgi.WSGIServer -> SCGIApp -> scgi.WSGIServer -> app

Anyway, it's a trade off and I really don't know how significant it is. But I was never really a fan of web server-launched FastCGI apps. Unless you configure mod_fastcgi explicitly, it has a tendency to launch an application multiple times. And if your application isn't multi-process safe and aware, it can lead to problems.

Consider these two modules experimental for now. I'm not really sure where I want them to live. But they're there for the time being to play around with.

flup Trac site available

nospam@example.com (Allan Saddi) — Mon, 20 Nov 2006 10:45:54 -0800

I briefly toyed with Trac some time ago. Though I had to patch it to use WSGI, I found it pretty easy to use and maintain. My efforts eventually stalled, however, probably due to the way my public repository was derived from my real repository (i.e. lots of empty changesets).

Anyway, yesterday I updated to Trac 0.10.2 and was pleasantly surprised to find that it supported WSGI and flup's ajp and scgi servers out-of-the-box. The empty changeset issue is still there (which is my problem to fix). I just might have to split my repository up in the future.

Anyhow, flup has its own Trac site now:

http://trac.saddi.com/flup

The old flup site, http://www.saddi.com/software/flup/ simply redirects there now. (And will probably do so for the foreseeable future.) Please continue to use this URL when referencing flup.

I locked down all the default pages, but all the flup-related pages (save for the front page) are editable by anyone. So feel free to contribute examples, FAQs, etc.

EPIPE no longer ignored

nospam@example.com (Allan Saddi) — Sun, 18 Jun 2006 09:10:02 -0700

For whatever reason, I had set up the AJP, FastCGI and SCGI servers to ignore EPIPE exceptions. (The exception that's thrown when writing to a disconnected socket, i.e. a web client that cancels a web page load or a download.) I think I had written the original version of fcgi (which I had written a millennia ago, it feels like), to do this... and since it was the basis for all the other servers... I never really questioned why EPIPE was being ignored. (To save the application from having to handle it?) I guess I just never tried downloading large files and then subsequently cancelling.

Anyhow, that behavior's been changed. It's still being ignored, but at a higher level. What this means is that WSGI applications may possibly see EPIPE socket.error exceptions, especially if they use the WSGI write callable. Also, the servers will stop iterating over the application's iterator should an EPIPE exception be thrown. (Which seems to be the correct thing to do anyway.)

Thanks to Ivan Sagalaev for bringing the problem to my attention!

New umask keyword parameter

nospam@example.com (Allan Saddi) — Thu, 18 May 2006 09:33:57 -0700

I added a umask keyword parameter to both fcgi.WSGIServer and fcgi_fork.WSGIServer. If given and binding to a UNIX socket, the umask will be changed right before the socket is bound. Afterwards, the umask will be restored.

This should allow one to change the file permissions of the socket without altering the umask of the entire application. (If you need it to be, say, group-writable or world-writable.)

paste.server_factory support

nospam@example.com (Allan Saddi) — Thu, 23 Feb 2006 16:18:11 -0800

I've added paste.server_factory-compliant factories as well as respective egg entry points for each server within flup. Thanks to Luis Bruno for providing the code.

Additionally, each server now accepts a 'debug' bool keyword, which is True by default. It affects the output of the server-level error handler (which really shouldn't be used, but it's there if you're lazy like me ).

Though I just noticed that PasteScript already includes server runners for the flup servers...

Conditional SIGHUP support

nospam@example.com (Allan Saddi) — Thu, 05 Jan 2006 12:57:45 -0800

Apparently, the use of signal.SIGHUP was the only thing preventing the use of the (threaded) servers on Windows. I made a minor change to conditionally install the SIGHUP handler. Thanks to Brad Miller for pointing this out.

Now whether the forked servers work on Windows, I really really doubt it.

Though I'm curious, how functional is flup under a CygWin-built Python?