17 September 2008

Python popen deadlocks

I have a python script that calls out to an external program using os.popen3(). That function returns file handle like objects for stdin, stdout, and stderr. I need to examine both stdout and stderr separately.

After running this script for a while I started to see it hang at times. This was especially true after I added in a parameter which increased the amount stderr. It turns out that there is a well known deadlock issue with the popen family of commands. Or, rather, it was not known to me until yesterday.

I started to tinker in my script using select.select() as a solution. That seemed a little foreign to my way of thinking so I changed directions and used separate threads for reading from the stderr and stdout file handles that os.popen2() returns. That seemed to do the trick so I'm pretty happy about that.

No comments:

Post a Comment