Pycon UK, Javascript and the GIL
Just got back from Pycon UK 2008 - here are some impressions.
Both the keynote speakers Mark Shuttleworth (Canonical) and
Ted Leung (Sun Microsystems) expressed their concerns about
Javascript becoming so fast and prominent that it could displace
Python in the future. They also highlighted the fact that
Multi-core systems get cheaper and more popular also on
desktop computers or notebooks. They challenged the community
to advance Python implementations to exploit it. Question was up
what PyPy can do here. As it stands, PyPy still uses the good old
Global Interpreter Lock (GIL) but our approaches should indeed
lend itself well to do experimentation with free threading.
During the 2-day conference we met many interesting people, most
notably the guys from Resolver, among them William Reade who is working on
IronClad -- which implements a fake python25.dll on top of
IronPython. He presented some good results for Numpy in his
lightning talk. This approach is surely something to follow
closely and potentially use for PyPy.
We also had lunch and a couple of chats with Jacob Kaplan-Moss from
Django fame - he is apparently up to try use PyPy's sandboxing features
for one of his projects, cool!
Conference itself was well organized for the 230 attending people - although
the venue might be a bit small for next year's EuroPython. Ah, and
we gave three well attended talks, find the slides here:
Holger, Maciej, Anto (associated through merlinux, btw)
Comments
Is the work done on this (https://code.google.com/p/python-safethread/) useful conceptually?
Is the new multiprocessing module going to offer improved multi-core performance?
Jython? About to hit 2.5, has a threading model for free from the JVM.
I wonder how the new javascript improvements compare to pypy technically. For example, the tracing techniques of Mozilla's Tracemonkey look impressive, but I don't know if these techniques are conceptually close or not to pypy's. Is there anything you can learn from them (tracemonkey, chrome's v8, etc).
Luis
ReneL:
Yes, the new multiprocessing module will improve multi-core performance if you use it.
That module allows a easy way to use multiple processes cooperatively in python. It tries to mimic the threading API.
If you use multiple processes instead of threads you will avoid the Global Interpreter Lock.
About new chalenges for PyPy, multicore isn't the major problem.. the absense of a powerfull JIT and GC is. Please keep working on a super-fast "VM+JIT" for python! (super linux performance is a must)
colin: true, jython and also ironpython can make use of multiple threads. should have mentioned it. Doesn't mean that pypy-c shouldn't go for it, rather the opposite i'd think :)
renel/multiprocessing module: i can imagine it helps with multi-core cpus. are there practical experiences using it yet?
luis, miguel: there are considerable efforts on the PyPy/JIT front - particularly from Armin, Carl Friedrich and Antonio - would be worth a dedicated blog post to relate this to tracing JITs, V8, squirrelfish, etc. One thing i know is that we probably want to apply for funding to help completing the JIT.
Miguel: We do have advanced GCs and are working on on improving them, currently.
to all: thanks for your feedback!