I just released Py4J 0.3, a major milestone. This release adds support for more Java collections such as array and set, but more importantly, it enables Java objects to call Python objects that implements a Java interface.
While working on Py4J, I constantly pushed the hard features and the nasty bug fixes to milestone 0.3. Let’s just say that I’m happy this release is now behind me and I am promoting Py4J to beta because all major features are now implemented and relatively well tested.
Enabling callback required a major rearchitecture of Py4J internals and I had to implement a multi-threaded and multi-socket RMI-like server on both the Python and the Java sides. Fortunately, this is not the first time I am involved in shady businesses like this (I remember a graduate software fault tolerance course where I created a redundant framework relying on an orgy of sockets, threads, and processes).
Another area that caused me some worries was the garbage collection in Python and in Java. It took me dozen of hours to reach a good level of understanding of Python handling of garbage collection, weak references, and circular references. I would have preferred to have spent these hours on Team Fortress 2, but in the end, I am glad I learned something valuable.
Getting this kind of code to work is hard and I expect it will take a few more releases to polish it and to uncover bugs, but I believe the time I took to design the new architecture on paper was worth it and the initial tests are green 🙂
Now, stop reading this post and go download Py4J 0.3!