Py4J 0.10.3 released

Py4J 0.10.3 has just been released on PyPI, Maven Central, and bintray (p2 eclipse repository).

This is a backward-compatible released with a focus on small utilities on the Python side and preparation steps for the major performance work of 0.11.0.

  • Python side: Added java_path option in launch_gateway. If None, will detect whether JAVA_HOME is set and use “JAVA_HOME/bin/java” instead of “java” to launch the JVM.
  • Python side: added “create_new_process_group” in “launch_gateway”. If True, will launch the JVM in a new process group which (1) prevents signals sent to the parent Python process to propagate to the child JVM process, and (2) does not kill the Java process if the Python process dies. This is a useful option if you want to interrupt a long-running Java method call from Python and you launched the JVM using launch_gateway. Such interruption has always been possible if you launched the JVM outside of Python.
  • Python side: introduced a small signals library. Users can now connect to signals emitted by the CallbackServer, which mirrors the events sent by GatewayServer on the Java side.
  • Python side: added “get_java_class” function which returns the java.lang.Class of a JavaClass. Equivalent to calling .class in Java, but from Python.
  • Python side: fixed the project root setup.py, which allows users to install Py4J with pip from the git repository. The root setup.py relied on compiled jars that are no longer provided. It now uses gradlew to build the required jars during the installation. Works on both Linux and Windows 🙂
  • Python side: fixed type conversion when passing a large negative integer.
  • Java side: added defensive programming to prevent concurrent modification of the listeners list (in case a listener removes itself after receiving an event).
  • Both sides: added more memory leak tests and fixed a potential memory leak related to listeners.
  • Both sides: added support for IPv6.
  • Created an official benchmark program to track Py4J speed. The results are available as a Google sheet and charts.
  • Eclipse: Replaced “Eclipse-BuddyPolicy: global” by “DynamicImport-Package: *” for greater compatibility with other OSGi frameworks. Thanks to @scottslewis and @jonahkichwacoders for their work on that.
  • GitHub 0.10.3 milesetone

In parallel to this release, I started testing various alternatives to the text protocol used by Py4J to support faster binary transfers. It is extremely easy to make small mistakes that double or triple the time to perform small operations, but I believe I found a new transport strategy that can significantly speed up large binary transfer while keeping the same performance for other operations.

0.11.0 will thus introduce a new transport mechanism and a non-backward compatible change that will allow Python classes implementing Java interfaces to easily implement equals/toString/hashCode methods. I hope to be able to make a release in two months, but considering the size of the task, this may take longer.

As always, this release has been made possible by the generous contributions from many users. Every bug reports, patches, pull requests, ideas or help on the mailing list is greatly appreciated.

Leave a Reply

Your email address will not be published. Required fields are marked *