2015-05-22

Getting started...

So I am at Gsoc now and really happy that everything worked out so well. Huge thanks to Jim Baker for making this possible!


Now let me introduce you to my project. I am the creator of the JyNI-project, see www.jyni.org for more details. Gsoc sponsors the development of the next important milestone: GC-support and hopefully also support for the ctypes-extension. But what am I talking here... just take look at my Gsoc-abstract:


JyNI is a compatibility layer with the goal to enable Jython to use native CPython extensions like NumPy or SciPy. It already supports a fair part of Python's C-API and is for instance capable of running basic TKinter code (currently linux-only). However, a main show-stopper for a production-release is its lack of garbage collection. The gap between Jython- (i.e. Java-)gc and CPython-gc goes far beyond the difference between reference-counting- and mark-and-sweep-based approaches. Even more important is the philosophy regarding gc in native interfaces. While CPython exposes its gc-mechanism in native API, allowing native extensions to leverage it by following some instructions (i.e. perform reference counting and gc-registration), Java's native interface (JNI) leaves memory management completely to the native code. As a preparation for this proposal I worked out a concept how native gc can be emulated for JyNI in a way that is (almost 100%) fully compatible with CPython's native gc. That means a native extension written for CPython would run without modification on JyNI, having gc-emulation behave consistently to Jython gc. This includes weak referencing, finalization and object resurrection (aspects that are famously known to make clean gc support so hard). JyNI-support for weak references includes providing the PyWeakReference builtin-type, which is currently the main show-stopper to support the original native ctypes-extension. Thus, as an optional/secondary goal I propose to complete JyNI-support of the ctypes-extension. This is proposed softly, under the assumption that no further, still unrecognized hard show-stoppers for that extension come up.



In a few days bonding period will be over and things will get serious. Since I did not have to learn much Jython- and JyNI-internals any more, I could use this period to close some open endings in JyNI which were actually unrelated to the Gsoc project, but -however- having this done feels much better now. I also fixed two related Jython-issues. Finally I reasoned about the Gsoc-project and think it would be a crucial debugging-feature if JyNI could dump a memory-allocation history of all its native references at any time (with Java GC-runs in time-line please!). To achieve this, I am currently writing the ReferenceMonitor-class, which will also expose this information in a way that allows to write gc-related unittests. (As this monitoring functionality was not considered in the timeline, it would be good to get it done before coding officially starts.)

Recently Jim Baker - mentor of this project - managed to get two JRuby developers into an email-discussion with us and we gathered some interesting details how JRuby handles/plans to handle C-extension API. Thanks for this interesting thread!