Modifications to Jython
To make this work, some modifications to Jython were required.
If activated, all Jython-style finalizers are not directly processed, but cached for a moment and processed right after the ordinary finalization process.
Rather similar to FORCE_DELAYED_FINALIZATION, but delays callbacks of Jython-style weak references.
JyNI always activates FORCE_DELAYED_WEAKREF_CALLBACKS and if there are native objects that can potentially cause a PyObject resurrection (JyNI-GC needs this sometimes), JyNI also activates FORCE_DELAYED_FINALIZATION.
FORCE_DELAYED_WEAKREF_CALLBACKS allows to restore weak references pointing to the resurrected object. This is done in a thread-safe manner and if someone calls a weak-refs get-method while the weakref is in a pending state it blocks until it was restored or finally cleared.
FORCE_DELAYED_FINALIZATION allows JyNI to prevent Jython-style finalizers from running, in case their objects were resurrected subsequently to an object-resurrection by JyNI.
This way the object-resurrection can be performed without any notable impact on Jython-level. (Raw Java-weak references would still break and also Java-finalizers would run too early, which is why PyObjects must not implement raw Java-finalizers.)
When working with Jython I took the opportunity and unified Py.EmptyTuple and PyTuple.EMPTY_TUPLE. These were two singleton-constants for the same purpose. JyNI also has a native counterpart constant for empty tuples, but until now it was not clear to which of the named Jython constants it should be bound.
JyNI's dependence on these features implies that JyNI requires Jython >2.7.0 from now on. I aim to sync JyNI 2.7-alpha3-release with Jython 2.7.1, so that JyNI's >2.7.0-requirement is fulfillable.
The more advanced GC-behavior is tested in test_JyNI_gc.py.
test_gc_list_modify_update demonstrates the case where the native reference graph is modified by an object that properly reports the modification to JyNI.
To test the edgy case of gc with silently modified native reference graph I added the listSetIndex-method to DemoExtension. This method modifies native reference graph without reporting it to JyNI. test_gc_list_modify_silent verifies that JyNI properly detects this issue and performs resurrection as needed.
Further it tests that Jython-style weak references that point to the resurrected object stay valid.
JyNI-support for weak references
Support for weak references is implemented, but not yet fully stable. Tests are needed and more debugging work. For now the code is included in JyNI, but not active. I will follow up on this -like on ctypes- as soon as possible.