As of middle of November 2015 I finally got some basic ctypes support working, after solving manifold issues. Now I know that ctypes support would not have been entirely feasible within the original GSoC period as it involved significant work.
A major improvement that was needed is support for subtypes. This required massive work in typeobject.c.
Also some more bridges for dict-API were needed, which is always tedious with Jython due to the split of PyDictionary type into PyDictionary.class and StringMap.class. Finally I took hand on Jython itself and unified these two under a common abstract class AbstractDict.class, which was a long-time outstanding design improvement. Now the bridge-code regarding PyDictionary is much simpler.
ThreadState API also had to be improved, wich required changes in pystate.c, i.e. the PyGILState_xxx method family is now supported in a Jython consistent way. The implementation takes care to attach or detach involved threads to the JVM as needed, i.e. use JNI methods AttachCurrentThread and DetachCurrentThread properly. The way how JyNI handles ThreadState was mostly redesigned.
Along with these changes came lots of subtle bugfixes, also regarding GC-support. Also some more API in abstract.c became implemented.
Temporarily required: Custom ctypes/__init__.py
Unfortunately it turned out that ctypes support is currently not feasible without providing a custom implementation of ctypes/__init__.py. Main reason for this is that the module sometimes checks for the current platform by looking at os.name. This has the value “java” in Jython-case, which is an unrecognized case in ctypes. I fixed this for now like this:
Original check:
if os.name == "posix":
New variant:
isPosix = os.name == "posix"
if os.name == "java":
from JyNI import JyNI
isPosix = JyNI.isPosix()
if isPosix:
This sort of issue cannot/should not be “fixed” in Jython, because “java” is actually the right value and changing it might break existing code. Rather ctypes code is invalid for this case and should be fixed to provide a proper behavior for os.name == "java" (i.e. determine platform in a Jython/JyNI compliant way). Maybe I will propose this once ctypes support in JyNI/Jython is more established.
Another current issue is that JyNI does not yet support new-style classes, which I worked around for now by changing new-style classes defined in ctypes/__init__.py to be old-style. This change can be reverted once new-style support is added to JyNI.
The consistency checks
_check_size(c_long)
_check_size(c_ulong)
had to be removed for now, because in Jython the struct module has currently hard-coded values for native primitive type sizes. This means some values are likely to be wrong for the actual platform, which was never much of an issue for the Jython-world without native extensions. This can be fixed e.g. using JNR to init org.python.modules.struct.native_table properly.
Results
It is now possible with activated JyNI to load original ctypes like bundeled with CPython. On posix systems this is possible by adding the following import path:
sys.path.append('/usr/lib/python2.7/lib-dynload')
This means JyNI can use the original _ctypes.so file in binary compatible fashion. (As said before the only overridden file is ctypes/__init__.py).
Now we can import ctypes and load e.g. libc.so.6 (libc.dylib on OSX):
import ctypes
import platform
if platform.java_ver()[-1][0] == 'Mac OS X' or platform.mac_ver()[0] != '':
# We're on OSX
libc = ctypes.CDLL('libc.dylib')
else:
libc = ctypes.CDLL('libc.so.6')
We can do simple function calls:
print libc.strlen("abcdef")
printf = libc.printf
printf("%d bottles of beer\n", 42)
Also using an object:
class Bottles:
def __init__(self, number):
self._as_parameter_ = number
printf("%d bottles of beer\n", Bottles(73))
Provide some type-checking using argtypes:
from ctypes import c_char_p, c_int, c_double
printf.argtypes = [c_char_p, c_char_p, c_int]
printf("String '%s', Int %d\n", "Hi", 22)
Use ctypes-pointers:
from ctypes import *
class cell(Structure):
pass
cell._fields_ = [("name", c_char_p), ("next", POINTER(cell))]
c1 = cell()
c1.name = "foo"
c2 = cell()
c2.name = "bar"
c1.next = pointer(c2)
c2.next = pointer(c1)
p = c1
print p.next[0]
for i in range(8):
print p.name,
p = p.next[0]
print ''
And also use Python-written callback functions:
IntArray6 = c_int * 6
ia = IntArray6(5, 1, 7, 33, 99, -7)
qsort = libc.qsort
def py_cmp_func(a, b):
print "py_cmp_func", a[0], b[0]
return a[0] - b[0]
CMPFUNC = CFUNCTYPE(c_int, POINTER(c_int), POINTER(c_int))
cmp_func = CMPFUNC(py_cmp_func)
qsort(ia, len(ia), sizeof(c_int), cmp_func)
for i in range(len(ia)):
print ia[i],
Output (of this part) with Debian's qsort implementation:
py_cmp_func 1 7
py_cmp_func 5 1
py_cmp_func 5 7
py_cmp_func 99 -7
py_cmp_func 33 -7
py_cmp_func 33 99
py_cmp_func 1 -7
py_cmp_func 1 33
py_cmp_func 5 33
py_cmp_func 7 33
-7 1 5 7 33 99
Despite this demonstration, unfortunately the majority of ctype's unit tests currently fails, however mostly due to secondary issues like lack of buffer protocol or new-style class support. However I plan to release the current state under the label “JyNI 2.7-alpha.3” right after Jython 2.7.1 beta 3 is released in near future (was originally planned for end of November 2015, but was delayed because of several blocking issues). It would not make sense to release JyNI before next Jython release, because it depends on Jython's current repository version.
