Java and C/C++ JNI Application Debugging - The GUI Way
Author: Nick Jancewicz
Date: February 23rd, 2006
Overview
This article details the procedures required to accomplish concurrent debugging of a Java application that calls native C language code in a shared library via the JNI (Java Native Interface). The solution presented utilizes the Eclipse IDE (Integrated Development Environment) Java debugger and the stand-alone Insight C debugger running on a Linux platform. Both debuggers provide the ease of use afforded by a GUI (Graphical User Interface) front end.
How the JNI and the Need for Dual Debuggers Came About
Java is a relatively new programming language that has become very popular for application development due to its object oriented approach, inherent safety and code portability to different platforms. At the same time, the C language has a much longer history than Java and is still widely used for its efficiency, particularly for implementing operating systems and embedded systems where the need for maximal performance far outweighs cost of software development. There is also a large resource of existing, time-proven code written in C and C++ (e.g. just take a look at the wealth of powerful algorithms in the book, "Numerical Recipes in C"). The undeniable advantages of both Java and C/C++, have lead to the wide utilization of "mixed" Java and C/C++ languages in many applications.
Realizing that Java could not replace the large body of existing code written in other languages, Sun wisely created the Java Native Interface, or JNI, in order to provide a standardized way for Java applications to utilize a variety of OS (Operating System) API's (Applications Program Interface) and libraries that were mainly geared toward C/C++ code. Thus, JNI is the key to Java's adaptability to many different platforms and to providing special niche functionality that Java libraries are lacking. However, just as early C++ compiler C-Front technology presented a challenge for existing debuggers in the 1980's because they could only handle C and assembly language, Java's JNI has created a similar problem for debugger technology in more recent years because they weren't designed to handle Java. Today many Java programmers are faced with the dilemma of how to debug programmatic interactions with the C/C++ code libraries and/or OS API's that their applications depend upon, because Sun's JDB (Java Debugger) and other Java debuggers only provide a view into the Java code, not the underlying C/C++ code. Thus, the need for using two debuggers came about—one for the Java code and another for the C/C++ code.
Command-Line Debuggers vs. GUI Debuggers
One article by Matthew White shows how to use the JDB and GDB command-line debuggers to debug a mixed-language Java and C program utilizing the JNI. However, today most programmers have come to expect GUI-based development tools and debuggers that permit more information to be made automatically available in a single glance. For example, by automatically showing all local variables for a given function or method context, developers can more efficiently debug code written by others since they don't need to remember the exact spellings of variable names in order to inspect their current values.
A Partial Eclipse Solution
Eclipse is an open source IDE that provides GUI debugger features mentioned above, as well as a sophisticated code-editing environment. Originally it was intended as a Java programming environment. However, the Eclipse architecture is highly extensible, and more recent add-ons, such as the CDT (C/C++ Development Tool) plug-ins, have added development support for other languages. Eclipse organizes IDE functions into so-called "perspectives". A perspective is a collection of related windows and screen objects for a particular development activity, one of which is for debugging, called the Debug Perspective.
The combination of the Eclipse Java Debug Perspective and the CDT Debug Perspective seemed like the ideal solution for working with a Java program utilizing C code through the JNI. Unfortunately, while the Java debugger integrated into the Eclipse IDE works quite well, the C/C++ Debug Perspective is really just a GUI "front end" for the open-source command-line driven GDB (Gnu Debugger). Although CDT uses the GDB MI (machine interface) protocol that is supposed to provide higher reliability than the older CLI (human command line interface), there are difficulties when attempting to debug a shared C/C++ library that is loaded and called from the Java code. The version 3.0.1 CDT plug-in's debug perspective almost works for this purpose but fails when attempting to set a breakpoint in a function contained in the library. Running Eclipse on Mandriva Linux, I found that the IDE was issuing the attach command redundantly, which may be related to the failure to later set a breakpoint (i.e. GDB complained it couldn't access the app's memory for setting the breakpoint).
The CDT plug-in is most often used for debugging stand-alone executables rather than shared libraries called from a Java app. It is more complicated for a debugger to deal with shared libraries because a library must be dynamically loaded into the application's process space before breakpoints can be set in the code. Some debuggers can defer setting breakpoints in a shared library until after the library code is loaded into the program address space but the 3.0.1 CDT debugger is not able to do this
successfully. That's why another approach is necessary.
A GUI Solution that Works
Since I was working under a deadline, I decided to simply use the stand-alone Insight debugger instead of trying to debug the CDT debugger interface plug-in. Unlike other debuggers such as DDD (Dynamic Data Display) and the Eclipse CDT, Insight is not just a GUI front-end to GDB. Insight and the GDB source code are compiled into a single, stand-alone program. Apparently this made a difference compared to the separate GUI and debugger approach since the breakpoint-setting problem above was not manifested by Insight. It worked well in concert with the Eclipse Java debugger, subject to a few simple rules. Figure 1 shows a snapshot of just such a debugging session with Eclipse and Insight. The rest of this paper describes the steps I took in order to debug a combined Java and C application using these tools.
How to Make Two Debuggers Cooperate
The concurrent debugging of one process shared by two separate debuggers requires careful synchronization between the two debuggers in order to avoid causing Java debugger timeout exceptions in Eclipse. When you launch a Java JNI application with java.exe, the OS creates a process containing the Java Virtual Machine (JVM). The JVM interprets and executes the Java byte-code generated from the previously compiled classes that comprise the Java program. When the JVM instantiates a Java class containing the System.loadLibrary() method, it loads the specified native C library code into its memory address space to make the functions it contains available for invocation by the Java code.
To use the Eclipse Java debugger and the Insight C debugger at the same time, you need to ensure that Insight never has the JVM process halted when the Eclipse debugger expects the JVM to respond to a command e.g. when single-stepping through the Java code or when continuing execution of the Java app from a halted state.
The basic procedure for debugging this way consists of five steps. These steps are explained in more detail under the Five-step Debugging Procedure toward the end of this article:
- Launch the JVM that will run the Java application.
- Launch the Eclipse IDE, load the JNI Java project and set a breakpoint in the Java code at a line after the native library is loaded.
- From Eclipse, run the Java application until it hits the breakpoint.
- Launch the Insight debugger from a command shell, attach it to the Java application's JVM and set a breakpoint in the native C code.
- Resume running the Java code and when it reaches the point where the native C code is called, the breakpoint set in step 4 will be hit and you can debug it as desired.
This is a straight-forward process with the help of a couple of Bash shell scripts. Most of the effort for starting a debug session like this is in the preparatory work, which I'll explain next.
Shell Scripts for Iterative Development Cycles
In my case, the initial "prep" work consisted of installing the Eclipse IDE and Insight debugger and creating a project and debug configuration for the JNI app. I also created two Bash shell scripts to streamline the debug procedure:
jvm – This script invokes the JNI app under a JVM to facilitate debug connections to Eclipse and Insight.
debug – This script invokes the Insight debugger and attaches it to the JVM.
The jvm script simply supplies the appropriate command-line arguments to the JVM to make it halt and wait for a connection on TCP/IP port 8000 from the Eclipse Java debugger before it runs the program. See Listing 1 below.
The debug script finds the process ID of the JVM and creates a corresponding Insight debugger command for automatically attaching the debugger to the JVM process. This script also specifies the function name for setting the breakpoint in the C code. See Listing 2.
Since the native C code that is called by the Java program through the JNI is contained in a shared (or dynamically loaded) library, it must first be loaded into the process space of the JVM before it can be called. The Java System.loadLibrary() method does this. If the call to System.loadLibrary() is contained in a static class initializer method, then it will be called when the class containing it is instantiated. In that case you can simply set the breakpoint with Eclipse (for step 2 listed above) on the first line of Java code after the class declaration.
This way, the native C code is loaded into the JVM's process space immediately so that the Insight debugger may connect and set breakpoints in the native C code before the Java application calls it. This approach takes advantage of the fact that static initializers are executed before the main() method of the Java class code. Alternatively, if the loadLibrary() call is not contained in a static class method, you could set a breakpoint in the System.loadLibrary() class itself and single-step out of it when the native C code is loaded into memory.
You could invoke the Insight debugger and set an initial breakpoint in the C code manually. However, I found that a debug script speeds up debugging by doing this automatically, thus allowing me to focus on my code rather than the details of issuing repetitive debugger commands. By setting the initial breakpoint in the first native C function that gets called, rather than on a specific line number, the script need not change even after you've edited the source code as you eliminate bugs. Once the initial function breakpoint is hit you can easily set multiple breakpoints elsewhere in your code on specific line numbers that do change as a result of iterative Edit/Compile/Debug cycles. In my test code, Java_MyJavaApp_MyCfunc() was the name of the first C function called.
Since this article only describes the "standard" JNI debug process itself, if you don't already have an existing JNI project then I suggest downloading Matthew White's example code for the standard JNI calling method and using that. He also has another example demonstrating the "invocation" JNI calling technique where the language roles are reversed i.e. the C code calls the Java code. Many other JNI resources are available on the web.
Software Installation and Configuration
The first step is to get Eclipse, CDT and Insight installed and configured as follows. This description assumes you installed Eclipse and Insight under /home/myhome, your main Java class is MyJavaApp and your C library is called libMyNativeClibrary.so
Eclipse IDE and CDT Plug-in Software Installation
Extract the eclipse-SDK-3.1.2-linux-qtk.tar.gz onto the eclipse directory under my home directory, /home/myhome. Then extract the files from the org.eclipse.cdt-3.0.1-linux.x86.tar.gz archive into their corresponding directories on the eclipse folder. To run eclipse, change directories to /home/myhome/eclipse and type eclipse.
Insight Debugger Executable Creation and Installation
Many Linux distributions do not include the Insight debugger executable so you may need to download and build it from the source code as follows: Download the source bz2 archive from the Insight web site. Click the insight-6.4.tar.bz2 archive with Konqueror to invoke File Roller to extract the contents to a directory you create e.g. /home/myhome. Alternatively, it can be decompressed with the command, "bunzip.exe insight-6.4.tar.bz2", and extracted with "tar –xf insight-6.4.tar". This will create a directory called "insight-6.4". Create another folder called /home/myhome/insight. From a shell, you can then create the executable with the following commands:
cd /home/myhome/insight
../insight-6.4/configure
make
make install
After these commands finish, the Insight debugger executable should now be available under /usr/local/bin.
Eclipse C Project Creation and Setup
Since we are using the Insight debugger instead of the CDT debug front-end for GDB, this step is not really required for debugging. However, to take advantage of the Eclipse CDT editor and navigator you may want to create a C project as follows. In Eclipse select File>New>Standard Make C Project. Enter a project name (e.g. MyNativeClibrary). Uncheck "Use Default" and browse to the /home/myhome/src/c directory (or the source root directory where your C source and build files are kept). This will make all C related files under this directory become part of the project. Click Next and specify "Local build output from file" under the "Discovery profile options" tab, browse to where your makefile is located and select it. Select the Environment tab and define a New variable called DEBUG. Check "Append to native environment". Assuming you have the automatic build feature enabled, the IDE will then rebuild your library with debug symbols.
Eclipse Java Project Creation and Setup
In my case, I already had an existing project that was being built by using the Ant build utility so all I needed to do is to create a Standard Java project in the Eclipse IDE by: Select File>New>Project and click the "Java Project from Existing Ant Buildfile" wizard and browse to /home/myhome/dev/ and select build.xml. The Project name will automatically be named after your main Java class (e.g. in my case it was MyJavaApp, which I'll refer to in the rest of this article). There are many other ways to create projects in Eclipse, so I recommend reading the Eclipse documentation, CDT FAQ as well as the Eclipse IDE's integral help.
Eclipse Java Run/Debug Configuration
Create an Eclipse Run Configuration for connecting to the JVM that will be started by the jvm script: From Eclipse with MyJavaApp project loaded, select Run>Debug then click "Remote java Application" under Configurations. Leave Project set to MyJavaApp and Connection Type set to Standard (Socket Attach). Under Connection Properties set Host to localhost and Port to 8000. Click "Allow termination of remote JVM".
Application Code Build Procedure for Debugging
Before rebuilding the JNI application, set the DEBUG environmental variable so that debug symbols are included in libMyNativeClibrary.so when your native C shared library is rebuilt. If rebuilding from within the Eclipse IDE, make sure your C library project settings have the DEBUG variable set via Project>Properties>Include Paths and C/C++ Symbols>Add Preprocessor Symbol. Then select Build->Clean.
After rebuilding the JNI app the first time, you may want to verify that the libMyNativeClibrary.so library contains debug symbols by using the following command, which displays all human readable text strings in the shared library file, including the names of the C functions you want to debug:
strings –a libMyNativeClibrary.so
Five-Step Debugging Procedure
- Launch the JVM that will run the app with java.exe, giving the command-line switches required to allow "remote" debugging via a TCP/IP socket: From the directory where you usually launch your Java app, run the "jvm" script to launch the JNI Java app under a JVM in remote debug mode.
- Launch the Eclipse IDE and load the previously created JNI Java project: From the directory where Eclipse is installed, launch Eclipse by typing "eclipse". Then select the JNI project, view the Java source module and set a breakpoint (e.g. in MyJavaApp class) at a point just after the application calls System.loadLibrary(libMyNativeClibrary) to load your native C code into the JVM's process space.
- Run the Eclipse Java application and allow it to execute up to the breakpoint. At this point the Eclipse debugger will have the Java app halted and the shared library will be loaded into the JVM's process address space.
- While the Eclipse debugger has the Java app's code halted (although the JVM itself is still running), launch the Insight debugger from a command shell and attach it to the Java application's JVM. When Insight attaches to the JVM, the JVM itself will come to a halt. In order to reach the point in the native C code that we want to debug, we need to set a breakpoint there with Insight and then resume executing the JVM. This is done as follows:
From the directory containing libMyNativeClibrary.so, run the"debug" script that launches the Insight debugger and passes it the debugger commands to attach it to the JVM and to set the breakpoint. A debugger message will pop up saying 'Could not load vsyscall page because no executable was specified try using the "file" command first'. This is not a problem, since we are debugging a shared library (vs. a stand-alone executable), so click the OK button to dismiss the pop-up.
- Now that Insight has the breakpoint set in the native C code, resume execution from the Eclipse IDE of the Java code that calls it so that the Java code will reach the point where the C code is called, the breakpoint set in step 4 will be hit and Insight can then debug the library as desired: From the Java debug session in Eclipse, click the "|>" continue icon to resume execution. After the Java app calls the native C Java_MyJavaApp_MyCfunc() function, the breakpoint will be hit and Insight's Source Window will appear showing the first line of the Java_MyJavaApp_MyCfunc() function highlighted in green. The C code can now be debugged by viewing local variables (View->Local Variables), single-stepping (Control->Step or Control->Next), and setting breakpoints (click the left column of the source line).
As an alternative to using the drop-down menus, Insight's Toolbar icons or single-letter commands can be used. You can also set breakpoints from the GDB debugger console (View->Console) by using the GDB break command e.g. "break function_name" or "break module_name:#" where "#" is the source code line number. When done debugging the C code, continue execution (Control->Continue). This way, the JVM will be free to keep running so you can continue debugging the Java side under Eclipse. The Eclipse debug session can be ended by clicking the red box icon.
Some day, perhaps there will be a Universal Debugger that can debug code written in multiple languages. Until that day comes, the dual debugger approach described in this article should let you debug bilingual applications that utilize both Java and C/C++.
Resources
The Java Native Interface Programmer's Guide and Specification by Sheng Liang
http://java.sun.com/docs/books/jni/download/jni.pdf
Debugging Integrated Java and C/C++ Code by Matthew White
http://www-128.ibm.com/developerworks/java/library/j-jnidebug/index.html?dwzone=java
Insight Debugger
http://sourceware.org/insight/
Eclipse IDE
http://www.eclipse.org/
Eclipse CDT (C/C++ Development Tools) http://www.eclipse.org/cdt/
Eclipse CDT FAQ (recommended) http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/cdt-home/user/faq.html?cvsroot=Tools_Project#debug_80
Code Listings
Listing 1:
Shell script file, jvm, for starting the JVM with the Java app initially halted while the JVM listens for a "remote" debugger connection via TCP/IP on port 8000 on the local host computer. The specific command-line switches that do this are: -Xdebug, which enables the JVM debugger; -Xnoagent, which disables the Java JIT (Just In Time) compiler; and the –Xrunjdwp: option which gives the connection type (TCP/IP rather than shared memory) and the port number, which you might need to change depending on any TCP/IP port restrictions that may apply to your network.
java -Xmx400m -Xms400m -Xdebug -Xnoagent \
-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=localhost:8000 \
-Djava.library.path=./lib -cp ./classes:./lib/ij.jar MyJavaApp
Listing 2:
Shell script file, debug, for starting the Insight debugger, attaching to the JVM and setting an initial breakpoint in the Java_MyJavaApp_MyCfunc function in the native C shared library that the Java app running on JVM calls. This works by using the ps (process status) command to list all processes and pipes that through an awk filter that selects only that process started by the "java –Xdebug" command (i.e. with the jvm script above). It then inserts that process ID into a GDB "attach" command piped into a GDB/Insight command file, breakpoint.cmd. It also appends a GDB/Insight break and continue command. In the final line, it invokes the Insight debugger, passing it the name of the command file just created to be executed upon starting up.
ps -af|awk 'java -Xdebug/ { print "attach " $2 "\nbreak \
Java_MyJavaApp_MyCfunc\ncont" }'>breakpoint.cmd
insight -x breakpoint.cmd
If you have any questions or comments, please email me at 
|
