static linking

**Nicholas Jordan** · 03-05-2009, 02:06 PM

{ message edit:

Java Code:

// Error 42: Symbol Undefined _UnlockData
zarf Zarf=new zarf("test_run.dat"));
try
{
  if(Zarf.imageUpdate())
   { 
       java.lang.System.out.println(new java.lang.String("returning from main ...."));//
       return;
   }
 }
}
catch(java.lang.Throwable t)
{
    java.lang.System.out.println(new java.lang.String("load failed ...."));//
    java.lang.System.out.println(t.getMessage());
    return;
}
.... prints:   zarfOCR.dll: %1 is not a valid Win32 application

completely re-wote zarfOCR.def and all the linker switches since I posted this morning

Had to run a tool on the import lib to get it in the format expected by compiler, the above link error is the only thing left before maybe it will load.

}

Originally Posted by toadaly

You should be casting imageMemoryReference (which should be a native array type such as signed char*) to a jbyte*, not a _jbyteArray. Also, don't forget to call ReleaseByteArrayElements at some point, so that the changes you made will be reflected on the java side. If you fail to do that, you'll have a heck of a time trying to figure out why the byte[] on the java side is always full of 0's.

made suggested change of making the cast to a pointer to an object type - rather than object without the "*" - still working on how to do the pointers as there are linker errors I need to find. I will be working on that momentarily, how do we do pointers? can I just declare what I want to be linked as 32-bit flat address space as being void* and just do casting and so on as need be? ( I know about putting the pointer back at the beginning of the offset - simple, obvious pre-intro stuff for a mechanic )

Java Code:

static jmethodID DataBufferByteConstructor=NULL;
// 2
jclass localRefzarfObject=(*JEnvironmentPointer)->GetObjectClass(JEnvironmentPointer, jayObject);
//
jclass localRefDataBufferIntClass =(*JEnvironmentPointer)->FindClass(JEnvironmentPointer,"[I");//
//
jclass localRefDataBufferByteClass =(*JEnvironmentPointer)->FindClass(JEnvironmentPointer,"Ljava/awt/image/DataBufferByte;");//
DataBufferByteConstructor=(*JEnvironmentPointer)->GetMethodID(JEnvironmentPointer,localRefDataBufferByteClass,"<init>","([B,I)V");
//
jfieldID localRefzarfMemberVar_1=(*JEnvironmentPointer)->GetStaticFieldID(JEnvironmentPointer,localRefzarfObject,"availablity","I");
//
jfieldID localRefzarfMemberVar_2=(*JEnvironmentPointer)->GetStaticFieldID(JEnvironmentPointer,localRefzarfObject,"dataLength" ,"I");
//
jfieldID localRefzarfMemberVar_3=(*JEnvironmentPointer)->GetStaticFieldID(JEnvironmentPointer,localRefzarfObject,"imageX","I");
//
jfieldID localRefzarfMemberVar_4=(*JEnvironmentPointer)->GetStaticFieldID(JEnvironmentPointer,localRefzarfObject,"imageY" ,"I");
//
jfieldID localRefzarfMemberVar_5=(*JEnvironmentPointer)->GetStaticFieldID(JEnvironmentPointer,localRefzarfObject,"dataBlock","[I");//private static volatile int [] dataBlock=new int[(64 / 4)];//localRefzarfMemberVar_5
//
jfieldID localRefzarfMemberVar_6=(*JEnvironmentPointer)->GetFieldID(JEnvironmentPointer,localRefzarfObject,"byteArray","[B");
//
fieldID localRefzarfMemberVar_7=(*JEnvironmentPointer)->GetFieldID(JEnvironmentPointer,localRefzarfObject,"dataBufferByte","Ljava/awt/image/DataBufferByte;");
//set some fields"
(*JEnvironmentPointer)->SetStaticIntField(JEnvironmentPointer,localRefzarfObject,localRefzarfMemberVar_1,availability);//
//set some fields"
(*JEnvironmentPointer)->SetStaticIntField(JEnvironmentPointer,localRefzarfObject,localRefzarfMemberVar_2,image_size);//
//set some fields"
(*JEnvironmentPointer)->SetStaticIntField(JEnvironmentPointer,localRefzarfObject,localRefzarfMemberVar_3,image_width);//
//set some fields"
(*JEnvironmentPointer)->SetStaticIntField(JEnvironmentPointer,localRefzarfObject,localRefzarfMemberVar_4,image_height);//
////****////****////
jintArray StaticIntArrayField=(jintArray)(*JEnvironmentPointer)->GetStaticObjectField(JEnvironmentPointer,localRefzarfObject,localRefzarfMemberVar_5);
jbyteArray StaticByteArrayField=(jbyteArray)(*JEnvironmentPointer)->GetStaticObjectField(JEnvironmentPointer,localRefzarfObject,localRefzarfMemberVar_6);
jsize StaticIntArraySize=(jsize)(*JEnvironmentPointer)->GetArrayLength(JEnvironmentPointer,StaticIntArrayField);
jsize StaticByteArraySize=(jsize)(*JEnvironmentPointer)->GetArrayLength(JEnvironmentPointer,StaticByteArrayField);
jboolean isCopyPointer='\0';//
void * StaticIntArrayFieldPointer=(*JEnvironmentPointer)->GetPrimitiveArrayCritical((JNIEnv*)JEnvironmentPointer,(jarray)StaticIntArrayField,(jboolean*)&isCopyPointer);// jboolean
// - check for null pointers
if((StaticIntArrayFieldPointer != (void *) NULL)&&(buffer64 != (int *)NULL))
{
memcpy((void *)StaticIntArrayFieldPointer,(void*)buffer64,StaticIntArraySize);
(*JEnvironmentPointer)->ReleasePrimitiveArrayCritical((JNIEnv *)(JEnvironmentPointer),(jarray*)StaticIntArrayFieldPointer,(void*)buffer64,(jint)0x00000000);
// ... wow,
}
else
{
return (jbyteArray) imageMemoryReference;
}
////****////****////
void * StaticByteArrayFieldPointer=(*JEnvironmentPointer)->GetPrimitiveArrayCritical((JNIEnv*)JEnvironmentPointer,(jarray*)StaticByteArrayField,(jboolean * )isCopyPointer);// jboolean
// - check for null pointers
if((StaticByteArrayFieldPointer != (void *) NULL)&&(buffer64 != (int *)NULL))
{
memcpy((void *)StaticByteArrayFieldPointer,(void*)imageMemoryReference,StaticByteArraySize);
(*JEnvironmentPointer)->ReleasePrimitiveArrayCritical((JNIEnv *)(JEnvironmentPointer),(jarray)StaticByteArrayFieldPointer,(void*)buffer64,(jint)0x00000000);
// ... wow,
}
else
{
return (jbyteArray) imageMemoryReference;
}
// Run constructor on dataBufferByte
////****////****////
jobject CdataBufferByte=(*JEnvironmentPointer)->AllocObject((JNIEnv *)JEnvironmentPointer,(jclass)localRefDataBufferByteClass);
jmethodID dataBufferByteConstructorID=(jmethodID)(*JEnvironmentPointer)->GetMethodID(JEnvironmentPointer,CdataBufferByte,"<init>","([B,I)V");
(*JEnvironmentPointer)->CallVoidMethod(JEnvironmentPointer,CdataBufferByte,dataBufferByteConstructorID,(jbyteArray)imageMemoryReference,(jsize)image_size);
(*JEnvironmentPointer)->SetStaticObjectField(JEnvironmentPointer,CdataBufferByte,localRefzarfMemberVar_7,CdataBufferByte);//
// end   image ops
return (jbyteArray)imageMemoryReference;

**toadaly** · 03-06-2009, 12:59 AM

Wow. You're a brave man. I would never have gotten that far without getting it to compile and run.

At this point, I would suggest writing the simplest possible hello world style c (or c++) application that does nothing but load the zarf dll. Make sure it compiles and runs. Then try the simplest possible method call into zarf. Maybe something like TWAIN_EasyVersion (I assume you're using eztwain ?).

Next, write the simplest possible hello world style JNI app that does nothing but return an int. Make sure it compiles and runs.

Then put these two together by having your jni call return the version number for eztwain (or whatever you're using).

If you can get that to compile and run, doing the rest is just more of the same.

**OrangeDog** · 03-08-2009, 12:26 AM

Nic, I'd love to try and help but this last post is practically indecipherable. I'm sure elephants and login shells etc. are completely irrelevant. Please try and keep to the point and avoid your industry-specific jargon.
If I am correct in understanding, this is your first JNI app? Try writing something simple before you try to solve all your problems in one massive code splurge.

**Nicholas Jordan** · 03-08-2009, 04:47 PM

Originally Posted by OrangeDog

.....If I am correct in understanding, this is your first JNI app? Try writing something simple before you try to solve all your problems in one massive code splurge.

Correct.

I deleted the post, I do not have any choice - I have to get this thing running.

Java Code:

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x28282828
// load address for EZTW32.dll
0x10000000 - 0x10010000 EZTW32.dll
// load address for zarfOCR.dll
0x0bcb0000 - 0x0bcc7000 EZTW32.dll

Can I put some printf's in the dll to see what is happening? Maybe a pusha and examine some values by GetProcAddress? ....

I got about three asm editors overnight, trying to get something to compile. I ended up removing a call that would not resolve in LibMain, 0x28282828 should actually point to

Java Code:

0x10000000 - 0x10010000 EZTW32.dll

in EZTW32.dll

I have in zarfDLL.c:

Java Code:

static int availability = (-1);

in LibMain

Java Code:

availability=TWAIN_IsAvailable();

Then in:

Java Code:

private native int    zarf_IsAvailable();

in zarfDLL.c does a check first thing:

Java Code:

if(availability > 0x00000000)

before it could possibly get to

Java Code:

TWAIN_AcquireNative(0x00000000,TWAIN_GRAY);//

Dump:

Java Code:

Registers:
EAX=0x0003657c, EBX=0x0bcb934c, ECX=0x03a7b138, EDX=0x03a7b168
ESP=0x0007f9c4, EBP=0x0007f9fc, ESI=0x000358c0, EDI=0x00036574
EIP=0x28282828, EFLAGS=0x00010202

I had to pull apart the object file with a dumper to see what assembly was being generated, I can look at that and probably get some idea how to see what to do, can you help me read this?

Java Code:

_Java_ocr_zarf_zarf_1AcquireNative@8:
		enter	080h,0
		push	EBX
		push	ESI
		push	EDI
;        {
;            //
		cmp	dword ptr FLAT:_DATA[018h],0
		jle	LDD3
;            if(availability > 0x00000000)
;            {
;                //  (void *) - cast removed for prototyping test
		push	2
		push	0
		call	dword ptr __imp__TWAIN_AcquireNative@8
		mov	-080h[EBP],EAX
;                HANDLE twainNativRef=TWAIN_AcquireNative(0x00000000,TWAIN_GRAY);//0x0002
		test	EAX,EAX
		je	LDDC
;                if(twainNativRef != ( HANDLE ) 0x00000000 )    // 0 == NULL == no entry
;                {
		push	0100h
		call	near ptr _malloc
		mov	FLAT:_DATA[0294h],EAX
;                    buffer64= (int *) malloc(64 * sizeof(int));//
		add	ESP,4
		test	EAX,EAX
		je	LDDC
;                    if( buffer64 != (int*)0x00000000)
;                    {

Walter's up - I looked in all the libs for UnlockData, which is the call in LibMain to make segments moveable, it has not shown up in anything I can find - may be built in as a compiler intrinsic - somehow the call to UnlockData is not external, nor will it link.....

I could do some copy ops with near pointers for some of the work but the image that will be coming in is

Java Code:

//C:\Documents and Settings\Owner\My Documents\Reports\sample_ocr_raw_scan.bmp - 493 KB (505,070 bytes)

so I'm up aginst the wall here.....

Hope deleting the post gains some redirection to the code ......

**Nicholas Jordan** · 03-08-2009, 10:25 PM

I know I am being a pest, but does this tell that I built several source files in to one loadable dll?

Java Code:

link EZTwain+zarfDLL,zarfOCR.dll,,kernel32+EZTW32+kernel32+EZTW32,zarfOCR/noi/DELEXECUTABLE/DELEXECUTABLE;

I was getting some really incorrect offsets into the loaded dll - to get this, I had to tweak EZTwain.c in several places...one #pragma push() / pop() and two casts to NULL so as to not get the un-intended assignment warning.

I did get one 'c' side call to return, not shabby for tabby ,....

**Nicholas Jordan** · 03-09-2009, 05:15 PM

Originally Posted by toadaly

Wow. You're a brave man. I would never have gotten that far without getting it to compile and run....

See attached prototyping result, I am at line 148 - several calls deep in bringing the dib in from eztwain - yesterday I had the JVM trying to do a read on an address from the

Java Code:

/**
 * Rijndael --pronounced Reindaal-- is a variable block-size (128-, 192- and
 * 256-bit), variable key-size (128-, 192- and 256-bit) symmetric cipher.*/

lookup table, today I get what may be an exception from my java code or could be calling JNI incorrectly on the "c" side.......

Can we skip the stuff that the parser-lexer removes that we would get from today's marching army of glass-eyed moaners and get on with it, as Lector would ask Clarice ,,,,....

What does "todaly" convey, in original form as the moniker came to monkey with your mind?

Like ftr says, this stuff is not hard.

More like what Ivor Horton says, ".... it's just that there's so much of it."

**Nicholas Jordan** · 03-10-2009, 01:34 AM

It's not that pointer arithemtic is challenging, it is that the compiler challenges you with a bunch of impediments to impeachment - let's get peachy:

Java Code:

zarfOCR.dll loaded ...
entering this.state == true
image_buffer_address
3110400
value of availability:  
1
value of image_width:  
1598
value of image_height:  
1944
value of image_planes:  
1
value of image_bit_count:  
8
value of image_compression:  
0
value of Pels_Per_Meter_X:  
7874
value of Pels_Per_Meter_Y:  
7874
Buffered Image loaded.
returning from main ....

Most of which was obtained from this supposedly difficult JNI - which I have to admit is not for the tame. As usual, memory ops are hidden behind a small, star looking symbol - and I cannot get the bye[] extracted from the dib.

This will be extracting bytes as data from a GlobalAlloc retuning a standard dib thus:

Java Code:

PBITMAPINFOHEADER pBitmapInfoPointer=(PBITMAPINFOHEADER)bitMapHandle;

known to contain standard 8-bit bytes as uncompressed data in the simplest possible format - that is where the proprietary gets to take the cake. Any sort of compression or fancy tools.

I need to extract the data and do:

Java Code:

(*JEnvironmentPointer)->SetIntArrayRegion

I would show the code but it is too blunderous,....

Should be really simple, maybe I should read Ivor Horton or even Herbert Schildt ....

**toadaly** · 03-10-2009, 06:33 AM

Nick, are you compiling with a c compiler or a c++ compiler? If c, putting

Java Code:

extern "C" {

...everything you write...

}

May fix some of these issues.

**Nicholas Jordan** · 03-10-2009, 02:17 PM

I stripped out the extern "C" and ( I think ) am writing in c not cpp - I had set a compiler switch but "-c" but that was something else { the switch "-cpp" turns on cpp for non cpp extensions ... I named all compilation units filename.c which is mapped to the c compiler in the build shell } I will be working on the dib stuff exclusively today, as soon as I get that I can get into ai/ocr as I think the DataBufferByte constructor is running on the c side

Java Code:

////****////****////
long const offset_to_image=(header_size+256);
//
unsigned char * base__bitMapHandle=(unsigned char *)pBitmapInfoPointer;
if(base__bitMapHandle != NULL )
{
signed char const * base_of_copy=(signed char const *)( base__bitMapHandle + header_size + 256 );
jsize base_1=image_size;//
jsize jayImageSize=(long)base__bitMapHandle;//
jbyteArray jayByteArray=(*JEnvironmentPointer)->NewByteArray(JEnvironmentPointer,jayImageSize);
(*JEnvironmentPointer)->SetByteArrayRegion(JEnvironmentPointer,jayByteArray,jayZero,jayImageSize,base_of_copy);
////****////****////
// Run constructor on dataBufferByte
jclass localRefDataBufferByteClass=(*JEnvironmentPointer)->FindClass(JEnvironmentPointer,"Ljava/awt/image/DataBufferByte;");//
jobject CdataBufferByte=(*JEnvironmentPointer)->AllocObject(JEnvironmentPointer,localRefDataBufferByteClass);
jmethodID dataBufferByteConstructorID=(*JEnvironmentPointer)->GetMethodID(JEnvironmentPointer,localRefDataBufferByteClass,"<init>","([B,I)V");
(*JEnvironmentPointer)->CallVoidMethod(JEnvironmentPointer,CdataBufferByte,dataBufferByteConstructorID,jayByteArray,jayImageSize);
(*JEnvironmentPointer)->SetStaticObjectField(JEnvironmentPointer,localRefDataBufferByteClass ,localRefzarfMemberVar_11,CdataBufferByte);//
// end   image ops
jint const good=0xffff;
return good;
}
else
{
return fail__;
}

I do not have any idea how to detemine the length of the pallete, notably whether it is even there. The header length is given in the dib header, then it is narrowly pointer arithmetic - which is not hard - code archaeology on what the compiler is doing is dicey at best for me.

0x0022566 + offset == where it should point, all I see is pointy head pencils as far as .....

maybe we should name this tool: "Horizons" ( folks, this todaly guy is smart - he will grasp what I am doing with this byte bite ) maybe we could require all source code to be underlined as a tribute to the body of code at the body farm.

**toadaly** · 03-11-2009, 05:22 AM

As far as the size of the pallet goes, there *should* be a method or #define somewhere in one of the libraries you are using. Finding it may be a chore though. The c world doesn't seem to believe in high quality API documentation.

**Nicholas Jordan** · 03-11-2009, 10:19 AM

Yep, + spike just took an interest in what I have so far.

I managed to do the pointer arithmetic by casting to long, doing + 40 + 256 and trying a bench run. With some tweaking to the JNI calls I now have a byte[] showing up on the Java side as non-null and length matching what the dib header shows.

I had to do some serious fix-ups on reported image size as it ( the famous they - who never shows at whuppin time ) seems to have been placing the pointer in the image size field or something.

I did

Java Code:

long const reCalcImageSize=(image_height*(image_width+(image_width % sizeof(int))));

to get some sanity, one of those Strange.os issues seems to be at hand as the image size matches the pointer value - before you dig in let me caution all that runnning a vast world empire ( such as the M$ Yacht - which is straight out of an Ian Fleming novel ) does not run in field engineering as cs is taught in academic facilities. Making a hardware transfer is done however they can get it to work, if that involves some processor fixups - then we may be able to proceed on the basis that it works - leaving the established computer science for the Master's level discussion on data structures.

Ever seen one of those people argue with someone who is actually running an installation?

Who says cat's can't dance?

**Nicholas Jordan** · 03-12-2009, 03:00 AM

Java Code:

TrainingSet.java:40: reference to getPixels is ambiguous, both method getPixels(int,int,int,int,float[]) in java.awt.image.Raster and method getPixels(int,int,int,int,double[]) in java.awt.image.Raster match

Back to same thing, code expects image to be supplied - not examined. ( I think )

{ message edit: I am looking at something from softwarepearls_com that may do it.}

**toadaly** · 03-13-2009, 07:32 AM

Well Nick, it looks like this is starting to fall together for you! I'm glad to see that, and hope we've helped in some way.

**OrangeDog** · 03-13-2009, 01:02 PM

Seriously, you'd get it working a lot quicker if you started from scratch and built it up in simple stages, testing each one as you go. If you didn't even know if you were using C or C++ then you definitely shouldn't be using either. The world of C is strange, complex and deadly compared to the cosy Java life.

As for your disassembled object code, I'd need to know what machine it's for before I can help. Looks like an x86 but all the work is being done on the stack rather than with registers. I seriously advise against trying to solve your problems by hacking around in assembler. Or hacking around in C for that matter.

It's your first go at this kind of system: take it slow and build up or you'll end up buried neck-deep in spaghetti with a bug at every line (which seems to be the case).

**Nicholas Jordan** · 03-14-2009, 02:07 PM

Originally Posted by OrangeDog

Seriously, you'd get it working a lot quicker if you started from scratch and built it up in simple stages, testing each one as you go. If you didn't even know if you were using C or C++ then you definitely shouldn't be using either. The world of C is strange, complex and deadly compared to the cosy Java life.

My first program was a sort of beginner's shift-or key generator, I blew into the copious STL error messages that run for pages. I dug until I discovered strlen having no ( as you are discussing ) Continuing work until transitioning to Java, I grasp what you are saying. If it would assist in the no "__extern__" on one of the calls that we see ( not sure which words to choose to please everybody - obviously you know enough to know where that is appended ) .... would assist in getting the effort to compile, I will try that approach. Treatment of pointers would be built in to the compiler so that would be the way to go.

Originally Posted by OrangeDog

As for your disassembled object code, I'd need to know what machine it's for before I can help. Looks like an x86 but all the work is being done on the stack rather than with registers. I seriously advise against trying to solve your problems by hacking around in assembler. Or hacking around in C for that matter.

I gave up on assembler the next day, sounds like you are saying the whole project will go smoother if I transition to cpp - so the question then would be that

Java Code:

(*EnvPtr)->object
// is coded as 
EnvPtr->object

In cpp?

Originally Posted by OrangeDog

It's your first go at this kind of system: take it slow and build up or you'll end up buried neck-deep in spaghetti with a bug at every line (which seems to be the case).

I get more done that way, good experiental basis for that assertion.

Development target is x-86, I am running on an amd semipron ... what I call a department store computer. iow - go to the nearest store and get a computer. I have a box to hand that is newly installed board by a EE who runs a compter shop nearby. I can develop the enitre system on that box if need be as that is the machine I intend the system to run on anyway.

I had one version going, moved it to backup by renaming the folder and doing copy-ops by the shell to a new folder named the same orginal name ... at which point I started over. Same problems came up, worked them to a clean compile, in figuring it out I ruined the checkshum on the system lib trying to pull the exports. Wiped the disk and started over this morning, go look at "D" and tell me if you think I should just go ahead and transition - I have the installer for that ready to run.

Attached file is from a tool I ran on what I think is the one that is running, renamed to zarfDLL.cod.txt to get the BB code to accept it. I note for what I can grasp of the matter:

Java Code:

_DllMain@12	COMDAT flags=x0 attr=x0 align=x0 
// and 
_LibMain@16	COMDAT flags=x0 attr=x0 align=x0

are showing up in too many builds as being defined already in a lib, there is an issue which I have not commited to memory - a dll is supposed to link against one lib and an exe links against another, a command line switch is supposed to pass the info to the linker....it may be that the one I got running just uses the two switches advised and no other.

The author of the compiler advises:

Avoid using a C++ DLL with a C application. Since C and C++ differ in how they pass parameters and handle function names, it is not easy to call a C++ function from a C program.

Additionally, most of java runs like "C" - you can see in their headers that they have done this plus wiki and others report that CPP style calls into Java are not formalized in a relaible fashion. Also, reading on dll's it one is advised that the dll uses the calling program's stack -

Of critical note on the attached .cod is that UnlockData(0); does not show up on a search in my editor. Another is that those calls that were being reported by the linker do not have a storage specifier that is either __implib__ or __extern__ ( don't know which one ) and as you look at the code there is a long stretch in the middle that looks like:

Java Code:

		mov	ESI,FLAT:_DATA[02B0h]

which is something I put in there as a get-going as the compiler was not accepting my pointer syntax.

Another issue is that sample files on the work I want to do show up as

Java Code:

//3.79 MB (3,975,678 bytes)

which will not go on sixteen bit pointers .....

**OrangeDog** · 03-14-2009, 09:17 PM

A few things:

I mean really start simple. Stuff like just returning an int through the JNI. Then do something simple with the STL if you're going to be using it, something that could only possible generate one error, and get it working before you proceed.

Java Code:

EnvPtr->object

is shorthand for

Java Code:

(*EvnPtr).object

in both C and C++. Using

Java Code:

(*EnvPtr)->object

is likely to cause difficult to trace pointer problems all over the place.

The calling style of Java is irrelevant: jni.h (via JNIEnv) deals with all of that. If you try and use binaries that are compiled to use different calling style (e.g. some C++ and C DLLs) then it won't work at all.