QB64 Community

Development => DECLARE LIBRARY Discussion Forum => Topic started by: mcalkins on April 04, 2013, 11:15:00 PM

Title: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: mcalkins on April 04, 2013, 11:15:00 PM
This works on QB64 0.954 win32. It'll probably work on GL also.

delme.h
Code: [Select]
extern "C" __declspec(dllexport) int32 __stdcall FUNC_F(int32*_FUNC_F_LONG_N);
extern "C" is to prevent C++ name mangling.
__declspec(dllexport) causes the function to be exported as a DLL symbol. This allows us to use GetProcAddress.
__stdcall causes the function to remove its own parameters. DECLARE DYNAMIC LIBRARY assumes that it does.
The rest is copied from regsf.txt.

Code: [Select]
DECLARE DYNAMIC LIBRARY "kernel32"
 FUNCTION GetModuleHandleA~%& (BYVAL lpModuleName~%&)
 FUNCTION GetProcAddress~%& (BYVAL hModule~%&, BYVAL lpProcName~%&)
END DECLARE

DECLARE DYNAMIC LIBRARY "delme"
 FUNCTION FUNC_F& (BYVAL n~%&)
END DECLARE

DECLARE LIBRARY "delme"
END DECLARE

DIM h AS _UNSIGNED _OFFSET
DIM t AS STRING
DIM n AS LONG

h = GetModuleHandleA(0)
t = "FUNC_F" + CHR$(0)
PRINT "0x" + LCASE$(HEX$(h))
PRINT "0x" + LCASE$(HEX$(GetProcAddress(h, _OFFSET(t))))

PRINT
n = 5
PRINT FUNC_F(_OFFSET(n))
PRINT n
PRINT
n = -11
PRINT FUNC_F(_OFFSET(n))
PRINT n
END

FUNCTION f& (n AS LONG)
f = n - 1
n = n + 1
END FUNCTION

DECLARE DYNAMIC LIBRARY doesn't allow you to specify an extension. (PE/COFF image files can have any extension, not just EXE or DLL.) Also, QB64 won't compile unless it can find the file.

Create a blank file named "delme.dll", so that the program will compile. Then compile the program. Then copy or rename the resulting executable to "delme.dll", overwriting the dummy file. Then run it from the command prompt. (From cmd.exe, in the correct folder, type "delme.dll" and press enter.)

The DECLARE DYNAMIC LIBRARY uses GetProcAddress at runtime. The explicit calls to GetModuleHandle and GetProcAddress in the program are just for demonstration.

Regards,
Michael
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: Galleon on April 05, 2013, 08:51:00 PM
Most interesting.

How do you plan to use this?
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: mcalkins on April 06, 2013, 01:02:48 AM
(I edited the function prototype to add __stdcall. It works without it, but would leak stack.)

At the moment, it's mostly for fun.  ;D 8) However, it is related to the subject of function pointers. (I still need to follow up on that discussion... (I procrastinate too much.))

I also have structured exception handling on my mind at the moment. However, I'm not planning to use this for it, at the moment. I'm working on some Asm wrapper functions to allow SEH to be used from QB64, despite MinGW's total lack of support. I might post today, or in the next few days, if it works.

In the "deliberate buffer overflow" topic, I told fatman that I would post a demo of easier and more reliable execution of machine code. I still need to get back to that.

I perceive these subjects as somewhat distinct, but loosely connected.

Regards,
Michael
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: mcalkins on April 11, 2013, 03:02:29 AM
If you go with CUSTOMTYPE, you don't need to use a header file.
(However, you wouldn't know the function's address. But you probably wouldn't need to. QB64 functions aren't generally very suitable for callbacks anyway.)

Code: [Select]
DECLARE CUSTOMTYPE LIBRARY
 SUB SUB_S (BYVAL p~%&)
END DECLARE

TYPE udt
 x AS LONG
 y AS LONG
 z AS LONG
END TYPE

DIM m AS _MEM
m = _MEMNEW(&HC)
_MEMPUT m, m.OFFSET, 1 AS LONG
_MEMPUT m, m.OFFSET + 4, -2 AS LONG
_MEMPUT m, m.OFFSET + 8, 3 AS LONG

SUB_S m.OFFSET

_MEMFREE m
END

SUB s (t AS udt)
PRINT t.x
PRINT t.y
PRINT t.z
END SUB

This demonstrates an obvious usage for this. The function can treat arbitrary memory as an object of a user defined type. So, you can use the normal BASIC UDT syntax within the function, even though the memory was not DIMmed using a UDT. This would have practical real world usage. It would not be necessary to use relatively clumsy peekpoke or _MEM to access memory that you don't own. Just use this method to pass byval an _OFFSET to a BASIC function that expects a byref object.

Regards,
Michael

P.S. I would expect this last example to work on Linux 32/64 also.
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: Mrwhy on April 11, 2013, 03:21:37 AM
It would have yet more more practical usage if described in the words human beings use when we talk to each other - words we use at least once or more every week!
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: mcalkins on April 11, 2013, 04:53:18 AM
Well, I used words that programmers use to talk to each other. I guess programmers are usually humans?

Which ones gave you trouble?

UDT = User Defined Type. These phrases "user-defined data type" and "user-defined type" appear in the QBASIC 1.1 Help. It is a structure type, a struct or a class. It is declared with the TYPE keyword:

So:

TYPE udt
 x AS LONG
 y AS LONG
 z AS LONG
END TYPE

declares a structure data type. This is just a declaration. It does not actually allocate any memory for the structure. It just describes the layout of the structure to the compiler.

"x", "y", and "z" are called elements or members.

If I were then to have:

DIM n AS udt

I would create an object/variable of that type. This would actually allocate the memory. However, I did not do that in the example.

SUB s (t AS udt)

declares that the parameter will be called "t", and it is an object of type "udt" passed by reference. (This means that, in reality, a pointer to the object is passed. The BASIC function hides the pointer from the programmer, allowing the object to be accessed using normal syntax.)

So, within the function:

PRINT t.y

prints the "y" member of the "t" object. "t" is the object that was passed by reference. (Really, it is an alias for the object.) (What is really happening is that the function's parameter was a pointer. Because the "y" member will be 4 bytes into the structure, the code just adds 4 to the address that it was given, and reads the memory.)

However, knowing that what is really passed is a pointer by value, we can use the DECLARE CUSTOMTYPE LIBRARY, as shown, to pass in a pointer to any arbitrary (but valid readable) memory that we choose.

In this case, I used _MEM to allocate 12 bytes, wrote values to the memory, and then passed the address by value to the function.

Normally, you wouldn't need to do this. You would just use the pure BASIC syntax, and pass in an object by reference:

DIM n AS udt
s n

Consider library functions that need to write structures. Ordinarily, you would allocate the memory yourself, using DIM. Then you would pass the address, the _OFFSET, of that object to the library function. The library function would write to your object, then you could read it normally.

However, what if instead of letting you specify the address, the library function returns a pointer to some other memory? You would have several options:

- You could use my peekpoke functions to read the memory. You would have to add offsets to the base address to read the members. This is efficient, but inconvenient.

- You could use _MEMGET to read the memory. You would have to add offsets to the base address to read the members. This is relatively efficient, but inconvenient.

- You could DIM an object, and then use memcpy or _MEMCOPY to copy the data into your object. Then you could read it normally. This is relatively convenient, but inefficient for large structures.

- You could use the method I demonstrated to pass the address to a BASIC function that expects a byref object. Then you could read it normally. This is relatively convenient and relatively efficient for large structures. The BASIC function overhead might make it comparatively inefficient for small structures.

For example, when using Component Object Model, the CoCreateInstance function gives you a pointer to a vtable. You could use the method demonstrated above to treat the vtable as a user-defined data type.

Perhaps you want to read a structure from an executable image file (an EXE or DLL) already loaded in your address space. For example, you might want to read the PE/COFF headers of your own executable. You could declare the header structures as TYPEs, and then conveniently read them using the demonstrated method. (GetModuleHandle returns the address of the start of your executable. At offset 0x3c from there is a value that tells you where the PE signature is.)

Also, perhaps you did use DIM to create the object, but it really should be a union. QB64 does not currently support unions. You could use the method that I demonstrated to access the various union members. For example, the INPUT_RECORD structure used by the ReadConsoleInput function contains a union of various structures.

(A union is where one of a number of different structures or data types could occupy the same memory.)

Perhaps later, I will post some code examples.

Regards,
Michael
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: Mrwhy on April 11, 2013, 08:22:07 AM
Good explanation and thank you Mcalkins

Yes I suppose it depends what your aim is.
To attract people is best to talk to them in their words.
So to attract  coders you talk code.

I agree an explanation in English takes a lot more space and thank you for your generosity spending the time to do it :D
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: DSMan195276 on May 07, 2013, 10:05:55 PM
Michael, I think you might like this idea, it's slightly related to what you have. Take a look at the following code:

test.bas:
Code: [Select]

SUB test_1 (k as INTEGER)
PRINT "Hello"
test_2 k
END SUB

SUB test_2 (k as INTEGER)
PRINT k
END SUB

test2.bas:
Code: [Select]
DECLARE STATIC LIBRARY "tes"
        SUB SUB_TEST_1 (x AS INTEGER)
END DECLARE

SUB_TEST_1 2

tes.h:
Code: [Select]
extern uint33_t r;
extern void evnt(uint32_t);

void SUB_TEST_1(int16*);
void SUB_TEST_2(int16*);

void SUB_TEST_1(int16*_SUB_TEST_1_INTEGER_K){
qbs *tqbs;
ptrszint tmp_long;
int32 tmp_fileno;
uint32 qbs_tmp_base=qbs_tmp_list_nexti;
uint8 *tmp_mem_static_pointer=mem_static_pointer;
uint32 tmp_cmem_sp=cmem_sp;
mem_lock *sf_mem_lock;
new_mem_lock();
sf_mem_lock=mem_lock_tmp;
sf_mem_lock->type=3;
if (new_error) goto exit_subfunc;
do{
tqbs=qbs_new(0,0);
qbs_set(tqbs,qbs_new_txt_len("Hello",5));
if (new_error) goto skip1;
makefit(tqbs);
qbs_print(tqbs,0);
qbs_free(tqbs);
qbs_print(nothingstring,1);
skip1:
qbs_cleanup(qbs_tmp_base,0);
if(!qbevent)break;evnt(5);}while(r);
do{
SUB_TEST_2(_SUB_TEST_1_INTEGER_K);
if(!qbevent)break;evnt(6);}while(r);
exit_subfunc:;
free_mem_lock(sf_mem_lock);
if ((tmp_mem_static_pointer>=mem_static)&&(tmp_mem_static_pointer<=mem_static_limit)) mem_static_pointer=tmp_mem_static_pointer; else mem_static_pointer=mem_static;
cmem_sp=tmp_cmem_sp;
}

void SUB_TEST_2(int16*_SUB_TEST_2_INTEGER_K){
qbs *tqbs;
ptrszint tmp_long;
int32 tmp_fileno;
uint32 qbs_tmp_base=qbs_tmp_list_nexti;
uint8 *tmp_mem_static_pointer=mem_static_pointer;
uint32 tmp_cmem_sp=cmem_sp;
mem_lock *sf_mem_lock;
new_mem_lock();
sf_mem_lock=mem_lock_tmp;
sf_mem_lock->type=3;
if (new_error) goto exit_subfunc;
do{
tqbs=qbs_new(0,0);
qbs_set(tqbs,qbs_add(qbs_str((int16)(*_SUB_TEST_2_INTEGER_K)),qbs_new_txt(" ")));
if (new_error) goto skip2;
makefit(tqbs);
qbs_print(tqbs,0);
qbs_free(tqbs);
qbs_print(nothingstring,1);
skip2:
qbs_cleanup(qbs_tmp_base,0);
if(!qbevent)break;evnt(10);}while(r);
exit_subfunc:;
free_mem_lock(sf_mem_lock);
if ((tmp_mem_static_pointer>=mem_static)&&(tmp_mem_static_pointer<=mem_static_limit)) mem_static_pointer=tmp_mem_static_pointer; else mem_static_pointer=mem_static;
cmem_sp=tmp_cmem_sp;
}

test.bas is there for reference, it's not technically needed though. Basically, create test2.bas and tes.h, then compile test2.bas. When it runs, it displays "Hello \n 2" (\n being new line).

The cool thing here is the header is just a slightly modified output from QB64, and you call run test_1 without having to declare test_2 as well. It's somewhat obvious, but the plan is to eventually be able to compile the QB64 outputted code here into a Object file, and then have an extra header declaring the functions inside. Then you can link together projects and have internal private QB64 functions without having them accessible to everything outside (And so you can much easier avoid naming conflicts and etc.).

Matt
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: mcalkins on May 09, 2013, 04:40:28 AM
You are working towards modules.

Something like that could be done to make a DLL from QB64 code, also. However, either libqb would then have to be a DLL, or else the DLL would have to be hard coded with the EXE name, which kind of defeats the purpose of using a DLL. (A 3rd option would be for the DLL to use GetProcAddress at runtime, but that adds complication.)

Regards,
Michael
Title: Re: Using DECLARE DYNAMIC LIBRARY to call a function exported from yourself.
Post by: DSMan195276 on May 09, 2013, 06:02:00 AM
Mhmm, that basically sums it up.

I think that libqb could be compiled into a DLL without problem (At least I can't see any reason for there to be issue). I think the issue more comes that QB64's routines probably won't play well with other programs.

Matt