heck
5 years ago
32 changed files with 347 additions and 271 deletions
@ -0,0 +1,26 @@ |
|||
pEpACIDgen |
|||
========== |
|||
|
|||
pEpACIDgen stands for "pEp Abstract C Interface Description Generator" |
|||
|
|||
pEpACIDgen generates an Abstract C Interface Defintion file (json or yml2) given the inputs: |
|||
* headerfile |
|||
* list of function names needed |
|||
* list of var names needed |
|||
|
|||
The generator searches each function/var in the headerfile (recursively) and collects all the |
|||
types needed (var type, return type, parm types). |
|||
As structs can contain further types, these dependent types need to be collected |
|||
recursively. |
|||
Finally, all the collected types will be resolved to their final underlying type. |
|||
If a type is primitive, nothing needs to be done, cause its already defined. |
|||
But types of typekind struct or enum need to be defined. |
|||
Their definition will be searched for in the headerfile and included in the interface definition |
|||
The ACID file (yml2) contains all the information needed to represent: |
|||
* functions |
|||
* vars |
|||
* structs |
|||
* enums |
|||
|
|||
well enough so they can be expressed in pyBind11. |
|||
use "pyBind11-CID" to generate the pyBind11 code out of the ACID file. |
|||
@ -1,10 +0,0 @@ |
|||
.PHONY: all clean |
|||
|
|||
all: |
|||
$(MAKE) -C synth_shed |
|||
$(MAKE) -C test_cid |
|||
|
|||
clean: |
|||
$(MAKE) -C synth_shed clean |
|||
$(MAKE) -C test_cid clean |
|||
|
|||
@ -0,0 +1,161 @@ |
|||
pEpACIDgen test library |
|||
----------------------- |
|||
This here is the test library for pEpACIDgen. |
|||
The C language is broken down into its components and common pattern. |
|||
It is a humble attempt to cover all possible combinations that can be created in the |
|||
C language. (like, struct containing` primitive, struct containing struct, typedef chains, |
|||
etc..). |
|||
Please note, that pointer types have not been addressed yet (at all) |
|||
|
|||
|
|||
The intended use of this test lib is: If you encounter a scenario using pEpACIDgen that poses problem |
|||
it is most likely that the scenario can be reproduced using this test lib pretty easily. |
|||
lets say you pEpACIDgen a function with sig: |
|||
|
|||
TT_E(*)(ACCS), where: |
|||
TT_E = typedef of typedef of an enum without an alias. |
|||
ACCS = Alias to a complex struct containing another complex struct |
|||
|
|||
These types exist already along with a systematic permutation of possible types that amount |
|||
to over 100 of types. |
|||
just write your func/var using these types and debug the generation. |
|||
|
|||
|
|||
The ACID format deals with items. Everything is an item. |
|||
There are qualifiers that can be used and combined to define an item. (list below) |
|||
e.g PS = primitive struct that possibly has a typedef |
|||
|
|||
Of all the possible combinations, not all are valid (C lang). Its a futile attempt |
|||
trying to cover all the valid ones. But an effort has been made to cover the most |
|||
of the valid combinations. |
|||
Combination of structs, enums and typedefs are systematically explored in the files: |
|||
* structs.h |
|||
* enums.h |
|||
* typedefs.h |
|||
|
|||
|
|||
For all the types defined the file: |
|||
* vars.h |
|||
|
|||
contains an instance of each. |
|||
|
|||
for all the possible function signature, well with a 100 types, |
|||
we can create 10000 different sigantures using 1 ret and 1 arg. |
|||
Therefore, no functions have been defined. |
|||
define them when you are interested in the combination, and pEpACIDgen the defintion. |
|||
|
|||
|
|||
Item Qualifier |
|||
=============== |
|||
V = Void already defined |
|||
P = Primitive Type already defined |
|||
S = Struct -> struct.h |
|||
E = Enum -> enums.h |
|||
A = Alias -> structs.h/enums.h |
|||
N = Nested (Structs only, cant be aliased) -> structs.h |
|||
H = Hosting (Structs only -> structs.h |
|||
containing the nested struct, cant be primitive) |
|||
T = typedef -> typedefs.h |
|||
I = Incomplete / (forward decl) -> struct.h/enums.h |
|||
|
|||
func_ = function -> functions.h |
|||
var_ = variable -> vars.h |
|||
|
|||
prefixed underline (e.g _E) means no typedef/alias exists for this item (enum in this case) |
|||
|
|||
Alias vs. Typedef |
|||
----------------- |
|||
aliased means typedef'd directly, like: |
|||
|
|||
typedef struct _X { |
|||
int x; |
|||
} X; |
|||
|
|||
(structs and enums only) |
|||
|
|||
|
|||
Typedef means, not typedef'd directly, but anywhere, like |
|||
|
|||
typedef _X X; |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
Structs |
|||
======= |
|||
For structs, a combination is needed to define the type of struct |
|||
IS = Struct (Empty or incomplete, neither primitive nor complex) |
|||
PS = Primitive struct (struct containing only primitive types) |
|||
CS = Complex struct (struct containing other structs and primitive) |
|||
NPS = Nested primitive struct |
|||
|
|||
Covered combinations |
|||
==================== |
|||
_IS = incomplete struct without a typedef (forward decl) |
|||
IS = incomplete struct |
|||
AIS = alias of an incomplete struct |
|||
(aliased) struct [ primitive | complex ] |
|||
---------------------------------------- |
|||
_PS = primitive struct without a an alias |
|||
PS = primitive struct |
|||
APS = Alias of primitive struct |
|||
_CS = complex struct without a an alias |
|||
CS = complex struct |
|||
ACS = alias of a complex struct |
|||
_CCS = complex complex struct without an alias |
|||
CCS = complex complex struct |
|||
ACCS = alias of a complex complex struct |
|||
|
|||
Nested structs |
|||
-------------- |
|||
there is always a hosting struct and a nesting struct |
|||
in case of double nesting it can be both |
|||
|
|||
Simply nested struct [ primitive | complex ] |
|||
-------------------------------------------- |
|||
(aliased) hosting struct (complex always) |
|||
_HS = hosting struct without an alias |
|||
HS = hosting struct |
|||
AHS = alias of a hosting struct |
|||
_NPS = nested primitive struct without an alias |
|||
_NCS = nested complex struct without an alias |
|||
|
|||
Doubly nested struct (complex always) |
|||
------------------------------------- |
|||
_HHS = hosting hosting struct without an alias |
|||
HHS = hosting hosting struct |
|||
AHHS = alias of a hosting hosting struct |
|||
_NHS = nested hosting struct |
|||
_NNPS = nested nested struct |
|||
_NNCS = nested nested complex struct |
|||
|
|||
enums in structs |
|||
_NEHS = nested enum in hosting struct |
|||
_NENHS = nested enum in nested hosting struct |
|||
|
|||
|
|||
|
|||
Enums |
|||
===== |
|||
Enums only have the item qualifier I and A, |
|||
so their combinations are simple. |
|||
|
|||
Covered combinations |
|||
==================== |
|||
_IE = incomplete enum without an alias |
|||
IE = incomplete enum |
|||
AIE = alias of an incomplete enum |
|||
_E = enum without an alias |
|||
E = enum |
|||
AE = alias of an enum |
|||
|
|||
|
|||
Typedefs |
|||
======== |
|||
Typedefs are simply applied to all defined enums and structs, and the |
|||
their respective "alias". |
|||
Then a typedef os applied to every typedef (TT*) |
|||
|
|||
|
|||
@ -1,4 +1,4 @@ |
|||
#include "test_lib.h" |
|||
#include "lib_test.h" |
|||
|
|||
struct _PS func__PS_V() { |
|||
return var__PS; |
|||
@ -0,0 +1,10 @@ |
|||
#ifndef MAIN_INCLUDE_H |
|||
#define MAIN_INCLUDE_H |
|||
|
|||
#include "typedefs.h" |
|||
#include "enums.h" |
|||
#include "structs.h" |
|||
#include "functions.h" |
|||
#include "vars.h" |
|||
|
|||
#endif //MAIN_INCLUDE_H
|
|||
@ -1,9 +1,9 @@ |
|||
#include "test_lib.h" |
|||
#include "lib_test.h" |
|||
#include <stdio.h> |
|||
|
|||
|
|||
int main() { |
|||
printf("test_lib starting...\n"); |
|||
printf("lib_test starting...\n"); |
|||
|
|||
|
|||
TT_HHS a; |
|||
@ -1,110 +0,0 @@ |
|||
#ifndef MAIN_INCLUDE_H |
|||
#define MAIN_INCLUDE_H |
|||
|
|||
#include "typedefs.h" |
|||
#include "enums.h" |
|||
#include "structs.h" |
|||
#include "functions.h" |
|||
#include "vars.h" |
|||
|
|||
// Gen-CID
|
|||
// =======
|
|||
// Gen-CID generates a C Interface Defintion file (json) given the inputs:
|
|||
// * headerfile
|
|||
// * list of function names needed
|
|||
// * list of var names needed
|
|||
// The generator searches each function/var in the headerfile (recursively) and collects all the
|
|||
// types needed (var type, return type, parm types).
|
|||
// As structs can contain further types, these dependent types need to be collected
|
|||
// recursively.
|
|||
// Finally, all the collected types will be resolved to their final underlying type.
|
|||
// If a type is primitive, nothing needs to be done, cause its already defined.
|
|||
// But types of typekind struct or enum need to be defined.
|
|||
// Their definition will be searched for in the headerfile and included in the interface definition
|
|||
// The CID file contains all the information needed to represent:
|
|||
// * functions
|
|||
// * vars
|
|||
// * structs
|
|||
// * enums
|
|||
// Well enough so they can be expressed in pyBind11
|
|||
// use "pyBind11-CID" to generate the pyBind11 code out of the CID file.
|
|||
//
|
|||
// Gen-CID test library
|
|||
// --------------------
|
|||
// This here is the test main test library for Gen-CID.
|
|||
// The C language is broken down into its components and common pattern.
|
|||
// It is a humble attempt to cover all possible combinations that can be created in the
|
|||
// C language. (like, struct containing` primitive, struct containing struct, typedef chains,
|
|||
// etc..).
|
|||
// Please note, that pointer types have not been addressed yet (at all)
|
|||
//
|
|||
// The intended use of this test lib is: If you encounter a scenario using Gen-CID that poses problem
|
|||
// it is most likely that the scenario can be reproduced using this test lib pretty easily.
|
|||
// lets say you Gen-CID a function with sig:
|
|||
// TT_E(*)(ACCS), where:
|
|||
// TT_E = typedef of typedef of an enum without an alias.
|
|||
// ACCS = Alias to a complex struct containing another complex struct
|
|||
// these types exist already along with a systematic permutation of possible types that amount
|
|||
// to over 100 of types.
|
|||
// just write your func/var using these types and debug the generation.
|
|||
|
|||
|
|||
// The CID format deals with items. Everything is an item.
|
|||
// There are qualifiers that can be used and combined to define an item. (list below)
|
|||
// e.g PS = primitive struct that possibly has a typedef
|
|||
//
|
|||
// Of all the possible combinations, not all are valid (C lang). Its a futile attempt
|
|||
// trying to cover all the valid ones. But an effort has been made to cover the most
|
|||
// of the valid combinations.
|
|||
// Combination of structs, enums and typedefs are systematically explored in the files:
|
|||
// * structs.h
|
|||
// * enums.h
|
|||
// * typedefs.h
|
|||
//
|
|||
// For all the types defined the file:
|
|||
// * vars.h
|
|||
// contains an instance of each.
|
|||
//
|
|||
// for all the possible function signature, well with a 100 types,
|
|||
// we can create 10000 different sigantures using 1 ret and 1 arg.
|
|||
// Therefore, no functions have been defined.
|
|||
// define them when you are interested in the combination, and Gen-CID the defintion.
|
|||
|
|||
|
|||
// Item Qualitfier
|
|||
// ===============
|
|||
// V = Void already defined
|
|||
// P = Primitive Type already defined
|
|||
// S = Struct -> struct.h
|
|||
// E = Enum -> enums.h
|
|||
// A = Alias -> structs.h/enums.h
|
|||
// N = Nested (Structs only, cant be aliased) -> structs.h
|
|||
// H = Hosting (Structs only -> structs.h
|
|||
// containing the nested struct, cant be primitive)
|
|||
// T = typedef -> typedefs.h
|
|||
// I = Incomplete / (forward decl) -> struct.h/enums.h
|
|||
//
|
|||
// func_ = function -> functions.h
|
|||
// var_ = variable -> vars.h
|
|||
//
|
|||
// prefixed underline (e.g _E) means no typedef/alias exists for this item (enum in this case)
|
|||
//
|
|||
// Alias vs. Typedef
|
|||
// -----------------
|
|||
// aliased means direct typedef directly, like
|
|||
// structs and enums only
|
|||
/*
|
|||
* typedef struct _X { |
|||
int x; |
|||
* } X; |
|||
* |
|||
*/ |
|||
|
|||
// Typedef means, not typdeffed directly, but anywhere, like
|
|||
/*
|
|||
* typedef _X X; |
|||
* |
|||
*/ |
|||
|
|||
|
|||
#endif //MAIN_INCLUDE_H
|
|||
@ -1,59 +0,0 @@ |
|||
# -*- coding: utf-8 -*- |
|||
# This file is under GNU Affero General Public License 3.0 |
|||
# see LICENSE.txt |
|||
|
|||
from . import c_2_ast |
|||
from . import ast_2_cid |
|||
from . import cid_2_yml |
|||
from . import utils |
|||
|
|||
import os |
|||
|
|||
|
|||
class CIDTools: |
|||
|
|||
def __init__(self, libclang_path, header_filename, pymodule_name, out_dir=None): |
|||
self.header = self._create_header(header_filename, out_dir) |
|||
self.c2ast = c_2_ast.C2AST(libclang_path) |
|||
self.pymodule_name = pymodule_name |
|||
|
|||
# out-dir is in-dir if not specified |
|||
def _create_header(self, path, out_dir=None): |
|||
header = {"path": "", |
|||
"dir": "", |
|||
"filename": "", |
|||
"out_dir": "", |
|||
"sourcecode": "", |
|||
"ast": "", |
|||
"cid": "", |
|||
"yml": ""} |
|||
|
|||
header["path"] = path |
|||
header["dir"] = os.path.dirname(path) |
|||
header["filename"] = os.path.basename(path) |
|||
|
|||
header["out_dir"] = header["dir"] |
|||
if out_dir: |
|||
header["out_dir"] = out_dir |
|||
|
|||
header["sourcecode"] = utils.read_file(path) |
|||
return header |
|||
|
|||
|
|||
def extract(self, var_names, func_names, debug_ast=False, debug_cid=False, debug_yml=False): |
|||
# ast |
|||
self.header["ast"] = self.c2ast.parse(self.header["path"], follow_includes=True) |
|||
if debug_ast: |
|||
utils.write_json(self.header["ast"], self.header["out_dir"] + "/" + self.header["filename"] + ".ast.json") |
|||
|
|||
# cid |
|||
self.header["cid"] = ast_2_cid.extract_cid_selectively(self.header["ast"], var_names, func_names) |
|||
if debug_cid: |
|||
utils.write_json(self.header["cid"], self.header["out_dir"] + "/" + self.header["filename"] + ".cid.json") |
|||
|
|||
# yml |
|||
self.header["yml"] = cid_2_yml.generate_yml(self.header["cid"], self.pymodule_name) |
|||
if debug_yml: |
|||
utils.write_string(self.header["yml"], self.header["out_dir"] + "/" + self.header["filename"] + ".cid.yml") |
|||
|
|||
return self.header |
|||
@ -0,0 +1,58 @@ |
|||
# -*- coding: utf-8 -*- |
|||
# This file is under GNU Affero General Public License 3.0 |
|||
# see LICENSE.txt |
|||
|
|||
from . import c_2_ast |
|||
from . import ast_2_acid |
|||
from . import acid_yml |
|||
from . import utils |
|||
|
|||
import os |
|||
|
|||
|
|||
class pEpACIDGen: |
|||
|
|||
def __init__(self, libclang_path, header_filename, pymodule_name, out_dir=None): |
|||
self.model = self._acid_model_create(header_filename, out_dir) |
|||
self.c2ast = c_2_ast.C2AST(libclang_path) |
|||
self.pymodule_name = pymodule_name |
|||
|
|||
# out-dir is in-dir if not specified |
|||
def _acid_model_create(self, path, out_dir=None): |
|||
model = {"path": "", |
|||
"dir": "", |
|||
"filename": "", |
|||
"out_dir": "", |
|||
"sourcecode": "", |
|||
"ast": "", |
|||
"acid": "", |
|||
"acid_yml": ""} |
|||
|
|||
model["path"] = path |
|||
model["dir"] = os.path.dirname(path) |
|||
model["filename"] = os.path.basename(path) |
|||
|
|||
model["out_dir"] = model["dir"] |
|||
if out_dir: |
|||
model["out_dir"] = out_dir |
|||
|
|||
model["sourcecode"] = utils.read_file(path) |
|||
return model |
|||
|
|||
def extract(self, var_names, func_names, debug_ast=False, debug_acid=False, debug_yml=False): |
|||
# ast |
|||
self.model["ast"] = self.c2ast.parse(self.model["path"], follow_includes=True) |
|||
if debug_ast: |
|||
utils.write_json(self.model["ast"], self.model["out_dir"] + "/" + self.model["filename"] + ".ast.json") |
|||
|
|||
# acid |
|||
self.model["acid"] = ast_2_acid.extract_acid_selectively(self.model["ast"], var_names, func_names) |
|||
if debug_acid: |
|||
utils.write_json(self.model["acid"], self.model["out_dir"] + "/" + self.model["filename"] + ".acid.json") |
|||
|
|||
# yml |
|||
self.model["acid_yml"] = acid_yml.generate_acid_yml(self.model["acid"], self.pymodule_name) |
|||
if debug_yml: |
|||
utils.write_string(self.model["acid_yml"], self.model["out_dir"] + "/" + self.model["filename"] + ".acid.yml") |
|||
|
|||
return self.model |
|||
Loading…
Reference in new issue