Mining Extensions are Java classes that implement a certain interface and register some additional functionality for the mining product. The extensions are registered automatically at runtime if they are put on the classpath of the mining-api-server (and they are located in the correct package).
We currently provide extension points to:
export mining data to a custom format
add additional custom pages to the Mining UI
provide additional MetricsContributor for Discover metrics process
Target audience is Mining and Adoption Pod DevOps team members, as well as mining power-users and delivery professionals. These readers typically configure, extend, or operationalize Mining beyond standard end-user workflows. The remaining sections assume familiarity with Java projects, server configuration, and Mining deployment concepts.
To access mining data within your application (for example, to implement an export extension), you will also want to include the "mining-data" dependency:
Disclaimer: There is currently no stable, public API available to access mining data. Therefore, include and use the "minig-data" project with the expectation that your project may break with a future update. A stable, approved API may come in the future.
To load your extension into the mining-api-server, you need to compile your extension to a jar file and then add the jar file (and all of its dependencies) to the classpath of mining-api-server. All extensions that are underneath the package innowake.mining.extensions will then be automatically registered.
To add your extension to the classpath, use the loader.path system property (or the LOADER_PATH environment variable) and launch the application using the org.springframework.boot.loader.launch.PropertiesLauncher class. More information on the topic can be found here.
On the command line, use the -cp option, not the -jar option to specify the mining-api-server jar file. Otherwise, the main class argument (org.springframework.boot.loader.launch.PropertiesLauncher) will be ignored.
You can show your extension on the export page, optionally. A "Mining Export Extension" will show itself on the export page by default, whereas a "Mining Job Extension" will not show. This option can be changed according to the requirement by overriding the method getShowOnPage() of the ShowOnPageExtension interface, inside your extension class, as shown below.
Info
Currently, Mining supports showing UI extensions only on the Export page and Customizable Tables.
@Override
public ShowOnPage getShowOnPage() {
return new ShowOnPage(true, "Extension label", "Extension Category", "Extension location");
}
It returns a ShowOnPage object, which can be created by providing values for the parameters :
boolean show: This accepts a boolean value. true if the extension should be shown on the page, false otherwise
String label: This accepts the string value for the label of the extension
String category: This accepts the string value for the category of the extension, defaults to 'Extension' if null or empty
String usage: This accepts the string value, which should be one of the Usage values from the Usages class if it needs to be shown in the table or 'ExportPage' to show in the export page, default to 'ExportPage' if null or empty.
'ExportPage' or Empty -> Export page in UI
Usages.MINING_UI_MODULES_TABLE -> Modules Reporting Page
Usages.MINING_UI_ANNOTATIONS_TABLE -> Annotation Reporting Page
Usages.MINING_UI_REACHABILITY_TABLE -> individual RB Table
Usages.MINING_UI_REACHABILITY_BLOCKS_OVERVIEW_TABLE -> Reachability Overview table
Usages.MINING_UI_DATADICTIONARY_TABLE -> Data dictionary Reporting Page
You can see the section "Pages / Locations on Mining UI" in the Usagses.Java (Mining-shared) for other usages that can be used.
The following extensions can currently be registered – each description also includes a sample implementation. Together, they cover the main extension patterns for adding export, job, or UI-related functionality. Review the type that matches the behavior you want to introduce before starting implementation.
This kind of extension contributes the implementation of a Job that can be executed on a mining project and runs on the server. Unlike an export extension, a job is executed asynchronously ("in background") - its status can be observed via the Job REST API. The Job API also provides a way to cancel a Job. The Job can optionally produce a result. These feature make it the most flexible type of extension that is suitable for long-running and computationally intensive tasks.
The implementation of a Job Extension must provide a factory method that creates a Job instance. The returned instance will then be scheduled for execution via the job framework. It can be triggered via an HTTP request to:
POST http://localhost:8080/api/v2/projects/<projectId>/job-extensions/<extensionId>
Optional query parameters are passed in the parameters Map to the createJob() method of your extension (see below). The request body is passed as the inputDataargument and may contain arbitrary data (or it can be empty, if the job requires no input data). This allows your extension to implement file upload / import functionality.
You can implement the getParameterDescriptions() and getUploadDescription() methods to describe the parameters that can be passed to your extension, respectively, and the file that can be uploaded. This is especially useful when also overriding getShowOnExportPage() to show the extension on the Export page of the Mining UI, as it will allow users to interactively input the parameters or choose a file through the UI. Note that even when not implementing getParameterDescriptions() or getUploadDescription() it is still possible to pass parameters or upload a file when manually sending the HTTP request or using the mining-cli.
package innowake.mining.extensions.job;
@Service
public class HelloWorldJobExtension implements MiningJobExtension<String> {
private static class HelloWorldJob extends Job<String> {
private final String greeting;
public HelloWorldJob(final String greeting) {
this.greeting = greeting;
}
@Override
protected Result<String> run(ProgressMonitor progressMonitor) {
return new Result<>("Hello " + greeting);
}
}
@Autowired
private ObjectMapper objectMapper;
@Override
public String getIdentifier() {
return "hello-world";
}
@Override
public String getDescription() {
return "Hello World Example Job";
}
@Override
public Job<String> createJob(final Long projectId, final Map<String, List<String>> parameters, HttpEntity<byte[]> inputData) {
/* read parameters and apply defaults */
final String greeting = Optional.ofNullable(parameters.get("greeting")).map(t -> t.get(0)).orElse("World");
/* check uploaded file and parse JSON */
if ( ! inputData.hasBody()) {
throw new IllegalArgumentException("Input data is mandatory and must contain the data to be imported in JSON format.");
}
final MyData data = objectMapper.readValue(inputData.getBody(), MyData.class);
/* MyData could now be passed to the Job (not shown) */
return new HelloWorldJob(greeting);
}
@Override
public NatureType getRequiredNature() {
return MINING;
}
@Override
public RoleType getRequiredRole() {
return VIEWER;
}
@Override
public ShowOnPage getShowOnPage() {
return new ShowOnPage(true, "Extension label", "Extension Category", "Extension location");
}
@Override
public List<ParameterDescription> getParameterDescriptions() {
return Arrays.asList(
new ParameterDescription("greeting", /* name */
"The greeting to be sent by the example extension", /* description */
ParameterDescription.ParameterType.STRING, /* parameter type */
false, /* mandatory parameter yes/no */
"World", /* default value for non-mandatory parameters */
false, /* parameter can be given multiple times yes/no */
"World", /* example value show to the user */
Collections.emptyList()) /* possible values - if specified allows to chose the values from a list, otherwise any value can be entered */
);
}
@Override
public UploadDescription getUploadDescription() {
return UploadDescription.with("JSON data file", /* short descriptive name */
"JSON file containing data to be imported", /* description */
true, /* file upload is mandatory yes/no */
".json,application/json"); /* possible file types (can be omitted to allow any type) */
}
}
This kind of extension contributes a new page within the Mining UI. The page will be displayed in the navigation menu on Mining UI. It can optionally receive the id of the currently selected project and client.
/*
* Copyright (c) 2021 innoWake gmbh Germany. All rights reserved.
*/
package innowake.mining.extensions.ui;
import innowake.mining.shared.extensions.MiningWebUiExtension;
@Service
public class HelloWorldExtension implements MiningWebUiExtension {
@Override
public String getName() {
return "Hello World";
}
@Override
public Kind getKind() {
/* Note: IFRAME is the only supported kind as of writing this */
return Kind.IFRAME;
}
@Value("${server.servlet.context-path:}")
private String servletContextPath;
@Override
public Map<Property, String> getProperties() {
final Map<Property, String> props = new HashMap<>();
final String prefix = "/".equals(servletContextPath) ? "" : servletContextPath; // Adding this prefix is not required if the Iframe is served externally.
props.put(Property.IFRAME_SRC, prefix + "/my/extension-url/index.html?clientId=${clientId}&projectId=${projectId}");
return props;
}
}
When using a path as IFRAME_SRC (as in the example above) then the static resources for your extension will be served from the classpath (from one of the default locations used by Spring Boot). You can also embed an external web page in the iFrame by specifying its full URL (for example. "http://my-other-domain.com/my/extension").
In order to serve the resources (HTML etc.) for your extension from the classpath, you need an additional configuration. The default security configuration of mining-api-server will set the X-Frame-Options: DENY header on all resources. This will effectively prevent your extension from functioning since it is displayed in an iframe. Also, we enforce Authentication which may also cause problems for your static resources.
To fix this, add another class to your extension project:
This extension enables you to provide custom contributors for Discovery. These contributors allow you to discover new types of Modules from source code, or to extend the already supported Modules with additional information. This will be helpful for clients who have different language dialects or need to support any client specific implementation.
Info
The class implementing DiscoveryContributor or DiscoveryContributorFromSource has to be public and needs to be annotated with @Component. It also needs to be underneath the innowake.mining.extensions package.
Here is an example for an extension that extends the already discovered Java classes with additional dependency information based on a customer-specific annotation:
/**
* Example contributor extension which creates additional dependecies from Java classes (JAVA_TYPE) to Stored Procedures (SQL_STORED_PROCEDURE)
* based on an "@StoredProcedure" annotation.
*/
@Component
public class JavaMetricsContributorExtension implements DiscoveryContributor {
private static final Pattern PATTERN = Pattern.compile("@StoredProcedure\\(\"(\\w+)\"\\)");
@Override
public void contribute(final DiscoveryBuilder builder, final DiscoveryContext context) {
/* anchorTo() is used to "attach" to existing Modules which were already discovered by another contributor.
* In this case, the built-in Java support of Discovery has already discovered the Java classes (they have the type JAVA_TYPE).
* So our contributor extension can simply "anchor to" the existing Modules and extend them with additional information.
*
* To do this, we use deferAction(). Discovery processes files in parallel for optimal performance. Using deferAction(), we can schedule
* some code to be executed after processing of all files is completed. This is ideal for our use-case, because we want our custom
* contributor to run after all Java classes and Stored Procedures have been discovered. */
builder.anchorTo(new ModuleFilter().setTypes(ModuleType.JAVA_TYPE))
.deferAction("analyzeStoredProcedures"); /* must match method name of method annotated with @DeferredAction */
}
@DeferredAction
public void analyzeStoredProcedures(final Module module, final ModuleBuilder moduleBuilder, final SourceObject sourceObject) {
String content = sourceObject.getContent();
/* the SourceObject is for the entire COMPILATION_UNIT (i.e. .java file) but this deferred action runs separately on each class
* since one COMPILATION_UNIT can contain multiple classes, we make sure to only look at the part of the file that contains the current class */
final ModuleLocation classLocation = module.getLocation();
if (classLocation != null) {
content = content.substring(classLocation.getOffset(), classLocation.getLength());
}
final Matcher matcher = PATTERN.matcher(content);
if (matcher.find()) {
final String storedProcedure = matcher.toMatchResult().group(1);
moduleBuilder.declareDependency(Relationship.CALLS, new ModuleFilter().setNames(storedProcedure).setTypes(ModuleType.SQL_STORED_PROCEDURE))
.setBinding(Binding.LATE);
}
}
}
This extension enables to provide custom Metrics contributors. This will be helpful for clients who have different dialects or to support any client specific implementation.
Info
The class implementing MetricsContributorExtension has to be public and needs to be annotated with @Component. It also needs to be underneath the innowake.mining.extensions package.
If a technical taxonomy has to be identified for a module based on some patterns that cannot be achieved through Taxonomy propagation (or) through the existing technical taxonomy logic, then you can write a custom logic that should implement TaxonomyIdentifier interface. Although, this interface is not introduced specifically as a mining extension, one can still use it like all other extension.
Info
The class implementing TaxonomyIdentifier<ModuleBasePojo> has to be public and needs to be annotated with @Component. It also needs to be underneath the innowake.mining.extensions package. At the time of writing, this can take AstNodePojo , AstModel , DependencyModel .
Let's say you need to identify all the Cobol modules that have a CICS statement in them as the "Access CICS" taxonomy, then:
Before a prompt is sent to a GenAI model, additional context is added to the prompt by context retrievers. It is possible to add custom context retrievers with a mining extension.
To do so you need to add a class implementing one of the ContextRetriever interfaces to the innowake.mining.server.genai.context package:
GenAiAnnotationContextRetriever
GenAiModuleContextRetriever
GenAiFunctionalBlockContextRetriever
GenAiReachabilityBlockContextRetriever
GenAiDataDictionaryContextRetriever
It is also possible to define multiple retrievers for the same kind of entity. Each retriever is supposed to return the context of the previously executed retriever + the newly retrieved context.
package innowake.mining.server.genai.context;
import innowake.mining.shared.entities.AnnotationPojoPrototype;
import org.springframework.stereotype.Service;
@Service
public class ExampleAnnotationContextRetriever implements GenAiAnnotationContextRetriever {
@Override
public AiGenerationContext retrieve(final AnnotationContext context) {
final AiGenerationContext previousContext = context.getGenerationContext();
final AnnotationPojoPrototype annotationPojoPrototype = context.getAnnotation();
if (annotationPojoPrototype.customProperties.isDefined()) {
/* Return context containing context from previous retrievers + new context */
return new AiGenerationContext(previousContext.getAdditionalPromptContext()
+ annotationPojoPrototype.customProperties.get().toString());
}
/* No custom properties are available -> we return the previous context */
return context.getGenerationContext();
}
}
The String value that gets passed to the AiGenerationContext constructor will get added to the prompt.
If you don't use GenAI capabilities for the context retrieval you can ignore this
The context object (f.e. AnnotationContext) which gets passed to retrieve has a method getRequestUUID
Purpose: associate the context retrieval with the description generation for token usage tracing purposes
If you access f.e. the Knowledge Service in your context retriever, you should add the UUID to the request as part of the requestMetadata map at key "uuid"