Merge remote-tracking branch 'origin/Ghidra_12.0'

Ghidra/Configurations/Public_Release/src/global/docs/WhatsNew.md

@@ -43,11 +43,11 @@ libraries and operating systems (e.g., CentOS 7.x) may also run into compatibili
 launching native executables such as the Decompiler and GNU Demangler which may necessitate a 
 rebuild of native components.
 
-**NOTE:** Ghidra Server: The Ghidra 12.0 server is compatible with Ghidra 9.2 and later Ghidra
+**NOTE:** Ghidra Server: The Ghidra 12.0 server is compatible with Ghidra 11.3.2 and later Ghidra
 clients, although the presence of any newer link-files within a repository may not be handled properly
 by client versions prior to 12.0 which lack support for the new storage format.  Ghidra 12.0 clients
 that introduce new link-files into a project will not be able to add such files into version 
-control if connected to older Ghidra Server versions.  
+control if connected to older Ghidra Server versions.
 
 **NOTE:** Ghidra Server: Due to potential Java version differences, it is 
 recommended that Ghidra Server installations older than 10.2 be upgraded. Those using 10.2 and newer
@@ -64,18 +64,19 @@ process that will provide better results than prior Ghidra versions.  You might
 fresh import of any program you will continue to reverse engineer to see if the latest Ghidra 
 provides better results.
 
-## Project Link Files
+## Project Data Link Files
 Support for link-files within a Ghidra Project has been significantly expanded with this release and
 with it a new file storage type has been introduced which can create some incompatibilities if
 projects and repositories containing such files are used by older version of Ghidra or the Ghidra 
 Server.
 
-Previously only external folder and file links were supported through the use of a Ghidra URL. With
-12.0 the ability to establish internal folder and file links has been introduced.  The new storage 
-format avoids the use of a database and relies only on a light-weight property file. Internal 
-project links also allow for either absolute or relative links.  Due to the fact that Ghidra allows 
-a folder or file to have the same pathname, some ambiguities can result.  It is highly recommended 
-that the use of conflicting folder and file pathnames be avoided.
+Previously, only external folder and file links were supported through the use of a Ghidra URL. With
+12.0 the ability to establish internal folder and file links has been introduced.  A new storage 
+format was adopted for link-files which avoids the use of a database and relies only on a 
+light-weight property file only. Internal project links also allow for either absolute or relative 
+links.  Due to Ghidra allowing a folder and file to have the same pathname, some ambiguities can 
+result for Ghidra URL usage.  It is highly recommended that the use of conflicting folder and file 
+pathnames be avoided.
 
 The use of internally linked folders and files allows batch import processing to more accurately
 reflect the native file-system and its use of symbolic links which allow for the same content to
@@ -89,9 +90,7 @@ Additional Ghidra API methods have been provided or refined on the following cla
 link-files: `DomainFolder`, `DomainFile`, `LinkFile`, `LinkHandler`, `DomainFileFilter`, 
 `DomainFileIterator`, etc.
 
-...TO BE CONTINUED...
-
-## Filesystem Mirroring
+## Importer Filesystem Mirroring
 An option has been added to mirror the local filesystem when importing programs and their libraries.
 Programs and libraries that exist on the local filesystem as symbolic links will have both their 
 corresponding link file and resolved program file mirrored in the project. Filesystem mirroring
@@ -113,7 +112,7 @@ We've added an experimental Z3-based symbolic emulator, which runs as an "auxili
 concrete emulator, effectively constructing what is commonly called a "concolic" emulator. The 
 symbolic emulator creates Z3 expressions and branching constraints, but it only follows the path 
 determined by concrete emulation. This is most easily accessed by installing the "SymbolicSummaryZ3"
-extension (**File** -> **Install Extensions**) and then enabling the `Z3SummaryPlugin` in the 
+extension (**File -> Install Extensions**) and then enabling the `Z3SummaryPlugin` in the 
 Debugger or Emulator tool, which includes a GUI for viewing and sorting through the results. The Z3
 emulator requires z3-4.13.0, available from https://github.com/Z3Prover/z3. Other versions may work,
 but our current test configuration uses 4.13.0. Depending on the release and your platform, the
@@ -138,10 +137,11 @@ that merely integrate with emulation should consider the compositional/callback-
 Extensions that incorporate new domains (e.g. Z3) or novel behaviors (e.g. JIT) should continue 
 using inheritance.
 
-## Other Improvements
- + Added the ability to toggle the displaying of function variables (parameters and locals) that are 
-   normally displayed just below the function signature. The variables display can be turned on/off 
-   globally or individually per function.
+## Hide Function Variables
+Added the ability to toggle the display of function variables (parameters and locals) within
+the Code Browser Listing just below the function signature. The Variables display can be turned 
+on/off globally via the popup menu toggle action (**Function -> Show/Hide All Variables**) or for
+individual functions via an adjacent expand/collapse(+/-) icon.
 
 ## Additional Bug Fixes and Enhancements
 Numerous other new features, improvements, and bug fixes are fully listed in the 

Ghidra/Extensions/BSimElasticPlugin/README.md

@@ -77,8 +77,8 @@ details.
 
 ## Version
 
-The current BSim plug-in was tested with Elasticsearch version `8.8.1`. A change to the 
-Elasticsearch scripting interface, starting with version `7.15`, makes the BSim plug-in incompatible
+The current BSim plug-in was tested with Elasticsearch version `8.19.7`. A change to the 
+Elasticsearch scripting interface, starting with version `8.16`, makes the BSim plug-in incompatible
 with previous versions, but the lsh plug-in jars may work without change across later Elasticsearch 
 versions.
 
@@ -93,7 +93,7 @@ elasticsearch/plugin-descriptor.properties
 
 The file format is fairly simple: edit the line
 ```
-elasticsearch.version=8.8.1
+elasticsearch.version=8.19.7
 ```
 
 The plugin may work with other nearby versions, but proceed at your own risk.

Ghidra/Extensions/BSimElasticPlugin/build.gradle

@@ -92,8 +92,8 @@ task elasticPluginZip(type: Zip) {
 	destinationDirectory = file("build/data")	
 }
 
-// Currently targeting elasticsearch-8.8.1 which by default runs with java 20
-compileElasticPluginJava.options.release = 20
+// Currently targeting elasticsearch-8.19.7 which by default runs with java 25
+compileElasticPluginJava.options.release = 21
 
 compileElasticPluginJava.dependsOn copyGenericTask
 compileElasticPluginJava.dependsOn copyUtilityTask

Ghidra/Extensions/BSimElasticPlugin/contribZipExclude/plugin-descriptor.properties

@@ -2,5 +2,5 @@ description=Feature Vector Plugin
 version=1.0
 name=lsh
 classname=org.elasticsearch.plugin.analysis.lsh.AnalysisLSHPlugin
-java.version=1.11
-elasticsearch.version=8.8.1
+java.version=1.21
+elasticsearch.version=8.19.7

Ghidra/Extensions/BSimElasticPlugin/src/org/elasticsearch/plugin/analysis/lsh/VectorCompareScriptFactory.java

@@ -4,9 +4,9 @@
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
- * 
+ *
  *      http://www.apache.org/licenses/LICENSE-2.0
- * 
+ *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -73,6 +73,11 @@ public class VectorCompareScriptFactory implements ScoreScript.Factory {
 			return false;
 		}
 
+		@Override
+		public boolean needs_termStats() {
+			return false;
+		}
+
 		private static int scanForFeatures(byte[] buffer, int offset) throws IOException {
 			int i = 0;
 			while (i < FEATURES_NAME.length()) {

Ghidra/Extensions/BSimElasticPlugin/srcdummy/org/elasticsearch/script/ScoreScript.java

@@ -5,9 +5,9 @@
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
  * You may obtain a copy of the License at
- * 
+ *
  *      http://www.apache.org/licenses/LICENSE-2.0
- * 
+ *
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -39,6 +39,8 @@ public abstract class ScoreScript {
 	public interface LeafFactory {
 		boolean needs_score();
 
+		boolean needs_termStats();
+
 		ScoreScript newInstance(DocReader reader) throws IOException;
 	}
 

Ghidra/Features/PyGhidra/src/main/py/README.md

@@ -220,9 +220,9 @@ def program_loader() -> "ProgramLoader.Builder":
     """
 ```
 
-### pyghidra.monitor()
+### pyghidra.task_monitor()
 ```python
-def monitor(
+def task_monitor(
         timeout: Optional[int] = None
     ) -> "PyGhidraTaskMonitor":
     """
@@ -288,22 +288,22 @@ with pyghidra.open_project(os.environ["GHIDRA_PROJECT_DIR"], "ExampleProject", c
         for f in fs.files(lambda f: "os/" in f.path and f.name.startswith("decompile")):
             loader = loader.source(f.getFSRL()).projectFolderPath("/" + f.parentFile.name)
             with loader.load() as load_results:
-                load_results.save(pyghidra.monitor())
+                load_results.save(pyghidra.task_monitor())
 
     # Analyze the windows decompiler program for a maximum of 10 seconds
     with pyghidra.program_context(project, "/win_x86_64/decompile.exe") as program:
         analysis_props = pyghidra.analysis_properties(program)
         with pyghidra.transaction(program):
             analysis_props.setBoolean("Non-Returning Functions - Discovered", False)
-        analysis_log = pyghidra.analyze(program, pyghidra.monitor(10))
-        program.save("Analyzed", pyghidra.monitor())
+        analysis_log = pyghidra.analyze(program, pyghidra.task_monitor(10))
+        program.save("Analyzed", pyghidra.task_monitor())
     
     # Walk the project and set a property in each decompiler program
     def set_property(domain_file, program):
         with pyghidra.transaction(program):
             program_info = pyghidra.program_info(program)
             program_info.setString("PyGhidra Property", "Set by PyGhidra!")
-        program.save("Setting property", pyghidra.monitor())
+        program.save("Setting property", pyghidra.task_monitor())
     pyghidra.walk_programs(project, set_property, program_filter=lambda f, p: p.name.startswith("decompile"))
 
     # Load some bytes as a new program
@@ -312,7 +312,7 @@ with pyghidra.open_project(os.environ["GHIDRA_PROJECT_DIR"], "ExampleProject", c
     loader = pyghidra.program_loader().project(project).source(my_bytes).name("my_bytes")
     loader = loader.loaders("BinaryLoader").language("DATA:LE:64:default")
     with loader.load() as load_results:
-        load_results.save(pyghidra.monitor())
+        load_results.save(pyghidra.task_monitor())
 
     # Run a GhidraScript
     pyghidra.ghidra_script(f"{os.environ['GHIDRA_SCRIPTS_DIR']}/HelloWorldScript.java", project)
@@ -564,6 +564,8 @@ __3.0.0:__
   via the `support/launch.properties` file), it will restore `sys.modules` to its prior state after
   a PyGhidra script is run so the next time the script is run, it freshly loads all of its imported
   modules again. This is experimental and should only be enabled if necessary.
+* Changed JPype dependency to be fixed at version 1.5.2 to avoid 
+  [a possible Windows crash on 1.6.0](https://github.com/jpype-project/jpype/issues/1316)
 
 __2.2.1:__
 * PyGhidra now launches with the current working directory removed from `sys.path` to prevent

Ghidra/Features/PyGhidra/src/main/py/pyproject.toml

@@ -30,7 +30,7 @@ classifiers = [
     "Programming Language :: Python :: 3.13",
 ]
 dependencies = [
-    "Jpype1>=1.5.2",
+    "Jpype1==1.5.2",
     "packaging"
 ]
 

Ghidra/Features/PyGhidra/src/main/py/src/pyghidra/__init__.py

@@ -58,7 +58,7 @@ from pyghidra.core import run_script, open_program
 from pyghidra.api import start, started
 from pyghidra.api import open_project, open_filesystem, consume_program, program_context, analyze
 from pyghidra.api import ghidra_script, transaction, analysis_properties, program_info
-from pyghidra.api import program_loader, monitor, walk_project, walk_programs
+from pyghidra.api import program_loader, task_monitor, walk_project, walk_programs
 from pyghidra.launcher import DeferredPyGhidraLauncher, GuiPyGhidraLauncher, HeadlessPyGhidraLauncher
 from pyghidra.script import get_current_interpreter
 from pyghidra.version import ApplicationInfo, ExtensionDetails
@@ -69,6 +69,6 @@ __all__ = [
     "DeferredPyGhidraLauncher", "ExtensionDetails", "GuiPyGhidraLauncher", 
     "HeadlessPyGhidraLauncher", "start", "started", "open_project", "open_filesystem", 
     "consume_program", "program_conext", "analyze", "ghidra_script", "transaction", 
-    "analysis_properties", "program_info", "program_loader", "monitor", "walk_project", 
+    "analysis_properties", "program_info", "program_loader", "task_monitor", "walk_project", 
     "walk_programs"
 ]

Ghidra/Features/PyGhidra/src/main/py/src/pyghidra/api.py

@@ -97,7 +97,7 @@ def open_filesystem(
     
     service = FileSystemService.getInstance()
     fsrl = service.getLocalFS().getLocalFSRL(File(path))
-    fs = service.openFileSystemContainer(fsrl, monitor())
+    fs = service.openFileSystemContainer(fsrl, task_monitor())
     if fs is None:
         raise ValueError(f'"{fsrl}" is not a supported GFileSystem!')
     return fs
@@ -131,7 +131,7 @@ def consume_program(
     df = project_data.getFile(path)
     if df is None:
         raise FileNotFoundError(f'"{path}" does not exist in the Project')
-    dobj = df.getDomainObject(consumer, True, False, monitor())
+    dobj = df.getDomainObject(consumer, True, False, task_monitor())
     program_cls = Program.class_
     if not program_cls.isAssignableFrom(dobj.getClass()):
         dobj.release(consumer)
@@ -174,7 +174,7 @@ def analyze(
     from ghidra.app.plugin.core.analysis import AutoAnalysisManager
     
     if monitor is None:
-        monitor = monitor()
+        monitor = task_monitor()
         
     with transaction(program, "Analyze"):
         GhidraScriptUtil.acquireBundleHostReference()
@@ -236,7 +236,7 @@ def ghidra_script(
         controls = ScriptControls(
             PrintWriter(stdout_string_writer, True),
             PrintWriter(stderr_string_writer, True),
-            monitor()
+            task_monitor()
         )
         script.setScriptArgs(script_args)
         script.execute(state, controls)
@@ -300,7 +300,7 @@ def program_loader() -> "ProgramLoader.Builder":
     from ghidra.app.util.importer import ProgramLoader
     return ProgramLoader.builder()
 
-def monitor(
+def task_monitor(
         timeout: Optional[int] = None
     ) -> "PyGhidraTaskMonitor":
     """