Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch extension index entry format from "s4ext" to "json" | ⚠️ Changes removed from main #7629

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions Docs/conf.py
Expand Up @@ -39,6 +39,7 @@
"sphinx_markdown_tables",
"notfound.extension", # Show a better 404 page when an invalid address is entered
"sphinx_rtd_theme",
"sphinx-jsonschema",
]

suppress_warnings = [
Expand Down
62 changes: 54 additions & 8 deletions Docs/developer_guide/extensions.md
Expand Up @@ -206,11 +206,9 @@ adding a playback button using [this free service](https://addplaybuttontoimage.
- Complete the [extension submission checklist](https://github.com/Slicer/ExtensionsIndex/blob/main/.github/PULL_REQUEST_TEMPLATE.md#todo-list-for-submitting-a-new-extension)) then submit it to the Slicer Extensions Index:
- Submit the extension to the Extensions Index:
- Fork ExtensionIndex repository on GitHub by clicking ''Fork'' button on the [Slicer Extensions Index](https://github.com/Slicer/ExtensionsIndex) page
- Create an [extension description (s4ext) file](#extension-description-file)
- If the extension was built then you can find the automatically generated extension description in the build folder
- If the extension was not built then create the extension description file manually, using a text editor
- Add your .s4ext file to your forked repository: it can be done using a git client or simply by clicking ''Upload files'' button
- To make the extension appear in the latest Slicer Preview Release: upload the file into the `master` branch.
- Create an [extension catalog entry file (json)](#extension-catalog-entry-file)
- Add your `.json` file to your forked repository: it can be done using a git client or simply by clicking ''Upload files'' button
- To make the extension appear in the latest Slicer Preview Release: upload the file into the `main` branch.
- To make the extension appear in the latest Slicer Stable Release: upload the file into the branch corresponding to the stable release version, for example: `4.10`.
- Create a pull request: by clicking ''Create pull request'' button
- Follow the instructions in the pull request template
Expand All @@ -227,8 +225,55 @@ After installing an extension, the directories are added to revision-specific se
- Folders containing modules bundled within an extension are added to Modules / AdditionalPaths. This ensures that libraries associated with modules are found.
- Folders containing third-party dynamic libraries, Python libraries, etc. are added to LibraryPaths, Paths, PYTHONPATH, QT_PLUGIN_PATH. This ensures that libraries associated with modules can be successfully loaded.

(extension-catalog-entry-file)=
## Extension catalog entry file

An extension catalog entry file is a JSON file describing how to build and package a Slicer extension.

It is named after the extension name (e.g `ExtensionName.json`) and may include the following entries:


```{list-table}
:header-rows: 1

* - Name
- Description
- Required
* - category
- Extension category.
- Y
* - scm_url
- Read-only url used to checkout the extension source code.
- Y
* - scm_revision
- Revision allowing to checkout the expected source code.
- N
* - scm_type
- Type of revision control system. Default to `git`.
- N
* - build_dependencies
- List of extensions required to build this extension. For example: `["extensionA", "extensionB"]`.
- N
* - build_subdirectory
- Name of the inner build directory in case of superbuild based extension. Default to `.`.
- N
* - enabled
- Specify if the extension should be enabled after its installation. Valid values are `true` (default) = enabled; `false` = disabled.
- N
```

The full schema for the extension catalog entry file is below:

```{jsonschema} ../../Schema/slicer-extension-catalog-entry-schema-v1.0.0.json
jcfr marked this conversation as resolved.
Show resolved Hide resolved
```

(extension-description-file)=
## Extension description file

:::{warning}
This file is used internally by [](#extensions-build-system) and it has been superseded by [](#extension-catalog-entry-file).
:::

An extension description file is a text file with `s4ext` extension allowing to specify metadata associated with an extensions.

The description file is automatically generated by the build system in the build tree of the extension from the metadata specified in the top-level `CMakeLists.txt` file in the source code of the extension. Since the description is a very simple text file, the description can be created using a text editor. It may be simpler to use a text editor to create the file if the extension only contains scripted modules, which can be developed without building the extension.
Expand Down Expand Up @@ -323,14 +368,15 @@ Until August 2021, a Midas-based server at `https://slicer.kitware.com/midas3` w

## Extensions Index

The ExtensionsIndex is a repository containing a list of [extension description files](#extension-description-file) `*.s4ext` used by the Slicer [Extensions build system](#extensions-build-system) to build, test, package and upload extensions on the [extensions server](#extensions-server).
The ExtensionsIndex is a repository containing a list of [extension catalog entry files](#extension-catalog-entry-file) `*.json` used by the Slicer [Extensions build system](#extensions-build-system) to build, test, package and upload extensions on the [extensions server](#extensions-server).

The ExtensionsIndex is hosted on GitHub: <https://github.com/Slicer/ExtensionsIndex>

Each branch of the repository contains extension description files that corresponds to the same branch in the Slicer repository. For example, `main` branch contains descriptions for Slicer `main` branch, and `4.11` branch contains extension descriptions for Slicer's `4.11` branch.

Extension developers have to make sure that the extension description in each branch of the Extensions index is compatible with the corresponding Slicer version. Extension developers often create the same branches (`main`, `4.11`, `4.13`, ...) in their repository and they specify this branch name in the extensions descriptor file.

(extensions-build-system)=
## Extensions build system

The extensions build system allows to drive the build, test, packaging and upload of Slicer extensions.
Expand Down Expand Up @@ -616,7 +662,7 @@ endif()

if(NOT Slicer_SOURCE_DIR)
set(EXTENSION_NAME EmptyExtensionTemplate)
set(EXTENSION_HOMEPAGE "https://www.slicer.org/wiki/Documentation/Nightly/Extensions/EmptyExtensionTemplate") set(EXTENSION_CATEGORY "Examples")
set(EXTENSION_HOMEPAGE "https://www.slicer.org/wiki/Documentation/Nightly/Extensions/EmptyExtensionTemplate")
set(EXTENSION_CONTRIBUTORS "Jean-Christophe Fillion-Robin (Kitware)")
set(EXTENSION_DESCRIPTION "This is an example of extension bundling N module(s)")
set(EXTENSION_ICONURL "http://viewvc.slicer.org/viewvc.cgi/Slicer4/trunk/Extensions/Testing/EmptyExtensionTemplate/EmptyExtensionTemplate.png?revision=21746&view=co")
Expand Down Expand Up @@ -644,7 +690,7 @@ find_package(Slicer COMPONENTS ConfigurePrerequisites)

project(EmptyExtensionTemplate)

set(EXTENSION_HOMEPAGE "https://www.slicer.org/wiki/Documentation/Nightly/Extensions/EmptyExtensionTemplate")set(EXTENSION_CATEGORY "Examples")
set(EXTENSION_HOMEPAGE "https://www.slicer.org/wiki/Documentation/Nightly/Extensions/EmptyExtensionTemplate")
set(EXTENSION_CONTRIBUTORS "Jean-Christophe Fillion-Robin (Kitware)")
set(EXTENSION_DESCRIPTION "This is an example of empty extension")
set(EXTENSION_ICONURL "http://viewvc.slicer.org/viewvc.cgi/Slicer4/trunk/Extensions/Testing/EmptyExtensionTemplate/EmptyExtensionTemplate.png?revision=21746&view=co")
Expand Down
9 changes: 9 additions & 0 deletions Extensions/CLIExtensionTemplate.json
@@ -0,0 +1,9 @@
{
jcfr marked this conversation as resolved.
Show resolved Hide resolved
"$schema": "https://raw.githubusercontent.com/Slicer/Slicer/main/Schemas/slicer-extension-catalog-item-schema-v1.0.0.json#",
"build_dependencies": [],
"build_subdirectory": ".",
"category": "Examples",
"scm_type": "local",
"scm_revision": "",
"scm_url": "Testing/CLIExtensionTemplate"
}
44 changes: 0 additions & 44 deletions Extensions/CLIExtensionTemplate.s4ext

This file was deleted.

2 changes: 1 addition & 1 deletion Extensions/CMake/CMakeLists.txt
Expand Up @@ -47,7 +47,7 @@ include(SlicerInitializeBuildType)
# Options
#-----------------------------------------------------------------------------
set(Slicer_BUILD_EXTENSIONS ON CACHE INTERNAL "Build Slicer extensions.")
set(Slicer_EXTENSION_DESCRIPTION_DIR "${default_extension_description_dir}" CACHE PATH "Path to folder containing *.s4ext files to consider.")
set(Slicer_EXTENSION_DESCRIPTION_DIR "${default_extension_description_dir}" CACHE PATH "Path to folder containing *.json files to consider.")
set(Slicer_LOCAL_EXTENSIONS_DIR "${default_local_extension_dir}" CACHE STRING "Path to extension sources locally available")
option(BUILD_TESTING "Test extensions." ${Slicer_BUILD_TESTING})

Expand Down
28 changes: 9 additions & 19 deletions Extensions/CMake/SlicerBlockBuildPackageAndUploadExtension.cmake
Expand Up @@ -23,7 +23,7 @@ set(expected_defined_vars
CDASH_PROJECT_NAME
EXTENSION_BUILD_OPTIONS_STRING
EXTENSION_BUILD_SUBDIRECTORY
EXTENSION_ENABLED
EXTENSION_CATALOG_ENTRY_FILE
EXTENSION_NAME
EXTENSION_SOURCE_DIR
EXTENSION_SUPERBUILD_BINARY_DIR
Expand Down Expand Up @@ -62,20 +62,6 @@ include(CMakeParseArguments)
include(SlicerCTestUploadURL)
include(UseSlicerMacros) # for slicer_setting_variable_message

#-----------------------------------------------------------------------------
set(optional_vars
EXTENSION_CATEGORY
EXTENSION_CONTRIBUTORS
EXTENSION_DESCRIPTION
EXTENSION_HOMEPAGE
EXTENSION_ICONURL
EXTENSION_SCREENSHOTURLS
EXTENSION_STATUS
)
foreach(var ${optional_vars})
slicer_setting_variable_message(${var})
endforeach()

#-----------------------------------------------------------------------------
# Set site name and force to lower case
if("${CTEST_SITE}" STREQUAL "")
Expand Down Expand Up @@ -152,7 +138,6 @@ CTEST_MODEL:STRING=${CTEST_MODEL}
GIT_EXECUTABLE:FILEPATH=${GIT_EXECUTABLE}
Slicer_DIR:PATH=${Slicer_DIR}
Slicer_EXTENSIONS_TRACK_QUALIFIER:STRING=${Slicer_EXTENSIONS_TRACK_QUALIFIER}
EXTENSION_DEPENDS:STRING=${EXTENSION_DEPENDS}
")

if(APPLE)
Expand All @@ -171,11 +156,11 @@ CMAKE_JOB_POOL_LINK:STRING=${CMAKE_JOB_POOL_LINK}")
endif()

# If needed, convert to a list
list(LENGTH EXTENSION_DEPENDS _count)
list(LENGTH EXTENSION_BUILD_DEPENDENCIES _count)
if(_count EQUAL 1)
string(REPLACE " " ";" EXTENSION_DEPENDS ${EXTENSION_DEPENDS})
string(REPLACE " " ";" EXTENSION_BUILD_DEPENDENCIES ${EXTENSION_BUILD_DEPENDENCIES})
endif()
foreach(dep ${EXTENSION_DEPENDS})
foreach(dep ${EXTENSION_BUILD_DEPENDENCIES})
set(cmakecache_content "${cmakecache_content}
${dep}_BINARY_DIR:PATH=${${dep}_BINARY_DIR}
${dep}_BUILD_SUBDIRECTORY:STRING=${${dep}_BUILD_SUBDIRECTORY}
Expand Down Expand Up @@ -237,6 +222,11 @@ else()
set(package_target "packageupload")
endif()
if(RUN_CTEST_PACKAGES)
get_filename_component(catalog_entry_filename ${EXTENSION_CATALOG_ENTRY_FILE} NAME)
file(COPY_FILE
${EXTENSION_CATALOG_ENTRY_FILE}
${EXTENSION_SUPERBUILD_BINARY_DIR}/${EXTENSION_BUILD_SUBDIRECTORY}/${catalog_entry_filename}
)
ctest_build(
TARGET ${package_target}
BUILD ${EXTENSION_SUPERBUILD_BINARY_DIR}/${EXTENSION_BUILD_SUBDIRECTORY}
Expand Down
48 changes: 20 additions & 28 deletions Extensions/CMake/SlicerBlockBuildPackageAndUploadExtensions.cmake
Expand Up @@ -48,57 +48,48 @@ include(SlicerFunctionExtractExtensionDescription)
include(SlicerBlockUploadExtensionPrerequisites) # Common to all extensions

#-----------------------------------------------------------------------------
# Collect extension description file (*.s4ext)
# Collect extension description file (*.json)
#-----------------------------------------------------------------------------
file(GLOB s4extfiles "${Slicer_EXTENSION_DESCRIPTION_DIR}/*.s4ext")
file(GLOB catalog_entry_files "${Slicer_EXTENSION_DESCRIPTION_DIR}/*.json")

# Get the dependency information of each extension
set(EXTENSION_LIST)
foreach(file ${s4extfiles})
foreach(file ${catalog_entry_files})
message(STATUS "Extension:${file}")

# Extract extension description info
slicerFunctionExtractExtensionDescription(EXTENSION_FILE ${file} VAR_PREFIX EXTENSION)
slicerFunctionExtractExtensionDescriptionFromJson(EXTENSION_FILE ${file} VAR_PREFIX EXTENSION)
jcfr marked this conversation as resolved.
Show resolved Hide resolved

# Extract file basename
get_filename_component(EXTENSION_NAME ${file} NAME_WE)
if("${EXTENSION_NAME}" STREQUAL "")
message(WARNING "Failed to extract extension name associated with file: ${file}")
else()
list(APPEND EXTENSION_LIST ${EXTENSION_NAME})
string(REGEX REPLACE "^NA$" "" EXTENSION_EXT_DEPENDS "${EXTENSION_EXT_DEPENDS}")
set(EXTENSION_${EXTENSION_NAME}_DEPENDS ${EXTENSION_EXT_DEPENDS})
set(EXTENSION_${EXTENSION_NAME}_BUILD_DEPENDENCIES ${EXTENSION_EXT_BUILD_DEPENDENCIES})
set(${EXTENSION_NAME}_BUILD_SUBDIRECTORY ${EXTENSION_FILE_BUILD_SUBDIRECTORY})
endif()
endforeach()

# Sort extensions
include(TopologicalSort)
topological_sort(EXTENSION_LIST "EXTENSION_" "_DEPENDS")
topological_sort(EXTENSION_LIST "EXTENSION_" "_BUILD_DEPENDENCIES")

foreach(EXTENSION_NAME ${EXTENSION_LIST})
# Set extension description filename using EXTENSION_NAME
set(file ${Slicer_EXTENSION_DESCRIPTION_DIR}/${EXTENSION_NAME}.s4ext)
set(file ${Slicer_EXTENSION_DESCRIPTION_DIR}/${EXTENSION_NAME}.json)

# Extract extension description info
slicerFunctionExtractExtensionDescription(EXTENSION_FILE ${file} VAR_PREFIX EXTENSION)
set(EXTENSION_CATEGORY ${EXTENSION_EXT_CATEGORY})
set(EXTENSION_STATUS ${EXTENSION_EXT_STATUS})
set(EXTENSION_ICONURL ${EXTENSION_EXT_ICONURL})
set(EXTENSION_CONTRIBUTORS ${EXTENSION_EXT_CONTRIBUTORS})
set(EXTENSION_DESCRIPTION ${EXTENSION_EXT_DESCRIPTION})
set(EXTENSION_HOMEPAGE ${EXTENSION_EXT_HOMEPAGE})
set(EXTENSION_SCREENSHOTURLS ${EXTENSION_EXT_SCREENSHOTURLS})
set(EXTENSION_ENABLED ${EXTENSION_EXT_ENABLED})
set(EXTENSION_DEPENDS ${EXTENSION_EXT_DEPENDS})
# Extract extension catalog entry fields setting "EXTENSION_EXT_*" variables
# in the current scope.
slicerFunctionExtractExtensionDescriptionFromJson(EXTENSION_FILE ${file} VAR_PREFIX EXTENSION)

# Ensure extensions depending on this extension can lookup the corresponding
# _DIR and _BUILD_SUBDIRECTORY variables.
set(${EXTENSION_NAME}_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/${EXTENSION_NAME}-build)
set(${EXTENSION_NAME}_BUILD_SUBDIRECTORY ${EXTENSION_EXT_BUILD_SUBDIRECTORY})

message(STATUS "Configuring extension: ${EXTENSION_NAME}")
if("${EXTENSION_EXT_SCM}" STREQUAL "" AND "${EXTENSION_EXT_SCMURL}" STREQUAL "")
if("${EXTENSION_EXT_SCM_TYPE}" STREQUAL "" AND "${EXTENSION_EXT_SCM_URL}" STREQUAL "")
message(WARNING "Failed to extract extension information associated to file: ${file}")
continue()
endif()
Expand All @@ -108,30 +99,30 @@ foreach(EXTENSION_NAME ${EXTENSION_LIST})

# Set external project DEPENDS parameter
set(EP_ARG_EXTENSION_DEPENDS)
if(NOT "${EXTENSION_DEPENDS}" STREQUAL "")
set(EP_ARG_EXTENSION_DEPENDS DEPENDS ${EXTENSION_DEPENDS})
if(NOT "${EXTENSION_EXT_BUILD_DEPENDENCIES}" STREQUAL "")
set(EP_ARG_EXTENSION_DEPENDS DEPENDS ${EXTENSION_EXT_BUILD_DEPENDENCIES})
endif()

#-----------------------------------------------------------------------------
# Configure extension source download wrapper script
#-----------------------------------------------------------------------------
set(ext_ep_options_repository)
set(ext_revision ${EXTENSION_EXT_SCMREVISION})
if("${EXTENSION_EXT_SCM}" STREQUAL "git")
set(ext_revision ${EXTENSION_EXT_SCM_REVISION})
if("${EXTENSION_EXT_SCM_TYPE}" STREQUAL "git")
set(EXTENSION_SOURCE_DIR ${CMAKE_CURRENT_BINARY_DIR}/${EXTENSION_NAME})
if("${ext_revision}" STREQUAL "")
set(ext_revision "origin/main")
endif()
set(ext_ep_options_repository
GIT_REPOSITORY ${EXTENSION_EXT_SCMURL} GIT_TAG ${ext_revision})
elseif("${EXTENSION_EXT_SCM}" STREQUAL "local")
set(EXTENSION_SOURCE_DIR ${EXTENSION_EXT_SCMURL})
GIT_REPOSITORY ${EXTENSION_EXT_SCM_URL} GIT_TAG ${ext_revision})
elseif("${EXTENSION_EXT_SCM_TYPE}" STREQUAL "local")
set(EXTENSION_SOURCE_DIR ${EXTENSION_EXT_SCM_URL})
if(NOT IS_ABSOLUTE ${EXTENSION_SOURCE_DIR})
set(EXTENSION_SOURCE_DIR ${Slicer_LOCAL_EXTENSIONS_DIR}/${EXTENSION_SOURCE_DIR})
endif()
set(ext_ep_download_command DOWNLOAD_COMMAND "")
else()
message(WARNING "Unknown type of SCM [${EXTENSION_EXT_SCM}] associated with extension named ${EXTENSION_NAME} - See file ${file}")
message(WARNING "Unknown type of SCM [${EXTENSION_EXT_SCM_TYPE}] associated with extension named ${EXTENSION_NAME} - See file ${file}")
continue()
endif()
if(NOT "${ext_ep_options_repository}" STREQUAL "")
Expand Down Expand Up @@ -189,6 +180,7 @@ foreach(EXTENSION_NAME ${EXTENSION_LIST})
if(NOT DEFINED CTEST_MODEL)
set(CTEST_MODEL "Experimental")
endif()
set(EXTENSION_CATALOG_ENTRY_FILE ${file})
include(SlicerBlockUploadExtension)
if(Slicer_UPLOAD_EXTENSIONS)
set(wrapper_command ${EXTENSION_UPLOAD_WRAPPER_COMMAND})
Expand Down
14 changes: 3 additions & 11 deletions Extensions/CMake/SlicerBlockUploadExtension.cmake
Expand Up @@ -5,7 +5,6 @@ set(expected_defined_vars
EXTENSION_BITNESS
EXTENSION_BUILD_SUBDIRECTORY
EXTENSION_COMPILER
EXTENSION_ENABLED
EXTENSION_NAME
EXTENSION_SOURCE_DIR
EXTENSION_SUPERBUILD_BINARY_DIR
Expand Down Expand Up @@ -87,18 +86,11 @@ set(CDASH_PROJECT_NAME \"${CDASH_PROJECT_NAME}\")
set(EXTENSION_BUILD_OPTIONS_STRING \"${EXTENSION_BITNESS}bits-Qt${Slicer_QT_VERSION_MAJOR}.${Slicer_QT_VERSION_MINOR}\")
set(EXTENSION_COMPILER \"${EXTENSION_COMPILER}\")
set(EXTENSION_NAME \"${EXTENSION_NAME}\")
set(EXTENSION_CATEGORY \"${EXTENSION_CATEGORY}\")
set(EXTENSION_STATUS \"${EXTENSION_STATUS}\")
set(EXTENSION_ICONURL \"${EXTENSION_ICONURL}\")
set(EXTENSION_CONTRIBUTORS \"${EXTENSION_CONTRIBUTORS}\")
set(EXTENSION_DESCRIPTION \"${EXTENSION_DESCRIPTION}\")
set(EXTENSION_HOMEPAGE \"${EXTENSION_HOMEPAGE}\")
set(EXTENSION_SCREENSHOTURLS \"${EXTENSION_SCREENSHOTURLS}\")
set(EXTENSION_CATALOG_ENTRY_FILE \"${EXTENSION_CATALOG_ENTRY_FILE}\")
set(EXTENSION_SOURCE_DIR \"${EXTENSION_SOURCE_DIR}\")
set(EXTENSION_SUPERBUILD_BINARY_DIR \"${EXTENSION_SUPERBUILD_BINARY_DIR}\")
set(EXTENSION_BUILD_SUBDIRECTORY \"${EXTENSION_BUILD_SUBDIRECTORY}\")
set(EXTENSION_ENABLED \"${EXTENSION_ENABLED}\")
set(EXTENSION_DEPENDS \"${EXTENSION_DEPENDS}\")
set(EXTENSION_BUILD_DEPENDENCIES \"${EXTENSION_BUILD_DEPENDENCIES}\")
set(Slicer_CMAKE_DIR \"${Slicer_CMAKE_DIR}\")
set(Slicer_DIR \"${Slicer_DIR}\")
set(Slicer_EXTENSIONS_TRACK_QUALIFIER \"${Slicer_EXTENSIONS_TRACK_QUALIFIER}\")
Expand All @@ -115,7 +107,7 @@ set(CMAKE_JOB_POOLS \"${CMAKE_JOB_POOLS}\")
set(CMAKE_JOB_POOL_COMPILE \"${CMAKE_JOB_POOL_COMPILE}\")
set(CMAKE_JOB_POOL_LINK \"${CMAKE_JOB_POOL_LINK}\")")
endif()
foreach(dep ${EXTENSION_DEPENDS})
foreach(dep ${EXTENSION_BUILD_DEPENDENCIES})
set(EXTENSION_COMMAND_ARG_LIST "${EXTENSION_COMMAND_ARG_LIST}
set(${dep}_BINARY_DIR \"${${dep}_BINARY_DIR}\")
set(${dep}_BUILD_SUBDIRECTORY \"${${dep}_BUILD_SUBDIRECTORY}\")
Expand Down