Writing an addon can be divided into two things:
- create the program
- define how to package and run it
The next sections describe both parts. For a quick start, check out the example addons.
As previously written, you can choose a language. The interaction with docspell happens by exchanging JSON data. So, whatever you choose, it should be possible to read and produce JSON with some convenience.
Writing the program🔗
Interface to Docspell🔗
The interface to Docspell is JSON data. The addon receives all inputs as JSON and may return a JSON object as output (via stdout).
An addon can be executed in different contexts. Depending on this, the available inputs differ. The addon always receives one argument, which is a file containing the user supplied data (it may be empty). A user is able to provide data to every addon from the web-ui.
All other things are provided as environment variables. There are environment variables that are always provided and some are only available for specific contexts.
For example, an addon that is executed in the context of an item (maybe after processing or when a user selects an addon to run "on an item"), Docspell prepares all data for the corresponding item and makes it available to the addon. In contrast, an addon executed periodically by a schedule, won't have this data available.
The following environment variables are always provided by Docspell:
ADDON_DIRpoints to the directory containing the extracted addon zip file
TMP_DIRa directory for storing temporary data
OUTPUT_DIRa directory for storing files that should be processed by docspell
CACHE_DIRa directory for storing data that should stay between addon runs
It is very much recommended to always use these environment variables when reading and writing data. This keeps Docspell in control about the exact location.
The working directory will be set to a directory that is also temporary, but please don't rely on that. Use the environment variables.
When executed in the context of an item. Meaning for triggers:
This environment variable points to a JSON file containing information about the current item. If it is run at processing time, it includes all information gathered so far by Docspell.
This environment variable points to a JSON file that contains the user supplied information with an upload request. That is, a user may specify tags or a language when uploading files. This would be in this file.
This is only available for uploads. Trigger
These JSON files contains a list of objects. Each object provides properties about a file - either the original file or the converted pdf. The structure is the same.
These environment variables point to directories that contain the attachment files.
ITEM_PDF_DIRcontains all converted pdf files, the attachment id is the filename
ITEM_ORIGINAL_DIRcontains all original files, the attachment id is the filename
For example, to obtain a converted pdf file, lookup the id in
ITEM_PDF_JSON and then construct the file name via
Session for dsc🔗
An addon may use dsc which requires for many
commands a valid session identifier. Usually this is obtained by
logging in (i.e. using
dsc login). This is not really feasible from
inside an addon, of course. Therefore you can configure an addon to
run on behalf of some user when creating the run configuration.
Docspell then generates a valid session identifier and puts it into
the environment. The dsc tool will pick them up
It will also setup the URL to connect to some restserver. (If you have multiple rest-servers running, it will pick one randomly).
DSC_SESSIONenv variable containing a session identifier. It's validity is coupled on the configured timeout.
DSC_DOCSPELL_URLthe base url to some rest server
That means when using an addon in this way, you can simply use
without worrying about authentication or the correct URL to connect
Docspell doesn't interpret the returncode of an addon, except checking
for being equal to
0 which indicates a successful run.
In order to do change data in Docspell, the addon program can run
dsc (for example) to change some state - like setting tags etc. But
the preferred approach would be to return instructions for Docspell.
Docspell will execute the instructions when the addon terminates
successfully - that is with return code
These instructions are in a JSON object which needs to go to stdout.
You can use stderr in an addon for logging/debugging purposes. But if
collectOutput: true in the descriptior, then stdout must
only return this specific JSON (or nothing, empty output is ignored).
You find the complete structure below. It consists of these parts:
commands: let's you declare actions to do for an item or attachment
files: defines files relative to
OUTPUT_DIRthat should be processed
newItems: declares files relative to
OUTPUT_DIRthat should be processed as new uploads
commands allows to set tags, fields and other things. All parts
are optional, you don't need to return the complete structure. Just
commands or only
files is ok.
An addon must provide an addon descriptior, which is a yaml or json file looking like this:
# The meta section is required. Name and version must not contain # whitespace meta: name: "name-of-addon" version: "2.21" description: | Describe the purpose and how it must be used here # Defining when this addon is run. This is used to guide the user # interface in selecting an addon. At least one is required to specify. # # Possible values: # - scheduled: requires to enter a timer to run this addon periodically # - final-process-item: the final step when processing an item # - final-reprocess-item: the final step when reprocessing an item # - existing-item: A user selects the addon to run on an item triggers: - final-process-item - final-reprocess-item - existing-item # How to build and run this addon (optional). If missing, auto # detection will enable a nix runner if a `flake.nix` is found in the # source root and docker if a `Dockerfile` is found. # # Both runners are compared to what is enabled at the server. runner: # Building the program using nix flakes. This requires a flake.nix # file in the source root with a default package and a flake-enabled # nix on the joex machine. # # The program is build via `nix build`. If the joex machine has # systemd-nspawn installed, it is used to run the addon inside a # container. Otherwise the addon is run directly on the machine. nix: enable: true # Docker based runner can define a custom image to use. If a `build` # key exists pointing to a Dockerfile, the image is build before. If # the docker image is complex, you can build it independently and # provide the pre-build image. # # The program is run via `docker run` passing the arguments to the # addon. Thus it expects the entrypoint to be correctly configured # to the executable. You may use `args` in order to prepend # additional arguments, like the path to an executable if the image # requires that. The joex machine must have docker installed and the # user running joex must be allowed to use docker. You must either # define an image with an appropriate entry point or a dockerfile. docker: enable: false #image: myorg/myimage:latest build: Dockerfile # Trivial runner that simply executes the file specified with # `exec`. Nothing is build before. This runner usually requires that # the joex machine contains all dependencies needed to run the # addon. You may need to install additional software on the machine # running joex. trivial: enable: false exec: src/addon.sh # Optional arguments/options given to the program. The program # receives at least one argument, which is a file to the user input as # supplied in the application. The arguments here are prepended. args: options: # If false, the program is run inside a private network, blocking # traffic to the host and networks reachable from there. This only # applies if the addon can be run inside a container. # # If the addon runs side effects (such as using dsc to set tags), # this must be set to `true`. # # Default is false. networking: true # If true, the stdout of the program is parsed into a JSON structure # that is interpreted as actions executed by the task that runs the # addon. If the addon runs side effects only, set this to `false` # and the output is ignored. # # It is recommended to use this approach, if possible. It allows # docspell itself to apply any changes and the addon can run # completely isolated. # # Default is true. collectOutput: true
Docspell can use different ways to build and run the addon:
trivial. The first two allow to package
the addon in a defined way (with a single dependency, either nix or
docker) and then execute it independently from the underlying system.
This makes it possible to execute the addon on a variety of systems.
This is especially useful for addons that are meant to be public and
reusable by different people.
The "trivial" runner is only executing some program specified in
docspell-addon.yaml, directly on the joex machine (or via
systemd-nspawn). The machine running joex must then provide all
necessary dependencies and it must be compatible to run the addon. It
may be useful especially for personal addons.
Using nix with flakes enabled, is the recommended approach. It is very flexible and reproducible while sharing most dependencies (in contrast to docker where each image contains the same packages again and again).
nix build to build the addon and then executes the
file produced to
For docker it is recommended to provide pre-build images. Docspell can
build images from provided
Dockerfile, but for larger images it
might be better to do this apriori.
Docspell will run the addon using
docker run … passing it only the
user-input file as argument. Thus the image must define an appropriate
The steps below create a minimal addon:
Create a bash script
addon.shwith this content:
Make it executable:
Create a yaml file
docspell-addon.yamlwith this content:
meta: name: "minimal-addon" version: "0.1.0" triggers: - existing-item - scheduled runner: trivial: enable: true exec: addon.sh
Create a zip file containing these two files:
The addon is now ready. Make it available via an url (use some file sharing tool, upload it somewhere etc) and then it can be installed and run.
The minimal example above is good to see what is required, but it is not very useful…. Please see this post about the audio file addon that walks through a more useful addon.
Advantages of "pure" addons🔗
Although the output structure is not set in stone, it is recommended
to use this in contrast to directly changing state via
- outputs of all addons are collected and only applied if all were successful; in contrast side effects are always applied even if the addon fails shortly after
- since addons are executed as joex tasks, their result can be send as events to another http server for further processing.
- addons can run in an isolated environment without network (no data can go out)
Use addons in other addons?🔗
This can be achieved very conveniently by using
nix. If addons are
defined as a nik flake, they can be easily consumed by each other.