consumerdir.sh is a bash script that works in two modes:
- Go through all files in given directories (recursively, if
-ris specified) and sent each to docspell.
- Watch one or more directories for new files and upload them to docspell.
It can watch or go through one or more directories. Files can be uploaded to multiple urls.
Run the script with the
--help option, to see a short help
text. The help text will also show the values for any given option.
The script requires
curl for uploading. It requires the
inotifywait command if directories should be watched for new
Example for watching two directories:
./tools/consumedir.sh --path ~/Downloads --path ~/pdfs -m -dv http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ
The script by default watches the given directories. If the
--once option is used, it will instead go through these directories
and upload all files in there.
Example for uploading all immediatly (the same as above only with
$ consumedir.sh -o --path ~/Downloads --path ~/pdfs/ -m -dv http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ
When given the
--integration option, the script changes its
behaviour slightly to work with the integration
-i is given, it implies
-r – so the directories are
watched or traversed recursively. The script then assumes that there
is a subfolder with the collective name. Files must not be placed
directly into a folder given by
-p, but below a sub-directory that
matches a collective name. In order to know for which collective the
file is, the script uses the first subfolder.
If the endpoint is protected, these credentials can be specified as
--iheader, respectively. The format is for
<name>:<value>, so the username cannot contain a colon
character (but the password can).
$ consumedir.sh -i -iheader 'Docspell-Integration:test123' -m -p ~/Downloads/ http://localhost:7880/api/v1/open/integration/item
The url is the integration endpoint url without the collective, as this is amended by the script.
This watches the folder
~/Downloads. If a file is placed in this
folder directly, say
~/Downloads/test.pdf the upload will fail,
because the collective cannot be determined. Create a subfolder below
~/Downloads with the name of a collective, for example
~/Downloads/family and place files somewhere below this
-m option, the script will not upload files that already
exist at docspell. For this the
sha256sum command is required.
So you can move and rename files in those folders without worring about duplicates. This allows to keep your files organized using the file-system and have them mirrored into docspell as well.
The script can be used with systemd to run as a service. This is an example unit file:
[Unit] After=networking.target Description=Docspell Consumedir [Service] Environment="PATH=/set/a/path" ExecStart=/bin/su -s /bin/bash someuser -c "consumedir.sh --path '/a/path/' -m 'http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ'"
This unit file is just an example, it needs some fiddling. It assumes
an existing user
someuser that is used to run this service. The url
http://localhost:7880/api/v1/open/upload/... is an anonymous upload
url as described here.
The provided docker image runs this script to watch a single
./docs in current directory, for new files. If a new file
is detected, it is pushed to docspell.
This utilizes the integration
endpoint, which is enabled in the
config file, to allow uploading documents for all collectives. A
subfolder must be created for each registered collective. The docker
containers are configured to use http-header protection for the
integration endpoint. This requires you to provide a secret, that is
shared between the rest-server and the
consumedir.sh script. This
can be done by defining an environment variable which gets picked up
by the containers defined in
export DOCSPELL_HEADER_VALUE="my-secret" docker-compose up
Now you can create a folder
./docs/<collective-name> and place all
files in there that you want to import. Once dropped in this folder
consumedir container will push it to docspell.