README.md 1.39 KB
Newer Older
Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
1
2
# Common CI/CD routines re-used in [CC Workflows](https://gitlab.astro.unige.ch/integral/cc-workflows)

Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
3
The workflows are mostly of INTEGRAL docker containers, which are [publicly available](https://hub.docker.com/orgs/integralsw). 
Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
4

Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
5
6
7
8
9
10
11
12
13
14
15
16
## Makefile:

Everybody likes to type "make". Also the make's target completing tracking is great and is a progenitor of similar techniques in other workflow mangment systems. But Makefile systax is another level of quoting hell, so all processes are actually described in [make-functions.sh](make-functions.sh)
In general, for every "make XXX" there is an "XXX" bash function in [make-functions.sh](make-functions.sh).

## make-functions.sh

- *build* build the container, if needed

- *run-one* runs notebook, using [oda evaluation](https://github.com/cdcihub/oda-kb).

## image
Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
17
18
19

integralsw/osa-python is the *base* docker image

Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
20
21
22
## nb2workflow

[nb2workflow](https://github.com/volodymyrss/nb2workflow) adapts notebooks as workflows, either [CWL](https://www.commonwl.org/) jobs or [OpenAPI](https://swagger.io/specification/) services.
Volodymyr SAVCHENKO's avatar
Volodymyr SAVCHENKO committed
23
24
25
26
27
28
29
30

extract of some of the functions useful here:

- *nbrun* runs notebook in a one-shot way. This can be done for test locally, and it is also done inside CWL container job (e.g. on REANA)

- *nb2service* runs a discoverable service that can execute the notebook on demand.
 
- *nb2worker* builds a container, either one-shot (for CWL) or service