Automating Publication on Zenodo

Introduction

When a project is hosted on GitHub, the integration with Zenodo is native, making it easy to generate a DOI (Digital Object Identifier) for each new version. However, GitLab does not benefit from this direct integration, requiring a tedious manual process to publish and assign a DOI to new project versions.

This documentation presents the automation solution implemented to simplify and secure the publication of RRINGG project versions on Zenodo via a CI/CD pipeline.

Challenges of the Manual Process

Before automation, publishing on Zenodo involved several time-consuming and error-prone steps. Here are the main steps of the manual process:

  1. Creating a tag corresponding to the version to be published.

  2. Creating a release following semantic versioning rules.

  3. Cloning the branch associated with the version.

  4. Archiving the repository as a compressed file (.zip).

  5. Creating a new project on Zenodo.

  6. Manually adding project metadata (title, authors, description, etc.) and generating the DOI.

  7. Uploading the previously generated archive.

  8. Validating and finalizing the publication on Zenodo.

This process, in addition to being lengthy, was prone to human errors, such as missing metadata or inconsistencies between the published version and the entered information.

Automation Solution

To make this process more efficient and reliable, we implemented a CI/CD pipeline to automatically manage the publication of project versions on Zenodo. This solution relies on the `gitlab2zenodo` tool, which interacts with the Zenodo API to publish repositories and assign DOIs automatically.

Benefits of Automation

  • Time-saving: The process is fully automated, from tag creation to publication on Zenodo.

  • Error reduction: Metadata is extracted directly from a structured file, avoiding input errors.

  • Reproducibility: Each version is published consistently and in compliance with semantic versioning rules.

  • Seamless integration: The pipeline integrates directly with GitLab, requiring no manual intervention.

Configuring the CI/CD Pipeline

GitLab Environment at the Université Grenoble Alpes

The CI/CD pipeline is configured on a dedicated GitLab instance named gricad-gitlab. This instance is equipped with runners to execute CI/CD jobs.

Available Runners

  • Instance runners: These runners are available for all groups and projects on the instance.

    • Number of available runners: 1

    • Machine name: #2774 (Mt_XxaLU)

    • New runner configured: new gricad-gitlab-ci nova

  • Group runners: These runners are shared across projects within the same group.

Installation and Functioning Principles

Setting Up the Automation Environment (One-Time Setup)

  1. Create a Zenodo Account

    • Go to Zenodo and create an account if you don’t already have one.

  2. Generate a Zenodo API Token

  3. Add the Token as a Variable in GitLab - In your GitLab project, navigate to:

    Settings → CI/CD → Variables → Add variable.

    • Fill in the following fields:

      • Key: Token name (e.g., ZENODO_TOKEN).

      • Value: Token value.

      • Options: Check Masked and Expand variable reference.

  4. Create a Metadata File

    • Create a metadata file in Citation File Format (CFF) or .zenodo.json. This file contains essential project information (title, authors, version, description, etc.).

    • Reference: Zenodo Documentation.

Functioning Principles (Pipeline Execution)

Once the setup steps are complete, the pipeline runs automatically when a tag is created. Here are the pipeline steps:

  1. Create a Docker Image

    • A Docker image is created with Python 3.9.18 to ensure an isolated environment.

  2. Create and Activate a Virtual Environment

    • A Python virtual environment is created and activated to install dependencies.

  3. Install `gitlab2zenodo`

    • The gitlab2zenodo tool is installed via PyPI: .. code-block:: bash

      pip install gitlab2zenodo

  4. Archive the Project - The project is archived into a .zip file for upload to Zenodo.

  5. Publish on Zenodo - Metadata is extracted from the CITATION.cff file and used to create a publication on Zenodo. - The .zip file is uploaded to Zenodo.

  6. Install `release-cli`

    • The release-cli tool is installed to create a release associated with the tag.

  7. Create a Release

    • A release is created in GitLab, associated with the tag.

  8. Add Additional Metadata

    • Additional metadata (DOI, development status, programming language) are added via the Zenodo interface, and the publication is finalized.

Usage

To publish a new version of the project, simply create a new tag. This can be done via the GitLab graphical interface or the command line.

Via the Graphical Interface

  1. Go to your project on GitLab.

  2. Navigate to: Code → Tags → New tag.

  3. Fill in the following fields:

    • Tag name: Tag name following semantic versioning (e.g., 1.0.0).

    • Create from: Select the source branch.

    • Message: Add a description of the tag.

Via the Command Line

  1. Create a tag locally: .. code-block:: bash

    git tag -a $TAG_NAME -m “TAG_DESCRIPTION”

  2. Push the tag to the remote repository: .. code-block:: bash

    git push origin –tags

Note

Semantic versioning is used to manage project versions. It follows the MAJOR.MINOR.PATCH format:

  • MAJOR (1.x.x): Incompatible changes with previous versions.

  • MINOR (x.1.x): Addition of new features, compatible with previous versions.

  • PATCH (x.x.1): Bug fixes with no impact on compatibility.

Examples:

  • 1.0.0: First stable version of the project.

  • 1.1.0: Addition of new features.

  • 1.1.1: Minor bug fixes in version 1.1.0.