Read more

How to checkout submodules in Gitlab CI

Emanuel
June 10, 2021Software engineer at makandra GmbH

Please prefer https://docs.gitlab.com/ee/ci/git_submodules.html with relative submodule paths

Accessing other repositories in Gitlab CI is not straight forward, since the access rights of the current pipeline might not be sufficient enough.

Illustration book lover

Growing Rails Applications in Practice

Check out our e-book. Learn to structure large Ruby on Rails codebases with the tools you already know and love.

  • Introduce design conventions for controllers and user-facing models
  • Create a system for growth
  • Build applications to last
Read more Show archive.org snapshot

One approach is to use project access tokens and clone the repositories via HTTPS.

  • Create a project access token Show archive.org snapshot for all submodules you want to have access to with the setting read_repository
  • Add the secrets as environment variable to the main project you want to have access to submodules:
    • Protected false (depending on your security settings), masked true
    • Add the project access token

Example configuration for the project manager with two submodules customer_1 and customer_2:


image:
  name: "example.com/manager/ci:v1"

before_script:
  # git clone without --recursive already happend
  - git submodule deinit --all --force # Enforce that the submodules are not cached before changing them
  - sed -i "s/git@code.example.com\/dev\/customer-1.git/https:\/\/gitlab-ci-token:$CI_CUSTOMER_1_ACCESS_TOKEN@code.example.com\/dev\/customer-1.git/" .gitmodules
  - sed -i "s/git@code.example.com\/dev\/customer-2/https:\/\/gitlab-ci-token:$CI_CUSTOMER_2_ACCESS_TOKEN@code.example.com\/dev\/customer-2.git/" .gitmodules
  - git submodule init
  - git submodule update
  - bundle install

rspec:
  script:
    - bundle exec rake db:create db:migrate
    - bundle exec rspec

This approach requires you to touch your CI configuration every time the gitmodules change. You also might prefer to remove the .gitmodules file entirely after the deinit and add the submodules one per line instead of using sed:

git submodule deinit --all --force

rm .gitmodules
git submodule add https://gitlab-ci-token:$CI_CUSTOMER_1_ACCESS_TOKEN@code.example.com/dev/customer-1.git customer-1
git submodule add https://gitlab-ci-token:$CI_CUSTOMER_2_ACCESS_TOKEN@code.example.com/dev/customer-2.git customer-2
git submodule update

Example error message when you try to clone via SSH with insufficient access rights:

Cloning into '/builds/manager/customer-1'...
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@code.example.com/dev/customer-1.git' into submodule path '/builds/manager/customer-1' failed
Posted by Emanuel to makandra dev (2021-06-10 16:02)