How to checkout submodules in Gitlab CI

Posted . Visible to the public. Deprecated.

Please prefer https://docs.gitlab.com/ee/ci/git_submodules.html with relative submodule paths

Accessing other repositories in Gitlab CI is not straight forward, since the access rights of the current pipeline might not be sufficient enough.

One approach is to use project access tokens and clone the repositories via HTTPS.

  • Create a project access token Show archive.org snapshot for all submodules you want to have access to with the setting read_repository
  • Add the secrets as environment variable to the main project you want to have access to submodules:
    • Protected false (depending on your security settings), masked true
    • Add the project access token

Example configuration for the project manager with two submodules customer_1 and customer_2:


image:
  name: "example.com/manager/ci:v1"

before_script:
  # git clone without --recursive already happend
  - git submodule deinit --all --force # Enforce that the submodules are not cached before changing them
  - sed -i "s/git@code.example.com\/dev\/customer-1.git/https:\/\/gitlab-ci-token:$CI_CUSTOMER_1_ACCESS_TOKEN@code.example.com\/dev\/customer-1.git/" .gitmodules
  - sed -i "s/git@code.example.com\/dev\/customer-2/https:\/\/gitlab-ci-token:$CI_CUSTOMER_2_ACCESS_TOKEN@code.example.com\/dev\/customer-2.git/" .gitmodules
  - git submodule init
  - git submodule update
  - bundle install

rspec:
  script:
    - bundle exec rake db:create db:migrate
    - bundle exec rspec

This approach requires you to touch your CI configuration every time the gitmodules change. You also might prefer to remove the .gitmodules file entirely after the deinit and add the submodules one per line instead of using sed:

git submodule deinit --all --force

rm .gitmodules
git submodule add https://gitlab-ci-token:$CI_CUSTOMER_1_ACCESS_TOKEN@code.example.com/dev/customer-1.git customer-1
git submodule add https://gitlab-ci-token:$CI_CUSTOMER_2_ACCESS_TOKEN@code.example.com/dev/customer-2.git customer-2
git submodule update

Example error message when you try to clone via SSH with insufficient access rights:

Cloning into '/builds/manager/customer-1'...
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@code.example.com/dev/customer-1.git' into submodule path '/builds/manager/customer-1' failed
Last edit
Florian Leinsinger
License
Source code in this card is licensed under the MIT License.
Posted by Emanuel to makandra dev (2021-06-10 14:02)