We want to use both go and ruby cache for test jobs since we run
go and rspec tests in that job.

We experienced a flaky failure of pipeline on `main` due to
caching changes.

To prevent that, we split the cache jobs specific to go and ruby
and use them accordingly. We also set the policy to `pull-push` on
specific jobs that need to update the cache.

We only use Ruby for running integration tests so we don't need
different versions of Ruby.

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>

This client bump includes plenty of improvement, espeically the support
for DNS service discovery in Gitaly/Praefect. This version requires Go
>= 1.18. As a result, we'll need to bump minimal Go version of GitLab
Shell accordingly.

What
---
Make the retryableHTTP client introduced in
https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/703 the
default HTTP client.

Why
---
In
https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1254964426
we've seen a 99% error reduction on `git` commands from `gitlab-shell`
when the retryableHTTP client is used.

This has been running in production for over 2 weeks in `us-east1-b` and
5 days fleet-wide so we should be confident that this client works as
expected.

Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979


Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>

What
---
- Update the `client.HttpClient` fields to have `http.Client` and
  `retryablehttp.Client`, one of them will be `nil` depending on the
  feature flag toggle.
- Create new method `newRetryableRequest` which will create a
  `retryablehttp.Request` and use that if the
  `FF_GITLAB_SHELL_RETRYABLE_HTTP` feature flag is turned on.
- Add checks for `FF_GITLAB_SHELL_RETRYABLE_HTTP` everywhere we use the
  http client to use the `retryablehttp.Client` or the default
  `http.Client`
- New job `tests-integration-retryableHttp` to run the integraiton tests
  with the new retryablehttp client. We didn't update go tests because
  some assertions are different and will break table driven tests.

Why
---
As discussed in
https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/703#note_1229645097
we want to put the client behind a feature flag, not just the retry
logic. This does bring extra risk for accessing a `nil` field but there
should be checks everytime we access `RetryableHTTP` and `HTTPClient`.

Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979


Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>

That would save us from installing additional
dependencies ourselves

To follow rubygems' security adisory
https://github.com/rubygems/rubygems.org/security/advisories/GHSA-hccv-rwq6-vh79:

It's been removed in:
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86704

Signed-off-by: Rémy Coutable <remy@rymai.me>

Our CI jobs and docker-compose pull in the "latest" tag of Gitaly. As it
turns out though, "latest" is pointing to Gitaly v13.3.0-rc5, which is
definitely not the latest versionat this point in time. This is because
CNG was converted to not use the "latest" tag anymore, but instead to
use a tag called "master" in gitlab-org/build/CNG!519.

Fix this by using the new "master" tag instead.

1.4.0 has introduced the format change
Since v1 means latest 1.* image, we're pulling the unsupported version

Signed-off-by: Takuya Noguchi <takninnovationresearch@gmail.com>