Bump go to 1.19.9

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/730



Merged-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: Patrick Bajao <ebajao@gitlab.com>
Co-authored-by: Ash McKenzie <amckenzie@gitlab.com>

Update golang-crypto fork version

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/729



Merged-by: Stan Hu <stanhu@gmail.com>
Approved-by: Joern Schneeweisz <jschneeweisz@gitlab.com>
Approved-by: Stan Hu <stanhu@gmail.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

It updates golang-crypto version and fixes the test that verifies
the order of the supported algorithms

Add build-package-and-qa job

Closes #645

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/728



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Vasilii Iakliushin <viakliushin@gitlab.com>
Co-authored-by: Balasankar "Balu" C <balasankar@gitlab.com>

Signed-off-by: Balasankar "Balu" C <balasankar@gitlab.com>

refactor: success api on acceptance tests

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/727



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Vasilii Iakliushin <viakliushin@gitlab.com>
Co-authored-by: Mohamed Saber <mohamed.saber@bosta.co>

Make the boringcrypto check POSIX shell compliant

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/725



Merged-by: Stan Hu <stanhu@gmail.com>
Approved-by: Stan Hu <stanhu@gmail.com>
Co-authored-by: DJ Mountney <dj@gitlab.com>

- Otherwise this fails on some of the omnibus builder images

Fix CGO_CFLAGS to use output from `brew --prefix`

Closes gitlab-development-kit#1790

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/724



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Co-authored-by: Gabriel Mazetto <gabriel@gitlab.com>

Acceptance test for Geo Push

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/719



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

It imitates a push to the secondary and verifies that the push
is redirected to the primary

Configure Gitaly storage acceptance tests

Closes #622

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/723



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Patrick Cyiza <jpcyiza@gitlab.com>

Merge branch '615-follow-up-from-add-dns-discovery-support-to-gitaly-via-client-side-load-balancing-options' into 'main'

Make golang 1.19 the default

Closes #615

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/718



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Ash McKenzie <amckenzie@gitlab.com>

Prepare for Go 1.19 FIPS support

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/721



Merged-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Reviewed-by: Patrick Bajao <ebajao@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>

https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/718 will
make Go 1.19 the default for gitlab-shell. Per
https://github.com/golang/go/issues/51940, the dev.boringcrypto branch
no longer exists, and to support FIPS we need to pass along
`GOEXPERIMENT=boringcrypto`.

To do this, we just see if this `GOEXPERIMENT` is available with `go
version` rather than do some more complicated version-specific
comparison.

Release v14.18.0

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/722



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>

Perform Git over HTTP request to primary repo

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/716



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Valery Sizov <valery@gitlab.com>
Reviewed-by: Alejandro Rodríguez <alejandro@gitlab.com>
Reviewed-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

Currently, we perform a request to Gitlab Rails that proxies
the request to primary

However, it causes timeouts on big pushes and consumes large
amount of memory. We can perform an HTTP request directly
from Gitlab Shell instead and stream the response to the user

sshd: exclude gssapi when building without cgo

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/720



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Vasilii Iakliushin <viakliushin@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Vasilii Iakliushin <viakliushin@gitlab.com>
Co-authored-by: Lorenz Brun <lorenz@brun.one>

MR #682 broke building without cgo enabled as it introduced a dependency
on a Kerberos library. This can only be disabled at runtime and thus
static builds of gitlab-sshd are no longer possible.

This change introduces an alternative implementation of the GSSAPI
structure which just rejects attempts to use it.
That alternative implementation gets automatically activated in case the
user is building without cgo.

Add DNS discovery support to Gitaly via client-side load-balancing options

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/717



Merged-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: Oscar Tovar <otovar@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Patrick Bajao <ebajao@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Reviewed-by: Oscar Tovar <otovar@gitlab.com>
Co-authored-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>

All the implementations of DNS discovery were done in this epic:
https://gitlab.com/groups/gitlab-org/-/epics/8971. Gitaly allows clients
to configure DNS discovery via dial option. This MR adds the exposed
dial options to client connection creation in Gitlab-shell.

Issue: https://gitlab.com/gitlab-org/gitaly/-/issues/4722
Changelog: added

This client bump includes plenty of improvement, espeically the support
for DNS service discovery in Gitaly/Praefect. This version requires Go
>= 1.18. As a result, we'll need to bump minimal Go version of GitLab
Shell accordingly.

Define Do function for Gitlab net client

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/715



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

Add bin/gitlab-sshd as an explicit Makefile target

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/714



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>

Since https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/682,
Kerberos headers and libraries are needed to build gitlab-sshd.  If
they are not available, `make build` successfully compiles
`bin/gitlab-shell` but fails to build `bin/gitlab-sshd`. However,
running `make build` again would do nothing and appear to be succeed
because `bin/gitlab-shell` existed. This led to Omnibus GitLab quietly
dropping the `gitlab-sshd` binary, as seen in
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6446#note_1265879416.

To ensure `make build` properly fails if `bin/gitlab-sshd` cannot
be built, we make the binary an explicit build target.

Changelog: changed

In future, we'll need to perform http requests for Geo related
code area.

We cannot use retryable requests because:

- It's not necessary for the to be retryable
- In order to retry, the whole request body is stored in RAM,
while we need to stream large blobs of data

This commit:

- Extracts logging into a separate round tripper in order to
reuse it for other http requests by default
- Defines Do function that accepts raw request as an argument

Release v14.17.0 version

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/713



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Ash McKenzie <amckenzie@gitlab.com>

Bump golang to 1.18.9

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/712



Merged-by: Ash McKenzie <amckenzie@gitlab.com>

Release v14.16.0 version

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/711



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>