- Otherwise this fails on some of the omnibus builder images

Fix CGO_CFLAGS to use output from `brew --prefix`

Closes gitlab-development-kit#1790

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/724



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Co-authored-by: Gabriel Mazetto <gabriel@gitlab.com>

Acceptance test for Geo Push

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/719



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

It imitates a push to the secondary and verifies that the push
is redirected to the primary

Configure Gitaly storage acceptance tests

Closes #622

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/723



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Patrick Cyiza <jpcyiza@gitlab.com>

Merge branch '615-follow-up-from-add-dns-discovery-support-to-gitaly-via-client-side-load-balancing-options' into 'main'

Make golang 1.19 the default

Closes #615

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/718



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Ash McKenzie <amckenzie@gitlab.com>

Prepare for Go 1.19 FIPS support

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/721



Merged-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: James Fargher <proglottis@gmail.com>
Reviewed-by: Patrick Bajao <ebajao@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>

https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/718 will
make Go 1.19 the default for gitlab-shell. Per
https://github.com/golang/go/issues/51940, the dev.boringcrypto branch
no longer exists, and to support FIPS we need to pass along
`GOEXPERIMENT=boringcrypto`.

To do this, we just see if this `GOEXPERIMENT` is available with `go
version` rather than do some more complicated version-specific
comparison.

Release v14.18.0

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/722



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>

Perform Git over HTTP request to primary repo

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/716



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Valery Sizov <valery@gitlab.com>
Reviewed-by: Alejandro Rodríguez <alejandro@gitlab.com>
Reviewed-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

Currently, we perform a request to Gitlab Rails that proxies
the request to primary

However, it causes timeouts on big pushes and consumes large
amount of memory. We can perform an HTTP request directly
from Gitlab Shell instead and stream the response to the user

sshd: exclude gssapi when building without cgo

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/720



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Vasilii Iakliushin <viakliushin@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Vasilii Iakliushin <viakliushin@gitlab.com>
Co-authored-by: Lorenz Brun <lorenz@brun.one>

MR #682 broke building without cgo enabled as it introduced a dependency
on a Kerberos library. This can only be disabled at runtime and thus
static builds of gitlab-sshd are no longer possible.

This change introduces an alternative implementation of the GSSAPI
structure which just rejects attempts to use it.
That alternative implementation gets automatically activated in case the
user is building without cgo.

Add DNS discovery support to Gitaly via client-side load-balancing options

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/717



Merged-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: Oscar Tovar <otovar@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Patrick Bajao <ebajao@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>
Reviewed-by: Oscar Tovar <otovar@gitlab.com>
Co-authored-by: Quang-Minh Nguyen <qmnguyen@gitlab.com>

All the implementations of DNS discovery were done in this epic:
https://gitlab.com/groups/gitlab-org/-/epics/8971. Gitaly allows clients
to configure DNS discovery via dial option. This MR adds the exposed
dial options to client connection creation in Gitlab-shell.

Issue: https://gitlab.com/gitlab-org/gitaly/-/issues/4722
Changelog: added

This client bump includes plenty of improvement, espeically the support
for DNS service discovery in Gitaly/Praefect. This version requires Go
>= 1.18. As a result, we'll need to bump minimal Go version of GitLab
Shell accordingly.

Define Do function for Gitlab net client

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/715



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

Add bin/gitlab-sshd as an explicit Makefile target

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/714



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Stan Hu <stanhu@gmail.com>

Since https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/682,
Kerberos headers and libraries are needed to build gitlab-sshd.  If
they are not available, `make build` successfully compiles
`bin/gitlab-shell` but fails to build `bin/gitlab-sshd`. However,
running `make build` again would do nothing and appear to be succeed
because `bin/gitlab-shell` existed. This led to Omnibus GitLab quietly
dropping the `gitlab-sshd` binary, as seen in
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6446#note_1265879416.

To ensure `make build` properly fails if `bin/gitlab-sshd` cannot
be built, we make the binary an explicit build target.

Changelog: changed

In future, we'll need to perform http requests for Geo related
code area.

We cannot use retryable requests because:

- It's not necessary for the to be retryable
- In order to retry, the whole request body is stored in RAM,
while we need to stream large blobs of data

This commit:

- Extracts logging into a separate round tripper in order to
reuse it for other http requests by default
- Defines Do function that accepts raw request as an argument

Release v14.17.0 version

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/713



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Ash McKenzie <amckenzie@gitlab.com>

Bump golang to 1.18.9

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/712



Merged-by: Ash McKenzie <amckenzie@gitlab.com>

Release v14.16.0 version

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/711



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>

feat: make retryable http default client

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/710



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Oscar Tovar <otovar@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Co-authored-by: Steve Azzopardi <sazzopardi@gitlab.com>

What
---
Make the retryableHTTP client introduced in
https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/703 the
default HTTP client.

Why
---
In
https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1254964426
we've seen a 99% error reduction on `git` commands from `gitlab-shell`
when the retryableHTTP client is used.

This has been running in production for over 2 weeks in `us-east1-b` and
5 days fleet-wide so we should be confident that this client works as
expected.

Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979


Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>

Stub retryable http values in tests

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/708



Merged-by: Ash McKenzie <amckenzie@gitlab.com>
Approved-by: Oscar Tovar <otovar@gitlab.com>
Approved-by: Ash McKenzie <amckenzie@gitlab.com>
Reviewed-by: Oscar Tovar <otovar@gitlab.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

Specify CGO_CFLAGS in Makefile to compile gssapi lib

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/709



Merged-by: Stan Hu <stanhu@gmail.com>
Co-authored-by: Igor Drozdov <idrozdov@gitlab.com>

Currently, the default values are used for retryable http.
That's why a test waits 1 second minimun to retry a request.
Client test takes 25 seconds to execute as a result.
When we stub the value to 1 millisecond instead, we get 0.5s of
execution

Add support for the gssapi-with-mic auth method

Closes #196

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/682



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Alejandro Rodríguez <alejandro@gitlab.com>
Approved-by: Patrick Bajao <ebajao@gitlab.com>
Approved-by: Costel Maxim <cmaxim@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Alejandro Rodríguez <alejandro@gitlab.com>
Reviewed-by: Igor Drozdov <idrozdov@gitlab.com>
Reviewed-by: Patrick Bajao <ebajao@gitlab.com>
Reviewed-by: Rohit Shambhuni <rshambhuni@gitlab.com>
Co-authored-by: Lee Tickett <ltickett@gitlab.com>
Co-authored-by: Marin Hannache <git@mareo.fr>

docs: Truncate pages, point users to GitLab repo

See merge request https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/705



Merged-by: Igor Drozdov <idrozdov@gitlab.com>
Approved-by: Torsten Linz <tlinz@gitlab.com>
Approved-by: Jerry Seto <jseto@gitlab.com>
Approved-by: Sean Carroll <scarroll@gitlab.com>
Approved-by: Igor Drozdov <idrozdov@gitlab.com>
Co-authored-by: Amy Qualls <aqualls@gitlab.com>