Gitlab Rails now restricts SSH protocol access via an
additional boolean setting rather than a separate Git
protocol

When a user is authenticated via SSH certificates, let's
send a different protocol rather than simple `ssh`

GitLab Shell detects that the passed key is a certificate and
requests /authorized_certs instead of /authorized_keys endpoint

This explicitly imports gitaly v16 instead of v15 and removes the
old one from `go.mod`.

Currently, we perform a request to Gitlab Rails that proxies
the request to primary

However, it causes timeouts on big pushes and consumes large
amount of memory. We can perform an HTTP request directly
from Gitlab Shell instead and stream the response to the user

This commit also excludes gitlab-shell from dependencies:

Gitaly specifies Gitlab Shell as a dependency as well in order
to use gitlabnet client to perform API endpoints to Gitlab Rails.
As a result, Gitlab Shell requires Gitaly -> Gitaly requires an
older version of Gitlab Shell -> that version requires an older
version of Gitlab Shell, etc. Let's use exclude to break the
chain earlier

While gitlab-shell currently has a major version of v14, the module path
it exposes is not using that major version like it is required by the Go
standard. This makes it impossible for dependents to import gitlab-shell
as a dependency without using a commit as version.

Fix this by changing the module path of gitlab-shell to instead be
`gitlab.com/gitlab-org/gitlab-shell/v14` and adjust all imports
accordingly.

Changelog: fixed

When gitlab-sshd were used with the PROXY protocol, the `check_ip`
argument passed to `/api/v4/internal/allowed` was the Go remote
address, which is a host and port combination
(e.g. 127.0.0.1:12345). As a result, This prevents IP restrictions
from working properly on Rails. We fix this by stripping out the port
if it is present.

When OpenSSH is used, this is not an issue because the IP address
is extracted from `SSH_CONNECTION`.

Changelog: fixed

Before this change, the GitLab internal API could use a boolean
response field to indicate whether gitlab-shell should make
sidechannel connections go Gitaly. We now ignore that response field
and always use sidechannel connections.

Jacob Vosmaer authored 3 years ago and Igor Drozdov committed 3 years ago

This field will act as a feature flag that controls whether
gitlab-shell uses the old SSHUploadPack RPC or the new
SSHUploadPackWithSidechannel.

This reduces coupling between tests in
internal/gitlabnet/accessverifier/client_test.go, and will make it
easier to add new test cases in the future.

Note that the test server had a special behavior for the username
"second", but this was never used. So we removed that behavior in this
commit.

Gitaly project now properly respects module release flow
and includes a module suffix in the package name. It requires
to re-write all non-suffixed imports with suffixed of a specific
version of tha module. With proper module versioning we don't
need to use a 'replace' directive to point to specific commit
and can use semantic versioning for the gitaly dependency.

Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/3177

In this case we don't need to propagate cleanup
function. It simplifies the code.

Previously, gitlab-shell did not pass a context through the application.
Correlation IDs were generated down the call stack instead of passed
around from the start execution.

This has several potential downsides:

1. It's easier for programming mistakes to be made in future that lead
to multiple correlation IDs being generated for a single request.
2. Correlation IDs cannot be passed in from upstream requests
3. Other advantages of context passing, such as distributed tracing is
not possible.

This commit changes the behavior:

1. Extract the correlation ID from the environment at the start of
the application.
2. If no correlation ID exists, generate a random one.
3. Pass the correlation ID to the GitLabNet API requests.

This change also enables other clients of GitLabNet (e.g. Gitaly) to
pass along the correlation ID in the internal API requests
(https://gitlab.com/gitlab-org/gitaly/-/issues/2725).

Fixes https://gitlab.com/gitlab-org/gitlab-shell/-/issues/474

Gitaly 1.68.0 upgrades gRPC to 1.24.0

Other functions are still expecting for `CommandArgs` instead
of `Shell`. They should be expecting `commandargs.Shell` now
since it has been renamed.