Contributes to https://gitlab.com/gitlab-org/gitlab-shell/-/issues/541

Changelog: removed

The option isn't required to accept self-signed certs

On the other hand, if the option set to true it makes
machine-in-the-middle attack possible

Let's clarify it in the code that the option is deprecated

Edited log_format description comment, if for 'text' if a user need 'text' logging

Add a basic monitoring endpoint to the sshd command.
* Listen on localhost port 9122 by default.
* Integrate build/version info.
* Update example config.

https://gitlab.com/gitlab-org/gitlab-shell/-/issues/121



Signed-off-by: Ben Kochie <superq@gmail.com>

The config.yml.example didn't include a field I was expecting to be
there, which lead me to believe the field didn't exist. This change adds
the `secret` YAML field, and describes how it interacts with the
secrets_file.

From
https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4498#note_397401883,
if you specify a relative path such as:

```
external_url 'http://gitlab.example.com/gitlab'
```

gitlab-shell doesn't have a way to pass the `/gitlab` to the host. For example, let's say we have:

```
gitlab_url: "http+unix://%2Fvar%2Fopt%2Fgitlab%2Fgitlab-workhorse%2Fsocket"
```

If we have `/gitlab` as the relative path, how do we specify what is the
UNIX socket path and what is the relative path? If we specify:

```
gitlab_url: "http+unix:///var/opt/gitlab/gitlab-workhorse.socket/gitlab
```

This is ambiguous. Is the socket in
`/var/opt/gitlab/gitlab-workhorse.socket/gitlab` or in
`/var/opt/gitlab/gitlab-workhorse.socket`?

To fix this, this merge request adds an optional
`gitlab_relative_url_root` config parameter:

```
gitlab_url: "http+unix://%2Fvar%2Fopt%2Fgitlab%2Fgitlab-workhorse%2Fsocket"
gitlab_relative_url_root: /gitlab
```

This is only used with UNIX domain sockets to disambiguate the socket
and base URL path. If `gitlab_url` uses `http://` or `https://`, then
`gitlab_relative_url_root` is ignored.

Relates to https://gitlab.com/gitlab-org/gitlab-shell/-/issues/476

The unicorn replacement 'puma' uses a unix socket in the example config [1] instead of a tcp port.
Using the non-existing tcp port results in "Internal API unreachable" on
git operations.

[1] https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/puma.rb.example#L34

It now lives within gitaly

In order to uncomment it in the Makefile of GDK

Adds distributed tracing instrumentation to GitLab-Shell using LabKit

Given the gitaly-* now proxy the data from the client to the Gitaly
server, the environment variables aren't used. Therefor we don't have to
set them either. Only exception to the rule, is the GITALY_TOKEN.

These changes also remove the `GIT_TRACE` options, introduced by
192e2bd3.

Part of: https://gitlab.com/gitlab-org/gitaly/issues/1300

Add a new configuration option, custom_hooks_dir. When this is set, we
will look for global custom hooks in:
    <custom_hooks_dir>/{pre-receive,update,post-receive}.d/*

When this is not set, default to <REPO_PATH>/hooks.

Enable GIT_TRACE/GIT_TRACE_PACKET/GIT_TRACE_PERFORMANCE by providing the git_trace_log_file config key

The value of the variable if present must be a writable absolute path. If it’s
not the case we log a proper message and not enable tracing to not throw output to the users.

[ci skip]

They do not play nice with gitlab-workhorse (or rather Golang net/http
DefaultServemux).

It is well known that UNIX sockets are faster than TCP over loopback.

E.g. on my machine according to lmbench[1] they have ~ 2 times
lower latency and ~ 2-3 times more throughput compared to TCP over
loopback:

    *Local* Communication latencies in microseconds - smaller is better
    ---------------------------------------------------------------------
    Host                 OS 2p/0K  Pipe AF     UDP  RPC/   TCP  RPC/ TCP
                            ctxsw       UNIX         UDP         TCP conn
    --------- ------------- ----- ----- ---- ----- ----- ----- ----- ----
    teco      Linux 4.2.0-1  13.8  29.2 26.8  45.0  47.9  48.5  55.5  45.

    *Local* Communication bandwidths in MB/s - bigger is better
    -----------------------------------------------------------------------------
    Host                OS  Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
                                 UNIX      reread reread (libc) (hand) read write
    --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- -----
    teco      Linux 4.2.0-1 1084 4353 1493 2329.1 3720.7 1613.8 1109.2 3402 1404.

The same ratio usually holds for servers.

Also UNIX sockets, since they reside on filesystem, besides being faster with
less latency, have one another nice property: access permissions to them are
managed the same way access to files is.

Because of lower latencies and higher throughput - for performance reasons, and
for easier security, it makes sense to interconnect services on one machine via
UNIX sockets and talk via TCP only to outside world.

All internal services inside GitLab can talk to each other via UNIX socket
already and only gitlab-shell was missing support to talk to Unicorn via UNIX
socket.

Let's teach gitlab-shell to talk via UNIX sockets.

[1] http://www.bitmover.com/lmbench/

~~~~

In this patch we

- add URI::HTTPUNIX to handle http+unix:// URI scheme
- add Net::HTTPUNIX to handle "connect via unix socket and then talk http"
- adjust GitlabNet#http_client_for() accordingly
- adjust documentation in config.yml.example

The http+unix:// scheme is not reinvented anew: the idea about its structure is
quite logical an was already established at least in requests-unixsocket python
package:

    http://fixall.online/theres-no-need-to-reinvent-the-wheelhttpsgithubcommsabramorequests-unixsocketurl/241810/
    https://github.com/msabramo/requests-unixsocket