S3 API sometimes returns duplicate keys, causing the number of file for a specific module greater than 3. This PR makes Athens handle that situation gracefully and log a warning message instead of treating the module as not found.
logging string content
Co-authored-by: Manu Gupta <manugupt1@gmail.com>
Co-authored-by: Ashish Ranjan <ashishranjan2912@gmail.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: Ashish Ranjan <ashishranjan2912@gmail.com>
Currently, when shutting down the server (via SIGINT or SIGTERM), the
shutdown closes any open connections after only 10 seconds (via a
context.WithTimeout). This does not provie a lot of time for longer
operations, such as listing versions, or downloading a larger module zip
file.
When running in Kubernetes, and scaling instances or changing config,
this causes a lot of dropped connections and gateway errors. 10 seconds
is arguably much too short, and should be configurable.
This commit increases that default to 60 seconds, and adds a config
variable to allow users to specify their desired timeout.
* read redis lock options from config to support custom TTL & timeout
* fix test
* fix typo
* downgrade to bsm/redislock@v0.7.2 to prevent usage of beta go-redis version
* revert test changes
* return error for invalid lock config
* update config parsing test
* udpate docs to include redis lock config
* fix test
* set default max retries to 10
* reduce default redis lock timeout to 15s
* update default TTL to 15mins
Co-authored-by: Manu Gupta <manugupt1@gmail.com>
When using Minio storage, memory usage constantly increases. After
profiling, it appears that memory usage is building in the minio client,
and not being released.
Finding a similar issue listed in the minio/warp project, I found that
they fixed the issue by closing the *minio.Object instances. In the
athens usage, the minio.Object instances are never closed after reading
the data from them.
This adds deferred 'Close' calls in the 'storageImpl.Info' and
'storageImpl.GoMod' functions so that the objects may be released and
garbage collected. The 'storageImpl.Zip' function does not use defer,
since it has the object embedded into the return value, so the 'Close'
is only called on error.
* chore(chart): update ingress to support apiVersion networking.k8s.io/v1
* refactor(chart): use API capabilities instead version comparisions in ingress
Co-authored-by: Manu Gupta <manugupt1@gmail.com>
* fix: flake on redis-sentinel by depending on redis container / step.
Signed-off-by: Manu Gupta <manugupt1@gmail.com>
* Add retries in an attempt to reduce the number of flakes
Also; update go-redis.
Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
* Set GO111MODULE to auto in golint script– #1743
As mentioned in #1743 the Go modules environment flag is set to `off`
in the script which appears to cause a warning message for each module
of the codebase that it is "not in GOROOT".
Set to `auto` as this allows the same build to be run the original way
should someone choose to delete the `go.mod` file from the project root.
* Remove redundant code – golint script
Go 1.17 ignores `GO111MODULE` and there are no directories in the
project root called `mock`.
* Replace `golint` with `go vet`
`golint` is deprecated (and frozen) replace with the current `go vet`.
This reported one issue on `main` branch:
```pkg/stash/with_etcd.go:33:28:
loop variable ep captured by func literal```
Fixed loop variable capture with extraction to parameterised anonymous
function passed loop variable and passed in to `errgroup.Go` call.
* Finish Removing `golint`
Removed remaining references to `golint` in dev doc, Makefile,
bash scripts, and powershell build scripts.
Removed `get_dev_tools.sh` and `get_dev_tools.ps1` as they only install
the removed `golint` package.
* Set GO111MODULE to auto in golint script– #1743
As mentioned in #1743 the Go modules environment flag is set to `off`
in the script which appears to cause a warning message for each module
of the codebase that it is "not in GOROOT".
Set to `auto` as this allows the same build to be run the original way
should someone choose to delete the `go.mod` file from the project root.
* Remove redundant code – golint script
Go 1.17 ignores `GO111MODULE` and there are no directories in the
project root called `mock`.
* Replace `golint` with `go vet`
`golint` is deprecated (and frozen) replace with the current `go vet`.
This reported one issue on `main` branch:
```pkg/stash/with_etcd.go:33:28:
loop variable ep captured by func literal```
Fixed loop variable capture with extraction to parameterised anonymous
function passed loop variable and passed in to `errgroup.Go` call.
