Currently Athens only supports connecting to Redis using a hostname:port combination in addition to a password. While this works in most cases it also means that if you have other options you wish to supply Athens has to be updated to support them. As a basic example Redis clusters that require TLS currently are not supported by Athens but with this change you can simply supply a [redis url](https://github.com/redis/redis-specifications/blob/master/uri/redis.txt) to connect over TLS. It also makes it easy to override the password, set a username and more all from a single configuration option:
`rediss://username:password@redis.example.com:6379/1?protocol=3`
* gcp/saver: Only return errors.KindAlreadyExists if all three exist
In #1124, a GCP lock type was added as a singleflight backend. As part of this work, the GCP backend's Save() was made serial, likely because moduploader.Upload requires a call to Exists() before it, rendering the GCP lock less useful, by doubling the calls to GCS.
However, by doing this, the existence check was now only checking the existence of the mod file, and not the info or zip. This meant that if during a Save, the zip or info uploads failed, on subsequent rquests, that when using the GCP singleflight backend, Athens would assume everything had been stashed and saved properly, and then fail to serve up the info or zip that had failed upload, meaning the cache was in an unhealable broklen state, requiring a manual intervention.
To fix this, without breaking the singleflight behavior, introduce a metadata key that is set on the mod file during its initial upload, indicating that a Stash is still in progress on subsequent files, which gets removed once all three files are uploaded successfully, which can be checked if it it is determined that the mod file already exists. That way we can return a errors.KindAlreadyExists if a Stash is in progress, but also properly return it when a Stash is *not* currently in progress if and only if all three files exist on GCS, which prevents the cache from becoming permanently poisoned.
One note is that it is possible the GCS call to remove the metadata key fails, which would mean it is left on the mod object forever. To avoid this, consider it stale after 2 minutes.
---------
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Co-authored-by: Matt <matt.ouille@protonmail.com>
A default homepage is baked into the server that uses the request host address, or in HTTP 2 the authority. This includes ports. It also checks for schema. The values are used to indicate to users how to configure their go env
Of course, this won't work on all installations - especially enterprise ones. For that, we've introduced ATHENS_HOME_TEMPLATE_PATH as an environment variable along with HomeTemplatePath in the config. This value defaults to /var/lib/athens/home.html but can be configured to any location that Athens can reliably read from. This is a Go HTML template so it should use Go HTML template formatting and logic.
* read redis lock options from config to support custom TTL & timeout
* fix test
* fix typo
* downgrade to bsm/redislock@v0.7.2 to prevent usage of beta go-redis version
* revert test changes
* return error for invalid lock config
* update config parsing test
* udpate docs to include redis lock config
* fix test
* set default max retries to 10
* reduce default redis lock timeout to 15s
* update default TTL to 15mins
Co-authored-by: Manu Gupta <manugupt1@gmail.com>
* Switching to using the main default branch
* changing links to main branch, not master branch
* Changing a few links to relative
* Bumping chart version
Co-authored-by: Marwan Sulaiman <marwan.sameer@gmail.com>
* Adding docs for setting the GONOSUMDB env var on the Athens side
Ref https://github.com/gomods/athens/issues/1363
* Updating with multiple repos, and adding a note
* Update docs/content/configuration/sumdb.md
Co-Authored-By: Ted Wexler <ted@stuckinacan.com>
* Qualifying "user"
Co-authored-by: Ted Wexler <ted@stuckinacan.com>
Also doing a few things in the process:
- Deprecating the documentation that references the filter file
- Changing the order of the configuration documentation to put the deprecated documents at the bottom of the config section
- Adding a note to the disk storage documentation about pre-filling the disk cache
Fixes https://github.com/gomods/athens/issues/1501
* Adding instructions for pre-filling the disk cache
* Fixing up titles
* Updating to the new pacmod
thanks to @jpreese for the advice on how to use the updated one!
* Making some wording changes
Thanks @jpreese!
* Updated mongo init to use db name env variable.
* Updated mongo init to use coll name env var.
* Fixed merge conflict.
* Updated formatting in mongo_test.
* Updating english docs to indicate the presence of hosted public proxies
* fixing spacing
* more spacing
* bump chart version again
since I pulled from master, the linter told me to do this again
* actions: support proxying sum db urls
* remove proxy prefix
* add docs for checksum db
* more docs
* typo
* typo
* typo
* typo
* typo
* typo
* typo
* typo
* move checksum db into its own section
* Add documentation about SSH_AUTH_SOCK from PR#1184
PR#1184 added support for propagating SSH_AUTH_SOCK if it points to a
valid unix socket. As requested, add a section explaining how to use
it.
* auth documentation: replace $(pwd) with $PWD
Replace references to `$(pwd)` with `$PWD` in shell snippets and add
more quoting. This should make these snippets reasonably friendly to
currnet working directory path components containing spaces.
Note: I neither endorse nor recommend spaces in directories, but this is
an easy enough change.
This commit is leaving powershell snippets untouched as I am not
familiar with powershell's word-splitting behavior (if there is any).
* Working tests
* More tests, a bugfix (yay tests) and docs
* Changed filter description adding v prefix to versions
* Updated docs to be more precise
* More robust, changed docs
* Fixed copy paste bug
* extend filtering logic and configuration to include version lists
Module filtering is very useful, but many deployments will need to
satisfy even more granular constrainsts. Enterprises may need
approved lists specific down to the minor (or patch) version element.
Version filtering logic is similar to the module filtering, in that
it's a prefix match of the version requested against each entry in the
version filter list.
Closes#1045
* include version filtering in documentation with example
* allow filtering when version is missing
Endpoints that do not specify a version, such as "@v/list", only need
to be filtered by module rules.
