* Add support for Redis Username and Password configuration
Introduced Redis master authentication parameters (username and password) to the Redis Sentinel setup. This enhances compatibility with Redis environments that require authentication for both sentinel and master nodes.
* Add support for protected Redis Sentinel configuration and related unit tests
Currently Athens only supports connecting to Redis using a hostname:port combination in addition to a password. While this works in most cases it also means that if you have other options you wish to supply Athens has to be updated to support them. As a basic example Redis clusters that require TLS currently are not supported by Athens but with this change you can simply supply a [redis url](https://github.com/redis/redis-specifications/blob/master/uri/redis.txt) to connect over TLS. It also makes it easy to override the password, set a username and more all from a single configuration option:
`rediss://username:password@redis.example.com:6379/1?protocol=3`
Uses the new Azure Identity package for current support. Support automated Azure storage token refresh. Sets the default value of Athens storage account key and Azure managed identity resource id should be empty.
* gcp/saver: Only return errors.KindAlreadyExists if all three exist
In #1124, a GCP lock type was added as a singleflight backend. As part of this work, the GCP backend's Save() was made serial, likely because moduploader.Upload requires a call to Exists() before it, rendering the GCP lock less useful, by doubling the calls to GCS.
However, by doing this, the existence check was now only checking the existence of the mod file, and not the info or zip. This meant that if during a Save, the zip or info uploads failed, on subsequent rquests, that when using the GCP singleflight backend, Athens would assume everything had been stashed and saved properly, and then fail to serve up the info or zip that had failed upload, meaning the cache was in an unhealable broklen state, requiring a manual intervention.
To fix this, without breaking the singleflight behavior, introduce a metadata key that is set on the mod file during its initial upload, indicating that a Stash is still in progress on subsequent files, which gets removed once all three files are uploaded successfully, which can be checked if it it is determined that the mod file already exists. That way we can return a errors.KindAlreadyExists if a Stash is in progress, but also properly return it when a Stash is *not* currently in progress if and only if all three files exist on GCS, which prevents the cache from becoming permanently poisoned.
One note is that it is possible the GCS call to remove the metadata key fails, which would mean it is left on the mod object forever. To avoid this, consider it stale after 2 minutes.
---------
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Co-authored-by: Matt <matt.ouille@protonmail.com>
A default homepage is baked into the server that uses the request host address, or in HTTP 2 the authority. This includes ports. It also checks for schema. The values are used to indicate to users how to configure their go env
Of course, this won't work on all installations - especially enterprise ones. For that, we've introduced ATHENS_HOME_TEMPLATE_PATH as an environment variable along with HomeTemplatePath in the config. This value defaults to /var/lib/athens/home.html but can be configured to any location that Athens can reliably read from. This is a Go HTML template so it should use Go HTML template formatting and logic.
Adds a log format setting as ATHENS_LOG_FORMAT that can be either plain or JSON when CloudRuntime is none (the default). Does not break or change any existing behavior.
Currently, when shutting down the server (via SIGINT or SIGTERM), the
shutdown closes any open connections after only 10 seconds (via a
context.WithTimeout). This does not provie a lot of time for longer
operations, such as listing versions, or downloading a larger module zip
file.
When running in Kubernetes, and scaling instances or changing config,
this causes a lot of dropped connections and gateway errors. 10 seconds
is arguably much too short, and should be configurable.
This commit increases that default to 60 seconds, and adds a config
variable to allow users to specify their desired timeout.
* read redis lock options from config to support custom TTL & timeout
* fix test
* fix typo
* downgrade to bsm/redislock@v0.7.2 to prevent usage of beta go-redis version
* revert test changes
* return error for invalid lock config
* update config parsing test
* udpate docs to include redis lock config
* fix test
* set default max retries to 10
* reduce default redis lock timeout to 15s
* update default TTL to 15mins
Co-authored-by: Manu Gupta <manugupt1@gmail.com>
* Reverting PR 1650
https://github.com/gomods/athens/pull/1650 was a big change to the
authentication/authorization code, which we have decided to pull
out and potentially move into a separate process/project
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
* removing commented, unused code
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
* removing more commented, unused code
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
* removing more unused code
Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
* add config options for redis password
* redis password test and failure test
* changed redis name and made a minimal redis.conf
add drone volume
volume fix
remove volumes from drone.
repoint redis path
Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
Co-authored-by: Marwan Sulaiman <marwan.sameer@gmail.com>
* Fixing the build failures from a missing GCS Key
Accidentally introduced in https://github.com/gomods/athens/pull/1428
* Adding a TODO to the drone build
* adding a step to test the Athens server startup
* Adding the JSONKey back to the default config
* use temporary image
* Switching back to the old env var for the GCS JSON Key
cc/ @marwan-at-work
* removing the ServiceAccount. we want to stick with JSONKey
* reverting to original key fetching code
* fixing build err
* bumping chart ver
* Add GCS to helm chart
* base64 encode serviceAccount in the chart
* Standardize GCP Service Account references
* pipe storage.gcp.serviceAccount to quote
* cmd/proxy/actions: pass NoSumPatterns to GoGetFetcher
* add gosum unit tests
* upgrade appveyor to 1.13
* fix test typo
* check appveyor version
* AppVeyor has no 1.13
* use logger
This CL addresses issue #1177 introducing two new configuration params
allowing to activate pprof.
- `ATHENS_ENABLE_PPROF`
- `ATHENS_PPROF_PORT`
pprof won't be exposed by default.
* actions: support proxying sum db urls
* remove proxy prefix
* add docs for checksum db
* more docs
* typo
* typo
* typo
* typo
* typo
* typo
* typo
* typo
* move checksum db into its own section
* add azblob sf stasher
* use http status const
* rm line
* cleanup, comments
* add test
* fix test
* skip test it no account key provided
* fix stash
* introduce stash timeout
* Rename config.dev.toml to config-example.toml
* Updating helm docs to point to the new helm chart repository (#1102)
* Updating helm docs to point to the new helm chart repository
Fixes https://github.com/gomods/athens/issues/1099
Follow-up to https://github.com/gomods/athens/pull/1097/files
* fixes
* Helm chart: Fix ingress setup (#1086)
* Helm chart: Fix ingress setup
The ingress part of the Helm chart contained references to
not-existing templates and values. Additionally, the deployment used
some no longer available health-check URLs which prevented it from
being marked as ready.
* Make readiness and liveness paths depend on the image.tag
* Add read from .athens.toml (from pwd, then home)
* Remove config lookup from HOME
* Revert "Rename config.dev.toml to config-example.toml"
This reverts commit 9c02956406.
* Fix comment
