* gcp/saver: Only return errors.KindAlreadyExists if all three exist
In #1124, a GCP lock type was added as a singleflight backend. As part of this work, the GCP backend's Save() was made serial, likely because moduploader.Upload requires a call to Exists() before it, rendering the GCP lock less useful, by doubling the calls to GCS.
However, by doing this, the existence check was now only checking the existence of the mod file, and not the info or zip. This meant that if during a Save, the zip or info uploads failed, on subsequent rquests, that when using the GCP singleflight backend, Athens would assume everything had been stashed and saved properly, and then fail to serve up the info or zip that had failed upload, meaning the cache was in an unhealable broklen state, requiring a manual intervention.
To fix this, without breaking the singleflight behavior, introduce a metadata key that is set on the mod file during its initial upload, indicating that a Stash is still in progress on subsequent files, which gets removed once all three files are uploaded successfully, which can be checked if it it is determined that the mod file already exists. That way we can return a errors.KindAlreadyExists if a Stash is in progress, but also properly return it when a Stash is *not* currently in progress if and only if all three files exist on GCS, which prevents the cache from becoming permanently poisoned.
One note is that it is possible the GCS call to remove the metadata key fails, which would mean it is left on the mod object forever. To avoid this, consider it stale after 2 minutes.
---------
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Co-authored-by: Matt <matt.ouille@protonmail.com>
Hugo is upgraded to a modern version, the reboot of the original theme is used, and links are replaced or removed. The readme is updated to state our utility in the first sentence. References to vGo and other old terminology has been moved to "modules" and more modern terminology.
A default homepage is baked into the server that uses the request host address, or in HTTP 2 the authority. This includes ports. It also checks for schema. The values are used to indicate to users how to configure their go env
Of course, this won't work on all installations - especially enterprise ones. For that, we've introduced ATHENS_HOME_TEMPLATE_PATH as an environment variable along with HomeTemplatePath in the config. This value defaults to /var/lib/athens/home.html but can be configured to any location that Athens can reliably read from. This is a Go HTML template so it should use Go HTML template formatting and logic.
* read redis lock options from config to support custom TTL & timeout
* fix test
* fix typo
* downgrade to bsm/redislock@v0.7.2 to prevent usage of beta go-redis version
* revert test changes
* return error for invalid lock config
* update config parsing test
* udpate docs to include redis lock config
* fix test
* set default max retries to 10
* reduce default redis lock timeout to 15s
* update default TTL to 15mins
Co-authored-by: Manu Gupta <manugupt1@gmail.com>
* Set GO111MODULE to auto in golint script– #1743
As mentioned in #1743 the Go modules environment flag is set to `off`
in the script which appears to cause a warning message for each module
of the codebase that it is "not in GOROOT".
Set to `auto` as this allows the same build to be run the original way
should someone choose to delete the `go.mod` file from the project root.
* Remove redundant code – golint script
Go 1.17 ignores `GO111MODULE` and there are no directories in the
project root called `mock`.
* Replace `golint` with `go vet`
`golint` is deprecated (and frozen) replace with the current `go vet`.
This reported one issue on `main` branch:
```pkg/stash/with_etcd.go:33:28:
loop variable ep captured by func literal```
Fixed loop variable capture with extraction to parameterised anonymous
function passed loop variable and passed in to `errgroup.Go` call.
* Finish Removing `golint`
Removed remaining references to `golint` in dev doc, Makefile,
bash scripts, and powershell build scripts.
Removed `get_dev_tools.sh` and `get_dev_tools.ps1` as they only install
the removed `golint` package.
* Switching to using the main default branch
* changing links to main branch, not master branch
* Changing a few links to relative
* Bumping chart version
Co-authored-by: Marwan Sulaiman <marwan.sameer@gmail.com>
* Adding docs for setting the GONOSUMDB env var on the Athens side
Ref https://github.com/gomods/athens/issues/1363
* Updating with multiple repos, and adding a note
* Update docs/content/configuration/sumdb.md
Co-Authored-By: Ted Wexler <ted@stuckinacan.com>
* Qualifying "user"
Co-authored-by: Ted Wexler <ted@stuckinacan.com>
* Add installation instructions for BOSH
* Fix weight and add link to try-out document
Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
Also doing a few things in the process:
- Deprecating the documentation that references the filter file
- Changing the order of the configuration documentation to put the deprecated documents at the bottom of the config section
- Adding a note to the disk storage documentation about pre-filling the disk cache
Fixes https://github.com/gomods/athens/issues/1501
* Add GCS to helm chart
* base64 encode serviceAccount in the chart
* Standardize GCP Service Account references
* pipe storage.gcp.serviceAccount to quote
* Adding instructions for pre-filling the disk cache
* Fixing up titles
* Updating to the new pacmod
thanks to @jpreese for the advice on how to use the updated one!
* Making some wording changes
Thanks @jpreese!
* adding documentation for installing on GAE for issue 1379
* turning draft status off for GAE install docs
* adding documentation for installing on google cloud run
* install documentation/scripts for gae
* updating docs on gae cloud run for clarity
* Changing install docs to include $ before shell commands
* adding a section for discussion on when to do vendor directories vs. when to use athens or another proxy server
* adding more detail and an outline
* Adding a bit more 😄
* adding to vendor vs athens faq
* removing reference material from FAQ when to use Athens
* Fixing awkward phrasing in faq docs.
* updating FAQ docs based on [slack message](https://gophers.slack.com/archives/C9LRAQN8N/p1571149854034400?thread_ts=1571093549.033500&cid=C9LRAQN8N)
* add index translation of install section
* add translation of build-from-source
* add chinese translation of install-on-aci.md
* add chinese translation of intall-on-kubernetes
* add zh translation of managing-private-repos
* fix some typos
* Adding note about office hours, removing note about developer meetings
@LinuxSuRen @lopetazor would you mind checking out the Chinese / Spanish documentation (respectively) and changing the wording in there, when you can?
* Changing title
thanks @robjloranger!
* Some updates
For @jpreese
* Making a proper link to the zoom download URL
Thanks for pointing this out @jpreese!
* adding a section for discussion on when to do vendor directories vs. when to use athens or another proxy server
* adding more detail and an outline
* Adding a bit more 😄
* adding to vendor vs athens faq
* removing reference material from FAQ when to use Athens
* Fixing awkward phrasing in faq docs.
