Commit Graph

74 Commits

Author SHA1 Message Date
Matt
b9c8fb7f8a Rework logging defaults (#1927)
Changes default logger back to plain.
2024-03-27 17:48:10 -07:00
Matt
08520bf894 Add log formatting settings (#1926)
Adds a log format setting as ATHENS_LOG_FORMAT that can be either plain or JSON when CloudRuntime is none (the default). Does not break or change any existing behavior.
2024-03-21 09:07:39 -07:00
LINKIWI
e5aa5974e1 Support Unix domain sockets for proxy server listener (#1865) 2023-05-06 11:35:06 +02:00
Rob Prentiss
a1553999bb Make shutdown timeout configurable (#1806)
Currently, when shutting down the server (via SIGINT or SIGTERM), the
shutdown closes any open connections after only 10 seconds (via a
context.WithTimeout). This does not provie a lot of time for longer
operations, such as listing versions, or downloading a larger module zip
file.

When running in Kubernetes, and scaling instances or changing config,
this causes a lot of dropped connections and gateway errors. 10 seconds
is arguably much too short, and should be configurable.

This commit increases that default to 60 seconds, and adds a config
variable to allow users to specify their desired timeout.
2023-01-23 22:02:45 -08:00
Ashish Ranjan
cc496afbf1 read redis lock options from config to support custom TTL & timeout (#1791)
* read redis lock options from config to support custom TTL & timeout

* fix test

* fix typo

* downgrade to bsm/redislock@v0.7.2 to prevent usage of beta go-redis version

* revert test changes

* return error for invalid lock config

* update config parsing test

* udpate docs to include redis lock config

* fix test

* set default max retries to 10

* reduce default redis lock timeout to 15s

* update default TTL to 15mins

Co-authored-by: Manu Gupta <manugupt1@gmail.com>
2022-09-28 22:53:06 -07:00
Manu Gupta
be203340be Remove deprecated GOPROXY in favor of GoBinaryEnvVars (#1759) 2022-02-18 10:24:44 -08:00
Abhay Krishna
38a3039a21 Fix typos and comments in config.toml (#1739)
Co-authored-by: Marwan Sulaiman <marwan.sameer@gmail.com>
2021-12-03 01:38:09 -05:00
Marwan Sulaiman
151c4922fb Enable Athens to work in offline mode (#1717)
Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2021-08-18 19:09:14 -04:00
Zdeněk Brabec
bb6c5f4c9c Update GCP Storage authentication comment (#1716) 2021-06-03 11:14:56 -07:00
Aaron Schlesinger
1acdf17a4a Reverting passthrough auth (#1700)
* Reverting PR 1650

https://github.com/gomods/athens/pull/1650 was a big change to the
authentication/authorization code, which we have decided to pull
out and potentially move into a separate process/project

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing commented, unused code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing more commented, unused code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>

* removing more unused code

Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net>
2021-03-11 11:28:15 -08:00
Marwan Sulaiman
dfb7887080 Allow Athens to Propagate Authentication to Mod Download (#1650)
* Allow Athens to Propagate Authentication to Mod Download

* update readme

* add pattern matching to auth propagation

* Propagate authentication to pre declared static host

* quote redis test

* fix flaky redis error message

* fix config tests

* fix config tests

* Update config.dev.toml

Co-authored-by: Ted Wexler <ted@stuckinacan.com>

* gofmt

Co-authored-by: Ted Wexler <ted@stuckinacan.com>
2020-07-30 17:06:53 -04:00
Marwan Sulaiman
ef15c3458b Allow users to specify the module fetcher's temporary directory (#1636) 2020-06-25 17:15:15 -04:00
Marwan Sulaiman
52934cfa46 implement /index endpoint (#1630)
* implement /index endpoint

* rename to Module to Path
2020-06-24 14:29:30 -04:00
Marwan Sulaiman
3c4db4ce86 pkg/storage: add External implementation (#1587)
* pkg/storage: add External implementation

* fix conflicts

* use newly instantiated client
2020-03-27 13:35:52 -04:00
Ted Wexler
701bf394a5 Move SingleFlight.Redis.Password back under SingleFlight.Redis (#1581)
Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2020-03-20 12:32:44 -07:00
Ted Wexler
939e695526 Adds redis sentinel support (#1554)
* Adds redis sentinel support

Fixes #1553

* Fix redis-sentinel test hostnames

* Fix redis master name again

* Fix redis sentinel port in tests

* Upgrade the redis client

* Rmoeve accidental config change

* Fix default config

* Addresses review comments

* Add documentation on single flight mechanisms

* Fix spelling issues

* Fix formatting

Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
2020-03-17 13:04:37 -07:00
Matthew Plachter
71c57ef7ee add config options for redis password (#1545)
* add config options for redis password

* redis password test and failure test

* changed redis name and made a minimal redis.conf

add drone volume

volume fix

remove volumes from drone.

repoint redis path

Co-authored-by: Aaron Schlesinger <70865+arschles@users.noreply.github.com>
Co-authored-by: Marwan Sulaiman <marwan.sameer@gmail.com>
2020-02-25 16:18:04 -08:00
vapod
4530a58359 Add forcepathstyle config for s3 (#1533)
* Add forcepathstyle config for s3

* Bump chart version

* Fix typos in s3 config struct
2020-02-10 16:03:25 -08:00
Marwan Sulaiman
ba40b8f76d config: use semi-colon instead of comma to separate ATHENS_GO_BINARY_… (#1531)
* config: use semi-colon instead of comma to separate ATHENS_GO_BINARY_ENV_VARS

* document behavior
2020-01-24 10:20:37 -05:00
Marwan Sulaiman
90d213f0af config: remove incorrect documentation (#1525) 2020-01-19 19:38:31 -05:00
Aleksandr Razumov
5b194688a0 storage/s3: add custom endpoint url support (#1467) 2019-12-05 13:04:03 -08:00
Aaron Schlesinger
707b7b5413 Fixing the build failures from a missing GCS Key (#1480)
* Fixing the build failures from a missing GCS Key

Accidentally introduced in https://github.com/gomods/athens/pull/1428

* Adding a TODO to the drone build

* adding a step to test the Athens server startup

* Adding the JSONKey back to the default config

* use temporary image

* Switching back to the old env var for the GCS JSON Key

cc/ @marwan-at-work

* removing the ServiceAccount. we want to stick with JSONKey

* reverting to original key fetching code

* fixing build err

* bumping chart ver
2019-12-05 10:49:56 -08:00
Aditya Sharma
e0ec46b221 Add GCS to helm chart (#1428)
* Add GCS to helm chart

* base64 encode serviceAccount in the chart

* Standardize GCP Service Account references

* pipe storage.gcp.serviceAccount to quote
2019-12-03 18:28:07 -08:00
Zdeněk Brabec
b6d8d2b015 added gcp configuration doc (#1465) 2019-11-25 16:47:07 -08:00
Marwan Sulaiman
298c6cd74d cmd/proxy/actions: pass NoSumPatterns to GoGetFetcher (#1391)
* cmd/proxy/actions: pass NoSumPatterns to GoGetFetcher

* add gosum unit tests

* upgrade appveyor to 1.13

* fix test typo

* check appveyor version

* AppVeyor has no 1.13

* use logger
2019-09-23 12:47:40 -04:00
oliverch
874d27158d feat: add /robots.txt (#1355)
* feat: add /robots.txt

* disallow all crawler by default.
2019-09-16 16:21:08 -05:00
Manu Gupta
f4f3afb4de Add docs for fargate url (#1352) 2019-08-28 11:19:44 -07:00
Manu Gupta
16739a1213 Add relative URI for AWS fargate. (#1337)
* f

* r

* f

* gofmt

* ep

* Container url

* Asd

* nit

* Update docs

* keep old stuff

* old is gold

* indent

* asd
2019-08-21 14:21:01 +01:00
Manu Gupta
5bff5fe74d endpoint creds (#1291)
* endpoint creds

* gofmt

* Add credential chain

* error handling

* Fix things

* add docs

* f

* asd

* fix
2019-07-22 22:47:36 +02:00
Yi Tang
7de77889ad support goproxy for list/download command (#1304) 2019-07-15 14:07:24 -04:00
Benjamin
e832c83ecd [debug] allow to expose activate via config (#1284)
This CL addresses issue #1177 introducing two new configuration params
allowing to activate pprof.
- `ATHENS_ENABLE_PPROF`
- `ATHENS_PPROF_PORT`

pprof won't be exposed by default.
2019-06-24 20:59:15 +02:00
Marwan Sulaiman
76fb786324 downloadProtocol: support multi-proxy environments with DownloadFile (#1230)
* downloadProtocol: support multi-proxy environments with DownloadFile

* remove debugging lines

* update config tests

* download/mode: add tests for DownloadFile and friends

* add documentation to Download File
2019-06-08 00:30:07 -04:00
Marwan Sulaiman
0cac0edbf1 actions: support proxying sum db urls (#1208)
* actions: support proxying sum db urls

* remove proxy prefix

* add docs for checksum db

* more docs

* typo

* typo

* typo

* typo

* typo

* typo

* typo

* typo

* move checksum db into its own section
2019-05-08 17:09:07 -07:00
Marwan Sulaiman
71aeca7f30 config: accept PORT env + ensure colon (#1181)
* config: accept PORT env + ensure colon

* ensure precedence
2019-04-08 12:54:25 -07:00
marpio
9974c16093 pkg/stash: Add Azure Blob implementation (#1162)
* add azblob sf stasher

* use http status const

* rm line

* cleanup, comments

* add test

* fix test

* skip test it no account key provided

* fix stash

* introduce stash timeout
2019-04-01 21:53:59 -07:00
Pontus Leitzler
d9e0df8643 Config improvements - rename file & remove default (#1103)
* Rename config.dev.toml to config-example.toml

* Updating helm docs to point to the new helm chart repository (#1102)

* Updating helm docs to point to the new helm chart repository

Fixes https://github.com/gomods/athens/issues/1099
Follow-up to https://github.com/gomods/athens/pull/1097/files

* fixes

* Helm chart: Fix ingress setup (#1086)

* Helm chart: Fix ingress setup

The ingress part of the Helm chart contained references to
not-existing templates and values. Additionally, the deployment used
some no longer available health-check URLs which prevented it from
being marked as ready.

* Make readiness and liveness paths depend on the image.tag

* Add read from .athens.toml (from pwd, then home)

* Remove config lookup from HOME

* Revert "Rename config.dev.toml to config-example.toml"

This reverts commit 9c02956406.

* Fix comment
2019-03-26 16:56:18 -04:00
Shivaram Radhkrishna
c59aa69848 Doc update to configure upstream proxy to fetch modules from a Go mod… (#1146)
* Doc update to configure upstream proxy to fetch modules from a Go modules repository

* review suggestions
2019-03-19 15:03:44 -07:00
Marwan Sulaiman
081ec9126e pkg/stash: Add GCS implementation (#1124)
* pkg/stash: Add GCS implementation

* fix docs
2019-03-13 11:07:07 -07:00
Marwan Sulaiman
fb9437ddff pkg/stash: implement a Redis lock for stashing (#1116)
* pkg/stash: implement a Redis lock for stashing

* fix tests

* fix op
2019-03-11 02:48:04 -04:00
Marwan Sulaiman
a4450c1142 config.toml: add cautionary note around basic auth (#1109) 2019-03-07 08:51:18 -05:00
Marwan Sulaiman
dbe44e627a storage/gcp: use base64 for json key (#1089) 2019-02-25 15:16:13 -08:00
Marwan Sulaiman
a27ef13b71 storage/gcp: allow json key to be used for GCS access (#1088) 2019-02-25 13:59:00 -08:00
Marwan Sulaiman
12ce2a6499 pkg/stash: add Etcd as a SingleFlight backend (#1070)
* pkg/stash: add Etcd as a SingleFlight backend

* fix config tests

* fmt

* pr fixes

* fix conflicts
2019-02-22 11:35:45 -08:00
Marwan Sulaiman
9d1078f901 config.toml: fix override typo (#1071) 2019-02-18 21:57:29 +01:00
Chris Mills
2cf10c3ead Allow mgo.ParseURL to set [/database] (#997)
* Allow mgo.ParseURL to set [/database]

As per Michael's solution we need to be able to set the database from
either the parsed connection string, or fallback to athens if not set.

Signed-off-by: Chris M <me@christophermills.co.uk>

* Remove TimeoutDuration which was removed #928

Signed-off-by: Chris M <me@christophermills.co.uk>

* Correct conf_test to accommodate for changes

I'd missed out the addition of DefaultDBName from the test, this has now
been corrected

Signed-off-by: Chris M <me@christophermills.co.uk>

* Make sure gofmt runs automatically on this machine

Y'know simple stuff.

Signed-off-by: Chris M <me@christophermills.co.uk>
2019-02-14 10:21:34 -08:00
Marwan Sulaiman
aa3fb3389c pkg/config: Update GoGetWorkers default/docs (#1052)
* pkg/config: Update GoGetWorkers default/docs

* fix tests
2019-02-11 11:51:38 -05:00
Joshua Rubin
84eff04ed2 Use ambient aws credential provider for s3 (#1041)
* use ambiant aws credentials

Signed-off-by: Joshua Rubin <joshua@rubixconsulting.com>

* better naming, docs and examples for default aws config

Signed-off-by: Joshua Rubin <joshua@rubixconsulting.com>
2019-01-16 08:47:21 +01:00
Manu Gupta
23b285d49d Catalog endpoint support for mongo (#985)
* Added new cataloger interface

* Implementing catalog protocol

* Propagated to protocol and over

* First round of fixes

* S3 almost ready, need to be tested

* Going on with testing s3

* Better testing with s3

* Simplified catalog tests

* Preparing gcp tests to access a gcp instance

* Fixing initialization errors

* Removed some prints

* Gcp ready, to be tested

* Gcp working

* Aligned bucket mock to catalog method

* Switched res payload to json

* Added catalog method to all storage instances

* Added catalog method to unsupported storages

* Fixed with pool test

* Restored tests

* Fixed gcp constructor

* Implemented catalog for fs

* Removed trace

* E2e tests, fixed fs

* Fixed module name return value

* Added cataloger method to azure storage

* Added docs

* Changed pagesize parameter name

* Fixed gofmt error

* Added json tags to result. Fixed lint warning

* Removed extra line

* Changed not implemented error to http.KindNotImplemented

* Checking for inequality on results

* Lower-cased json keys

* Added cleaning of path separator

* Fixed review comments

* Add catalog endpoint for mongo

* Add omitempty to ID

* Fix catalog tests

* update for next token

* fix e2e

* Make query readable

* Fix language and e2e script

* remove new line
2019-01-02 22:13:56 -08:00
z2665
aa76528450 fix toml note (#1014) 2018-12-23 02:06:07 -05:00
Marwan Sulaiman
5870aeee8d Remove Buffalo (#1010)
* Remove Buffalo

* gofmt

* pr fixes

* fix subrouter

* bring back secure middleware + pr fixes

* better place for subrouter

* vendor
2018-12-22 20:24:25 -05:00