Commit Graph

103 Commits

Author SHA1 Message Date
Marwan Sulaiman
2d589f1945 Remove CSRF option from config (#814) 2018-10-24 10:59:33 +02:00
Henry Jenkins
d26b99d41c Upgrade Buffalo (#789)
* Upgrade buffalo

* Switch to go modules everywhere

* Fixes from buffalo fix

* Add missing modules from module list

* Update vendored modules in /vendor

* Stop using vendor directory for tests

* Check go.mod and go.sum files on verify

* Upgrade Buffalo from v0.13.0 to v0.13.1

* Fix test for new Buffalo

Allow for new Buffalo code

* Add test for endpoint with trailing slash
2018-10-23 16:49:32 -07:00
Henry Jenkins
532e93e119 Remove Olympus code (#779)
* Remove nolonger used moniker package

Usage was removed in 97d8013

* Remove reference to Zeus

Use name Proxy for now.

* Remove Olympus code

See github issue #777

* Remove Olympus related vendor modules

* Mention Olympus removal in docs

* Add note about no registries existing

* Remove eventlog

* Update docs for Olympus removal
2018-10-22 11:52:02 -07:00
Tomasz Tomalak
480d8c8e8c Implement storage.Backend for S3 storage (#750)
* Implement storage.Backend for S3 storage

* Adapt S3 storage test after refactor, Fix skiped minio tests

* Add defer to Close calls in s3.getter methods

* CR fixes

* Fix doc align

* Tweak err checks

* Fix parse test for S3 configs and cleanup error messages in storage.s3
2018-10-15 21:46:08 -04:00
Marwan Sulaiman
6753def0b7 Authenticate by providing Github Token only (#739)
* Authenticate by providing Github Token only

* docs: add backticks to env var

* add optional word to docs
2018-10-04 19:35:16 -04:00
Marwan Sulaiman
337488c202 Add stackdriver to traces (#737) 2018-10-04 18:47:38 -04:00
Bhavesh Praveen
73e45cb413 Tracing Exporter for Datadog #714 (#723)
* Tracing Exporter for Datadog #714

* Tracing Exporter for Datadog #714
2018-10-03 22:48:10 -04:00
Tomasz Tomalak
dfe939e14c Proxy: Add HGRC file mount (#720)
* Proxy: Add HGRC file mount

* Rename and move dotFile initializer, lower permission for authFiles
2018-10-03 20:13:57 +02:00
Zachary Romero
cdced929ab Add Host to configuration (#666) 2018-10-01 20:58:34 +02:00
Zachary Romero
e5b307354c Patch points of test flag leakage (#667) 2018-09-29 19:26:19 +02:00
Manu Gupta
536257ce25 Change error log level to debug (#696) 2018-09-22 01:36:45 -04:00
Marwan Sulaiman
83281a80fb Proxy remove assets and config scripts (#693)
* Proxy remove assets and config scripts

* update olympus dockerfile - tho its outdated anyway
2018-09-21 16:31:47 -04:00
marpio
bcc896f226 rename example to dev and rm test config (#691) 2018-09-20 16:38:08 -04:00
Marwan Sulaiman
d314c7847f Make example config the default for dev (#683) 2018-09-20 12:35:46 -04:00
Marwan Sulaiman
fe65b9703f Use config for trace url (#681)
* Use config for trace url

* pr updates

* fix typo

* fix typo
2018-09-20 12:00:13 -04:00
Marwan Sulaiman
7e881bfaec Dockerfile updates: better naming and prefer CMD (#688) 2018-09-20 11:32:50 -04:00
Marwan Sulaiman
9128fd65f1 Ensure Simple Worker is off (#687) 2018-09-20 11:15:39 -04:00
Marwan Sulaiman
e1b7fbcff0 Git ignore .env file (#684) 2018-09-20 10:46:58 -04:00
Aravind
d4941647af Refactored to remove pkg/cdn (#672) 2018-09-19 08:36:23 +02:00
Marwan Sulaiman
b6ca4149e5 Default to example config (#678) 2018-09-19 02:20:37 -04:00
Gil
5fe85f7cf5 changed base to use 1.11-alpine and addded GO111MODULE=on to environment (#635)
* changed base to use 1.11-alpine and addded GO111MODULE=on to environment

* change go build to be one line with modules on

* add vcs systems that were in original stretch image

* created multistage build with copy from and path environment update

* added env GO111MODULE=on

* add and run create default, prereq bash installed, moved go binary, created GOROOT path

* update to preferred docker entry point and shorter config path

* added fossil and info to links for compare of VCS
2018-09-19 00:48:05 -04:00
marpio
6af44242fb Fix list (#607)
* merge list from storage with the one from go list

* fix and rename

* move stuff to semver pkg

* fix gofmt

* move union out of semver pkg

* add tests

* fix err msg

* fix tests

* error handling

* error handling, go list refactoring

* fix list

* cleanup

* fix gofmt
2018-09-13 14:12:17 -07:00
Manu Gupta
57b11e8489 Shorten op names and remove spans from handlers (#647) 2018-09-11 23:20:32 -04:00
Rohan Chakravarthy
0e470d0294 Plumb config - Latest (#627)
* switch proxy to config file

pull in single flight changes

* changes for single-flight

* intermediate stage. All tests passing. pkg still has env refs

* remove all env references

* delete config/env entirely

* fix failing tests

* create the config.toml file as part of dev setup

* create config file only if it doesn't exist

* update Dockerfiles to use config file

* move composing elements to the top

* verbose parameter naming

* newline

* add flag for config file path

* update docs with config file flag

* remove unnecessary nil check

* use filepath.join

* rename redis port to address

* fix path.join

* fix issues after merge

* add vendor dir
2018-09-11 15:04:20 -07:00
Manu Gupta
bb250437da Instrumenting observability with opencensus (#604)
* Initial chanages for observability

* Fix some panics to start testing

* Export tracing properly

* First example of child spans using opencensus

* Add spans to download protocol

* Add url to traces

* Remove opentracing

* Remove gopkg.* files

* Start deprecating opentracing

* Resolve stupid build errors

* Use observability package

* Fix test errors

* Convert buffalo spans to observercontext

* change package name

* defer flush to the end of the app execution

* Change op names to the correct package

* Rename pkg/observability to pkg/observ

* Show traces for the package

* Keep tracing in the earlier way

* Add info from request headers

* Remove whitespace

* Move exporter url to env var

* Add to env file for documentation

* Remove opentracing stuff

* Use stdlib

* Shorten service name

* Add a service name to olympus as well

* Add test to test if there is a recursion or not

* Add Ops Suite

* Move around code

* Make sure the service is not instantiated if the exporter is not found
2018-09-11 10:31:20 -07:00
Marwan Sulaiman
ea96158f8f Athens: introduce Single Flight (#573)
* Athens: introduce singleflight

* fix ci

* small fixes

* Remove unnecessary method

* simplify folder structure

* dcoument stasher iface

* fixes

* Fix olympus

* Remove Version from Download Protocol

* fix olympus

* refactor fetcher

* consolodate wrapping styles

* fix olympus

* Further refactoring

* fix typo
2018-09-03 16:31:13 -04:00
Aaron Schlesinger
da34539226 Making port configurable for the proxy (#596)
* Making port configurable for the proxy

Fixes https://github.com/gomods/athens/issues/595

* simple is good
2018-09-03 08:38:30 +02:00
Komu Wairagu
748ed7dda9 Remove dep files (#378)
* remove vendor directory
athens will be using Go modules for dependency management

* remove Gopkg.* files. athens will be using Go modules for dependency management

* add note about Go version required for developing on Athens
To use Go modules for dep management, you require atleast Go version 1.11

* update documentation regarding supported version of Go

* dont install dep

* re-add vendor directory
go mod vendor

* use vendored directory to carry out Go commands

* remove the use of -mod=vendor
this is necessary because;
1. we are running ci in a travis with a matrix of GO111MODULE=on & GO111MODULE=auto
and -mod=vendor is not available as an option in GO111MODULE=auto(inside GOPATH)

* rm vendor

* run go mod tidy
this is beacuse the master branch had changes to Gopkg.lock

* match gopkg.in/go-playground/validator.v9 version
with master branhc

* go mod tidy

* go mod vendor
so as to vendor the new changes that were added in master

* set GO111MODULE=on globally in ci

* build with -mod=vendor when building docker image

* use -mod=vendor to avoid getting deps from vcs in ci

* use -mod=vendor in .drone.yml

* use go1.11 in drone.yml
2018-09-02 11:54:42 -04:00
Rohan Chakravarthy
697a7a5073 remove olympus storage implementation and proxy worker (#513)
* remove olympus storage implementation and proxy worker

* cleanup after merge conflicts

* remove newline
2018-08-30 03:47:04 -04:00
Arthur Mello
e5ce3f7c6e Set a default value for ATHENS_MONGO_CONNECTION_STRING env var (#580)
* Set a default value for ATHENS_MONGO_CONNECTION_STRING env var

- ATHENS_MONGO_CONNECTION_STRING will default to mongodb://127.0.0.1:27017 if not set

* Fix Get function call

* Update user of env.MongoConnectionString
2018-08-28 11:56:15 +02:00
Komu Wairagu
69a5340c91 update to go1.11 (#563)
* update travis to Go 1.11

* update check_deps ci script
ci for checking dependencies should now check Go module files and use go mod verify

* remove uneeded comment

* set GO111MODULE

* debug GO111MODULE propagation

* rm dep installation

* debug GO111MODULE propagation

* rm debug statements

* use Go point releases in ci

* remove references to Go1.11 RCs

* run ci with GO111MODULE auto and also on

* use matrix
2018-08-28 08:46:12 +02:00
Marwan Sulaiman
9860bd51e7 Proxy: return 200 on / regardless of prefix (#572) 2018-08-26 22:11:04 -04:00
Andrey Dyachkov
062f24d32f Issue 427: error check for storage exists method (#458)
* The commit brings error check for storage exists

The commit opened against ticket #427, and adds error response to
checker exists method and all corresponding places. The major changes related to renaming and processing of additional errors, because the signature of Checker.Exists was changed to return error in response.

* improved error processing

* go fmt'ed files

* added check to minio storage for no key error

* moved our constant + changed check for err

* go fmt'ed

* deleted rdbms packege, which brought merge with upsteam

* process error from minio checker
2018-08-22 15:49:36 -04:00
Manu Gupta
3a8c6429e9 Removal of cdn rdbms driver and pop support from olympus (#551)
* Inital removal of cdn rdbms driver and pop support from olympus

* Update travis and other unit test scripts

* Remove postgres from Makefile

* Update comments and remove function that is not required

* Update .env file

* clean commented code

* Update docs to remove references to rdbms

* Remove pop and run dep ensure to make sure that the tests pass without pop

* Keeping GO_ENV=test for parametter logging
2018-08-22 20:10:19 +02:00
Manu Gupta
c0635b748c Remove pop from Proxy & RDBMS storage getter (#511)
* Remove pop from models for now

* Remove rdbms storage from pkg/storage

* Completely remove pop

* Some improvements to tests

- Not sourcing from cmd/proxy/.env, because that doesn't export any variables
- Removing mysql support (I feel like 1 database is enough)
- Pruning networks on teardown

I'm happy to split these changes up into separate PRs - it's the end of the day for me so I wanted to get everything in 😄

* update to point to test instance

* Update databasy.yml for travis??

* Remove database.yml

* Do not use db migrations and creation

* Remove pop dependency from cdn

* add schema for olympus assuming cdn driver is part of olympus

* Move CI/CD and test scripts to point pop to olympus

* Update database.yml and keep it only for olympus

* Add migrations to travis

* Add debug logs to see what travis is doing

* Update travis to use olympus database.yml

* Moarrrr logs to see where travis is pointint the db towards

* Change env to test

* Use test as the default environment if not specified otherwise

* Check if test new rdbms storage succeeds

* Try to see which connection string rdbms test suite is using on travis

* Update db tags for cdn driver

* create db by using travis go_env

* Remove sql from database config as it is removed from docker-compose

* Remove extra logs that I missed

* Add newline

* remove models.go and test file from proxy

* removing extraneous network prune from Makefile
2018-08-21 19:21:40 -04:00
Federico Paolinelli
c8b64fd258 Proxy 'admission controller' (#492)
* The proxy middleware calls an hook if set to check if the module needs
to be filtered

* Fixed error handling

* Added a middleware func that passes the log entry down to other
middlewares

* Split the filter middleware in two

* Moved validation middleware in its own file

* Renamed filter to middleware, all the middleware stuff in a single file

* Moved default hook to empty string

* Split tests of filter middleware from validation middleware

* Installed the validation middleware only if the hook is set

* Moved filter test to middleware test

* Fixed gofmt issue caused by manual merge

* Fixed wrong logrus import

* Removed logger from filter middleware

* Moved middleware functions to own package

* Removed unused argument, fixed errors

* Fixed gofmt errors

* Made middleware func private

* Changed debuglevel from string to enum

* Added comments to now public functions

* Changed module alias to a shorter one

* Removed white space

* Split middleware in separate files, injected the hook as param

* Removed commented code
2018-08-21 22:51:26 +02:00
Michal Pristas
042cbcaddf SSL Support for Mongo (#527)
* tested mongo ssl

* Added a todo
2018-08-21 18:09:49 +02:00
Marwan Sulaiman
8554f91286 Proxy: MVP Dockerfile (#536)
* Proxy: MVP Dockerfile

* remove comments

* add new line
2018-08-21 10:37:17 -04:00
Marwan Sulaiman
87c92dfe28 Proxy: add NETRC file mount (#528) 2018-08-20 13:58:28 -04:00
Marwan Sulaiman
622aba0fe1 Athens: disable buffalo logger in production (#516)
* Athens: disable buffalo logger in production

* fix gofmt

* pkg/log: pass down the level instead of environment

* pkg/log: remove nop writer

* pkg/log: use indepnedent buffalo log level
2018-08-20 13:23:27 -04:00
Marwan Sulaiman
d749465e35 pkg/download: use worker pool to limit concurrent gogets (#524)
* pkg/download: use worker pool to limit concurrent gogets

* fix test closure

* use workers env
2018-08-20 11:30:12 -04:00
Marwan Sulaiman
db5eb0ef69 proxy: add path prefix (#520) 2018-08-20 09:24:51 +02:00
Marwan Sulaiman
493945c13a Athens: add health check endpoint (#514) 2018-08-19 16:34:46 -04:00
Marwan Sulaiman
4c55c813d0 Proxy: make Force SSL configurable (#515)
* Proxy: make Force SSL configurable

* fix godoc
2018-08-19 15:57:55 -04:00
Marwan Sulaiman
1ac2cf1317 Proxy: remove unused arguments (#512)
* Proxy: remove unused arguments

* remove debugging info
2018-08-19 15:17:08 -04:00
Marwan Sulaiman
70167dc486 Proxy: use a global switch for Filter Middleware (#501)
* Proxy: do not redirect on filter.Include

* Filter: add Off opt

* Filter: set Off to true when there's no config

* Proxy: use a global switch for filter
2018-08-17 18:06:37 -04:00
Marwan Sulaiman
231cd85728 Proxy: add basic authentication middleware (#500)
* Proxy: add basic authentication middleware

* Proxy: fix ok
2018-08-17 09:01:35 -04:00
kteb
cdab5c515c Remove storage connector (#457)
* get rid of fake storage.Connector

* get rid of the storage.Connector for mango & rdbms

* Remove all the connector interfaces

* Updating all the test to get rid of the connect

* Updating connection to storage for proxy

* Updating connection to storage for olympus

* get rid of fake storage.Connector

* get rid of the storage.Connector for mango & rdbms

* Remove all the connector interfaces

* Updating all the test to get rid of the connect

* Updating connection to storage for proxy

* Updating connection to storage for olympus

* Fix reviews

* Fix reviews

* fixing op Name

* fix replace the check of an error insted of the object exists

* Fix error message in the test
2018-08-15 09:51:10 -07:00
Marwan Sulaiman
feb17a7a61 Athens: turn off worker to avoid nil logs (#483) 2018-08-14 21:43:39 -04:00
Carolyn Van Slyck
afd01813e9 Default to in-memory storage for the proxy (#475)
* Default to in-memory storage for the proxy

This makes it easier for someone to try out or just run the proxy by
executing the binary without further environment setup.

* Remove obsolete comment
2018-08-14 17:54:26 -04:00