Commit Graph

42 Commits

Author SHA1 Message Date
Marwan Sulaiman
322ff26694 Do not download modules during CI build (#815)
* Do not download modules during CI build

* install golint in non module system

* Use vendor in tests + update vendor dir

* mod tidy
2018-10-24 14:34:06 -07:00
Henry Jenkins
d26b99d41c Upgrade Buffalo (#789)
* Upgrade buffalo

* Switch to go modules everywhere

* Fixes from buffalo fix

* Add missing modules from module list

* Update vendored modules in /vendor

* Stop using vendor directory for tests

* Check go.mod and go.sum files on verify

* Upgrade Buffalo from v0.13.0 to v0.13.1

* Fix test for new Buffalo

Allow for new Buffalo code

* Add test for endpoint with trailing slash
2018-10-23 16:49:32 -07:00
Henry Jenkins
532e93e119 Remove Olympus code (#779)
* Remove nolonger used moniker package

Usage was removed in 97d8013

* Remove reference to Zeus

Use name Proxy for now.

* Remove Olympus code

See github issue #777

* Remove Olympus related vendor modules

* Mention Olympus removal in docs

* Add note about no registries existing

* Remove eventlog

* Update docs for Olympus removal
2018-10-22 11:52:02 -07:00
Marwan Sulaiman
97d8013876 Refactor storage tests (#727)
* Refactor storage tests

* minio: check object err

* remove inaccurate test

* storage/fs: use memory
2018-10-04 16:16:24 -07:00
Marwan Sulaiman
337488c202 Add stackdriver to traces (#737) 2018-10-04 18:47:38 -04:00
Bhavesh Praveen
73e45cb413 Tracing Exporter for Datadog #714 (#723)
* Tracing Exporter for Datadog #714

* Tracing Exporter for Datadog #714
2018-10-03 22:48:10 -04:00
Manu Gupta
d663e733a3 clean up mod and sum files (#701) 2018-09-25 14:03:30 +02:00
Marwan Sulaiman
5fd3e12270 Update OpenCensus to v0.17.0 (#686)
* Update OpenCensus to v0.17.0

* fix typo

* Rename go file to match pkg name
2018-09-20 12:50:32 -04:00
Marwan Sulaiman
e1b7fbcff0 Git ignore .env file (#684) 2018-09-20 10:46:58 -04:00
Marwan Sulaiman
6910a5ea19 fix gobuffalo/suite hash mismatch (#674)
* fix gobuffalo/suite hash mismatch

* update vendor
2018-09-19 01:50:51 -04:00
Rohan Chakravarthy
0e470d0294 Plumb config - Latest (#627)
* switch proxy to config file

pull in single flight changes

* changes for single-flight

* intermediate stage. All tests passing. pkg still has env refs

* remove all env references

* delete config/env entirely

* fix failing tests

* create the config.toml file as part of dev setup

* create config file only if it doesn't exist

* update Dockerfiles to use config file

* move composing elements to the top

* verbose parameter naming

* newline

* add flag for config file path

* update docs with config file flag

* remove unnecessary nil check

* use filepath.join

* rename redis port to address

* fix path.join

* fix issues after merge

* add vendor dir
2018-09-11 15:04:20 -07:00
Manu Gupta
bb250437da Instrumenting observability with opencensus (#604)
* Initial chanages for observability

* Fix some panics to start testing

* Export tracing properly

* First example of child spans using opencensus

* Add spans to download protocol

* Add url to traces

* Remove opentracing

* Remove gopkg.* files

* Start deprecating opentracing

* Resolve stupid build errors

* Use observability package

* Fix test errors

* Convert buffalo spans to observercontext

* change package name

* defer flush to the end of the app execution

* Change op names to the correct package

* Rename pkg/observability to pkg/observ

* Show traces for the package

* Keep tracing in the earlier way

* Add info from request headers

* Remove whitespace

* Move exporter url to env var

* Add to env file for documentation

* Remove opentracing stuff

* Use stdlib

* Shorten service name

* Add a service name to olympus as well

* Add test to test if there is a recursion or not

* Add Ops Suite

* Move around code

* Make sure the service is not instantiated if the exporter is not found
2018-09-11 10:31:20 -07:00
Tobias Klauser
ec2748ecc3 Remove unused file in pkg/module and clean up dependencies (#625)
* pkg/module: Remove unused file parser.go

PR #618 removed the last remaining user of the functions/types defined
in parser.go

* go.mod: update

The last user of github.com/sabhiram/go-gitignore was removed. Run 'go
mod tidy' to remove the dependency.

* vendor: remove unused dependency

The last user of github.com/sabhiram/go-gitignore was removed. Remove
the vendored copy as well by running 'go mod vendor'.
2018-09-07 02:04:23 -04:00
Komu Wairagu
748ed7dda9 Remove dep files (#378)
* remove vendor directory
athens will be using Go modules for dependency management

* remove Gopkg.* files. athens will be using Go modules for dependency management

* add note about Go version required for developing on Athens
To use Go modules for dep management, you require atleast Go version 1.11

* update documentation regarding supported version of Go

* dont install dep

* re-add vendor directory
go mod vendor

* use vendored directory to carry out Go commands

* remove the use of -mod=vendor
this is necessary because;
1. we are running ci in a travis with a matrix of GO111MODULE=on & GO111MODULE=auto
and -mod=vendor is not available as an option in GO111MODULE=auto(inside GOPATH)

* rm vendor

* run go mod tidy
this is beacuse the master branch had changes to Gopkg.lock

* match gopkg.in/go-playground/validator.v9 version
with master branhc

* go mod tidy

* go mod vendor
so as to vendor the new changes that were added in master

* set GO111MODULE=on globally in ci

* build with -mod=vendor when building docker image

* use -mod=vendor to avoid getting deps from vcs in ci

* use -mod=vendor in .drone.yml

* use go1.11 in drone.yml
2018-09-02 11:54:42 -04:00
Rohan Chakravarthy
7c745fb3d9 add a configuration file (#453)
* complete updated config package

* use envconfig+toml instead of viper. Add descriptions in example config file

* add unit tests

* debug gofmt on build server

* force dummy commit

* skip gofmt to validate other tests are passing

* unset env vars for example file parsing test

* cleanup tests

* test improvements

* re-enable gofmt

* naming

* PR comments

* fix failing test after olympus default endpoint change

* remove rdbms config

* set defaults in code

* add support for proxyfilteroff

* add basic auth params

* update gopkg.lock

* undo gopkg.lock changes made during merge

* remove defaults

* explicitly specify all env variables

* remove rdbms from example

* remove user and pass to disable basic auth by default

* switch to memory by default for the proxy

* fix tests after config file change
2018-08-31 14:23:41 -04:00
Komu Wairagu
e124f7b9a0 update uuid (#419) 2018-08-06 17:51:23 -07:00
Manu Gupta
5568175254 Update buffet to the tip of the master in go (#388) 2018-08-03 12:46:12 -07:00
Komu Wairagu
94bbcd1996 update redigo (#266)
* dep ensure -v -update github.com/gocraft/work

* rm -rf vendor/github.com/garyburd

* sed /garyburd/redigo / gomodule/redigo

* dont constrain redigo version

* update redigo ver

* rm deps

* update deps

* add vendor
2018-07-15 10:37:29 -07:00
Rob j Loranger
2018538c94 finish gcp implementation (#208)
adds gcp implementation of the (./pkg/storage.Backend) interface
2018-07-14 14:14:24 -07:00
Rob j Loranger
46699719a0 fix merge conflict artifact and remove corba from Gopkg.toml (#265) 2018-07-13 19:22:43 -07:00
Michal Pristas
7795def18b SkipDead for workers (#256)
* skip dead

* refactor

* real redis tests of workers

* configurable maxfails
2018-07-13 12:18:24 -07:00
marpio
9ee0478780 Stream zip up to AWS S3 storage (#254)
* add s3manager dep and remove cobra

* use env timeout

* use s3manager  for uploads

* ups
2018-07-13 09:36:16 -07:00
marpio
58d42a1b57 Stream zips (#200)
* storage.saver interface - use io.reader instead of []byte

* stream zip - backendconnector

* stream zip - azurecdn

* stream zip - fs

* storage - switch to io.Reader

* update callers of Save

* rm empty line

* makezip stream

* refactor mergedb

* add multierror dep

* mergedb multierror

* scope err

* update storages

* return pipereader

* fix test

* use multierror for all strorage drivers

* review feedback

* azure cdn storage - use UploadStreamToBlockBlob

* use numUpload

* use timeout ctx
2018-07-07 08:46:43 -07:00
Michal Pristas
5caa8a369d AWS S3 Storage (#194)
* basic aws impl

* iface client

* comments

* s3 mock

* Added test

* exclude mocks from linter

* get rid of races

* subpackaging
2018-06-29 16:02:16 -06:00
Rob j Loranger
9e76722acc GCP storage saver (#183)
* Add Dependencies for GCP

cloud.google.com/go/storage
google.golang.org/api/option // allows authetication options for storage
github.com/golang/protobuf // override to fix dependency issue

issue was with a protobuf library required by the storage library.
it was importing v1.0.0 but referencing code from v1.1.0

* WIP mostly roughed out

some small descisions to be made and testing is missing

* rename gcp.go to saver.go

* rename gcp.go to saver.go

* Requested Changes

+ closing the storage.Client is not required
+ move increment of count into 'for' line
+ add tracing span for save helper
+ removed projectID from New signiture, unused

* basic test, just testing generator so far

* Readability fixes

+ change calls to save so they send directly to the errs channel
+ return wc.Close instead of checking the error first
+ also remove failing part of tests for now, needs authentication

* Happy Path Tests

- removed use of buffalo.Context
- (storage).Save no longer accepts a context
+ includes basic happy path tests
+ private key for test project praxis-cab-207400.appspot.com

* messy gitwork

* remove keys

* ch-ch-changes🎶

+ defer close of storage.Writer, logging to be decided
+ correct check on length of errsOut
+ set inital length of errsOut to 0

* more stuff is in here

+ switch to context being passed again
+ use context with timeout instead of time.After
+ tests pull creds from env variable

missing skipping tests when no variable is set and teardown is no
implemented

* skip gcp storage tests when no key file present

* speed up tests a little

* need aetest pkg

* add aetest to Gopkg.toml explicitly

* tear down suite deletes storage artifacts

* should setup whole suite

* log error on wc.close and close errs chan

* cleaner check for env var

* use envy where appropriate and pull gcp bucket name from env var

* one more for consistency

I like the comma ok idiom better in this case but everything else is
using envy.

* don't panic in test set up and tear down

* use shared context with timeout and switch to seconds for duration

* no one needed the context.Cancel func. use `file` instead of `content` for consistency

* defer cancellation of context for uploads

* Update saver.go

+ rename `save` to `upload`
+ clarify comments
2018-06-25 15:49:56 -06:00
Aaron Schlesinger
a8e71934eb [WIP] Implementing a CDN driver for Azure (#170)
* Implementing the CDN getter and storage saver for Azure CDN

Fixes https://github.com/gomods/athens/issues/29

* Adding azure dependencies

* adding GoGet middleware into olympus

just using azure blob storage for now. will parameterize later

* cleaning up compile errs

* adding fmt.Sprintf param

* removing newline in imports

* following the rules for container names

* removing unused import
2018-06-14 21:08:49 +00:00
Michal Pristas
6c4a0641df Background job for pulling modules from olympus (#132)
* basic structure

* added olympus endpoint for fetching log

* broken test

* added backing buffalo job

* slightly reshuffled, todo more shuffling

* reshuffle more

* breaking things up

* more breaking up

* hook to 101

* added redis to docker compose file

* working!

* a

* build fix

* first batch of changes towards new arch"

* first batch of comments resolved

* override for olympus endpoint

* styling

* osFs fix

* fixed in memory storage

* in-mem storage and test (#166)
2018-06-11 18:12:10 +02:00
marpio
cde250728a Implement Minio storage (#128)
* add minio dep

* implement minio storage

* fix travis.yml

* start minio in background
2018-05-07 18:13:11 -07:00
Brian Ketelsen
e76125d3f1 Feature: Add OpenTracing (#112)
* tracing

* oops
2018-03-26 17:14:49 -04:00
Aaron Schlesinger
b7e15a845c Mongo cdn getter (#62)
* Implementing a MongoDB backed CDN getter

Fixes https://github.com/gomods/athens/issues/49

* removing the base URL from the CDN getter
2018-03-23 13:41:22 -07:00
Brian Ketelsen
cf6a6f4e3f UI: Start UI (#35)
* UI: Start UI

* link to github auth

* save point

* simplify the UI a lot

* make a separate renderer for proxy vs registry

* get rid of debug

* update deps

* remove unneeded assets
2018-03-21 17:09:45 -04:00
marpio
2694401b7e Use afero for as abstraction layer for FS (#70)
* add afero dependency

* implement memory and disk storage using afero fs

* pass the appropriate fs

* remove unnecessary interface in the fs package
2018-03-20 18:39:55 -07:00
Michal Pristas
1df7400819 not tested version (#54) 2018-03-15 20:01:54 +01:00
Brian Ketelsen
733fa62ec5 athens: add userStore interface and mongoDB implementation (#34)
* athens: save and retrieve OAUTH users from github to mongodb storage interface

* deps

* add mongo to travis

* updates based on PR feedback
2018-03-07 15:29:15 -05:00
Aaron Schlesinger
353c1e35d1 Moving to cobra for the CLI (#7)
* [WIP] Moving to cobra for the CLI

Fixes https://github.com/gomods/athens/issues/6

* dep ensure

* dep ensure

* travis ignoring vendor for 1.8

* disabled 1.8 for travis
2018-03-07 09:16:09 -05:00
Michal Pristas
7fc0e6f3b5 zip creation fixed (#30) 2018-03-05 22:43:29 +01:00
Brian Ketelsen
ef2a8230b9 basic github oauth (#23) 2018-03-02 11:22:38 -08:00
Brian Ketelsen
10bbd589b7 athens: update to buffalo 0.11.0, add cors and csrf support (#13) 2018-03-01 10:22:48 -08:00
Aaron Schlesinger
89eeef7da1 adding CLI and refactoring payload defns 2018-02-25 14:34:01 -08:00
Aaron Schlesinger
cf3d44a62c pruning
after the go-git dependency was dropped
2018-02-25 14:06:34 -08:00
Aaron Schlesinger
a7f1d27491 adding list and admin API 2018-02-25 14:05:36 -08:00
Aaron Schlesinger
5c889a06f4 more initial things 2018-02-25 12:32:51 -08:00