Commit Graph

54 Commits

Author SHA1 Message Date
Mark Jung
a4cf4eb991 removed flag from test command (#1257)
fixes make test-unit by removing an outdated flag from the test command
2019-05-29 22:52:00 -04:00
Scott Lee
3f2c99f6a2 #1149 Remove references to Buffalo (#1203)
* #1149 Remove references to Buffalo

* #1149 Add Makefile
2019-04-30 09:59:52 -04:00
Arpit Gogia
974077e73b Mongo go driver beta integration (#1042)
* Changed mongo.go to use new driver

* Modified mongo cataloger

* More new driver related changes

* Change lister.go

* Change saver.go

* Change imports

* Remove unnecessary Count query

* Use IndexView for indexing

* Rename ModuleStore fields

* Use map of key:sorting-order for creating the index

* Minor changes

* Use client options to configure mongo client

* Use method chaining

* gofmt changes

* Change imports

* Fix some build errors

* Use new GridFS API

* Fix more build errors

* Add Go Mongo driver to dependency modules

* Use multierror

* Leave download stream open

* Remove mgo error handling

* Copy zip instead of loading all in memory

* Use context.WithTimeout() wherever possible

* Raise KindNotFound when mod@ver isn't found

* NopCloser not needed

* Fix IndexView error

* Fix build errors

* Remove another mgo error usage

* Fix build error

* Changes according to review

* Formatting changes as per gofmt

* Modify gofmt argument to show the expected formatting (diff)

* Handle ErrNoDocument error and error arising from query execution

* Fix kind of returned error

* Minor changes

* Bug fixes

* gofmt related changes

* Minor change

* Use Insecure from MongoConfig, remove Insecure from global Config

* Remove stray print statement
2019-04-17 19:59:01 +02:00
Carolyn Van Slyck
d379c20f44 Publish charts to azure storage (#1097)
* Add chart-push target and script

* Rename chart dir to match helm standards
2019-03-01 15:08:30 -08:00
Manu Gupta
53dd1c605e Do not fetch buffalo on travis (#1029) 2019-01-05 12:44:53 -05:00
Manu Gupta
23b285d49d Catalog endpoint support for mongo (#985)
* Added new cataloger interface

* Implementing catalog protocol

* Propagated to protocol and over

* First round of fixes

* S3 almost ready, need to be tested

* Going on with testing s3

* Better testing with s3

* Simplified catalog tests

* Preparing gcp tests to access a gcp instance

* Fixing initialization errors

* Removed some prints

* Gcp ready, to be tested

* Gcp working

* Aligned bucket mock to catalog method

* Switched res payload to json

* Added catalog method to all storage instances

* Added catalog method to unsupported storages

* Fixed with pool test

* Restored tests

* Fixed gcp constructor

* Implemented catalog for fs

* Removed trace

* E2e tests, fixed fs

* Fixed module name return value

* Added cataloger method to azure storage

* Added docs

* Changed pagesize parameter name

* Fixed gofmt error

* Added json tags to result. Fixed lint warning

* Removed extra line

* Changed not implemented error to http.KindNotImplemented

* Checking for inequality on results

* Lower-cased json keys

* Added cleaning of path separator

* Fixed review comments

* Add catalog endpoint for mongo

* Add omitempty to ID

* Fix catalog tests

* update for next token

* fix e2e

* Make query readable

* Fix language and e2e script

* remove new line
2019-01-02 22:13:56 -08:00
Manu Gupta
81547dd19e reduce sleep time in e2e between http calls (#994)
* reduce sleep time

* Check for version faster

* sleep to 1 sec to reduce hits

* Update scripts
2018-12-18 12:52:57 +01:00
Federico Paolinelli
0258e17d89 Catalogendpoint (#955)
* Added new cataloger interface

* Implementing catalog protocol

* Propagated to protocol and over

* First round of fixes

* S3 almost ready, need to be tested

* Going on with testing s3

* Better testing with s3

* Simplified catalog tests

* Preparing gcp tests to access a gcp instance

* Fixing initialization errors

* Removed some prints

* Gcp ready, to be tested

* Gcp working

* Aligned bucket mock to catalog method

* Switched res payload to json

* Added catalog method to all storage instances

* Added catalog method to unsupported storages

* Fixed with pool test

* Restored tests

* Fixed gcp constructor

* Implemented catalog for fs

* Removed trace

* E2e tests, fixed fs

* Fixed module name return value

* Added cataloger method to azure storage

* Added docs

* Changed pagesize parameter name

* Fixed gofmt error

* Added json tags to result. Fixed lint warning

* Removed extra line

* Changed not implemented error to http.KindNotImplemented

* Checking for inequality on results

* Lower-cased json keys

* Added cleaning of path separator

* Fixed review comments

* Fixed docs
2018-12-12 21:17:26 +01:00
Carolyn Van Slyck
1295637a7e Fix remaining docker image names (#910)
I missed a few (very important steps) that were setting the image name
2018-11-12 14:09:10 -08:00
Carolyn Van Slyck
bee463053a Rename docker image (#909)
* Rename docker image to gomods/athens

* Bump chart version to match new release version

* Switch from stable tag to latest tag

Our docs used latest, and it's awkard when you don't have a latest tag anyway
2018-11-12 15:15:58 -06:00
Rob j Loranger
2f524d27c3 add build details to binary (#866)
* poc

* finish JSON, remove uneeded methods, update docker file, finish script details

* build.Details didn't need to be exported

* typo

* somehow missed this

* should default to false

* Fix some things

remove commit SHA as build script will now create a hybrid for version
as in the push-docker-images.sh script, code shamelessly stolen.

change JSON function to Data and return a struct, the caller can do what
they like with that data, i.e. marshall JSON for a response body.

* add struct tags

* use build script in docker image

* newline in build.sh

* use previously generated version for binary build

* Working docker args

removed script build.sh, was not really needed.

date is generated automatically during docker build process.

VERSION will be set to 'Not Specified', or the version set during
our CI docker build/push step. A user can set their own version during
build using ldflags.

* remove old comment re: script

* Yikes

Almost forgot to remove my testing junk

* GitHub on my phone is hard
2018-11-09 10:32:55 -08:00
marpio
1a20f8b6a3 tag with stable on semver git tag (#858) 2018-11-04 14:26:35 -05:00
marpio
2f41b2f81b Update test_unit.ps1 so that it matches the bash version. (#799)
* add default env vars

* export GO111MODULE var

* rm dups
2018-10-29 14:15:19 +01:00
Marwan Sulaiman
322ff26694 Do not download modules during CI build (#815)
* Do not download modules during CI build

* install golint in non module system

* Use vendor in tests + update vendor dir

* mod tidy
2018-10-24 14:34:06 -07:00
Henry Jenkins
d26b99d41c Upgrade Buffalo (#789)
* Upgrade buffalo

* Switch to go modules everywhere

* Fixes from buffalo fix

* Add missing modules from module list

* Update vendored modules in /vendor

* Stop using vendor directory for tests

* Check go.mod and go.sum files on verify

* Upgrade Buffalo from v0.13.0 to v0.13.1

* Fix test for new Buffalo

Allow for new Buffalo code

* Add test for endpoint with trailing slash
2018-10-23 16:49:32 -07:00
Gabriel Schneider
ccf59405cf Updated according to test_unit.sh (#805) 2018-10-23 16:04:59 -07:00
Tomasz Tomalak
f665b0c2eb Fix skipped mongo and minio tests (#791) 2018-10-18 14:36:28 -07:00
Tomasz Tomalak
f5259a388d Add jaeger tracer to helm charts (#768)
* Add jaeger tracer to helm charts

* Add new lines to files
2018-10-15 20:34:36 -04:00
Tomasz Tomalak
4b6da42480 Fix issue with golint (#774) 2018-10-11 22:20:34 +02:00
Brent Pendergraft
524912a3b0 Clean up e2e binary (#698)
* clean up test binary after e2e tests

* continue test cleanup if binary removal fails
2018-09-23 00:31:16 -04:00
Marwan Sulaiman
83281a80fb Proxy remove assets and config scripts (#693)
* Proxy remove assets and config scripts

* update olympus dockerfile - tho its outdated anyway
2018-09-21 16:31:47 -04:00
marpio
bcc896f226 rename example to dev and rm test config (#691) 2018-09-20 16:38:08 -04:00
marpio
29b6b6ffab Minio conf + tests refactoring (#680)
* cleanup tests and change minio port

* fix cleanup

* cleanup

* fix config test

* add comment to travis

* revert to generic minio addr

* fix test

* switch to test config

* adapt timeouts

* use example config

* fix test... again

* add new lines
2018-09-20 21:31:21 +02:00
Marwan Sulaiman
5449649420 Do not use modules in go get of tools (#690)
* Do not use modules in go get of tools

* temp remove validation

* put back verify minus gofmt

* include module off in list
2018-09-20 13:18:51 -04:00
marpio
d6a1cdac84 do not read .env file , sync with the bash version (#689) 2018-09-20 16:23:45 +02:00
Marwan Sulaiman
d03eb7ba03 Travis: do not fetch modules during CI (#677) 2018-09-19 10:29:17 +02:00
Marwan Sulaiman
86d94c3e5a Olympus: stop docker push (#675)
* Olympus: stop docker push

* remove lines completely
2018-09-19 02:04:30 -04:00
marpio
d745dfbae4 Fix end 2 end test (#649)
* set GOPATH to the temp dir so that the cache is cleared

* cleanup

* fix unbound var

* add sudo

* remove sudo
2018-09-13 14:04:10 -07:00
marpio
f25c427239 Fix for Windows 10 (#648)
* ps: add cdefault conf

* init - add config

* add required env vars for win10

* update test_e2e.ps1 to use go build -mod=vendor
2018-09-13 13:57:37 -07:00
Rohan Chakravarthy
0e470d0294 Plumb config - Latest (#627)
* switch proxy to config file

pull in single flight changes

* changes for single-flight

* intermediate stage. All tests passing. pkg still has env refs

* remove all env references

* delete config/env entirely

* fix failing tests

* create the config.toml file as part of dev setup

* create config file only if it doesn't exist

* update Dockerfiles to use config file

* move composing elements to the top

* verbose parameter naming

* newline

* add flag for config file path

* update docs with config file flag

* remove unnecessary nil check

* use filepath.join

* rename redis port to address

* fix path.join

* fix issues after merge

* add vendor dir
2018-09-11 15:04:20 -07:00
Brent Pendergraft
a400d16674 remove buffalo dev in favor of go build in e2e test script (#636)
* remove buffalo dev in favor of go build in e2e test script

* re-write comment for clarity
2018-09-07 08:51:20 +02:00
Komu Wairagu
748ed7dda9 Remove dep files (#378)
* remove vendor directory
athens will be using Go modules for dependency management

* remove Gopkg.* files. athens will be using Go modules for dependency management

* add note about Go version required for developing on Athens
To use Go modules for dep management, you require atleast Go version 1.11

* update documentation regarding supported version of Go

* dont install dep

* re-add vendor directory
go mod vendor

* use vendored directory to carry out Go commands

* remove the use of -mod=vendor
this is necessary because;
1. we are running ci in a travis with a matrix of GO111MODULE=on & GO111MODULE=auto
and -mod=vendor is not available as an option in GO111MODULE=auto(inside GOPATH)

* rm vendor

* run go mod tidy
this is beacuse the master branch had changes to Gopkg.lock

* match gopkg.in/go-playground/validator.v9 version
with master branhc

* go mod tidy

* go mod vendor
so as to vendor the new changes that were added in master

* set GO111MODULE=on globally in ci

* build with -mod=vendor when building docker image

* use -mod=vendor to avoid getting deps from vcs in ci

* use -mod=vendor in .drone.yml

* use go1.11 in drone.yml
2018-09-02 11:54:42 -04:00
lpdyck
c4417371ef Fix powershell scripts (#615)
* add check_deps.ps1

* fix check_gofmt select-string regex pattern

* capitalize JoinPath

* fix typo

* fail check_deps.ps1 on error
2018-09-01 11:59:55 +02:00
Arthur Mello
e5ce3f7c6e Set a default value for ATHENS_MONGO_CONNECTION_STRING env var (#580)
* Set a default value for ATHENS_MONGO_CONNECTION_STRING env var

- ATHENS_MONGO_CONNECTION_STRING will default to mongodb://127.0.0.1:27017 if not set

* Fix Get function call

* Update user of env.MongoConnectionString
2018-08-28 11:56:15 +02:00
Komu Wairagu
69a5340c91 update to go1.11 (#563)
* update travis to Go 1.11

* update check_deps ci script
ci for checking dependencies should now check Go module files and use go mod verify

* remove uneeded comment

* set GO111MODULE

* debug GO111MODULE propagation

* rm dep installation

* debug GO111MODULE propagation

* rm debug statements

* use Go point releases in ci

* remove references to Go1.11 RCs

* run ci with GO111MODULE auto and also on

* use matrix
2018-08-28 08:46:12 +02:00
marpio
b62ebddb1b fix mongo env var name after rename (#578) 2018-08-27 14:11:32 -07:00
Rob j Loranger
37f0687761 No conflicts (#569)
* add script and step to verify stage of makefile

* this commit should fail the build

* build should now pass

* fix typo in excludes for grep

* better output on failure

also removes tilde from initial regex, git does not use tilde in diffs
adds output of filenames with detected regex on failure
2018-08-27 10:48:24 -06:00
marpio
1de9ea4c2a Powershell - clean up ENV after tests (#568)
* clean up ENV after tests
2018-08-26 22:40:28 +02:00
marpio
fa16c6950b Powershell scripts (#525)
* port scripts to ps

* remove ps core features

* init script

* make it work on ps 5.0

* sync with make file

* cleanup

* cleanup

* review feedback

* newline
2018-08-24 19:44:36 +02:00
Carolyn Van Slyck
26e8197b82 Publish docker images during CI builds (#544)
master builds publish using the git describe output  as the immutable tag and
canary for the mutable tag. Examples:

  gomods/proxy:abc1234
  gomods/proxy:canary

tagged builds publish using the tag as the immutable tag, which is
usually going to be a semver value and with latest for the mutable tag.
Examples:

  gomods/proxy:v1.0.0
  gomods/proxy:latest

branch builds publish using the git describe output as the immutable tag and the
branch name for the mutable tag. Examples:

  gomods/proxy:def4567
  gomods/proxy:beta-1
2018-08-24 10:36:42 -05:00
Manu Gupta
3a8c6429e9 Removal of cdn rdbms driver and pop support from olympus (#551)
* Inital removal of cdn rdbms driver and pop support from olympus

* Update travis and other unit test scripts

* Remove postgres from Makefile

* Update comments and remove function that is not required

* Update .env file

* clean commented code

* Update docs to remove references to rdbms

* Remove pop and run dep ensure to make sure that the tests pass without pop

* Keeping GO_ENV=test for parametter logging
2018-08-22 20:10:19 +02:00
Manu Gupta
c0635b748c Remove pop from Proxy & RDBMS storage getter (#511)
* Remove pop from models for now

* Remove rdbms storage from pkg/storage

* Completely remove pop

* Some improvements to tests

- Not sourcing from cmd/proxy/.env, because that doesn't export any variables
- Removing mysql support (I feel like 1 database is enough)
- Pruning networks on teardown

I'm happy to split these changes up into separate PRs - it's the end of the day for me so I wanted to get everything in 😄

* update to point to test instance

* Update databasy.yml for travis??

* Remove database.yml

* Do not use db migrations and creation

* Remove pop dependency from cdn

* add schema for olympus assuming cdn driver is part of olympus

* Move CI/CD and test scripts to point pop to olympus

* Update database.yml and keep it only for olympus

* Add migrations to travis

* Add debug logs to see what travis is doing

* Update travis to use olympus database.yml

* Moarrrr logs to see where travis is pointint the db towards

* Change env to test

* Use test as the default environment if not specified otherwise

* Check if test new rdbms storage succeeds

* Try to see which connection string rdbms test suite is using on travis

* Update db tags for cdn driver

* create db by using travis go_env

* Remove sql from database config as it is removed from docker-compose

* Remove extra logs that I missed

* Add newline

* remove models.go and test file from proxy

* removing extraneous network prune from Makefile
2018-08-21 19:21:40 -04:00
Rob j Loranger
43d7501420 Update e2e tests to not use sudo (#477)
* Update e2e tests to not use sudo

Using a system temporary directory via mktemp we can place our GOMOD_CACHE
in a place that does not require root permission to clear.

As well we use the environment variable for GO_BINARY_PATH to point to
a module compatible go version. At this time, 1.11beta3.

* add GO_BINARY_PATH to travis

* Use conditional GO_BINARY_PATH

Co-authored-by: robbie <hello@robloranger.ca>
Co-authored-by: carolyn <me@carolynvanslyck.com>

* no longer needs to set GO_BINARY_PATH in env
2018-08-15 19:16:14 +02:00
Fazlul Shahriar
5a445aacf8 scripts/get_buffalo.sh: use 'go get' to install outside of Travis (#415)
Since CONTRIBUTING.md and DEVELOPMENT.md asks the reader to run
'make setup-dev-env', which runs this script, it needs to be support
other platforms besides Linux.

Helps #397
2018-08-13 18:21:31 -07:00
Michal Pristas
5260639171 remove tempdir (#460) 2018-08-13 15:06:01 +02:00
Andrey Dyachkov
051f3884e1 Bring more clarity on how to run unit tests (#449)
* The commit brings more clarity on how to run unut tests.

The commit chages behaivour of `make test-unit` by adding sourcing of
env vars. Documentation is updated to provide more clarity on how to run
unit tests.

* changed list of commands to run before unit tests

* moved env vars to .env

* removed dduplicated env var

* returned conventional file name in script

* link to DEVELOPMENT.md

* fixed typo

* moved env var export to script

* removed export of tmp folder
2018-08-11 12:12:24 -07:00
Dinesh Kumar
4cc1b71007 Benchmark storages (#426)
* Adding benchmark for mongo and fs

* Adding delete, exists, save with other storages

* Fixing typos, adding nonexistent module exits benchmark

* Fixing delete operation in delete benchmark

* Separating non testing existing modules benchmark, shortening the benchmark name

* running benchmark in ci

* Revert "running benchmark in ci" - mongo index make duplicate records to
fail
idempotent, will fix in a separate PR.

This reverts commit f7d78262d2.

* Adding reset timer, fixing duplicate record issue

* Revert "Revert "running benchmark in ci" - mongo index make duplicate records to"

This reverts commit 40a7b33199.

* removing benchmark from ci

* Adding operation for returning errors
2018-08-10 01:34:42 -04:00
Rob j Loranger
dd16ce86f1 Fix up e2e script (#410)
* small fixes

* set GO111MODULES=auto

* set GO111MODULES=on after running buffalo

* use travis for go1.11beta3 not build from source

* set GO111MODULES=on before running buffalo

* revert GO111MODULE placement to after buffalo dev, comment w/ issue link

* oops

* spaces too
2018-08-09 16:09:39 -07:00
Marwan Sulaiman
44625459fc Download: move cache location (#404)
* Download: move cache location

* beta3

* trying differnet repo

* avoid go breaking stuff
2018-08-04 11:29:30 -07:00
Carolyn Van Slyck
f1efadeed3 Run E2E tests during CI (#363)
* Use go1.11beta2 instead of vgo

* Make the go 1.11 source directory configurable

Don't error out if it's been cloned already

* Add end-to-end (e2e) tests

* Make sure the repo bin is on the path

* Call the dev target from setup-dev-env

* Use the same dev setup on travis and locally
2018-08-03 16:40:33 +02:00