Commit Graph

24 Commits

Author SHA1 Message Date
Vanes Angelo
900cb4f7c1 Upgrade to AWS SDK v2 (#1938) (#1950)
Upgrades the AWS SDK to v2. AWS S3 bucket urls will now error if they are not prefixed with a schema (example: https://).
2024-04-28 21:38:42 -07:00
Nicholas Wiersma
d932d50232 chore: lint code with golangci-lint (#1828)
* feat: add golangci-lint linting

* chore: fix linter issues

* feat: add linting into the workflow

* docs: update lint docs

* fix: cr suggestions

* fix: remove old formatting and vetting scripts

* fix: add docker make target

* fix: action go caching

* fix: depreciated actions checkout version

* fix: cr suggestion

* fix: cr suggestions

---------

Co-authored-by: Manu Gupta <manugupt1@gmail.com>
2023-02-24 20:39:17 -08:00
Zhongpeng Lin
e97cb5f5df Passing http client to S3 client (#1786) 2022-08-26 09:03:56 -04:00
Marwan Sulaiman
52934cfa46 implement /index endpoint (#1630)
* implement /index endpoint

* rename to Module to Path
2020-06-24 14:29:30 -04:00
Marwan Sulaiman
38a6a6fe0b Add client tracing to external storage requests (#1626) 2020-06-14 11:03:48 -04:00
Marwan Sulaiman
3c4db4ce86 pkg/storage: add External implementation (#1587)
* pkg/storage: add External implementation

* fix conflicts

* use newly instantiated client
2020-03-27 13:35:52 -04:00
Aravind
48f7ca7780 Implement storage.Backend for azureblob (#917) 2018-11-29 17:06:19 +01:00
marpio
4785e9c2b3 Remove storage timeouts (#928)
* remove storage timeouts

* fix tests
2018-11-16 16:00:08 -05:00
marpio
54f92c55d6 Config cleanup (#860)
* add default env vars

* del setRuntimeDefaults

* replace cdn conf with azure config and remove baseURL
2018-11-05 09:04:38 +01:00
Tomasz Tomalak
480d8c8e8c Implement storage.Backend for S3 storage (#750)
* Implement storage.Backend for S3 storage

* Adapt S3 storage test after refactor, Fix skiped minio tests

* Add defer to Close calls in s3.getter methods

* CR fixes

* Fix doc align

* Tweak err checks

* Fix parse test for S3 configs and cleanup error messages in storage.s3
2018-10-15 21:46:08 -04:00
Rohan Chakravarthy
0e470d0294 Plumb config - Latest (#627)
* switch proxy to config file

pull in single flight changes

* changes for single-flight

* intermediate stage. All tests passing. pkg still has env refs

* remove all env references

* delete config/env entirely

* fix failing tests

* create the config.toml file as part of dev setup

* create config file only if it doesn't exist

* update Dockerfiles to use config file

* move composing elements to the top

* verbose parameter naming

* newline

* add flag for config file path

* update docs with config file flag

* remove unnecessary nil check

* use filepath.join

* rename redis port to address

* fix path.join

* fix issues after merge

* add vendor dir
2018-09-11 15:04:20 -07:00
Arthur Mello
e5ce3f7c6e Set a default value for ATHENS_MONGO_CONNECTION_STRING env var (#580)
* Set a default value for ATHENS_MONGO_CONNECTION_STRING env var

- ATHENS_MONGO_CONNECTION_STRING will default to mongodb://127.0.0.1:27017 if not set

* Fix Get function call

* Update user of env.MongoConnectionString
2018-08-28 11:56:15 +02:00
Manu Gupta
c0635b748c Remove pop from Proxy & RDBMS storage getter (#511)
* Remove pop from models for now

* Remove rdbms storage from pkg/storage

* Completely remove pop

* Some improvements to tests

- Not sourcing from cmd/proxy/.env, because that doesn't export any variables
- Removing mysql support (I feel like 1 database is enough)
- Pruning networks on teardown

I'm happy to split these changes up into separate PRs - it's the end of the day for me so I wanted to get everything in 😄

* update to point to test instance

* Update databasy.yml for travis??

* Remove database.yml

* Do not use db migrations and creation

* Remove pop dependency from cdn

* add schema for olympus assuming cdn driver is part of olympus

* Move CI/CD and test scripts to point pop to olympus

* Update database.yml and keep it only for olympus

* Add migrations to travis

* Add debug logs to see what travis is doing

* Update travis to use olympus database.yml

* Moarrrr logs to see where travis is pointint the db towards

* Change env to test

* Use test as the default environment if not specified otherwise

* Check if test new rdbms storage succeeds

* Try to see which connection string rdbms test suite is using on travis

* Update db tags for cdn driver

* create db by using travis go_env

* Remove sql from database config as it is removed from docker-compose

* Remove extra logs that I missed

* Add newline

* remove models.go and test file from proxy

* removing extraneous network prune from Makefile
2018-08-21 19:21:40 -04:00
Michal Pristas
042cbcaddf SSL Support for Mongo (#527)
* tested mongo ssl

* Added a todo
2018-08-21 18:09:49 +02:00
kteb
cdab5c515c Remove storage connector (#457)
* get rid of fake storage.Connector

* get rid of the storage.Connector for mango & rdbms

* Remove all the connector interfaces

* Updating all the test to get rid of the connect

* Updating connection to storage for proxy

* Updating connection to storage for olympus

* get rid of fake storage.Connector

* get rid of the storage.Connector for mango & rdbms

* Remove all the connector interfaces

* Updating all the test to get rid of the connect

* Updating connection to storage for proxy

* Updating connection to storage for olympus

* Fix reviews

* Fix reviews

* fixing op Name

* fix replace the check of an error insted of the object exists

* Fix error message in the test
2018-08-15 09:51:10 -07:00
Carolyn Van Slyck
afd01813e9 Default to in-memory storage for the proxy (#475)
* Default to in-memory storage for the proxy

This makes it easier for someone to try out or just run the proxy by
executing the binary without further environment setup.

* Remove obsolete comment
2018-08-14 17:54:26 -04:00
Marwan Sulaiman
15326af2bb GCP: get credentials implicitly (#380)
* GCP: get credentials implicitly

* update name
2018-08-02 12:37:19 -04:00
Michal Pristas
8dc02d6b3b Env and config refactor (#209)
* reduce envy just to one package
2018-07-05 22:32:17 +02:00
Rob j Loranger
5101e1ea6d basic root dir pre check on fs storage.New (#199)
* basic root dir pre check on fs storage.New

* update uses of fs.storage.New to check for error

* use testify to check for error
2018-06-26 12:37:21 -06:00
Michal Pristas
6c4a0641df Background job for pulling modules from olympus (#132)
* basic structure

* added olympus endpoint for fetching log

* broken test

* added backing buffalo job

* slightly reshuffled, todo more shuffling

* reshuffle more

* breaking things up

* more breaking up

* hook to 101

* added redis to docker compose file

* working!

* a

* build fix

* first batch of changes towards new arch"

* first batch of comments resolved

* override for olympus endpoint

* styling

* osFs fix

* fixed in memory storage

* in-mem storage and test (#166)
2018-06-11 18:12:10 +02:00
marpio
a1e76e4569 in-mem storage and test (#166) 2018-06-11 09:56:04 +02:00
marpio
cde250728a Implement Minio storage (#128)
* add minio dep

* implement minio storage

* fix travis.yml

* start minio in background
2018-05-07 18:13:11 -07:00
lcd1232
700d6ee7af Add rdbms backend (#74)
* Add rdbms backend

* Remove unused imports

* Fix ci

* Fix problems

* Now test works

* Fix lint errors

* Change name to more descriptive

* Add logic to support rdbms storage

* Update to new newStorage logic
2018-03-29 14:23:08 -07:00
Michal Pristas
9c14b1fd12 Proposal: move buffalo app to cmd (#91)
* buffalo app moved to cmd

* ooops:

* added copy of proxy as olympus

* removed buffalo test

* added blank endpoint for feed

* forgot db config file and storage setup

* resolved comments

* newlines

* resolved conficts
2018-03-28 10:30:56 -04:00