* Changed mongo.go to use new driver
* Modified mongo cataloger
* More new driver related changes
* Change lister.go
* Change saver.go
* Change imports
* Remove unnecessary Count query
* Use IndexView for indexing
* Rename ModuleStore fields
* Use map of key:sorting-order for creating the index
* Minor changes
* Use client options to configure mongo client
* Use method chaining
* gofmt changes
* Change imports
* Fix some build errors
* Use new GridFS API
* Fix more build errors
* Add Go Mongo driver to dependency modules
* Use multierror
* Leave download stream open
* Remove mgo error handling
* Copy zip instead of loading all in memory
* Use context.WithTimeout() wherever possible
* Raise KindNotFound when mod@ver isn't found
* NopCloser not needed
* Fix IndexView error
* Fix build errors
* Remove another mgo error usage
* Fix build error
* Changes according to review
* Formatting changes as per gofmt
* Modify gofmt argument to show the expected formatting (diff)
* Handle ErrNoDocument error and error arising from query execution
* Fix kind of returned error
* Minor changes
* Bug fixes
* gofmt related changes
* Minor change
* Use Insecure from MongoConfig, remove Insecure from global Config
* Remove stray print statement
* Allow mgo.ParseURL to set [/database]
As per Michael's solution we need to be able to set the database from
either the parsed connection string, or fallback to athens if not set.
Signed-off-by: Chris M <me@christophermills.co.uk>
* Remove TimeoutDuration which was removed #928
Signed-off-by: Chris M <me@christophermills.co.uk>
* Correct conf_test to accommodate for changes
I'd missed out the addition of DefaultDBName from the test, this has now
been corrected
Signed-off-by: Chris M <me@christophermills.co.uk>
* Make sure gofmt runs automatically on this machine
Y'know simple stuff.
Signed-off-by: Chris M <me@christophermills.co.uk>
* Added new cataloger interface
* Implementing catalog protocol
* Propagated to protocol and over
* First round of fixes
* S3 almost ready, need to be tested
* Going on with testing s3
* Better testing with s3
* Simplified catalog tests
* Preparing gcp tests to access a gcp instance
* Fixing initialization errors
* Removed some prints
* Gcp ready, to be tested
* Gcp working
* Aligned bucket mock to catalog method
* Switched res payload to json
* Added catalog method to all storage instances
* Added catalog method to unsupported storages
* Fixed with pool test
* Restored tests
* Fixed gcp constructor
* Implemented catalog for fs
* Removed trace
* E2e tests, fixed fs
* Fixed module name return value
* Added cataloger method to azure storage
* Added docs
* Changed pagesize parameter name
* Fixed gofmt error
* Added json tags to result. Fixed lint warning
* Removed extra line
* Changed not implemented error to http.KindNotImplemented
* Checking for inequality on results
* Lower-cased json keys
* Added cleaning of path separator
* Fixed review comments
* Add catalog endpoint for mongo
* Add omitempty to ID
* Fix catalog tests
* update for next token
* fix e2e
* Make query readable
* Fix language and e2e script
* remove new line
* Added new cataloger interface
* Implementing catalog protocol
* Propagated to protocol and over
* First round of fixes
* S3 almost ready, need to be tested
* Going on with testing s3
* Better testing with s3
* Simplified catalog tests
* Preparing gcp tests to access a gcp instance
* Fixing initialization errors
* Removed some prints
* Gcp ready, to be tested
* Gcp working
* Aligned bucket mock to catalog method
* Switched res payload to json
* Added catalog method to all storage instances
* Added catalog method to unsupported storages
* Fixed with pool test
* Restored tests
* Fixed gcp constructor
* Implemented catalog for fs
* Removed trace
* E2e tests, fixed fs
* Fixed module name return value
* Added cataloger method to azure storage
* Added docs
* Changed pagesize parameter name
* Fixed gofmt error
* Added json tags to result. Fixed lint warning
* Removed extra line
* Changed not implemented error to http.KindNotImplemented
* Checking for inequality on results
* Lower-cased json keys
* Added cleaning of path separator
* Fixed review comments
* Fixed docs
* Add check if GCP bucket exists, Remove bucket creation
* Update GCP section in Storage Configuration docs
* Add more explicit info if bucket not exists
* Remove inefficient assignment of variable cxt in pkg/storage/fs
* Remove inefficient assignment of variable cxt in pkg/storage/minio
* Remove inefficient assignment of variable cxt in pkg/storage/mongo
* Fix commonly misspelled English words successfully and accessible
* Remove inefficient assignment of variable cxt in pkg/download
* Remove inefficient assignment of variable cxt in pkg/module
* Add missing error checks in a few unit-test files
* Revert removal of inefficient assignment of variable cxt
The variable, although is not being used, it maintains the readability
in the functions where it’s being assigned. The removal of `cxt` in
these files may introduce errors when a new function taking these
values is called. Two other contributors agree that it’s easy to miss
this detail when passing the variable to other functions and the traces
are not being recorded.
* Modify unit-test error reporting for the config package
* Modify config test to exit when ioutil.TempFile or os.Chmod fail
* Set up CI with Azure Pipelines
* trial & error
* use robocopy
* fix slash
* handle robocopy exit code
* rm goroot
* build proxy
* build & test proxy
* Skip Test_checkFilePerms on Win until issue resolved
* start minio and mongo
* do not create a new network on win 2016
* use win docker img for mongo
* download and start minio since there is no win docker img
* fix minio download & exec
* add minio access key and secret
* use new mongodb img
* fix docker-compose mongo
* trying to fix mongo tests
* ipconfig
* lets try that
* use docker inspect to get the container ip
* echo mongo container ip
* debuging docker inspect ipaddress
* set mongo env var to internal addr
* debug
* quotation is difficult
* without docker
* pass db dir
* split steps
* add some echos to know what takes so long
* cleanup
* add downloadURL var for minio and mongo
* rm windows docker compose file
* Remove nolonger used moniker package
Usage was removed in 97d8013
* Remove reference to Zeus
Use name Proxy for now.
* Remove Olympus code
See github issue #777
* Remove Olympus related vendor modules
* Mention Olympus removal in docs
* Add note about no registries existing
* Remove eventlog
* Update docs for Olympus removal
* Implement storage.Backend for S3 storage
* Adapt S3 storage test after refactor, Fix skiped minio tests
* Add defer to Close calls in s3.getter methods
* CR fixes
* Fix doc align
* Tweak err checks
* Fix parse test for S3 configs and cleanup error messages in storage.s3
When creating files, don't set the permission bits as 0777
(os.ModePerm), since that will cause them to be marked as executable.
Instead set them to 0666.
Add a comment explaining the umask will be subtracted, which is why 0666
(and not e.g. 0644 or 0600) is appropriate.
