diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 00000000..263ba27e --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,54 @@ +# Development Guide for Athens + +Both the registry and the proxy are written using the [Buffalo](https://gobuffalo.io/) framework. We chose +this framework to make it as straightforward as possible to get your development environment up and running. + +You'll need Buffalo [v0.11.0](https://github.com/gobuffalo/buffalo/releases/tag/v0.11.0) or later to get started on Athens, +so be sure to download the CLI and put it into your `PATH`. + +# Services that Athens Needs + +Both the proxy and the registry rely on several services (i.e. databases, etc...) to function +properly. We use [Docker](http://docker.com/) images to configure and run those services. + +If you're not familiar with Docker, that's ok. In the spirit of Buffalo, we've tried to make +it easy to get up and running: + +1. [Download and install docker-compose](https://docs.docker.com/compose/install/) (docker-compose is a tool for easily starting and stopping lots of services at once) +2. Run `make dev` from the root of this repository + +That's it! After the `make dev` command is done, everything will be up and running and you can move +on to the next step. + +If you want to stop everything at any time, run `make dev-teardown`. + +# Run the Proxy or the Registry + +As you know from reading the [README](./README.md) (if you didn't read the whole thing, that's ok. Just read the +introduction), the Athens project is made up of two components: + +1. [Package Registry](https://github.com/gomods/athens/wiki/The-Central-Package-Registry-(Olympus)) +2. [Edge Proxy](https://github.com/gomods/athens/wiki/Proxies-(Zeus)) + +To run the registry: + +```console +cd cmd/olympus +buffalo dev +``` + +To run the proxy: + +```consols +cd cmd/proxy +buffalo dev +``` + +After either `buffalo dev` command, you'll see some console output like: + +```console +Starting application at 127.0.0.1:3000 +``` + +And you'll be up and running. As you edit and save code, the `buffalo dev` command will notice and automatically +re-compile and restart the server. diff --git a/README.md b/README.md index 7579e36d..35075155 100644 --- a/README.md +++ b/README.md @@ -1,84 +1,40 @@ -_Note: this document is out of date. Updates coming soon, but until then, see the following two design docs:_ -- [Design of Olympus, the central package repository](https://github.com/gomods/athens/wiki/The-Central-Package-Registry-(Olympus)) -- [Design of Zeus, the edge proxy](https://github.com/gomods/athens/wiki/Proxies-(Zeus)) +# Welcome Gophers! +Welcome to the Athens project! We're building all things Go package repository in here. -# Athens +1. [Package Registry](https://github.com/gomods/athens/wiki/The-Central-Package-Registry-(Olympus)) +2. [Edge Proxy](https://github.com/gomods/athens/wiki/Proxies-(Zeus)) -*This is a very early alpha release, and the API will be changing as the proxy API changes.* -_Do not run this in production. This warning will be changed or removed as the project and the proxy API changes._ +If you want more of a tl;dr on the project, check out [this quick introduction](https://medium.com/@arschles/project-athens-c80606497ce1) -Athens is a proxy server for [vgo modules](https://github.com/golang/go/wiki/vgo). It implements the download protocol specified [here](https://research.swtch.com/vgo-module) (under "Download Protocol"), and a few additional API endpoints to make it more useful. See [API.md](./API.md) for more information. +# Project Status -# Architecture +Project Athens is in a very early alpha release and everything might change. +Don't run it in production, but do play around with it and [contribute](#contributing) +when you can! -Athens is composed roughly of three logical pieces. The below list contains links to a description of each: +# More Details Please! -* [Module proxy](./PROXY.md) -* [Module registry](./REGISTRY.md) -* [CLI](./CLI.md) +Although the project is in development, here's where we're going: + +The package registry and the edge proxy both implement the [vgo download protocol](https://medium.com/@arschles/project-athens-the-download-protocol-2b346926a818), but each one +is intended for different purposes. + +The registry will be hosted globally, and will be "always on" for folks. Anyone will be able to +configure their machine to do a `go get` (right now, it's a `vgo get`) and have it request +packages from the registry. + +On the other hand, the registry will only host _public_ code. If you have private code, the +edge proxy is your jam. The proxy will store your private code for you, in your database +of choice. It will be designed to also cache packages from the registry, subject to +an exclude list. # Development -The server is written using [Buffalo](https://gobuffalo.io/), so it's fairly straightforward -to get started on development. You'll need Buffalo v0.11.0 or later to do development on Athens. +See [DEVELOPMENT.md](./DEVELOPMENT.md) for details on how to set up your development environment +and start contributing code. -Download [v0.11.0](https://github.com/gobuffalo/buffalo/releases/tag/v0.11.0) or later, untar/unzip the binary into your PATH, and then run the -following to run [Olympus](https://github.com/gomods/athens/wiki/The-Central-Package-Registry-(Olympus)): - -```console -cd cmd/olympus -buffalo dev -``` - -... and the following to run [Athens](https://github.com/gomods/athens/wiki/Proxies-(Athens): - -```console -cd cmd/proxy -buffalo dev -``` - -After you see something like `Starting application at 127.0.0.1:3000`, the server -is up and running. As you edit and save code, Buffalo will automatically restart the server. - -## Dependencies and Set-up - -To run the development server, or run tests (tip: run `make test` to easily run tests), you'll need a running MongoDB server. We plan to add more service dependencies in the future, so we are using [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) to create and destroy -development environments. - -To create, run the following from the repository root: - -```console -docker-compose -p athens up -d -``` - -To destroy: - -```console -docker-compose -p athens down -``` - -A few environment variables are expected by the application and tests. They are -stored in `cmd/olympus/.env` and `cmd/proxy/.env`. Below is a table of the -default values: - -|Variable |Value | -|---|---:| -|POP_PATH |$PWD/cmd/proxy | -|GO_ENV |test_postgres | -|MINIO_ACCESS_KEY |minio | -|MINIO_SECRET_KEY |minio123 | -|ATHENS_MONGO_STORAGE_URL |mongodb://127.0.0.1:27017 | - -To set in bash/zsh/osx: `export POP_PATH=$PWD/cmd/proxy` -To set in fish: `set -Ux POP_PATH $PWD/cmd/proxy` - -Lastly you will need to create and initialize the database. - -```console -buffalo db create -buffalo db migrate up -``` +Speaking of contributing, read on! # Contributing @@ -92,49 +48,6 @@ in the [queue](https://github.com/gomods/athens/issues) that you'd like to work * ["Go and Versioning"](https://research.swtch.com/vgo) papers * [vgo wiki](https://github.com/golang/go/wiki/vgo) -# Does it Work? - -Great question (especially for an alpha project)! The short answer is this: - -> The basic pieces are in place for a proxy, but the CLI and the server makes -> it near-impossible to use this thing in the real world - -## Some Additional Details - -The basic API and storage system work, but the proxy is limited for real world use right now. - -First, it only stores modules in memory, so that's a major issue if you want to use it for anything real. - -Second, it doesn't hold any packages other than the ones you upload to it. A package proxy is pretty much only as useful as the packages it stores. You can work around that by declaring dependencies as `file:///` URLs if you want, but that defeats much of the purpose of this project. - -When athens has better storage drivers (at least persistent ones!), it'll be easier to load it up -with modules (i.e. by running a background job to crawl your `GOPATH`). At that point, it'll be more practical to successfully run `vgo get` inside a less-trivial project. - -Finally, here's what the whole workflow looks like in the real world (spoiler alert: the CLI needs work). The setup: - -* First, I uploaded a basic module to the server using the CLI (see above) using the following command from the root of this repo: `console ./athens ./testmodule arschles.com testmodule v1.0.0` -* Then I created a new module with the following files in it: - * A single `go.mod` file with only the following line in it: `module "foo.bar/baz"` - * A `main.go` file with the following in it: - -```go -package main -func main() {} -``` - -Finally, from the root of the new module, I ran `vgo get arschles.com/testmodule@v1.0.0` and got the -following output: - -```console -$ vgo get arschles.com/testmodule@v1.0.0 -vgo: downloading arschles.com/testmodule v1.0.0 -vgo: import "arschles.com/testmodule": zip for arschles.com/testmodule@v1.0. has unexpected file testmodule/.DS_Store -``` - -As you can see, the CLI uploaded a file to athens that's not `.go`, `go.mod`, or anything else that vgo, so at least the CLI needs some work (and the server needs some sanity checks too). - -You can get around all of this by manually zipping up your code and uploading it with `curl` or similar, but like I said, that's super impractical. Yay alpha software! - # Code of Conduct This project follows the [Contributor Covenant](https://www.contributor-covenant.org/) (English version [here](https://www.contributor-covenant.org/version/1/4/code-of-conduct)) code of conduct. @@ -142,4 +55,3 @@ This project follows the [Contributor Covenant](https://www.contributor-covenant If you have concerns, notice a code of conduct violation, or otherwise would like to talk about something related to this code of conduct, please reach out to me, Aaron Schlesinger on the [Gophers Slack](https://gophers.slack.com/. My username is `arschles`. Note that in the future, we will be expanding the ways that you can contact us regarding the code of conduct. -