Browse Source

Merge branch 'master' into pull/722

pull/837/head
Hironsan 4 years ago
parent
commit
8ade54c777
16 changed files with 243 additions and 87 deletions
  1. 73
      CONTRIBUTING.md
  2. 51
      README.md
  3. 21
      app/Dockerfile
  4. 10
      app/api/tests/test_api.py
  5. 3
      app/api/urls.py
  6. 11
      app/api/views.py
  7. 2
      app/requirements.txt
  8. 2
      app/server/static/assets/css/annotation.css
  9. 3
      app/server/static/components/annotation.pug
  10. 6
      app/server/static/package-lock.json
  11. 26
      docs/faq.md
  12. 84
      docs/project_structure.md
  13. 15
      frontend/Dockerfile
  14. 17
      frontend/components/organisms/layout/TheHeader.vue
  15. 4
      frontend/nuxt.config.js
  16. 2
      tools/install-mssql.sh

73
CONTRIBUTING.md

@ -0,0 +1,73 @@
# Contributing
When contributing to this repository, please first discuss the change you wish to make via issue with the owners of this repository before making a change.
Please note we have a code of conduct, please follow it in all your interactions with the project.
## How to contribute to doccano
### Reporting Bugs
#### Before submitting a bug report
* Check the [FAQs](https://github.com/doccano/doccano/blob/master/docs/faq.md) for a list of common questions and problems.
* Ensure the bug was not already reported by searching on GitHub under [Issues](https://github.com/doccano/doccano/issues).
* [Open a new issue](https://github.com/doccano/doccano/issues/new/choose) if you're unable to find an open one addressing the problem.
* Use the relevant bug report templates to create the issue.
#### How do I submit a good bug report?
Explain the problem and include additional details to help maintainers reproduce the problem:
* Use a clear and descriptive title for the issue to identify the problem.
* Describe the exact steps which reproduce the problem in as many details as possible.
* Provide specific examples to demonstrate the steps.
* Describe the behavior you observed after following the steps and point out what exactly is the problem with that behavior.
* Explain which behavior you expected to see instead and why.
* Include screenshots and animated GIFs which show you following the described steps and clearly demonstrate the problem.
* If the problem is related to performance or memory, include a CPU profile capture with your report.
* If the problem is related to network, include a network activity in Chrome/Firefox/Safari DevTools.
* If the problem wasn't triggered by a specific action, describe what you were doing before the problem happened and share more information using the guidelines below.
### Suggesting Enhancements
#### Before submitting an enhancement suggestion
* Ensure the suggestion was not already reported by searching on GitHub under [Issues](https://github.com/doccano/doccano/issues).
* [Open a new issue](https://github.com/doccano/doccano/issues/new/choose) if you're unable to find an open one addressing the suggestion.
* Use the relevant issue templates to create one.
#### How do I submit a good enhancement suggestion?
Explain the suggestion and include additional details to help developers understand it:
* Use a clear and descriptive title for the issue to identify the suggestion.
* Provide a step-by-step description of the suggested enhancement in as many details as possible.
* Provide specific examples to demonstrate the steps.
* Describe the current behavior and explain which behavior you expected to see instead and why.
* Include screenshots and animated GIFs which help you demonstrate the steps or point out the part of doccano which the suggestion is related to.
* Explain why this enhancement would be useful to most doccano users.
* List some other annotation tools or applications where this enhancement exists.
* Specify which version of doccano you're using.
* Specify the name and version of the OS you're using.
### Pull Request Process
Please follow these steps to have your contribution considered by the maintainers:
1. Open a related issue before making a pull request as much as possible.
2. Follow all instructions in [the template](PULL_REQUEST_TEMPLATE.md)
3. Follow the [styleguides](#styleguides)
4. After you submit your pull request, verify that all [status checks](https://help.github.com/articles/about-status-checks/) are passing <details><summary>What if the status checks are failing?</summary>If a status check is failing, and you believe that the failure is unrelated to your change, please leave a comment on the pull request explaining why you believe the failure is unrelated. A maintainer will re-run the status check for you. If we conclude that the failure was a false positive, then we will open an issue to track that problem with our status check suite.</details>
5. You may merge the Pull Request in once you have the sign-off of the project members([@Hironsan](https://github.com/Hironsan) or [@icoxfog417](https://github.com/icoxfog417)).
While the prerequisites above must be satisfied prior to having your pull request reviewed, the reviewer(s) may ask you to complete additional design work, tests, or other changes before your pull request can be ultimately accepted.
## Styleguides
### Git Commit Messages
* Use the present tense ("Add feature" not "Added feature")
* Use the imperative mood ("Move cursor to..." not "Moves cursor to...")
* Limit the first line to 72 characters or less
* Reference issues and pull requests liberally after the first line

51
README.md

@ -28,17 +28,29 @@ You can try the [annotation demo](http://doccano.herokuapp.com).
Two options to run doccano:
- Production
- Development
- production,
- development.
In any case, you need Docker and Docker Compose. Anyway, you need to clone the repository:
To use doccano, please follow:
### Install dependencies
You need to install dependencies:
- [Git](https://git-scm.com),
- [Docker](https://www.docker.com),
- [Docker Compose](https://docs.docker.com/compose).
### Get the code
You need to clone the repository:
```bash
$ git clone https://github.com/doccano/doccano.git
$ cd doccano
```
_Note for Windows developers: Be sure to configure git to correctly handle line endings or you may encounter `status code 127` errors while running the services in future steps. Running with the git config options below will ensure your git directory correctly handles line endings._
_Note for Windows developers:_ Be sure to configure git to correctly handle line endings or you may encounter `status code 127` errors while running the services in future steps. Running with the git config options below will ensure your git directory correctly handles line endings.
```bash
git clone https://github.com/doccano/doccano.git --config core.autocrlf=input
@ -46,20 +58,20 @@ git clone https://github.com/doccano/doccano.git --config core.autocrlf=input
### Production
```bash
$ docker-compose -f docker-compose.prod.yml up
```
Go to <http://0.0.0.0/>.
Note the superuser account credentials located in the `docker-compose.prod.yml` file:
Set the superuser account credentials in the `docker-compose.prod.yml` file:
```yml
ADMIN_USERNAME: "admin"
ADMIN_PASSWORD: "password"
```
> Note: If you want to add annotators, see [Frequently Asked Questions](./docs/faq.md)
Run doccano:
```bash
$ docker-compose -f docker-compose.prod.yml up
```
Go to <http://0.0.0.0/>.
<!--
@ -91,12 +103,25 @@ Go to <http://127.0.0.1:8000/>.
### Development
Set the superuser account credentials in the `docker-compose.dev.yml` file:
```yml
ADMIN_USERNAME: "admin"
ADMIN_PASSWORD: "password"
```
Run Doccano:
```bash
$ docker-compose -f docker-compose.dev.yml up
```
Go to <http://127.0.0.1:3000/>.
### Add annotators (optionally)
If you want to add annotators/annotation approvers, see [Frequently Asked Questions](./docs/faq.md)
## One-click Deployment
| Service | Button |
@ -104,7 +129,7 @@ Go to <http://127.0.0.1:3000/>.
| AWS[^1] | [![AWS CloudFormation Launch Stack SVG Button](https://cdn.rawgit.com/buildkite/cloudformation-launch-stack-button-svg/master/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home?#/stacks/create/review?stackName=doccano&templateURL=https://s3-external-1.amazonaws.com/cf-templates-10vry9l3mp71r-us-east-1/2019290i9t-AppSGl1poo4j8qpq) |
| Azure | [![Deploy to Azure](https://azuredeploy.net/deploybutton.svg)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fdoccano%2Fdoccano%2Fmaster%2Fazuredeploy.json) |
| GCP[^2] | [![GCP Cloud Run PNG Button](https://storage.googleapis.com/gweb-cloudblog-publish/images/run_on_google_cloud.max-300x300.png)](https://console.cloud.google.com/cloudshell/editor?shellonly=true&cloudshell_image=gcr.io/cloudrun/doccano&cloudshell_git_repo=https://github.com/doccano/doccano.git) |
| Heroku | [![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy) |
| Heroku | [![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://dashboard.heroku.com/new?template=https%3A%2F%2Fgithub.com%2Fdoccano%2Fdoccano) |
> [^1]: (1) EC2 KeyPair cannot be created automatically, so make sure you have an existing EC2 KeyPair in one region. Or [create one yourself](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair). (2) If you want to access doccano via HTTPS in AWS, here is an [instruction](https://github.com/doccano/doccano/wiki/HTTPS-setting-for-doccano-in-AWS).
> [^2]: Although this is a very cheap option, it is only suitable for very small teams (up to 80 concurrent requests). Read more on [Cloud Run docs](https://cloud.google.com/run/docs/concepts).

21
app/Dockerfile

@ -1,20 +1,17 @@
FROM python:3.6
FROM alpine:3.9.6
CMD ["python3"]
# set work directory
WORKDIR /app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apt-get update \
&& apt-get install --no-install-recommends -y python3-dev libpq-dev unixodbc-dev
COPY . /app/
# install dependencies
RUN pip install --upgrade pip setuptools
COPY ./requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
RUN apk add -U bash python3 python3-dev libpq postgresql-dev unixodbc-dev musl-dev g++ libffi-dev \
&& pip3 install --upgrade pip setuptools \
&& pip3 install --no-cache-dir -r requirements.txt \
&& ln -s /usr/bin/python3 /usr/bin/python \
&& apk del python3-dev postgresql-dev unixodbc-dev musl-dev g++ libffi-dev
# copy project
COPY . /app/

10
app/api/tests/test_api.py

@ -29,6 +29,16 @@ def remove_all_role_mappings():
RoleMapping.objects.all().delete()
class TestHealthEndpoint(APITestCase):
@classmethod
def setUpTestData(cls):
cls.url = reverse(viewname='health')
def test_returns_green_status_on_health_endpoint(self):
response = self.client.get(self.url, format='json')
self.assertEqual(response.data['status'], 'green')
class TestUtilsMixin:
def _patch_project(self, project, attribute, value):
old_value = getattr(project, attribute, None)

3
app/api/urls.py

@ -2,7 +2,7 @@ from django.urls import path
from rest_framework.authtoken.views import obtain_auth_token
from rest_framework.urlpatterns import format_suffix_patterns
from .views import Me, Features, Users
from .views import Me, Features, Users, Health
from .views import ProjectList, ProjectDetail
from .views import LabelList, LabelDetail, ApproveLabelsAPI, LabelUploadAPI
from .views import DocumentList, DocumentDetail
@ -12,6 +12,7 @@ from .views import StatisticsAPI
from .views import RoleMappingList, RoleMappingDetail, Roles
urlpatterns = [
path('health', Health.as_view(), name='health'),
path('auth-token', obtain_auth_token),
path('me', Me.as_view(), name='me'),
path('features', Features.as_view(), name='features'),

11
app/api/views.py

@ -11,7 +11,7 @@ from libcloud.base import DriverType, get_driver
from libcloud.storage.types import ContainerDoesNotExistError, ObjectDoesNotExistError
from rest_framework import generics, filters, status
from rest_framework.exceptions import ParseError, ValidationError
from rest_framework.permissions import IsAuthenticated
from rest_framework.permissions import IsAuthenticated, IsAuthenticatedOrReadOnly
from rest_framework.response import Response
from rest_framework.views import APIView
from rest_framework.parsers import MultiPartParser
@ -30,6 +30,13 @@ IsInProjectReadOnlyOrAdmin = (IsAnnotatorAndReadOnly | IsAnnotationApproverAndRe
IsInProjectOrAdmin = (IsAnnotator | IsAnnotationApprover | IsProjectAdmin)
class Health(APIView):
permission_classes = (IsAuthenticatedOrReadOnly,)
def get(self, request, *args, **kwargs):
return Response({'status': 'green'})
class Me(APIView):
permission_classes = (IsAuthenticated,)
@ -212,7 +219,7 @@ class AnnotationList(generics.ListCreateAPIView):
class AnnotationDetail(generics.RetrieveUpdateDestroyAPIView):
lookup_url_kwarg = 'annotation_id'
permission_classes = [IsAuthenticated & (((IsAnnotator | IsAnnotationApprover) & IsOwnAnnotation) | IsProjectAdmin)]
permission_classes = [IsAuthenticated & (((IsAnnotator & IsOwnAnnotation) | IsAnnotationApprover) | IsProjectAdmin)]
swagger_schema = None
def get_serializer_class(self):

2
app/requirements.txt

@ -4,7 +4,7 @@ colour==0.1.5
chardet==3.0.4
coverage==4.5.3
dj-database-url==0.5.0
Django==2.2.10
Django==2.2.13
django-cloud-browser==0.5.0
django-cors-headers==3.1.1
django-filter==2.0.0

2
app/server/static/assets/css/annotation.css

@ -162,7 +162,7 @@ body {
}
.content .text-sequence {
white-space: normal; /* Not render newlines for sequence labelling */
white-space: pre-wrap; /* Not render newlines for sequence labelling */
height: auto;
}

3
app/server/static/components/annotation.pug

@ -73,7 +73,8 @@ div.columns(v-cloak="")
href="#"
)
span.icon
i.fa.fa-check(v-show="annotations[index] && annotations[index].length")
i.fa.fa-thumbs-up(v-show="annotations[index] && docs[index].annotation_approver")
i.fa.fa-check(v-show="annotations[index] && annotations[index].length && !docs[index].annotation_approver")
span.name {{ doc.text.slice(0, 60) }}...
div.column.is-7.is-offset-1.message.hero.is-fullheight#message-pane

6
app/server/static/package-lock.json

@ -9194,9 +9194,9 @@
}
},
"websocket-extensions": {
"version": "0.1.3",
"resolved": "https://registry.npmjs.org/websocket-extensions/-/websocket-extensions-0.1.3.tgz",
"integrity": "sha512-nqHUnMXmBzT0w570r2JpJxfiSD1IzoI+HGVdd3aZ0yNi3ngvQ4jv1dtHt5VGxfI2yj5yqImPhOK4vmIh2xMbGg==",
"version": "0.1.4",
"resolved": "https://registry.npmjs.org/websocket-extensions/-/websocket-extensions-0.1.4.tgz",
"integrity": "sha512-OqedPIGOfsDlo31UNwYbCFMSaO9m9G/0faIHj5/dZFDMFqPTcx6UwqyOy3COEaEOg/9VsGIpdqn62W5KhoKSpg==",
"dev": true
},
"which": {

26
docs/faq.md

@ -16,8 +16,30 @@ Please check the following list.
**You don't need your real & all data to validate file format. The picked data & masked data is suitable if your data is large or secret.**
## I want to add annotators
## I want to add annotators annotators/annotation approvers
1. Add a user from [Django Admin site](https://djangobook.com/django-admin-site/).
![Add a user](./images/faq/add_user.png)
2. Add the user to the project in the member page(`/projects/{project_id}/members`).
2. **Logout from Django Admin site.** [You'll face login error without logout of Django Admin site](https://github.com/doccano/doccano/issues/723).
3. Add the user to the project in the member page (`/projects/{project_id}/members`).
## I want to update to the latest doccano image
1. Execute `git pull` to reflect the latest doccano.
2. Delete the volume that `doccano_node_modules`, `doccano_static_volume`, `doccano_venv` and `doccano_www`.
**Do not delete `doccano_postgres_data` because it stores your projects data.**
3. Rebuild the doccano image.
The following commands are the procedure for 2~3.
```
❯ docker volume ls
DRIVER VOLUME NAME
local doccano_node_modules
local doccano_postgres_data
local doccano_static_volume
local doccano_venv
local doccano_www
❯ docker volume rm doccano_node_modules doccano_static_volume doccano_venv doccano_www
❯ docker-compose -f docker-compose.prod.yml build --no-cache
```

84
docs/project_structure.md

@ -1,70 +1,82 @@
# Project Structure
In doccano v1.x, the application consists of a frontend and backend API. They are stored in the `frontend` and `app` directories.
The important files/directories are as follows:
```bash
.
├── app
```
/
├── app/
├── frontend/
├── nginx/
├── tools/
├── docker-compose.dev.yml
├── docker-compose.prod.yml
├── frontend
├── nginx
└── tools
└── docker-compose.prod.yml
```
The other important files/directories are as follows:
Consider them:
**[app/](https://github.com/doccano/doccano/tree/master/app)**
The `app/api` directory contains backend code. See [below](#Backend).
- [docker-compose.dev.yml](https://github.com/doccano/doccano/blob/master/docker-compose.dev.yml)
- [docker-compose.prod.yml](https://github.com/doccano/doccano/blob/master/docker-compose.prod.yml)
- [nginx](https://github.com/doccano/doccano/tree/master/nginx)
- [tools](https://github.com/doccano/doccano/tree/master/tools)
**[frontend/](https://github.com/doccano/doccano/tree/master/frontend)**
The `app/api` directory contains frontend code. See [below](#Frontend).
**[docker-compose.dev.yml](https://github.com/doccano/doccano/blob/master/docker-compose.dev.yml)**
The `docker-compose.dev.yml` file contains configuration to run a development environment. Once we run the command `docker-compose -f docker-compose.dev.yml up`, compose runs backend API and frontend development containers.
The `docker-compose.dev.yml` file contains [Docker Compose](https://docs.docker.com/compose) configuration to run a development environment.
Once we run the command `docker-compose -f docker-compose.dev.yml up`, compose runs backend API and frontend development containers.
**[docker-compose.prod.yml](https://github.com/doccano/doccano/blob/master/docker-compose.prod.yml)**
The `docker-compose.prod.yml` file contains configuration to run a production environment. We adopted the three tier architecture. Once we run the command `docker-compose -f docker-compose.prod.yml up`, compose builds frontend and runs DBMS, backend API and web server containers.
The `docker-compose.prod.yml` file contains [Docker Compose](https://docs.docker.com/compose) configuration to run a production environment.
We adopted the three tier architecture. Once we run the command `docker-compose -f docker-compose.prod.yml up`, compose builds frontend and runs DBMS, backend API and web server containers.
**[nginx](https://github.com/doccano/doccano/tree/master/nginx)**
The `nginx` directory contains a nginx configuration file and Docker container. They are used only in `docker-compose.prod.yml`.
The `nginx` directory contains a NGINX configuration file and Docker container. They are used only in `docker-compose.prod.yml`.
**[tools](https://github.com/doccano/doccano/tree/master/tools)**
The `tools` directory contains some shell scripts. They are used for CI, CD and so on.
## Frontend
The directory structure of the frontend follows Nuxt.js one. See the Nuxt.js documentation for details:
Also, there are directories and files contain doccano v0.x codes.
In the future, they will be integrated into the currect code or removed:
- [Nuxt.js/Directory Structure](https://nuxtjs.org/guide/directory-structure/)
```
/
├── app/
├── └── server/
└── docker-compose.yml
```
## Backend API
## Backend
The directory structure of the backend api follows Django one. The important directories are as follows:
The directory structure of the backend follows [Django](https://www.djangoproject.com) one.
The important directories are as follows:
```bash
.
├── api
├── app
├── authentification
└── server
```
/
├── app/
├── ├── api/
├── ├── app/
└── └── authentification/
```
**[app/api](https://github.com/doccano/doccano/tree/master/app/api)**
**[app/api/](https://github.com/doccano/doccano/tree/master/app/api)**
The `api` directory contains backend API application. We use Django Rest Framework to implement the API. If you want to add new API, change the contents of this directory.
The `app/api` directory contains backend API application. We use [Django Rest Framework](https://www.django-rest-framework.org) to implement the API.
If you want to add new API, change the contents of this directory.
**[app/app](https://github.com/doccano/doccano/tree/master/app/app)**
**[app/app/](https://github.com/doccano/doccano/tree/master/app/app)**
The `app` directory contains Django project settings. See [Writing your first Django app, part 1](https://docs.djangoproject.com/en/3.0/intro/tutorial01/#creating-a-project).
The `app/app` directory contains Django project settings. See [Writing your first Django app, part 1](https://docs.djangoproject.com/en/3.0/intro/tutorial01/#creating-a-project).
**[app/authentification](https://github.com/doccano/doccano/tree/master/app/authentification)**
**[app/authentification/](https://github.com/doccano/doccano/tree/master/app/authentification)**
The `authentification` directory contains authentification application. It is mainly used for user signup.
The `app/authentification` directory contains authentification application. It is mainly used for user signup.
**[app/server](https://github.com/doccano/doccano/tree/master/app/server)**
## Frontend
The `server` directory contains doccano v0.x codes. In the future, this directory will be integrated into the `api` directory.
The `frontent` directory structure of the frontend follows [Nuxt.js](https://ru.nuxtjs.org) one.
See the [Nuxt.js documentation](https://nuxtjs.org/guide/directory-structure/) for details.

15
frontend/Dockerfile

@ -1,14 +1,11 @@
FROM node:13.2.0
FROM node:13.2-alpine
# set work directory
WORKDIR /app
# copy project
COPY . /app/
# install dependencies
RUN npm install -g npm@latest
RUN npm install
# build project
RUN npm run build
RUN apk add -U git python3 make g++ \
&& npm install -g npm@latest \
&& npm install \
&& npm run build \
&& apk del git make g++

17
frontend/components/organisms/layout/TheHeader.vue

@ -17,6 +17,16 @@
>
doccano
</v-toolbar-title>
<v-btn
v-if="isAuthenticated && isIndividualProject"
text
style="text-transform:none"
>
<v-icon small class="mr-1">
mdi-hexagon-multiple
</v-icon>
<span> {{ currentProject.name }}</span>
</v-btn>
<div class="flex-grow-1" />
<the-color-mode-switcher />
<v-btn
@ -104,7 +114,12 @@ export default {
},
computed: {
...mapGetters('auth', ['isAuthenticated', 'getUsername'])
...mapGetters('auth', ['isAuthenticated', 'getUsername']),
...mapGetters('projects', ['currentProject']),
isIndividualProject() {
return this.$route.name.startsWith('projects-id')
}
},
methods: {

4
frontend/nuxt.config.js

@ -26,10 +26,6 @@ export default {
]
},
serverMiddleware: [
'~/api/index.js'
],
server: {
host: '0.0.0.0' // default: localhost
},

2
tools/install-mssql.sh

@ -12,7 +12,7 @@ set -eo pipefail
# install build dependencies
apt-get update
apt-get install --no-install-recommends -y \
apt-get install --no-install-recommends -y --allow-downgrades \
curl=7.52.1-5+deb9u10 \
gnupg=2.1.18-8~deb9u4 \
apt-transport-https=1.4.9

Loading…
Cancel
Save