You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

272 lines
10 KiB

6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
5 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
6 years ago
  1. # doccano
  2. [![Build Status](https://travis-ci.org/chakki-works/doccano.svg?branch=master)](https://travis-ci.org/chakki-works/doccano)
  3. doccano is an open source text annotation tool for human. It provides annotation features for text classification, sequence labeling and sequence to sequence. So, you can create labeled data for sentiment analysis, named entity recognition, text summarization and so on. Just create project, upload data and start annotation. You can build dataset in hours.
  4. ## Demo
  5. You can enjoy [annotation demo](http://doccano.herokuapp.com).
  6. ### [Named entity recognition](https://doccano.herokuapp.com/demo/named-entity-recognition/)
  7. First demo is one of the sequence labeling tasks, named-entity recognition. You just select text spans and annotate it. Since doccano supports shortcut key, so you can quickly annotate text spans.
  8. ![Named Entity Recognition](./docs/named_entity_annotation.gif)
  9. ### [Sentiment analysis](https://doccano.herokuapp.com/demo/text-classification/)
  10. Second demo is one of the text classification tasks, topic classification. Since there may be more than one category, you can annotate multi-labels.
  11. ![Text Classification](./docs/text_classification.gif)
  12. ### [Machine translation](https://doccano.herokuapp.com/demo/translation/)
  13. Final demo is one of the sequence to sequence tasks, machine translation. Since there may be more than one responses in sequence to sequence tasks, you can create multi responses.
  14. ![Machine Translation](./docs/translation.gif)
  15. ## Deployment
  16. ### Azure
  17. Doccano can be deployed to Azure ([Web App for Containers](https://azure.microsoft.com/en-us/services/app-service/containers/) +
  18. [PostgreSQL database](https://azure.microsoft.com/en-us/services/postgresql/)) by clicking on the button below:
  19. [![Deploy to Azure](https://azuredeploy.net/deploybutton.svg)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fchakki-works%2Fdoccano%2Fmaster%2Fazuredeploy.json)
  20. ### Heroku
  21. Doccano can be deployed to [Heroku](https://www.heroku.com/) by clicking on the button below:
  22. [![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
  23. Of course, you can deploy doccano by using [heroku-cli](https://devcenter.heroku.com/articles/heroku-cli).
  24. ```bash
  25. heroku create
  26. heroku stack:set container
  27. git push heroku master
  28. ```
  29. ### AWS
  30. Doccano can be deployed to AWS ([Cloudformation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html)) by clicking on the button below:
  31. [![AWS CloudFormation Launch Stack SVG Button](https://cdn.rawgit.com/buildkite/cloudformation-launch-stack-button-svg/master/launch-stack.svg)](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://s3-external-1.amazonaws.com/cf-templates-10vry9l3mp71r-us-east-1/20190732wl-new.templatexloywxxyimi&stackName=doccano)
  32. > Notice: (1) EC2 KeyPair cannot be created automatically, so make sure you have an existing EC2 KeyPair in one region. Or [create one yourself](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#having-ec2-create-your-key-pair). (2) If you want to access doccano via HTTPS in AWS, here is an [instruction](https://github.com/chakki-works/doccano/wiki/HTTPS-setting-for-doccano-in-AWS).
  33. ## Features
  34. * Collaborative annotation
  35. * Multi-Language support
  36. * Emoji :smile: support
  37. * (future) Auto labeling
  38. ## Requirements
  39. * Python 3.6+
  40. * Django 2.1.7+
  41. * Node.js 8.0+
  42. * Google Chrome(highly recommended)
  43. ## Installation
  44. First of all, you have to clone the repository:
  45. ```bash
  46. git clone https://github.com/chakki-works/doccano.git
  47. cd doccano
  48. ```
  49. To install doccano, there are two options:
  50. **Option1: Pull the Docker image**
  51. ```bash
  52. docker pull chakkiworks/doccano
  53. ```
  54. **Option2: Setup Python environment**
  55. First we need to install the dependencies. Run the following commands:
  56. ```bash
  57. pip install -r requirements.txt
  58. cd app
  59. ```
  60. Next we need to start the webpack server so that the frontend gets compiled continuously.
  61. Run the following commands in a new shell:
  62. ```bash
  63. cd server
  64. npm install
  65. npm run build
  66. # npm start # for developers
  67. ```
  68. Next we need to make migration. Run the following command:
  69. ```bash
  70. python manage.py migrate
  71. ```
  72. Next we need to create a user who can login to the admin site. Run the following command:
  73. ```bash
  74. python manage.py createsuperuser
  75. ```
  76. Enter your desired username and press enter.
  77. ```bash
  78. Username: admin
  79. ```
  80. You will then be prompted for your desired email address:
  81. ```bash
  82. Email address: admin@example.com
  83. ```
  84. The final step is to enter your password. You will be asked to enter your password twice, the second time as a confirmation of the first.
  85. ```bash
  86. Password: **********
  87. Password (again): *********
  88. Superuser created successfully.
  89. ```
  90. ## Usage
  91. ### Start the development server
  92. Let’s start the development server and explore it.
  93. Depending on your installation method, there are two options:
  94. **Option1: Running the Docker image as a Container**
  95. First, run a Docker container:
  96. ```bash
  97. docker run -d --name doccano -p 8000:80 chakkiworks/doccano
  98. ```
  99. Then, execute `create-admin.sh` script for creating a superuser.
  100. ```bash
  101. docker exec doccano tools/create-admin.sh "admin" "admin@example.com" "password"
  102. ```
  103. **Option2: Running Django development server**
  104. ```bash
  105. python manage.py runserver
  106. ```
  107. Now, open a Web browser and go to <http://127.0.0.1:8000/login/>. You should see the login screen:
  108. <img src="./docs/login_form.png" alt="Login Form" width=400>
  109. ### Create a project
  110. Now, try logging in with the superuser account you created in the previous step. You should see the doccano project list page:
  111. <img src="./docs/projects.png" alt="projects" width=600>
  112. There is no project created yet. To create your project, make sure you’re in the project list page and select `Create Project` button. You should see the following screen:
  113. <img src="./docs/create_project.png" alt="Project Creation" width=400>
  114. In this step, you can select three project types: text classificatioin, sequence labeling and sequence to sequence. You should select a type with your purpose.
  115. ### Import Data
  116. After creating a project, you will see the "Import Data" page, or click `Import Data` button in the navigation bar. You should see the following screen:
  117. <img src="./docs/upload.png" alt="Upload project" width=600>
  118. You can upload two types of files:
  119. - `CSV file`: file must contain a header with a `text` column or be one-column csv file.
  120. - `JSON file`: each line contains a JSON object with a `text` key. JSON format supports line breaks rendering.
  121. > Notice: Doccano won't render line breaks in annotation page for sequence labeling task due to the indent problem, but the exported JSON file still contains line breaks.
  122. `example.txt` (or `example.csv`)
  123. ```python
  124. EU rejects German call to boycott British lamb.
  125. President Obama is speaking at the White House.
  126. He lives in Newark, Ohio.
  127. ...
  128. ```
  129. `example.json`
  130. ```JSON
  131. {"text": "EU rejects German call to boycott British lamb."}
  132. {"text": "President Obama is speaking at the White House."}
  133. {"text": "He lives in Newark, Ohio."}
  134. ...
  135. ```
  136. Any other columns (for csv) or keys (for json) are preserved and will be exported in the `metadata` column or key as is.
  137. Once you select a TXT/JSON file on your computer, click `Upload dataset` button. After uploading the dataset file, we will see the `Dataset` page (or click `Dataset` button list in the left bar). This page displays all the documents we uploaded in one project.
  138. ### Define labels
  139. Click `Labels` button in left bar to define your own labels. You should see the label editor page. In label editor page, you can create labels by specifying label text, shortcut key, background color and text color.
  140. <img src="./docs/label_editor.png" alt="Edit label" width=600>
  141. ### Annotation
  142. Now, you are ready to annotate the texts. Just click the `Annotate Data` button in the navigation bar, you can start to annotate the documents you uploaded.
  143. <img src="./docs/annotation.png" alt="Edit label" width=600>
  144. ### Export Data
  145. After the annotation step, you can download the annotated data. Click the `Edit data` button in navigation bar, and then click `Export Data`. You should see below screen:
  146. <img src="./docs/export_data.png" alt="Edit label" width=600>
  147. You can export data as CSV file or JSON file by clicking the button. As for the export file format, you can check it here: [Export File Formats](https://github.com/chakki-works/doccano/wiki/Export-File-Formats).
  148. Each exported document will have metadata column or key, which will contain
  149. additional columns or keys from the imported document. The primary use-case for metadata is to allow you to match exported data with other system
  150. by adding `external_id` to the imported file. For example:
  151. Input file may look like this:
  152. `import.json`
  153. ```JSON
  154. {"text": "EU rejects German call to boycott British lamb.", "external_id": 1}
  155. ```
  156. and the exported file will look like this:
  157. `output.json`
  158. ```JSON
  159. {"doc_id": 2023, "text": "EU rejects German call to boycott British lamb.", "labels": ["news"], "username": "root", "metadata": {"external_id": 1}}
  160. ```
  161. ### Tutorial
  162. We prepared a NER annotation tutorial, which can help you have a better understanding of doccano. Please first read the README page, and then take the tutorial. [A Tutorial For Sequence Labeling Project](https://github.com/chakki-works/doccano/wiki/A-Tutorial-For-Sequence-Labeling-Project).
  163. I hope you are having a great day!
  164. ## Contribution
  165. As with any software, doccano is under continuous development. If you have requests for features, please file an issue describing your request. Also, if you want to see work towards a specific feature, feel free to contribute by working towards it. The standard procedure is to fork the repository, add a feature, fix a bug, then file a pull request that your changes are to be merged into the main repository and included in the next release.
  166. Here are some tips might be helpful. [How to Contribute to Doccano Project](https://github.com/chakki-works/doccano/wiki/How-to-Contribute-to-Doccano-Project)
  167. ## Contact
  168. For help and feedback, please feel free to contact [the author](https://github.com/Hironsan).
  169. **If you are favorite to doccano, please follow my [GitHub](https://github.com/Hironsan) and [Twitter](https://twitter.com/Hironsan13) account.**