Hironsan
a6928fd0ad
Remove duplication from export task
2 years ago
Hironsan
e83bc924f1
Replace zip_files with shutil.make_archive
2 years ago
Hironsan
45ddacb504
Boost performance for data export
Remove is_text_project from ExportedExample because this queries db every time
2 years ago
Hironsan
cfcdcf247b
Rename parameter name
2 years ago
Hironsan
12573ec4bb
Update factories of data export
2 years ago
Hironsan
dac6fb05f4
Move filter_examples feature to ExportedExampleManager
2 years ago
Hironsan
04f9ddf7d7
Add remove files function
2 years ago
Hironsan
bf94da0ddd
Change attribute name from field_name to column
2 years ago
Hironsan
96b9984dde
Remove unused code
2 years ago
Hironsan
e332381133
Update typing to pass mypy
2 years ago
Hironsan
ce18476180
Add test cases for export task
2 years ago
Hironsan
d5338962af
Add test cases for exporting relation data
2 years ago
Hironsan
549e909070
Enable to accept multiple label types in export dataset
2 years ago
Hironsan
aa13d5c97f
Enable to export dataset
2 years ago
Hironsan
a361f17d84
Update factory for export repository
2 years ago
Hironsan
3200168d30
Apply isort
2 years ago
Hironsan
99f77b2bd2
Move project models to projects app
2 years ago
Hironsan
be5faaf304
Separate data export app
2 years ago
Hironsan
fd7bc43947
Move data import's code and data to data_import app
2 years ago
Hironsan
a719996ac3
Rename tasks to celery_tasks
2 years ago
Hironsan
a85336885c
Add test cases for builder
2 years ago
Hironsan
4b2bfabee3
Rename factory.py to factories.py
2 years ago
Hironsan
d4e216b188
Update ingest_task function
2 years ago
Hironsan
1e372f2243
Add BulkWriter
2 years ago
Hironsan
2c1d0ba5eb
Remove Labels
2 years ago
Hironsan
3d74106321
Add ExampleManager
2 years ago
Hironsan
a52fa97cd7
Update DataFactory
2 years ago
Hironsan
2a325904ec
Update Span model
3 years ago
Hironsan
b81c5e4aab
Enable to ingest lines without errors even if an exception occurs during parsing
3 years ago
Hironsan
fed94b70f3
Fix typo: injest -> ingest
3 years ago
Hironsan
5552da55ed
Revert primary key change
3 years ago
Hironsan
f381184f21
Change example id from auto field to uuid field
3 years ago
Hironsan
1977ff165f
Fix flake8 warnings
3 years ago
zanussbaum
df61e8a012
removing debugging statement
3 years ago
Roland Szabo
62bc03f6c1
Remove unneed import
3 years ago
Roland Szabo
0243c73462
Remove unneeded query
Bulk create returns the created objects in the same order as they have been added.
In Postgres, the query was wrong, because ordering was not guaranteed.
3 years ago
zanussbaum
9bf6c52347
Fixing Data Annotation Issues
When uploading datasets, the code uses a `bulk_create` to upload
Examples and Labels. It then filters the data from the database
based on when it was created. However, [Django doesn't enforce
the list order when calling filter](https://stackoverflow.com/questions/7163640/what-is-the-default-order-of-a-list-returned-from-a-django-filter-call )
unless ordering is specified. The previous behavior mismatched labels
and examples. When this was shown in the UI, the data would show labels
for incorrect examples (i.e. a label for message #2 would be shown on
message #1 ). This fix enforces that the data is returned in the order
it was inserted so that the data, label pair is as expected.
3 years ago
Hironsan
07d4455810
Update task
3 years ago
Hironsan
9acc447f23
Remove document and image models
3 years ago
Hironsan
a3a054c393
Rename app dir to backend for pip installation
3 years ago
Hironsan
1309c07af5
Add celery task for exporting dataset
3 years ago
Hironsan
9066a62ab9
Support bulk create
3 years ago
Hironsan
aecf44296e
Fix bug to pass annotation validation
3 years ago
Hironsan
586748531c
Update injest_data task to handle exception
3 years ago
Hironsan
500351835a
Apply isort
3 years ago
Hironsan
aa57786388
Update injest_data to avoid unique constraint error
3 years ago
Hironsan
60fef6e41d
Update injest_data
3 years ago
Hironsan
c629ecd489
Add inject_data task
3 years ago
Hironsan
e25a4905ee
Add example celery tasks
3 years ago