feat: Celery support and asynchronous draft submission API#4037
Merged
rjsparks merged 111 commits intoietf-tools:mainfrom Aug 22, 2022
Merged
feat: Celery support and asynchronous draft submission API#4037rjsparks merged 111 commits intoietf-tools:mainfrom
rjsparks merged 111 commits intoietf-tools:mainfrom
Conversation
Move some PlaintextDraft methods into the Draft base class and implement for the XMLDraft class. Use xml2rfc code from ietf.submit as a model for the parsing. This leaves some mismatch between the PlaintextDraft and the Draft class spec for the get_author_list() method to be resolved.
This adds an api_upload() that behaves analogously to the api_submit() endpoint. Celery tasks to handle asynchronous processing are added but are not yet functional enough to be useful.
This substantially speeds up submission rate threshold checks.
After checking that a submission is not in progress, remove any files in staging that have the same name/rev with any extension. This should guard against stale files confusing the submission process if the usual cleanup fails or is skipped for some reason.
Minimizing the amount of validation done when accepting a file. The data extraction will be moved to asynchronous processing.
* remove get_draftname() from Draft api; set filename during init
* further XMLDraft work
- remember xml_version after parsing
- extract filename/revision during init
- comment out long broken get_abstract() method
* adjust form clean() method to use changed API
First basically working pass!
Not bulletproof but should prevent
Will be used to coordinate package versions with the celery container in production.
…ifer/submit-async
Setting to an empty password is really not a good plan!
rjsparks
approved these changes
Aug 22, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This rather large pull request adds support for Celery asynchronous task handling and uses this to implement an asynchronous draft submission API.
The new API has two endpoints. Most of it is in
api_submission(), which is a refactored and slightly renamedapi_submit(). The other piece isapi_submission_status(), which provides a way to check whether a submission is still in the newvalidatingstate.For development, docker containers for rabbitmq and the celery worker are started up alongside the app and db containers. The celery container is currently pulled from the github package repo under the painless-security fork, to be changed when this is merged. That docker image is based on the existing test image.
Fixes #3791. Fixes #3286.