FastAPI conversion #143

Kilo59 · 2020-03-22T20:57:23Z

I thought I should open up my in-progress PR so that the maintainers can see what a transition to
#130 FastAPI would look like.

I didn't want to heavily redesign the application without input or discussion from the original creators.
But for example, some of the existing code could be refactored to take advantage of FastAPI's dependency injection system.

I've added 2 pipenv scripts, that can be used to run a local development server.
To run it just type.
pipenv run dev_app or pipenv run app

TODO:

Kilo59 · 2020-03-22T21:01:40Z

Swagger/OpenAPI docs

ReDocs

ExpDev07 · 2020-03-22T21:07:14Z

Looks good. I don't like that everything is in main.py. Hopefully we can for example put models under /models, etc.

Kilo59 · 2020-03-22T21:08:48Z

Looks good. I don't like that everything is in main.py. Hopefully we can for example put models under /models, etc.

Yes absolutely, I planned on refactoring and moving those into a models.py, just thought I'd talk to you first about exactly how you'd like that done.

ExpDev07 · 2020-03-22T21:11:08Z

just thought I'd talk to you first about exactly how you'd like that done.

That's considerate of you, thank you. I really like dividing it up as much as possible, so you have the /models directory, then /models/location.py, /models/timeline.py, etc, each containing max 1-2 classes each. You can really just look at how much I've divided it with the current version running :).

It's really just some simple refactoring that can be done at the end though.

Kilo59 · 2020-03-22T22:07:48Z

app/main.py

+
+
+# mount the existing Flask app to /v2
+APP.mount("/v2", WSGIMiddleware(create_app()))


@ExpDev07
I'm not sure how this behaves with the /v1 endpoints. But (at least locally), the /v2 endpoints appear to be functional, including the redirect to the github page when hitting /v2.

So FastAPI works with flask or? The v1 endpoints can be found at:

https://coronavirus-tracker-api.herokuapp.com/all

https://coronavirus-tracker-api.herokuapp.com/confirmed

https://coronavirus-tracker-api.herokuapp.com/deaths

https://coronavirus-tracker-api.herokuapp.com/recovered

I have checked the logs, and they are still in use by a lot of apps, so they need to work.

@ExpDev07
It's mashing v1 and v2 together 😦
So because /v1 and /v2 are defined into the same app, this appears to put both /v2 and /v1 behind the /v2 prefix.

In order to separate them, I believe I need to create 2 separate flask apps and mount them separately.
This should be relatively simple.

For example /confirmed is part of /v1, but I can access it locally at /v2/confirmed when it should be /v1/confirmed.

I probably need to do something like...

APP.mount("/v1", WSGIMiddleware(create_app("v1"))) APP.mount("/v2", WSGIMiddleware(create_app("v2")))

It should be /confirmed without any prefix :). Hmm, two different flask apps? Not really the biggest fan of that. FastAPI doesn't support blueprinting?

I mean if the old v1 endpoints work then, then I guess it's fine.

I'd be happy to get on a screen share, discord etc. to talk about this stuff (and deployment configuration, etc.).

My Discord # is: Marius Big Ounce#2997.

Just sent you an invite.
Kilo59#8819

I think I accidentally declined it (lol), sent you one now.

Kilo59 · 2020-03-22T22:23:50Z

@ExpDev07
Here's a response comparison between the currently deployed application and this one.
The only difference is the format of last_updated.

Working on fixing that now.
https://fastapi.tiangolo.com/tutorial/extra-data-types/#other-data-types
https://pydantic-docs.helpmanual.io/usage/types/#datetime-types

GET /locations?country_code=IT

v2

{
  "latest": {
    "confirmed": 53578,
    "deaths": 4825,
    "recovered": 6072
  },
  "locations": Array[1][
    {
      "coordinates": {
        "latitude": "43",
        "longitude": "12"
      },
      "country": "Italy",
      "country_code": "IT",
      "id": 16,
      "last_updated": "2020-03-22T22:02:46.078243Z",
      "latest": {
        "confirmed": 53578,
        "deaths": 4825,
        "recovered": 6072
      },
      "province": ""
    }
  ]
}

v2-1

{
  "latest": {
    "confirmed": 53578,
    "deaths": 4825,
    "recovered": 6072
  },
  "locations": Array[1][
    {
      "coordinates": {
        "latitude": "43",
        "longitude": "12"
      },
      "country": "Italy",
      "country_code": "IT",
      "id": 16,
      "last_updated": "2020-03-22T22:12:42.421409+00:00",
      "latest": {
        "confirmed": 53578,
        "deaths": 4825,
        "recovered": 6072
      },
      "province": ""
    }
  ]
}

ExpDev07 · 2020-03-22T22:26:15Z

Awesome, how about the v1 endpoints?

https://www.starlette.io/requests/#other-state

ExpDev07

Looks good!

ExpDev07 · 2020-03-24T23:11:08Z

Pipfile

 flask = "*"
 python-dotenv = "*"
 requests = "*"
 gunicorn = "*"


Can be removed now that we're using uvicorn.

Actually we are running uvicorn workers via gunicorn.
https://www.uvicorn.org/deployment/

This also seems to imply that it's okay to run directly it with uvicorn.
It might be worth doing a performance test on both deployment methods.
https://fastapi.tiangolo.com/deployment/#alternatively-deploy-fastapi-without-docker

…ided + doc changes + some cleanup

ExpDev07

There's an issue where when using alternative data-sources for locations, their fields won't show up. For example, /v2/locations?source=csbs won't display the county field as defined in app/location/csbs.py - CSBSLocation.

Kilo59 · 2020-03-25T00:44:52Z

There's an issue where when using alternative data-sources for locations, their fields won't show up. For example, /v2/locations?source=csbs won't display the county field as defined in app/location/csbs.py - CSBSLocation.

Fixed it by adding the county field to the mode.
Also added some very minimal tests of the v2/location endpoint itself.
I certainly can add more.

… renamed models + some cleanup

ExpDev07 · 2020-03-25T03:07:37Z

To start the app, now use

pipenv run dev for development (with live-reload), and
pipenv run start for starting without livereload.

ExpDev07

Awesome work dude! Looks like it's time for merge.

ExpDev07 · 2020-03-25T04:05:19Z

@all-contributors please add @Kilo59 for code, infrastructure, testing, and documentation.

allcontributors · 2020-03-25T04:05:30Z

@ExpDev07

I've put up a pull request to add @Kilo59! 🎉

ExpDev07 linked an issue Mar 22, 2020 that may be closed by this pull request

Consider switching (from Flask) to ASGI framework #130

Closed

Kilo59 force-pushed the FastAPI-conversion branch from 988a375 to 9ae7ff5 Compare March 22, 2020 22:03

Kilo59 commented Mar 22, 2020

View reviewed changes

Kilo59 added 21 commits March 22, 2020 20:44

add FastAPI uvicorn

5f1a422

define app

8166a4d

create provisional models

d79965e

define latest, location endpoints

cd64e47

update models

21f4472

change model name to match pre-existing model

b362750

defaults

033d924

update versions and response models

b1a4742

divide sections

bdae7f0

create middleware to attach the "data_source"

825de31

https://www.starlette.io/requests/#other-state

validation exception handler

e31aad6

get location by id

20c49c2

WIP get_all_locations

1cfdc2c

add Totals

e8e50e8

Mount v2 of the application

94a2f4e

add FIXME for timelines

091b461

end of file newlines

ce9374c

move models to models.py

0d2f5f3

exclude unset

04a35d8

gunicorn with FastAPI

6ecbc26

change version name

43bdeb0

Kilo59 force-pushed the FastAPI-conversion branch from 4561003 to dc943b1 Compare March 24, 2020 22:58

add api_client testing client fixture and basic swagger doc tests

129a153

Kilo59 force-pushed the FastAPI-conversion branch from dc943b1 to 129a153 Compare March 24, 2020 22:59

Kilo59 requested a review from ExpDev07 March 24, 2020 23:09

Kilo59 and others added 2 commits March 24, 2020 19:15

add Swagger information to the Readme

d343a35

Improve middleware.

8b7d299

ExpDev07 approved these changes Mar 24, 2020

View reviewed changes

ExpDev07 added 3 commits March 25, 2020 00:21

fix tests

2a6892b

cleanup and use single quotes to follow same conventions

6b4447b

fixes source middleware not returning 404 when invalid source is prov…

1dd1cf0

…ided + doc changes + some cleanup

ExpDev07 requested changes Mar 24, 2020

View reviewed changes

Kilo59 added 3 commits March 24, 2020 20:26

add county to Country model

a0542f2

test /locations codes

8978af1

better swagger description

c7cb10c

Kilo59 requested a review from ExpDev07 March 25, 2020 00:41

forgot to import pytest

2854739

Kilo59 and others added 2 commits March 24, 2020 22:41

add /latest test

45dc4d4

changed start dev script to "pipenv run dev" and "pipenv run start" +…

304de58

… renamed models + some cleanup

ExpDev07 and others added 4 commits March 25, 2020 04:42

fixed querying of all properties

88a6faf

update README with doc

a55e03e

small edit in models file

0e3d352

add github link

d7d72d9

ExpDev07 approved these changes Mar 25, 2020

View reviewed changes

ExpDev07 merged commit 1bc26b7 into ExpDev07:master Mar 25, 2020

ExpDev07 deleted the FastAPI-conversion branch March 25, 2020 03:55

allcontributors bot mentioned this pull request Mar 25, 2020

docs: add Kilo59 as a contributor #170

Merged



		# mount the existing Flask app to /v2
		APP.mount("/v2", WSGIMiddleware(create_app()))

Uh oh!

FastAPI conversion #143

FastAPI conversion #143

Uh oh!

Conversation

Kilo59 commented Mar 22, 2020 • edited by ExpDev07 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO:

Uh oh!

Kilo59 commented Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ExpDev07 commented Mar 22, 2020

Uh oh!

Kilo59 commented Mar 22, 2020

Uh oh!

ExpDev07 commented Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kilo59 Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExpDev07 Mar 22, 2020

Choose a reason for hiding this comment

Uh oh!

Kilo59 Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kilo59 Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExpDev07 Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExpDev07 Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kilo59 Mar 22, 2020

Choose a reason for hiding this comment

Uh oh!

ExpDev07 Mar 22, 2020

Choose a reason for hiding this comment

Uh oh!

Kilo59 Mar 22, 2020

Choose a reason for hiding this comment

Uh oh!

ExpDev07 Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kilo59 commented Mar 22, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

v2

v2-1

Uh oh!

ExpDev07 commented Mar 22, 2020

Uh oh!

ExpDev07 left a comment

Choose a reason for hiding this comment

Uh oh!

ExpDev07 Mar 24, 2020

Choose a reason for hiding this comment

Uh oh!

Kilo59 Mar 25, 2020

Choose a reason for hiding this comment

Uh oh!

Kilo59 Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ExpDev07 left a comment

Choose a reason for hiding this comment

Uh oh!

Kilo59 commented Mar 25, 2020

Uh oh!

Kilo59 commented Mar 22, 2020 •

edited by ExpDev07

Loading

Kilo59 commented Mar 22, 2020 •

edited

Loading

ExpDev07 commented Mar 22, 2020 •

edited

Loading

Kilo59 Mar 22, 2020 •

edited

Loading

Kilo59 Mar 22, 2020 •

edited

Loading

Kilo59 Mar 22, 2020 •

edited

Loading

ExpDev07 Mar 22, 2020 •

edited

Loading

ExpDev07 Mar 22, 2020 •

edited

Loading

ExpDev07 Mar 22, 2020 •

edited

Loading

Kilo59 commented Mar 22, 2020 •

edited

Loading

Kilo59 Mar 25, 2020 •

edited

Loading