Client improve docs (canada-ca#1656)

* First pass at fleshed out README

* Remove duplicate IPython notice in README

* Improve consistency of headers

* Fix examples match output shown

* Convert existing docstrings to rst style

* revise function docstrings in client

* Module docstrings, document queries

* More test docstrings

* Refactor formatting functions to own module

This change is necessary to reduce the length of client.py in preparation
for adding examples to docstrings. Otherwise it would become unmanageably
long after examples are added.

* Add examples for all get functions in client.py

* Run black on last commit

Author: ThomasNickerson

clients/python/README.md

@@ -1,12 +1,21 @@
 # Tracker Python API Client
 
-The Tracker Python API Client will provide a Python wrapper for key features of Tracker. 
+The Tracker Python API Client provides a simple Python interface for the [Tracker GraphQL API](https://github.com/canada-ca/tracker/blob/master/api-js/README.md), with the aim of allowing users to easily integrate data from Tracker into existing workflows and platforms. It allows access to the JSON data served by the API without requiring specific knowledge of [GraphQL](https://graphql.org/) or the Tracker API. This is done by providing functions that execute canned queries against the API using [gql](https://github.com/graphql-python/gql). Responses are formatted to remove pagination related structures, and to ensure useful keys are always present.
 
-It makes use of [gql](https://github.com/graphql-python/gql), a Python GraphQL client, to query the Tracker API.
 
-#### Installing Dependencies
+## Installation
 
-Install [pipenv](https://pypi.org/project/pipenv/) if you don't already have it.
+### For Users
+
+The client will soon be available to install as a package via pip or pipenv. Until then, follow the instructions for developers below.
+
+### For Developers
+
+Install [pipenv](https://pypi.org/project/pipenv/) if you don't already have it. The following instructions assume you are using pipenv.
+
+#### Installing Dependencies 
+
+Make sure you have pulled the most recent version from the repo, then run:
 
 ```shell
 pipenv install --dev
@@ -18,7 +27,12 @@ If you run into issues, ensure pipenv has installed the most recent GQL version.
 pipenv install -e git+https://github.com/graphql-python/gql.git#egg=gql
 ```
 
-#### Authentication
+## Usage
+
+
+### Authentication
+
+You must have a Tracker account that is a member of one or more organizations to make use of the Python client. You can manage your account in the [Tracker web interface](https://tracker.alpha.canada.ca/).
 
 The client will attempt to draw credentials from its environment in order to obtain an authentication token. Pipenv makes this easy to set up by importing environment variables from a `.env` file present in this directory whenever `pipenv run` or `pipenv shell` are used. The `.env` file can be created like so:
 
@@ -29,7 +43,122 @@ TRACKER_PASS=YOURPASSWORDHERE
 EOF
 ```
 
-#### Testing
+You should be mindful that setting these variables manually can result in credentials being stored in your shell command history.
+
+### Basic Usage
+
+You will generally start by creating a client with `create_client(auth_token=get_auth_token())` and storing the result. All functions that make queries expect such a client to be passed as the first argument.
+
+### Examples
+
+#### Get all domains in my organizations
+
+Supposing I belong to two organizations with the acronyms "FOO" and "BAR":
+
+```python
+>>> import tracker_client.client as tracker_client
+>>> client = tracker_client.create_client(auth_token=tracker_client.get_auth_token())
+>>> print(tracker_client.get_all_domains(client))
+{
+    "FOO": [
+        "foo.bar",
+        "foo.bar.baz"
+    ],
+    "BAR": [
+        "fizz.buzz",
+        "buzz.bang",
+        "ab.cd.ef",
+    ]
+}
+```
+
+The following examples continue the previous one (assume the package has been imported as above and a `client` object with a valid token exists).
+
+#### Get a DMARC summary for a domain
+
+```python
+>>> print(tracker_client.get_dmarc_summary(client, "foo.bar", "september", 2020))
+{
+    "foo.bar": {
+        "month": "SEPTEMBER",
+        "year": "2020",
+        "categoryPercentages": {
+            "fullPassPercentage": 87,
+            "passSpfOnlyPercentage": 0,
+            "passDkimOnlyPercentage": 6,
+            "failPercentage": 8,
+            "totalMessages": 10534
+        }
+    }
+}
+```
+
+#### Get summary metrics for an organization
+
+```python
+>>> print(tracker_client.get_summary_by_acronym(client, "foo"))
+{
+    "FOO": {
+        "domainCount": 10,
+        "summaries": {
+            "web": {
+                "total": 10,
+                "categories": [
+                    {
+                        "name": "pass",
+                        "count": 1,
+                        "percentage": 10
+                    },
+                    {
+                        "name": "fail",
+                        "count": 9,
+                        "percentage": 90
+                    }
+                ]
+            },
+            "mail": {
+                "total": 10,
+                "categories": [
+                    {
+                        "name": "pass",
+                        "count": 5,
+                        "percentage": 50
+                    },
+                    {
+                        "name": "fail",
+                        "count": 5,
+                        "percentage": 50
+                    }
+                ]
+            }
+        }
+    }
+}
+```
+
+#### Get the status of a domain 
+
+```python
+>>> print(tracker_client.get_domain_status(client, "foo.bar"))
+{
+    "foo.bar": {
+        "lastRan": "2021-01-23 22:33:26.921529",
+        "status": {
+            "https": "FAIL",
+            "ssl": "FAIL",
+            "dmarc": "PASS",
+            "dkim": "PASS",
+            "spf": "PASS"
+        }
+    }
+}
+```
+
+> **NOTE**: Because of gql limitations, the client is not currently compatible with IPython or Jupyter.
+
+## Development
+
+### Testing
 
 Pytest is used for testing. To run tests, run the following in the project root (the folder containing this README.md):
 
@@ -41,6 +170,9 @@ Alternatively, if you are already in a pipenv shell, just run `pytest`.
 
 If tests are failing with ModuleNotFoundError, make sure tracker_client/ is on your PYTHONPATH. The .env file used to store your credentials is a good way to set this.
 
-#### Note about IPython/Jupyter 
+When additions or significant changes are made, check test coverage with:
+
+```shell
+pipenv run pytest --cov=tracker_client
+```
 
-Because of a limitation in gql, the client is not currently compatible with IPython or Jupyter.

clients/python/tests/test_formatting.py

@@ -1,4 +1,8 @@
-from tracker_client.client import (
+"""Tests for query response formatting functions.
+
+The queries named in ALL_CAPS can be found in tracker_client/queries.py
+"""
+from tracker_client.formatting import (
     format_all_domains,
     format_acronym_domains,
     format_name_domains,
@@ -11,41 +15,52 @@
     format_domain_status,
 )
 
+
 def test_format_all_domains(all_domains_input, all_domains_output):
+    """Test formatting of ALL_DOMAINS_QUERY results"""
     assert format_all_domains(all_domains_input) == all_domains_output
 
 
 def test_format_acronym_domains(all_domains_input, org_domains_output):
+    """Test formatting + filtering by acronym of ALL_DOMAINS_QUERY results"""
     assert format_acronym_domains(all_domains_input, "def") == org_domains_output
 
 
 def test_format_name_domains(name_domain_input, org_domains_output):
+    """Test formatting of DOMAINS_BY_SLUG results"""
     assert format_name_domains(name_domain_input) == org_domains_output
 
 
 def test_format_dmarc_monthly(monthly_dmarc_input, monthly_dmarc_output):
+    """Test formatting of DMARC_SUMMARY results"""
     assert format_dmarc_monthly(monthly_dmarc_input) == monthly_dmarc_output
 
 
 def test_format_dmarc_yearly(yearly_dmarc_input, yearly_dmarc_output):
+    """Test formatting of YEARLY_DMARC_SUMMARIES results"""
     assert format_dmarc_yearly(yearly_dmarc_input) == yearly_dmarc_output
 
 
 def test_format_all_summaries(all_summaries_input, all_summaries_output):
+    """Test formatting of ALL_ORGS_SUMMARIES results"""
     assert format_all_summaries(all_summaries_input) == all_summaries_output
 
 
 def test_format_acronym_summary(all_summaries_input, org_summary_output):
+    """Test formatting + filtering by acronym of ALL_ORGS_SUMMARIES results"""
     assert format_acronym_summary(all_summaries_input, "def") == org_summary_output
 
 
 def test_format_name_summary(name_summary_input, org_summary_output):
+    """Test formatting of SUMMARY_BY_SLUG results"""
     assert format_name_summary(name_summary_input) == org_summary_output
 
 
 def test_format_domain_results(scan_results_input, scan_results_output):
+    """Test formatting of DOMAIN_RESULTS results"""
     assert format_domain_results(scan_results_input) == scan_results_output
 
 
 def test_format_domain_status(domain_status_input, domain_status_output):
+    """Test formatting of DOMAIN_STATUS results"""
     assert format_domain_status(domain_status_input) == domain_status_output

clients/python/tests/test_get_dmarc.py

@@ -1,3 +1,4 @@
+"""Tests for functions that get DMARC summaries for a domain"""
 import json
 
 from gql import Client

clients/python/tests/test_get_domains.py

@@ -1,3 +1,4 @@
+"""Tests for functions that get domain lists"""
 import json
 
 from gql import Client

clients/python/tests/test_get_results.py

@@ -1,3 +1,4 @@
+"""Tests for functions that get scan results for a domain"""
 import json
 
 from gql import Client

clients/python/tests/test_get_summaries.py

@@ -1,3 +1,4 @@
+"""Tests for functions that get summary metrics"""
 import json
 
 from gql import Client

clients/python/tests/test_gql.py

@@ -1,3 +1,4 @@
+"""Tests for gql related utility functions"""
 import re
 import pytest