Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 140 additions & 8 deletions clients/python/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,21 @@
# Tracker Python API Client

The Tracker Python API Client will provide a Python wrapper for key features of Tracker.
The Tracker Python API Client provides a simple Python interface for the [Tracker GraphQL API](https://github.com/canada-ca/tracker/blob/master/api-js/README.md), with the aim of allowing users to easily integrate data from Tracker into existing workflows and platforms. It allows access to the JSON data served by the API without requiring specific knowledge of [GraphQL](https://graphql.org/) or the Tracker API. This is done by providing functions that execute canned queries against the API using [gql](https://github.com/graphql-python/gql). Responses are formatted to remove pagination related structures, and to ensure useful keys are always present.

It makes use of [gql](https://github.com/graphql-python/gql), a Python GraphQL client, to query the Tracker API.

#### Installing Dependencies
## Installation

Install [pipenv](https://pypi.org/project/pipenv/) if you don't already have it.
### For Users

The client will soon be available to install as a package via pip or pipenv. Until then, follow the instructions for developers below.

### For Developers

Install [pipenv](https://pypi.org/project/pipenv/) if you don't already have it. The following instructions assume you are using pipenv.

#### Installing Dependencies

Make sure you have pulled the most recent version from the repo, then run:

```shell
pipenv install --dev
Expand All @@ -18,7 +27,12 @@ If you run into issues, ensure pipenv has installed the most recent GQL version.
pipenv install -e git+https://github.com/graphql-python/gql.git#egg=gql
```

#### Authentication
## Usage


### Authentication

You must have a Tracker account that is a member of one or more organizations to make use of the Python client. You can manage your account in the [Tracker web interface](https://tracker.alpha.canada.ca/).

The client will attempt to draw credentials from its environment in order to obtain an authentication token. Pipenv makes this easy to set up by importing environment variables from a `.env` file present in this directory whenever `pipenv run` or `pipenv shell` are used. The `.env` file can be created like so:

Expand All @@ -29,7 +43,122 @@ TRACKER_PASS=YOURPASSWORDHERE
EOF
```

#### Testing
You should be mindful that setting these variables manually can result in credentials being stored in your shell command history.

### Basic Usage

You will generally start by creating a client with `create_client(auth_token=get_auth_token())` and storing the result. All functions that make queries expect such a client to be passed as the first argument.

### Examples

#### Get all domains in my organizations

Supposing I belong to two organizations with the acronyms "FOO" and "BAR":

```python
>>> import tracker_client.client as tracker_client
>>> client = tracker_client.create_client(auth_token=tracker_client.get_auth_token())
>>> print(tracker_client.get_all_domains(client))
{
"FOO": [
"foo.bar",
"foo.bar.baz"
],
"BAR": [
"fizz.buzz",
"buzz.bang",
"ab.cd.ef",
]
}
```

The following examples continue the previous one (assume the package has been imported as above and a `client` object with a valid token exists).

#### Get a DMARC summary for a domain

```python
>>> print(tracker_client.get_dmarc_summary(client, "foo.bar", "september", 2020))
{
"foo.bar": {
"month": "SEPTEMBER",
"year": "2020",
"categoryPercentages": {
"fullPassPercentage": 87,
"passSpfOnlyPercentage": 0,
"passDkimOnlyPercentage": 6,
"failPercentage": 8,
"totalMessages": 10534
}
}
}
```

#### Get summary metrics for an organization

```python
>>> print(tracker_client.get_summary_by_acronym(client, "foo"))
{
"FOO": {
"domainCount": 10,
"summaries": {
"web": {
"total": 10,
"categories": [
{
"name": "pass",
"count": 1,
"percentage": 10
},
{
"name": "fail",
"count": 9,
"percentage": 90
}
]
},
"mail": {
"total": 10,
"categories": [
{
"name": "pass",
"count": 5,
"percentage": 50
},
{
"name": "fail",
"count": 5,
"percentage": 50
}
]
}
}
}
}
```

#### Get the status of a domain

```python
>>> print(tracker_client.get_domain_status(client, "foo.bar"))
{
"foo.bar": {
"lastRan": "2021-01-23 22:33:26.921529",
"status": {
"https": "FAIL",
"ssl": "FAIL",
"dmarc": "PASS",
"dkim": "PASS",
"spf": "PASS"
}
}
}
```

> **NOTE**: Because of gql limitations, the client is not currently compatible with IPython or Jupyter.

## Development

### Testing

Pytest is used for testing. To run tests, run the following in the project root (the folder containing this README.md):

Expand All @@ -41,6 +170,9 @@ Alternatively, if you are already in a pipenv shell, just run `pytest`.

If tests are failing with ModuleNotFoundError, make sure tracker_client/ is on your PYTHONPATH. The .env file used to store your credentials is a good way to set this.

#### Note about IPython/Jupyter
When additions or significant changes are made, check test coverage with:

```shell
pipenv run pytest --cov=tracker_client
```

Because of a limitation in gql, the client is not currently compatible with IPython or Jupyter.
17 changes: 16 additions & 1 deletion clients/python/tests/test_formatting.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
from tracker_client.client import (
"""Tests for query response formatting functions.

The queries named in ALL_CAPS can be found in tracker_client/queries.py
"""
from tracker_client.formatting import (
format_all_domains,
format_acronym_domains,
format_name_domains,
Expand All @@ -11,41 +15,52 @@
format_domain_status,
)


def test_format_all_domains(all_domains_input, all_domains_output):
"""Test formatting of ALL_DOMAINS_QUERY results"""
assert format_all_domains(all_domains_input) == all_domains_output


def test_format_acronym_domains(all_domains_input, org_domains_output):
"""Test formatting + filtering by acronym of ALL_DOMAINS_QUERY results"""
assert format_acronym_domains(all_domains_input, "def") == org_domains_output


def test_format_name_domains(name_domain_input, org_domains_output):
"""Test formatting of DOMAINS_BY_SLUG results"""
assert format_name_domains(name_domain_input) == org_domains_output


def test_format_dmarc_monthly(monthly_dmarc_input, monthly_dmarc_output):
"""Test formatting of DMARC_SUMMARY results"""
assert format_dmarc_monthly(monthly_dmarc_input) == monthly_dmarc_output


def test_format_dmarc_yearly(yearly_dmarc_input, yearly_dmarc_output):
"""Test formatting of YEARLY_DMARC_SUMMARIES results"""
assert format_dmarc_yearly(yearly_dmarc_input) == yearly_dmarc_output


def test_format_all_summaries(all_summaries_input, all_summaries_output):
"""Test formatting of ALL_ORGS_SUMMARIES results"""
assert format_all_summaries(all_summaries_input) == all_summaries_output


def test_format_acronym_summary(all_summaries_input, org_summary_output):
"""Test formatting + filtering by acronym of ALL_ORGS_SUMMARIES results"""
assert format_acronym_summary(all_summaries_input, "def") == org_summary_output


def test_format_name_summary(name_summary_input, org_summary_output):
"""Test formatting of SUMMARY_BY_SLUG results"""
assert format_name_summary(name_summary_input) == org_summary_output


def test_format_domain_results(scan_results_input, scan_results_output):
"""Test formatting of DOMAIN_RESULTS results"""
assert format_domain_results(scan_results_input) == scan_results_output


def test_format_domain_status(domain_status_input, domain_status_output):
"""Test formatting of DOMAIN_STATUS results"""
assert format_domain_status(domain_status_input) == domain_status_output
1 change: 1 addition & 0 deletions clients/python/tests/test_get_dmarc.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Tests for functions that get DMARC summaries for a domain"""
import json

from gql import Client
Expand Down
1 change: 1 addition & 0 deletions clients/python/tests/test_get_domains.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Tests for functions that get domain lists"""
import json

from gql import Client
Expand Down
1 change: 1 addition & 0 deletions clients/python/tests/test_get_results.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Tests for functions that get scan results for a domain"""
import json

from gql import Client
Expand Down
1 change: 1 addition & 0 deletions clients/python/tests/test_get_summaries.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Tests for functions that get summary metrics"""
import json

from gql import Client
Expand Down
1 change: 1 addition & 0 deletions clients/python/tests/test_gql.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
"""Tests for gql related utility functions"""
import re
import pytest

Expand Down
Loading