update example for binder

Author: AntSimi

notebooks/python_module/01_general_things/pet_storage.ipynb

@@ -15,7 +15,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "\n# How data is stored\n\nGeneral information about eddies storage.\n\nAll eddies files have same structure with more or less field and a way of ordering.\n\nThere are 3 class of files:\n\n- Eddies collections which contains a list of eddies without link between observations\n- Track eddies collections which manage eddies when there are merged in trajectory\n  (track field allow to separate each track)\n- Network eddies collections which manage eddies when there are merged in network\n  (track/segment field allow to separate observations)\n"
+        "\n# How data is stored\n\nGeneral information about eddies storage.\n\nAll files have the same structure, with more or less fields and possible different order.\n\nThere are 3 class of files:\n\n- **Eddies collections** : contain a list of eddies without link between them\n- **Track eddies collections** : manage eddies associated in trajectories, the ```track``` field allows to separate each trajectory\n- **Network eddies collections** : manage eddies associated in networks, the ```track``` and ```segment``` fields allow to separate observations\n"
       ]
     },
     {
@@ -33,14 +33,14 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Eddies could be store in 2 formats with same structures:\n\n- zarr (https://zarr.readthedocs.io/en/stable/), which allow efficiency in IO,...\n- NetCDF4 (https://unidata.github.io/netcdf4-python/), well-known format\n\nEach field are stored in column, each row corresponds at 1 observation,\narray field like contour/profile are 2D column.\n\n"
+        "Eddies can be stored in 2 formats with the same structure:\n\n- zarr (https://zarr.readthedocs.io/en/stable/), which allow efficiency in IO,...\n- NetCDF4 (https://unidata.github.io/netcdf4-python/), well-known format\n\nEach field are stored in column, each row corresponds at 1 observation,\narray field like contour/profile are 2D column.\n\n"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "Eddies files (zarr or netcdf) could be loaded with `load_file` method:\n\n"
+        "Eddies files (zarr or netcdf) could be loaded with ```load_file``` method:\n\n"
       ]
     },
     {
@@ -58,7 +58,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Field access\n\n"
+        "## Field access\nTo access the total field, here ```amplitude```\n\n"
       ]
     },
     {
@@ -69,7 +69,7 @@
       },
       "outputs": [],
       "source": [
-        "eddies_collections.amplitude"
+        "eddies_collections.amplitude\n\n# To access only a specific part of the field\neddies_collections.amplitude[4:15]"
       ]
     },
     {
@@ -105,14 +105,14 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Contour storage\nContour are stored to fixed size for all, contour are resample with an algorithm before to be store in object\n\n"
+        "## Contour storage\nAll contours are stored on the same number of points, and are resampled if needed with an algorithm to be stored as objects\n\n"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Tracks\nTracks add several field like:\n\n- track : ID which allow to identify path\n- observation_flag : True if it's an observation to filled a missing detection\n- observation_number : Age of eddies\n- cost_association : result of cost function which allow to associate the observation with eddy path\n\n"
+        "## Trajectories\nTracks eddies collections add several fields :\n\n- **track** : Trajectory number\n- **observation_flag** : Flag indicating if the value is interpolated between two observations or not (0: observed eddy, 1: interpolated eddy)\"\n- **observation_number** : Eddy temporal index in a trajectory, days starting at the eddy first detection\n- **cost_association** : result of the cost function to associate the eddy with the next observation\n\n"
       ]
     },
     {
@@ -123,14 +123,14 @@
       },
       "outputs": [],
       "source": [
-        "eddies_tracks = TrackEddiesObservations.load_file(\n    py_eddy_tracker_sample.get_path(\"eddies_med_adt_allsat_dt2018/Cyclonic.zarr\")\n)\n# In this example some field are removed like effective_contour_longitude, ... in order to save time for doc building\neddies_tracks.field_table()"
+        "eddies_tracks = TrackEddiesObservations.load_file(\n    py_eddy_tracker_sample.get_path(\"eddies_med_adt_allsat_dt2018/Cyclonic.zarr\")\n)\n# In this example some fields are removed (effective_contour_longitude,...) in order to save time for doc building\neddies_tracks.field_table()"
       ]
     },
     {
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "## Network\nNetwork files use some specific field:\n\n- track :  ID of network (ID 0 are for lonely eddies/trash)\n- segment :  ID of path in network (from 0 to N)\n- previous_obs : Index of the previous observation in the full dataset, if -1 there are no previous observation\n- next_obs : Index of the next observation in the full dataset, if -1 there are no next observation\n- previous_cost : Result of cost_function (1 good <> 0 bad) with previous observation\n- next_cost : Result of cost_function (1 good <> 0 bad) with next observation\n\n"
+        "## Networks\nNetwork files use some specific fields :\n\n- track :  ID of network (ID 0 correspond to lonely eddies)\n- segment :  ID of a segment within a network (from 1 to N)\n- previous_obs : Index of the previous observation in the full dataset, if -1 there are no previous observation (the segment starts)\n- next_obs : Index of the next observation in the full dataset, if -1 there are no next observation (the segment ends)\n- previous_cost : Result of the cost function (1 is a good association, 0 is bad) with previous observation\n- next_cost : Result of the cost function (1 is a good association, 0 is bad) with next observation\n\n"
       ]
     },
     {
@@ -154,6 +154,13 @@
       "source": [
         "sl = slice(70, 100)\nTable(\n    eddies_network.network(651).obs[sl][\n        [\n            \"time\",\n            \"track\",\n            \"segment\",\n            \"previous_obs\",\n            \"previous_cost\",\n            \"next_obs\",\n            \"next_cost\",\n        ]\n    ]\n)"
       ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Networks are ordered by increasing network number (`track`), then increasing segment number, then increasing time\n\n"
+      ]
     }
   ],
   "metadata": {

notebooks/python_module/02_eddy_identification/pet_eddy_detection_ACC.ipynb

@@ -15,7 +15,7 @@
       "cell_type": "markdown",
       "metadata": {},
       "source": [
-        "\n# Eddy detection : Antartic circum polar\n\nScript will detect eddies on adt field, and compute u,v with method add_uv(which could use, only if equator is avoid)\n\nTwo ones with filtering adt and another without\n"
+        "\n# Eddy detection : Antartic Circumpolar Current\n\nThis script detect eddies on the ADT field, and compute u,v with the method add_uv (use it only if the Equator is avoided)\n\nTwo detections are provided : with a filtered ADT and without filtering\n"
       ]
     },
     {
@@ -26,7 +26,7 @@
       },
       "outputs": [],
       "source": [
-        "from datetime import datetime\n\nfrom matplotlib import pyplot as plt\n\nfrom py_eddy_tracker import data\nfrom py_eddy_tracker.dataset.grid import RegularGridDataset\n\n\ndef quad_axes(title):\n    fig = plt.figure(figsize=(13, 8.5))\n    fig.suptitle(title, weight=\"bold\")\n    axes = list()\n    for position in (\n        [0.05, 0.53, 0.44, 0.44],\n        [0.53, 0.53, 0.44, 0.44],\n        [0.05, 0.03, 0.44, 0.44],\n        [0.53, 0.03, 0.44, 0.44],\n    ):\n        ax = fig.add_axes(position)\n        ax.set_xlim(5, 45), ax.set_ylim(-60, -37)\n        ax.set_aspect(\"equal\"), ax.grid(True)\n        axes.append(ax)\n    return axes"
+        "from datetime import datetime\n\nfrom matplotlib import pyplot as plt\nfrom matplotlib import style\n\nfrom py_eddy_tracker import data\nfrom py_eddy_tracker.dataset.grid import RegularGridDataset\n\npos_cb = [0.1, 0.52, 0.83, 0.015]\npos_cb2 = [0.1, 0.07, 0.4, 0.015]\n\n\ndef quad_axes(title):\n    style.use(\"default\")\n    fig = plt.figure(figsize=(13, 10))\n    fig.suptitle(title, weight=\"bold\", fontsize=14)\n    axes = list()\n\n    ax_pos = dict(\n        topleft=[0.1, 0.54, 0.4, 0.38],\n        topright=[0.53, 0.54, 0.4, 0.38],\n        botleft=[0.1, 0.09, 0.4, 0.38],\n        botright=[0.53, 0.09, 0.4, 0.38],\n    )\n\n    for key, position in ax_pos.items():\n        ax = fig.add_axes(position)\n        ax.set_xlim(5, 45), ax.set_ylim(-60, -37)\n        ax.set_aspect(\"equal\"), ax.grid(True)\n        axes.append(ax)\n        if \"right\" in key:\n            ax.set_yticklabels(\"\")\n    return fig, axes\n\n\ndef set_fancy_labels(fig, ticklabelsize=14, labelsize=14, labelweight=\"semibold\"):\n    for ax in fig.get_axes():\n        ax.grid()\n        ax.grid(which=\"major\", linestyle=\"-\", linewidth=\"0.5\", color=\"black\")\n        if ax.get_ylabel() != \"\":\n            ax.set_ylabel(ax.get_ylabel(), fontsize=labelsize, fontweight=labelweight)\n        if ax.get_xlabel() != \"\":\n            ax.set_xlabel(ax.get_xlabel(), fontsize=labelsize, fontweight=labelweight)\n        if ax.get_title() != \"\":\n            ax.set_title(ax.get_title(), fontsize=labelsize, fontweight=labelweight)\n        ax.tick_params(labelsize=ticklabelsize)"
       ]
     },
     {
@@ -80,7 +80,14 @@
       },
       "outputs": [],
       "source": [
-        "axs = quad_axes(\"General properties field\")\nm = g_raw.display(axs[0], \"adt\", vmin=-1, vmax=1, cmap=\"RdBu_r\")\naxs[0].set_title(\"ADT(m)\")\nm = g.display(axs[1], \"adt_low\", vmin=-1, vmax=1, cmap=\"RdBu_r\")\naxs[1].set_title(\"ADT (m) large scale with cut at 700 km\")\nm = g.display(axs[2], \"adt\", vmin=-1, vmax=1, cmap=\"RdBu_r\")\naxs[2].set_title(\"ADT (m) high scale with cut at 700 km\")\ncb = plt.colorbar(\n    m, cax=axs[0].figure.add_axes([0.03, 0.51, 0.94, 0.01]), orientation=\"horizontal\"\n)\ncb.set_label(\"ADT(m)\", labelpad=-2)"
+        "kw_adt = dict(vmin=-1.5, vmax=1.5, cmap=plt.get_cmap(\"RdBu_r\", 30))\nfig, axs = quad_axes(\"General properties field\")\nm = g_raw.display(axs[0], \"adt\", **kw_adt)\naxs[0].set_title(\"Total ADT (m)\")\nm = g.display(axs[1], \"adt_low\", **kw_adt)\naxs[1].set_title(\"ADT (m) large scale, cutoff at 700 km\")\nm2 = g.display(axs[2], \"adt\", cmap=plt.get_cmap(\"RdBu_r\", 20), vmin=-0.5, vmax=0.5)\naxs[2].set_title(\"ADT (m) high-pass filtered, a cutoff at 700 km\")\ncb = plt.colorbar(m, cax=axs[0].figure.add_axes(pos_cb), orientation=\"horizontal\")\ncb.set_label(\"ADT (m)\", labelpad=0)\ncb2 = plt.colorbar(m2, cax=axs[2].figure.add_axes(pos_cb2), orientation=\"horizontal\")\ncb2.set_label(\"ADT (m)\", labelpad=0)\nset_fancy_labels(fig)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "The large-scale North-South gradient is removed by the filtering step.\n\n"
       ]
     },
     {
@@ -91,7 +98,14 @@
       },
       "outputs": [],
       "source": [
-        "axs = quad_axes(\"\")\naxs[0].set_title(\"Without filter\")\naxs[0].set_ylabel(\"Contours used in eddies\")\naxs[1].set_title(\"With filter\")\naxs[2].set_ylabel(\"Closed contours but not used\")\ng_raw.contours.display(axs[0], lw=0.5, only_used=True)\ng.contours.display(axs[1], lw=0.5, only_used=True)\ng_raw.contours.display(axs[2], lw=0.5, only_unused=True)\ng.contours.display(axs[3], lw=0.5, only_unused=True)"
+        "fig, axs = quad_axes(\"\")\naxs[0].set_title(\"Without filter\")\naxs[0].set_ylabel(\"Contours used in eddies\")\naxs[1].set_title(\"With filter\")\naxs[2].set_ylabel(\"Closed contours but not used\")\ng_raw.contours.display(axs[0], lw=0.5, only_used=True)\ng.contours.display(axs[1], lw=0.5, only_used=True)\ng_raw.contours.display(axs[2], lw=0.5, only_unused=True)\ng.contours.display(axs[3], lw=0.5, only_unused=True)\nset_fancy_labels(fig)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Removing the large-scale North-South gradient reveals closed contours in the\nSouth-Western corner of the ewample region.\n\n"
       ]
     },
     {
@@ -102,7 +116,14 @@
       },
       "outputs": [],
       "source": [
-        "kw = dict(ref=-10, linewidth=0.75)\nkw_a = dict(color=\"r\", label=\"Anticyclonic ({nb_obs} eddies)\")\nkw_c = dict(color=\"b\", label=\"Cyclonic ({nb_obs} eddies)\")\nkw_filled = dict(vmin=0, vmax=100, cmap=\"Spectral_r\", lut=20, intern=True, factor=100)\naxs = quad_axes(\"Comparison between two detection\")\n# Match with intern/inner contour\ni_a, j_a, s_a = a_.match(a, intern=True, cmin=0.15)\ni_c, j_c, s_c = c_.match(c, intern=True, cmin=0.15)\n\na_.index(i_a).filled(axs[0], s_a, **kw_filled)\na.index(j_a).filled(axs[1], s_a, **kw_filled)\nc_.index(i_c).filled(axs[0], s_c, **kw_filled)\nm = c.index(j_c).filled(axs[1], s_c, **kw_filled)\n\ncb = plt.colorbar(\n    m, cax=axs[0].figure.add_axes([0.03, 0.51, 0.94, 0.01]), orientation=\"horizontal\"\n)\ncb.set_label(\"Similarity index\", labelpad=-5)\na_.display(axs[0], **kw, **kw_a), c_.display(axs[0], **kw, **kw_c)\na.display(axs[1], **kw, **kw_a), c.display(axs[1], **kw, **kw_c)\n\naxs[0].set_title(\"Without filter\")\naxs[0].set_ylabel(\"Detection\")\naxs[1].set_title(\"With filter\")\naxs[2].set_ylabel(\"Contours' rejection criteria\")\n\ng_raw.contours.display(axs[2], lw=0.5, only_unused=True, display_criterion=True)\ng.contours.display(axs[3], lw=0.5, only_unused=True, display_criterion=True)\n\nfor ax in axs:\n    ax.legend()"
+        "kw = dict(ref=-10, linewidth=0.75)\nkw_a = dict(color=\"r\", label=\"Anticyclonic ({nb_obs} eddies)\")\nkw_c = dict(color=\"b\", label=\"Cyclonic ({nb_obs} eddies)\")\nkw_filled = dict(vmin=0, vmax=100, cmap=\"Spectral_r\", lut=20, intern=True, factor=100)\nfig, axs = quad_axes(\"Comparison between two detections\")\n# Match with intern/inner contour\ni_a, j_a, s_a = a_.match(a, intern=True, cmin=0.15)\ni_c, j_c, s_c = c_.match(c, intern=True, cmin=0.15)\n\na_.index(i_a).filled(axs[0], s_a, **kw_filled)\na.index(j_a).filled(axs[1], s_a, **kw_filled)\nc_.index(i_c).filled(axs[0], s_c, **kw_filled)\nm = c.index(j_c).filled(axs[1], s_c, **kw_filled)\n\ncb = plt.colorbar(m, cax=axs[0].figure.add_axes(pos_cb), orientation=\"horizontal\")\ncb.set_label(\"Similarity index (%)\", labelpad=-5)\na_.display(axs[0], **kw, **kw_a), c_.display(axs[0], **kw, **kw_c)\na.display(axs[1], **kw, **kw_a), c.display(axs[1], **kw, **kw_c)\n\naxs[0].set_title(\"Without filter\")\naxs[0].set_ylabel(\"Detection\")\naxs[1].set_title(\"With filter\")\naxs[2].set_ylabel(\"Contours' rejection criteria\")\n\ng_raw.contours.display(axs[2], lw=0.5, only_unused=True, display_criterion=True)\ng.contours.display(axs[3], lw=0.5, only_unused=True, display_criterion=True)\n\nfor ax in axs:\n    ax.legend()\n\nset_fancy_labels(fig)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Very similar eddies have Similarity Indexes >= 40%\n\n"
       ]
     },
     {
@@ -120,7 +141,7 @@
       },
       "outputs": [],
       "source": [
-        "i_a, j_a = i_a[s_a >= 0.4], j_a[s_a >= 0.4]\ni_c, j_c = i_c[s_c >= 0.4], j_c[s_c >= 0.4]\nfig = plt.figure(figsize=(12, 12))\nfig.suptitle(f\"Scatter plot (A : {i_a.shape[0]}, C : {i_c.shape[0]} matches)\")\n\nfor i, (label, field, factor, stop) in enumerate(\n    (\n        (\"speed radius (km)\", \"radius_s\", 0.001, 120),\n        (\"outter radius (km)\", \"radius_e\", 0.001, 120),\n        (\"amplitude (cm)\", \"amplitude\", 100, 25),\n        (\"speed max (cm/s)\", \"speed_average\", 100, 25),\n    )\n):\n    ax = fig.add_subplot(2, 2, i + 1, title=label)\n    ax.set_xlabel(\"Without filter\")\n    ax.set_ylabel(\"With filter\")\n\n    ax.plot(\n        a_[field][i_a] * factor,\n        a[field][j_a] * factor,\n        \"r.\",\n        label=\"Anticyclonic\",\n    )\n    ax.plot(\n        c_[field][i_c] * factor,\n        c[field][j_c] * factor,\n        \"b.\",\n        label=\"Cyclonic\",\n    )\n    ax.set_aspect(\"equal\"), ax.grid()\n    ax.plot((0, 1000), (0, 1000), \"g\")\n    ax.set_xlim(0, stop), ax.set_ylim(0, stop)\n    ax.legend()"
+        "i_a, j_a = i_a[s_a >= 0.4], j_a[s_a >= 0.4]\ni_c, j_c = i_c[s_c >= 0.4], j_c[s_c >= 0.4]\nfig = plt.figure(figsize=(12, 12))\nfig.suptitle(f\"Scatter plot (A : {i_a.shape[0]}, C : {i_c.shape[0]} matches)\")\n\nfor i, (label, field, factor, stop) in enumerate(\n    (\n        (\"Speed radius (km)\", \"radius_s\", 0.001, 120),\n        (\"Effective radius (km)\", \"radius_e\", 0.001, 120),\n        (\"Amplitude (cm)\", \"amplitude\", 100, 25),\n        (\"Speed max (cm/s)\", \"speed_average\", 100, 25),\n    )\n):\n    ax = fig.add_subplot(2, 2, i + 1, title=label)\n    ax.set_xlabel(\"Without filter\")\n    ax.set_ylabel(\"With filter\")\n\n    ax.plot(\n        a_[field][i_a] * factor,\n        a[field][j_a] * factor,\n        \"r.\",\n        label=\"Anticyclonic\",\n    )\n    ax.plot(\n        c_[field][i_c] * factor,\n        c[field][j_c] * factor,\n        \"b.\",\n        label=\"Cyclonic\",\n    )\n    ax.set_aspect(\"equal\"), ax.grid()\n    ax.plot((0, 1000), (0, 1000), \"g\")\n    ax.set_xlim(0, stop), ax.set_ylim(0, stop)\n    ax.legend()\n\nset_fancy_labels(fig)"
       ]
     }
   ],

src/py_eddy_tracker/dataset/grid.py

@@ -634,7 +634,6 @@ def eddy_identification(
         self.init_speed_coef(uname, vname)
 
         # Get unit of h grid
-
         h_units = (
             self.units(grid_height) if force_height_unit is None else force_height_unit
         )