Eddy tracking on 3-hourly files #92
Description
Hello,
I'm applying the tracking on model outputs with a time step of 3 hours. The detection gave files such as:
Anticyclonic_20190101000000.nc
Anticyclonic_20190101030000.nc
...
Cyclonic_20191231180000.nc
Cyclonic_20191231210000.nc
on which I apply EddyTracking conf.yaml
1st issue (solved)
...
raise ValueError("unconverted data remains: %s" %
ValueError: unconverted data remains: 210000
the problem came from the fact that the tracker was expecting files with year-month-day.
Solution (bad)
I hard-coded the date_model as "%Y%m%d%H%M%S" in eddies.py.
Question: how could I do it in a cleaner way?
2nd issue
I re-run EddyTracking conf.yaml, it works for some periods of time, for example if I only take FILES_PATTERN: /data/CAPCO/EddyTracking/CORSE400/Cyclonic_201902*.nc and I get the output as specified. However for some periods, the tracking stops with this message:
Traceback (most recent call last):
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/bin/EddyTracking", line 8, in <module>
sys.exit(eddies_tracking())
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/py_eddy_tracker/appli/eddies.py", line 206, in eddies_tracking
track(
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/py_eddy_tracker/appli/eddies.py", line 325, in track
c.track()
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/py_eddy_tracker/tracking.py", line 385, in track
i_previous, i_current, association_cost = self.previous_obs.tracking(
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/py_eddy_tracker/featured_tracking/area_tracker.py", line 46, in tracking
i_self, i_other = self.solve_function(cost_mat)
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/py_eddy_tracker/observations/observation.py", line 1431, in solve_function
return numba_where(self.solve_simultaneous(cost_matrix))
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/py_eddy_tracker/observations/observation.py", line 1331, in solve_simultaneous
max_links = max(self_links.max(), other_links.max())
File "/home/ctroupin/Software/PythonEnvs/EddyCorsica/lib/python3.8/site-packages/numpy/core/_methods.py", line 39, in _amax
return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity
I check visually that the detection worked properly and I suspect the failure is due to the fact that in my application the anticyclonic eddies are really intermittent and not frequent (just popping up once in a while)
Question: is this a plausible explanation and if yes, maybe is possible to have a more explicit error message?
This is the .yaml I use:
PATHS:
# Files produces with EddyIdentification
FILES_PATTERN: /data/CAPCO/EddyTracking/CORSE400/Cyclonic_201903*.nc
SAVE_DIR: /data/CAPCO/EddyTracking/CORSE400/Tracking
# Number of timestep for missing detection
VIRTUAL_LENGTH_MAX: 24 # 24 X 3 hours = 72 hours = 3 days
# Minimal time to consider as a full track
TRACK_DURATION_MIN: 10
CLASS:
MODULE: py_eddy_tracker.featured_tracking.area_tracker
CLASS: AreaTracker
OPTIONS:
cmin: 0.2