zero-size array in i_keep_track

[koldunovn]

Hi @AntSimi

First of all many thanks for making the software available!

I have the following error message when running the tracking:

Traceback (most recent call last):
  File "/mnt/lustre01/work/ab0995/a270088/miniconda3/envs/eddy/bin/EddyTracking", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/mnt/lustre01/pf/a/a270088/PYTHON/EDDY/py-eddy-tracker/src/scripts/EddyTracking", line 198, in <module>
    SHORT_CORRESPONDANCES.shorter_than(size_max=NB_OBS_MIN)
  File "/mnt/lustre01/pf/a/a270088/PYTHON/EDDY/py-eddy-tracker/src/py_eddy_tracker/tracking.py", line 515, in shorter_than
    translate = empty(i_keep_track.max() + 1, dtype='u4')
  File "/mnt/lustre01/work/ab0995/a270088/miniconda3/envs/eddy/lib/python3.6/site-packages/numpy/core/_methods.py", line 30, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity

It looks like the reason is that I get an empty array as a result of:

i_keep_track = where(self.nb_obs_by_tracks < size_max)[0]

My time series is quite small, only 30 days, so this might be the reason.

[AntSimi]

Could you run again with -V DEBUG and post here log.
Could you also show me the yaml file for tracking.
It's look like you said, an empty dataset, but with the current information i couldn't saw reasons

[koldunovn]

The complete output is quite long, but what I get at the end is:

INFO 2020-06-17 17:03:13,283 observation.write_file :
					Store in ./out//Cyclonic_untracked.nc
	DEBUG 2020-06-17 17:03:13,285 	observation.	to_netcdf :
						Create Dimensions "obs" : 0
	DEBUG 2020-06-17 17:03:13,285 	observation.	to_netcdf :
						Create Variable amplitude
WARNING 2020-06-17 17:03:13,286 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,286 	observation.	to_netcdf :
						Create Variable effective_contour_height
WARNING 2020-06-17 17:03:13,287 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,287 	observation.	to_netcdf :
						Create Variable effective_contour_latitude
WARNING 2020-06-17 17:03:13,288 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,288 	observation.	to_netcdf :
						Create Variable effective_contour_longitude
WARNING 2020-06-17 17:03:13,289 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,289 	observation.	to_netcdf :
						Create Variable effective_contour_shape_error
WARNING 2020-06-17 17:03:13,290 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,290 	observation.	to_netcdf :
						Create Variable effective_radius
WARNING 2020-06-17 17:03:13,291 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,291 	observation.	to_netcdf :
						Create Variable inner_contour_height
WARNING 2020-06-17 17:03:13,291 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,291 	observation.	to_netcdf :
						Create Variable latitude
WARNING 2020-06-17 17:03:13,292 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,292 	observation.	to_netcdf :
						Create Variable latitude_max
WARNING 2020-06-17 17:03:13,292 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,293 	observation.	to_netcdf :
						Create Variable longitude
WARNING 2020-06-17 17:03:13,293 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,293 	observation.	to_netcdf :
						Create Variable longitude_max
WARNING 2020-06-17 17:03:13,294 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,294 	observation.	to_netcdf :
						Create Variable num_contours
WARNING 2020-06-17 17:03:13,294 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,294 	observation.	to_netcdf :
						Create Variable speed_average
WARNING 2020-06-17 17:03:13,295 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,295 	observation.	to_netcdf :
						Create Variable speed_contour_height
WARNING 2020-06-17 17:03:13,296 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,296 	observation.	to_netcdf :
						Create Variable speed_contour_latitude
WARNING 2020-06-17 17:03:13,297 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,297 	observation.	to_netcdf :
						Create Variable speed_contour_longitude
WARNING 2020-06-17 17:03:13,298 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,298 	observation.	to_netcdf :
						Create Variable speed_contour_shape_error
WARNING 2020-06-17 17:03:13,299 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,299 	observation.	to_netcdf :
						Create Variable speed_radius
WARNING 2020-06-17 17:03:13,299 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,299 	observation.	to_netcdf :
						Create Variable time
WARNING 2020-06-17 17:03:13,300 observation.create_variable :
					Data is empty
	DEBUG 2020-06-17 17:03:13,300 	observation.	to_netcdf :
						Create Variable uavg_profile
WARNING 2020-06-17 17:03:13,301 observation.create_variable :
					Data is empty
INFO 2020-06-17 17:03:13,311 tracking.prepare_merging :
					1537 tracks identified
INFO 2020-06-17 17:03:13,311 tracking.prepare_merging :
					49184 observations will be join
	DEBUG 2020-06-17 17:03:13,311 	tracking.	_copy :
						Copy done
Traceback (most recent call last):
  File "/mnt/lustre01/work/ab0995/a270088/miniconda3/envs/eddy/bin/EddyTracking", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/mnt/lustre01/pf/a/a270088/PYTHON/EDDY/py-eddy-tracker/src/scripts/EddyTracking", line 198, in <module>
    SHORT_CORRESPONDANCES.shorter_than(size_max=NB_OBS_MIN)
  File "/mnt/lustre01/pf/a/a270088/PYTHON/EDDY/py-eddy-tracker/src/py_eddy_tracker/tracking.py", line 515, in shorter_than
    translate = empty(i_keep_track.max() + 1, dtype='u4')
  File "/mnt/lustre01/work/ab0995/a270088/miniconda3/envs/eddy/lib/python3.6/site-packages/numpy/core/_methods.py", line 30, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity

[koldunovn]

Sorry, and here is the yaml:

PATHS:
  # Files produces with EddyIdentification
  FILES_PATTERN: "./Cyclonic*.nc"
  SAVE_DIR: "./out/"

# Number of timestep for missing detection
VIRTUAL_LENGTH_MAX: 3
# Minimal time to consider as a full track
TRACK_DURATION_MIN: 10

[koldunovn]

I think I know what might happen. I have a ton of files, that basically represent the same time step. Please give me some time to check it.

[AntSimi]

At the top of log there 0 obs for untracked, and it is strange! I never see that. In which region did you run it? Take a look at log between two date of tracking, and then to see how many link his found and death, maybe tracker used is too laxist for your area, but again it s strange

[koldunovn]

This is global (interpolated model data) on a 1/4 degree. I obviously created a bunch of similar Anticyclonic and Cyclonic files. The reason is that I have all my input data stored in one file, and for some strange reason was thinking that the date argument for EddyId will work with the specified date from the input file. Silly me :( So I did the loop where only the dates were changing, and created a bunch of files where only the first time step was analyzed :)

One related thing that stil might be worth to fix. I now get this error, when the longer_than is computed:

Traceback (most recent call last):
  File "/mnt/lustre01/work/ab0995/a270088/miniconda3/envs/eddy/bin/EddyTracking", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/mnt/lustre01/pf/a/a270088/PYTHON/EDDY/py-eddy-tracker/src/scripts/EddyTracking", line 200, in <module>
    CORRESPONDANCES.longer_than(size_min=NB_OBS_MIN)
  File "/mnt/lustre01/pf/a/a270088/PYTHON/EDDY/py-eddy-tracker/src/py_eddy_tracker/tracking.py", line 496, in longer_than
    translate = empty(i_keep_track.max() + 1, dtype='u4')
  File "/mnt/lustre01/work/ab0995/a270088/miniconda3/envs/eddy/lib/python3.6/site-packages/numpy/core/_methods.py", line 30, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
ValueError: zero-size array to reduction operation maximum which has no identity

The thing is I only have 7 records now, and I have:

TRACK_DURATION_MIN: 10

So, there are no tracks available for analysis. Maybe it is worth to add an error message with something like:

f`There are no tracks longer than {size_min}. Check that your TRACK_DURATION_MIN is not shorter than the time period you analyze.`

[AntSimi]

Hi
Well done, i didn't think about that (unique file for input data).

With EddyId script it's not available to select index (it could/must be an option to add).
But it's possible to do when you use python module like you could see in #5 or in code.

Time selection is not automatic :( sorry, it's my choice because i saw lot of different case of time definition, or in few case no units. I prefer user put date explicitly, to avoid painful case.

You are right for message i will add this.

Antoine

[AntSimi]

If you have feedback about documentation and/or tools from py eddy tracker , i am interested!

[koldunovn]

I think it's very wise not to try to deal with time, it's always a way to hell :)

I hope to use your code more over the next few days and will try to report if something in the docs will be unclear :)

I think this issue can be closed.