add ucmc track

JackWoo0831 · JackWoo0831 · commit 995813cf81e3 · 2024-09-16T20:31:55.000+08:00
diff --git a/README.md b/README.md
@@ -37,6 +37,7 @@ and the tracker supports:
 - C_BIoU Track ([arxiv2211](https://arxiv.org/pdf/2211.14317v2.pdf))
 - Strong SORT ([IEEE TMM 2023](https://arxiv.org/pdf/2202.13514))
 - Sparse Track ([arxiv 2306](https://arxiv.org/pdf/2306.05238))
+- UCMC Track ([AAAI 2024](http://arxiv.org/abs/2312.08952))
 
 and the reid model supports:
 
@@ -52,9 +53,7 @@ The highlights are:
 
 ## 🗺️ Roadmap
 
-- [ x ] Add StrongSort and SparseTrack
-- [ x ] Add save video function
-- [ x ] Add timer function to calculate fps
+- [ x ] Add UCMC Track
 - [] Add more ReID modules.
 
 ##  🔨 Installation
@@ -189,6 +188,15 @@ For example:
 
 - Sparse Track: `python tracker/track.py --dataset uavdt --detector yolov8 --tracker sparsetrack --kalman_format bot --detector_model_path weights/yolov8l_UAVDT_60epochs_20230509.pt`
 
+- UCMC Track: `python tracker/track.py --dataset mot17 --detector yolox --tracker ucmctrack --kalman_format ucmc --detector_model_path weights/bytetrack_m_mot17.pth.tar --camera_parameter_folder ./tracker/cam_param_files`
+
+> **Important notes for UCMC Track:**
+> 
+> 1. Camera parameters. The UCMC Track need the intrinsic and extrinsic parameter of camera. Please organize like the format of `tracker/cam_param_files/uavdt/M0101.txt`. One video sequence corresponds to one txt file. If you do not have the labelled parameters, you can refer to the estimating toolbox in original repo ([https://github.com/corfyi/UCMCTrack](https://github.com/corfyi/UCMCTrack)).
+> 
+> 2. The code does not contain the camera motion compensation part between every two frame, please refer to [https://github.com/corfyi/UCMCTrack/issues/12](https://github.com/corfyi/UCMCTrack/issues/12). From my perspective, since the algorithm name is 'uniform', the update of compensation between every two frames is not necessary.
+
+
 ### ✅ Evaluation 
 
 Coming Soon. As an alternative, after obtaining the result txt file, you can use the [Easier to use TrackEval repo](https://github.com/JackWoo0831/Easier_To_Use_TrackEval).
diff --git a/README_CN.md b/README_CN.md
@@ -33,6 +33,7 @@ git checkout v2  # change to v2 branch !!
 - C_BIoU Track ([arxiv2211](https://arxiv.org/pdf/2211.14317v2.pdf))
 - Strong SORT ([IEEE TMM 2023](https://arxiv.org/pdf/2202.13514))
 - Sparse Track ([arxiv 2306](https://arxiv.org/pdf/2306.05238))
+- UCMC Track ([AAAI 2024](http://arxiv.org/abs/2312.08952))
 
 REID模型支持：
 
@@ -48,10 +49,8 @@ REID模型支持：
 
 ## 🗺️ 路线图
 
-- [ x ] Add StrongSort and SparseTrack
-- [ x ] Add save video function
-- [ x ] Add timer function to calculate fps
-- [] Add more ReID modules.mer function to calculate fps
+- [ x ] Add UCMC Track
+- [] Add more ReID modules.
 
 ##  🔨 安装
 
@@ -181,6 +180,14 @@ python tracker/track.py --dataset ${dataset name, related with the yaml file} --
 
 - BoT-SORT: `python tracker/track.py --dataset uavdt --detector yolox --tracker botsort --kalman_format bot --detector_model_path weights/yolox_m_uavdt_50epochs.pth.tar`
 
+- UCMC Track: `python tracker/track.py --dataset mot17 --detector yolox --tracker ucmctrack --kalman_format ucmc --detector_model_path weights/bytetrack_m_mot17.pth.tar --camera_parameter_folder ./tracker/cam_param_files`
+
+>**UCMC Track的重要提示：**
+> 
+> 1. 相机参数. UCMC Track需要相机的内参和外参. 请按照`tracker/cam_ram_files/uavdt/M0101.txt`的格式组织. 一个视频序列对应一个txt文件. 如果您没有标记的参数, 可以参考原始仓库中的估算工具箱([https://github.com/corfyi/UCMCTrack](https://github.com/corfyi/UCMCTrack)).
+> 
+> 2. 该代码不包含每两帧之间的相机运动补偿部分, 请参阅[https://github.com/corfyi/UCMCTrack/issues/12](https://github.com/corfyi/UCMCTrack/issues/12). 在我看来, 既然算法叫"统一相机运动补偿", 因此不需要每两帧之间再更新补偿. 
+
 ### ✅ 评估
 
 马上推出！作为备选项，你可以使用这个repo： [Easier to use TrackEval repo](https://github.com/JackWoo0831/Easier_To_Use_TrackEval).
diff --git a/tracker/cam_param_files/uavdt/M0101.txt b/tracker/cam_param_files/uavdt/M0101.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.8979905814549864 -0.39754048571901224 0.18861197690610634 
+-0.11475152919221074 -0.6253973983846266 -0.7718226354816174 
+0.4247881850595499 0.6714459444423146 -0.607219352067917 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M0202.txt b/tracker/cam_param_files/uavdt/M0202.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.9032157300195275 -0.42859525452700087 0.02252671396874304 
+-0.19012553250747607 -0.44662008618260757 -0.8742898721287973 
+0.3847773732095512 0.7853894616145389 -0.4848811881799312 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M0203.txt b/tracker/cam_param_files/uavdt/M0203.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.9184529838275687 -0.3941594264594937 -0.032900805937865915 
+-0.33168018518486475 -0.7222023632569826 -0.6069695225147331 
+0.21548171907141328 0.5683855144522528 -0.7940437870211415 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M0402.txt b/tracker/cam_param_files/uavdt/M0402.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.9900225296679271 0.13563763189861872 0.03818145574562663 
+0.09507946439276654 -0.4430504618529409 -0.8914405104675692 
+-0.10399656821599908 0.8861764615835999 -0.45152629240661585 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M0601.txt b/tracker/cam_param_files/uavdt/M0601.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.9965360219425241 0.07908685280531619 -0.025714328385584462 
+0.009986822246025843 -0.42077701740785656 -0.9071091251898942 
+-0.08256040426498443 0.9037101146577359 -0.4201092813934298 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+725 0 512 
+0 725 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M0604.txt b/tracker/cam_param_files/uavdt/M0604.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.999783748579928 -0.008142098538121654 -0.019135367957401395 
+-0.020538893739341778 -0.242485078238191 -0.9699376993785647 
+0.003257287127912365 0.9701209682628312 -0.24259987019236565 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M1002.txt b/tracker/cam_param_files/uavdt/M1002.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.958995837118094 0.2784639208312455 -0.05277148079653312 
+0.0558475988637905 -0.3682090522307261 -0.9280641893514154 
+-0.2778633298787693 0.887062533675411 -0.36866248962708675 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M1003.txt b/tracker/cam_param_files/uavdt/M1003.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.997881807190502 -0.009330599286301164 -0.06438042245263502 
+-0.058971643889622644 -0.5475526162640636 -0.8346906478686194 
+-0.027463504786840724 0.8367192314864867 -0.5469430350278591 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M1008.txt b/tracker/cam_param_files/uavdt/M1008.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.9941156935168154 0.06039358449950929 -0.08992554061491932 
+-0.036689756467594016 -0.5933614281160473 -0.8040995444560548 
+-0.10192080098749955 0.8026673224788619 -0.5876540817102196 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/cam_param_files/uavdt/M1202.txt b/tracker/cam_param_files/uavdt/M1202.txt
@@ -0,0 +1,12 @@
+RotationMatrices
+0.9993294906377356 -0.012829960883434719 -0.03429229134160642 
+-0.03522538794712723 -0.5923724138185908 -0.80489384106894 
+-0.009987050901165704 0.8055621114789755 -0.592427162918643 
+
+TranslationVectors
+0 1000 33640 
+
+IntrinsicMatrix
+1000 0 512 
+0 1000 270 
+0 0 1 
diff --git a/tracker/track.py b/tracker/track.py
@@ -28,6 +28,7 @@
 from trackers.deepsort_tracker import DeepSortTracker
 from trackers.strongsort_tracker import StrongSortTracker
 from trackers.sparse_tracker import SparseTracker
+from trackers.ucmc_tracker import UCMCTracker
 
 # YOLOX modules
 try:
@@ -70,7 +71,8 @@
     'ocsort': OCSortTracker, 
     'deepsort': DeepSortTracker, 
     'strongsort': StrongSortTracker, 
-    'sparsetrack': SparseTracker
+    'sparsetrack': SparseTracker, 
+    'ucmctrack': UCMCTracker, 
 }
 
 def get_args():
@@ -115,6 +117,9 @@ def get_args():
     
     parser.add_argument('--track_eval', type=bool, default=True, help='Use TrackEval to evaluate')
 
+    """camera parameter"""
+    parser.add_argument('--camera_parameter_folder', type=str, default='./tracker/cam_param_files', help='folder path of camera parameter files')
+
     return parser.parse_args()
 
 def main(args, dataset_cfgs):
@@ -204,6 +209,9 @@ def main(args, dataset_cfgs):
 
         data_loader = torch.utils.data.DataLoader(dataset, batch_size=1, shuffle=False)
 
+        # store the seq name, for conveniently reading the camera param file w.r.t. each sequence
+        args.cam_param_file = os.path.join(args.camera_parameter_folder, args.dataset, seq + '.txt')  
+
         tracker = TRACKER_DICT[args.tracker](args, )
 
         process_bar = enumerate(data_loader)
diff --git a/tracker/trackers/kalman_filters/ucmctrack_kalman.py b/tracker/trackers/kalman_filters/ucmctrack_kalman.py
@@ -2,26 +2,60 @@
 import numpy as np 
 
 class UCMCKalman(BaseKalman):
-    def __init__(self, ):
+    def __init__(self, 
+                 sigma_x=1.0,  # noise factor in x axis (Eq. 11)
+                 sigma_y=1.0,  # noise factor in y axis (Eq. 11)
+                 vmax=1.0,  # TODO
+                 dt=1/30,  # interval between frames
+                 **kwargs
+                 ):
 
-        state_dim = 8
-        observation_dim = 4 
+        state_dim = 4  # [x, dx, y, dy]  where x, y is the corr on ground
+        observation_dim = 2   # [x, y]
 
-        F = np.eye(state_dim, state_dim)
-        '''
-        [1, 0, 0, 0, 1, 0, 0]
-        [0, 1, 0, 0, 0, 1, 0]
-        ...
-        '''
-        for i in range(state_dim // 2):
-            F[i, i + state_dim // 2] = 1
+        F = np.array([[1, dt, 0, 0], 
+                      [0, 1, 0, 0], 
+                      [0, 0, 1, dt], 
+                      [0, 0, 0, 1]])
 
-        H = np.eye(state_dim // 2, state_dim)
+        H = np.array([[1, 0, 0, 0], 
+                      [0, 0, 1, 0]])
+        
+        P = np.array([[1, 0, 0, 0], 
+                      [0, vmax**2 / 3.0, 0, 0], 
+                      [0, 0, 1, 0], 
+                      [0, 0, 0, vmax**2 / 3.0]])
+        
+        # noise compensation to initialize Q by Eq. 10 and 11
+        G = np.array([[0.5*dt*dt, 0], 
+                      [dt, 0], 
+                      [0, 0.5*dt*dt], 
+                      [0, dt]])
+        Q0 = np.array([[sigma_x, 0], [0, sigma_y]])
+        Q = np.dot(np.dot(G, Q0), G.T)
     
         super().__init__(state_dim=state_dim, 
                        observation_dim=observation_dim, 
                        F=F, 
-                       H=H)
+                       H=H, 
+                       P=P,
+                       Q=Q)
+        
+    def initialize(self, observation, R):
+        """
+        observation: [x, y]  where x y is the grounding coordinate
+        R: the cov matrix of observation (2, 2)
+        """
         
-        self._std_weight_position = 1. / 20
-        self._std_weight_velocity = 1. / 160
+        self.kf.x[0] = observation[0]
+        self.kf.x[1] = 0
+        self.kf.x[2] = observation[1]
+        self.kf.x[3] = 0
+
+        self.kf.R = R
+
+    def predict(self):
+        self.kf.predict()
+
+    def update(self, z, R):
+        self.kf.update(z=z, R=R)
diff --git a/tracker/trackers/sparse_tracker.py b/tracker/trackers/sparse_tracker.py
@@ -1,5 +1,5 @@
 """
-Bot sort
+Sparse Track
 """
 
 import numpy as np  
diff --git a/tracker/trackers/strongsort_tracker.py b/tracker/trackers/strongsort_tracker.py
@@ -1,5 +1,5 @@
 """
-Deep Sort
+Strong Sort
 """
 
 import numpy as np  
diff --git a/tracker/trackers/tracklet.py b/tracker/trackers/tracklet.py
diff --git a/tracker/trackers/ucmc_tracker.py b/tracker/trackers/ucmc_tracker.py
