Background: 

Deformable registration is required to generate a time-integrated activity (TIA) map which is essential for voxel-based dosimetry. The conventional iterative registration algorithm using anatomical images (e.g., computed tomography (CT)) could result in registration errors in functional images (e.g., single photon emission computed tomography (SPECT) or positron emission tomography (PET)). Various deep learning-based registration tools have been proposed, but studies specifically focused on the registration of serial hybrid images were not found.

Purpose: 

In this study, we introduce CoRX-NET, a novel unsupervised deep learning network designed for deformable registration of hybrid medical images. The CoRX-NET structure is based on the Swin-transformer (ST), allowing for the representation of complex spatial connections in images. Its self-attention mechanism aids in the effective exchange and integration of information across diverse image regions. To augment the amalgamation of SPECT and CT features, cross-stitch layers have been integrated into the network.

Methods: 

Two different ¹⁷⁷ Lu DOTATATE SPECT/CT datasets were acquired at different medical centers. 22 sets from Seoul National University and 14 sets from Sunway Medical Centre are used for training/internal validation and external validation respectively. The CoRX-NET architecture builds upon the ST, enabling the modeling of intricate spatial relationships within images. To further enhance the fusion of SPECT and CT features, cross-stitch layers have been incorporated within the network. The network takes a pair of SPECT/CT images (e.g., fixed and moving images) and generates a deformed SPECT/CT image. The performance of the network was compared with Elastix and TransMorph using L1 loss and structural similarity index measure (SSIM) of CT, SSIM of normalized SPECT, and local normalized cross correlation (LNCC) of SPECT as metrics. The voxel-wise root mean square errors (RMSE) of TIA were compared among the different methods.

Results: 

The ablation study revealed that cross-stitch layers improved SPECT/CT registration performance. The cross-stitch layers notably enhance SSIM (internal validation: 0.9614 vs. 0.9653, external validation: 0.9159 vs. 0.9189) and LNCC of normalized SPECT images (internal validation: 0.7512 vs. 0.7670, external validation: 0.8027 vs. 0.8027). CoRX-NET with the cross-stitch layer achieved superior performance metrics compared to Elastix and TransMorph, except for CT SSIM in the external dataset. When qualitatively analyzed for both internal and external validation cases, CoRX-NET consistently demonstrated superior SPECT registration results. In addition, CoRX-NET accomplished SPECT/CT image registration in less than 6 s, whereas Elastix required approximately 50 s using the same PC's CPU. When employing CoRX-NET, it was observed that the voxel-wise RMSE values for TIA were approximately 27% lower for the kidney and 33% lower for the tumor, compared to when Elastix was used.

Conclusion: 

This study represents a major advancement in achieving precise SPECT/CT registration using an unsupervised deep learning network. It outperforms conventional methods like Elastix and TransMorph, reducing uncertainties in TIA maps for more accurate dose assessments.

1. Zhang, J., Song, Q., Cai, L., Xie, Y., & Chen, Y. (2020). The efficacy of Lu-DOTATATE peptide receptor radionuclide therapy (PRRT) in patients with metastatic neuroendocrine tumours: A systematic review and meta-analysis. Journal of Cancer Research and Clinical Oncology, 146(6), 1533–1543.
2. Sandström, M., Garske-Román, U., Granberg, D., Johansson, S., Widström, C., Eriksson, B., ... & Lundqvist, H. (2013). Individualized dosimetry of kidney and bone marrow in patients undergoing $^{177}\text{Lu}$-DOTA-octreotate treatment. Journal of Nuclear Medicine, 54(1), 33–41.
3. Hagmarker, L., Svensson, J., Rydén, T., Van Essen, M., Bernhardt, P., & Gjertsson, P. (2019). Bone marrow absorbed doses and correlations with hematologic response during $^{177}\text{Lu}$-DOTATATE treatments are influenced by image-based dosimetry method and presence of skeletal metastases. Journal of Nuclear Medicine, 60(10), 1406–1413.
4. Roth, D., Gustafsson, J., Warfvinge, C. F., Sundlöv, A., Åkesson, A., Tennvall, J., & Gleisner, K. S. (2022). Dosimetric quantities in neuroendocrine tumors over treatment cycles with Lu-DOTATATE. Journal of Nuclear Medicine, 63(3), 399–405.
5. Dewaraja, Y. K., Frey, E. C., Sgouros, G., Brill, A. B., Roberson, P., Zanzonico, P. B., ... & Ljungberg, M. (2012). MIRD pamphlet No. 23: Quantitative SPECT for patient-specific 3-dimensional dosimetry in internal radionuclide therapy. Journal of Nuclear Medicine, 53(8), 1310–1325.
6. Ljungberg, M., Celler, A., Konijnenberg, M. W., Eckerman, K. F., Dewaraja, Y. K., Sgouros, G., ... & SNMMI MIRD Committee. (2016). MIRD pamphlet no. 26: Joint EANM/MIRD guidelines for quantitative $^{177}\text{Lu}$ SPECT applied for dosimetry of radiopharmaceutical therapy. Journal of Nuclear Medicine, 57(1), 151–162.
7. Sarrut, D., Halty, A., Badel, J. N., Ferrer, L., & Bardiès, M. (2017). Voxel-based multimodel fitting method for modeling time activity curves in SPECT images. Medical Physics, 44(12), 6280–6288.
8. Papavasileiou, P., Divoli, A., Hatziioannou, K., & Flux, G. D. (2007). The importance of the accuracy of image registration of SPECT images for 3D targeted radionuclide therapy dosimetry. Physics in Medicine & Biology, 52(24), N539–N548.
9. He, B., & Frey, E. C. (2010). The impact of 3D volume of interest definition on accuracy and precision of activity estimation in quantitative SPECT and planar processing methods. Physics in Medicine & Biology, 55(12), 3535–3544.
10. Mok, G. S. P., & Dewaraja, Y. K. (2021). Recent advances in voxel-based targeted radionuclide therapy dosimetry. Quantitative Imaging in Medicine and Surgery, 11(2), 483–489.
11. Lyu, Y. Q., Chen, G. L., Lu, Z. L., Chen, Y., & Mok, G. S. P. (2023). The effects of mismatch between SPECT and CT images on quantitative activity estimation—A simulation study. Zeitschrift für Medizinische Physik, 33(1), 54–69.
12. Yang, D., Li, H., Low, A. D., Deasy, J. O., & El Naqa, I. (2008). A fast inverse consistent deformable image registration method based on symmetric optical flow computation. Physics in Medicine & Biology, 53(21), 6143–6165.
13. Vercauteren, T., Pennec, X., Perchant, A., & Ayache, N. (2009). Diffeomorphic demons: Efficient non-parametric image registration. NeuroImage, 45(1), S61–S72.
14. Klein, S., Staring, M., Murphy, K., Viergever, M. A., & Pluim, J. P. (2010). Elastix: A toolbox for intensity-based medical image registration. IEEE Transactions on Medical Imaging, 29(1), 196–205.
15. Yang, X., Kwitt, R., Styner, M., & Niethammer, M. (2017). Quicksilver: Fast predictive image registration—A deep learning approach. NeuroImage, 158, 378–396.
16. Lv, J., Yang, M., Zhang, J., & Wang, X. (2018). Respiratory motion correction for free-breathing 3D abdominal MRI using CNN-based image registration: A feasibility study. The British Journal of Radiology, 91(1083), Article 20170788.
17. Eppenhof, K. A. J., & Pluim, J. P. W. (2019). Pulmonary CT registration through supervised learning with convolutional neural networks. IEEE Transactions on Medical Imaging, 38(5), 1097–1105.
18. Jaderberg, M., Simonyan, K., Zisserman, A., & Kavukcuoglu, K. (2015). Spatial transformer networks. Advances in Neural Information Processing Systems, 28, 2017–2025.
19. de Vos, B. D., Berendsen, F. F., Viergever, M. A., Staring, M., & Išgum, I. (2017). End-to-end unsupervised deformable image registration with a convolutional neural network. In International Workshop on Biomedical Image Registration (pp. 204–212). Springer, Cham.
20. Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2019). VoxelMorph: A learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 38(8), 1788–1800.
21. Yoo, I., Hildebrand, D. G. C., Tobin, W. F., Lee, W. C. A., & Jeong, W. K. (2017). ssEMnet: Serial-section electron microscopy image registration using a spatial transformer network with learned features. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 249–257). Springer, Cham.
22. Mocanu, S., Moody, A. R., & Khademi, A. (2021). Flowreg: Fast deformable unsupervised medical image registration using optical flow. arXiv preprint arXiv:2101.09639.
23. O’Briain, T., Uribe, C., Yi, K. M., Teuwen, J., Sechopoulos, I., & Bazalova-Carter, M. (2022). FlowNet-PET: Unsupervised learning to perform respiratory motion correction in PET imaging. arXiv preprint arXiv:2205.14147.
24. Chen, J., Frey, E. C., He, Y., Segars, W. P., Li, Y., & Du, Y. (2022). TransMorph: Transformer for unsupervised medical image registration. Medical Image Analysis, 82, Article 102615.
25. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., ... & Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 9992–10002).
26. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., ... & Cao, Y. (2022). Swin transformer V2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11999–12009).
27. Misra, I., Shrivastava, A., Gupta, A., & Hebert, M. (2016). Cross-stitch networks for multi-task learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3994–4003).
28. Elmahdy, M. S., Beljaards, L., Yousefi, S., Sokooti, H., Verbeek, F., & Staring, M. (2021). Joint registration and segmentation via multi-task learning for adaptive radiotherapy of prostate cancer. IEEE Access, 9, 95551–95568.
29. Beljaards, L., Elmahdy, M. S., Verbeek, F., & Staring, M. (2020). A cross-stitch architecture for joint registration and segmentation in adaptive radiotherapy. In Medical Imaging with Deep Learning (pp. 62–74). PMLR.
30. Avants, B. B., Epstein, C. L., Grossman, M., & Gee, J. C. (2008). Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis, 12(1), 26–41.
31. Tran-Gia, J., Salas-Ramirez, M., & Lassmann, M. (2020). What you see is not what you get: On the accuracy of voxel-based dosimetry in molecular radiotherapy. Journal of Nuclear Medicine, 61(8), 1178–1186.
32. Brosch-Lenz, J., Ke, S., Wang, H., Uribe, C., Shcherbinin, S., Celler, A., ... & Saboury, B. (2023). An international study of factors affecting variability of dosimetry calculations, part 2: Overall variabilities in absorbed dose. Journal of Nuclear Medicine, 64(7), 1109–1116.
33. Gustafsson, J., Brolin, G., Cox, M., Ljungberg, M., Johansson, L., & Gleisner, K. S. (2015). Uncertainty propagation for SPECT/CT-based renal dosimetry in $^{177}\text{Lu}$ peptide receptor radionuclide therapy. Physics in Medicine & Biology, 60(21), 8329–8346.
34. Sarrut, D., Halty, A., Badel, J. N., Ferrer, L., & Bardiès, M. (2017). Voxel-based multimodel fitting method for modeling time activity curves in SPECT images. Medical Physics, 44(12), 6280–6288.
35. Peters, S. M. B., Mink, M. C. T., Privé, B. M., de Bakker, M., de Lange, F., de Geus-Oei, L. F., ... & Konijnenberg, M. W. (2023). Optimization of the radiation dosimetry protocol in Lutetium-177-PSMA therapy: Toward clinical implementation. EJNMMI Research, 13(1), 1–10.
36. Finocchiaro, D., Gear, J. I., Fioroni, F., Flux, G. D., Iori, M., & Spezi, E. (2020). Uncertainty analysis of tumour absorbed dose calculations in molecular radiotherapy. EJNMMI Physics, 7(1), Article 63.
37. Mora-Ramirez, E., Santoro, L., Cassol, E., Chouaf, S., Trauchessec, D., Malibert, B., ... & Bardiès, M. (2020). Comparison of commercial dosimetric software platforms in patients treated with $^{177}\text{Lu}$-DOTATATE for peptide receptor radionuclide therapy. Medical Physics, 47(9), 4602–4615.
38. Ao, E. C., Wu, N. Y., Wang, S. J., Song, N., & Mok, G. S. (2015). Improved dosimetry for targeted radionuclide therapy using nonrigid registration on sequential SPECT images. Medical Physics, 42(2), 1060–1070.
39. Yue, J. T., Mauxion, T., Reyes, D. K., Lodge, M. A., Hobbs, R. F., Roundhill, E. P., ... & Dewaraja, Y. K. (2016). Comparison of quantitative $^{90}\text{Y}$ SPECT and non-time-of-flight PET imaging in post-therapy radioembolization of liver cancer. Medical Physics, 43(10), 5779–5790.