Fig. 1. Illustration of 3-D volume reconstruction through recursive light propagation. Due to the spatial invariance of light propagation function with respect to the axial location of the image, an input image $I_0$ can be recursively fed to the light propagation function for a unit distance (i.e., $f_{\Delta }\left(\cdot \right)$) to obtain the image at other axial locations. Similarly, the recursive light propagation network for unit distance propagation can be applied recursively to input images to infer 3-D volumes.

Fig. 2. (a) RLP-Net architecture. The network consists of 8 residual and dense blocks (RDBs) and a memory module (MM) in the middle. It takes two adjacent image planes and produces the subsequent plane in a single step. The hidden state of the MM (which is a Conv-LSTM) is used for the inference in the next axial step. (b) 3-D virtual refocusing with RLP-Net. The 3-D volume is reconstructed through recursive inferences using two adjacent images. After the recursive inference, the input images $I_0$ and $I_1$ are swapped to perform the inference towards the other side (as the concatenation order of two inputs determines the inference direction). (c) RLP-Net training procedure. The loss function is evaluated based on the inference results where the number of recursive inference steps is gradually increased so that the network can progressively learn to propagate the image further. The cycle loss was introduced to exploit the additive property of the light propagation function.

Fig. 3. Schematic of the custom-designed microscope used for imaging larval zebrafish brain. An electrically tunable lens (ETL) is conjugated to the back pupil plane of the objective lens. Rapid axial scanning can be performed by electrically modulating the focal length of the ETL.

Fig. 4. Virtual refocusing of fluorescent beads. (a) Two adjacent images applied as the input. (b) Oblique maximum intensity projection (MIP) of the reconstructed volume obtained with RLP-Net. (c) As in b, but for the ground truth image. (d) Lateral MIP (top) and axial MIP (bottom) of reconstructed volume. (e) As in d, but for the ground truth image. Scale bars, 50 $\mathrm{\mu }$m.

Fig. 5. Virtual refocusing of C. elegans images. (a) Two adjacent images used as the input. Each region-of-interest shows head ganglia, body, and tail ganglia, respectively. (b) MIPs of the reconstructed volume using RLP-Net (left) and the ground-truth (right). In each panel, lateral ($xy$) MIP (top-left), axial ($xz$) MIP (bottom-left), and axial ($yz$) MIP (right) are shown. (c) The ground truth images (1st row) are compared with the refocusing results from Deep-Z (2nd row) and RLP-Net (3rd row). Different neurons are in focus at different axial locations. Scale bars, 20 $\mathrm{\mu }$m.

Fig. 6. Virtual refocusing of the images of a larval zebrafish brain. (a) Two adjacent images used as the input to RLP-Net. (b) Refocusing results obtained using RLP-Net (top) and the corresponding ground truth (bottom) at multiple axial locations: $z$ = 30, −30, 60, −60 $\mathrm{\mu }$m. The cell bodies of different sets of neurons are in focus at different axial locations in the boxed area. Scale bar, 40 $\mathrm{\mu }$m.

Fig. 7. MIPs of the volume images of a larval zebrafish brain: (left) RLP-Net output, (right) ground truth. The volumes were deconvolved using Richardson–Lucy iteration. For volume reconstruction, virtual refocusing was performed over 230 $\mathrm{\mu }$m of axial range. Scale bar, 40 $\mathrm{\mu }$m.