How to build the ‘optical inverse’ of a multimode ﬁbre

When light propagates through a multimode optical ﬁbre (MMF), the spatial information it carries is scrambled. Wavefront shaping can undo this scrambling, typically one spatial mode at a time – enabling deployment of MMFs as ultra-thin micro-endoscopes. In this work we go beyond serial wavefront shaping by showing how to simultaneously unscramble all spatial modes emerging from an MMF in parallel. We introduce a passive multiple-scattering element – crafted through the process of inverse design – that is complementary to an MMF and undoes its optical effects. This ‘optical inverter’ makes possible both single-shot wide-ﬁeld imaging and super-resolution imaging through MMFs. Our design consists of a cascade of diffractive elements, and can be understood from the perspective of both multi-plane light conversion , and as a physically inspired deep diffractive neural network . This physical architecture can outperform state-of-the-art electronic neural networks tasked with unscrambling light, as it preserves the phase and coherence information of the optical signals ﬂow-ing through it. Here we demonstrate our MMF inversion concept through numerical simulations, and efﬁciently sort and unscramble up to ∼ 400 step-index ﬁbre modes, reforming incoherent images of scenes at arbitrary dis-tances from the distal ﬁbre facet. We also describe how our optical inverter can dynamically adapt to see through ﬂexible ﬁbres with a range of experimentally realistic TMs – made possible by moulding optical memory effects into the structure of our design. Although complex, our inversion scheme is based on current fabrication technology so could be realised in the near future. Beyond imaging through scattering media, these concepts open up a range of new avenues for optical multiplexing, communications and computation in the realms of classical and quantum photonics. we phase mode. is a Hadamard product by element between each input mode and the conjugate of the corresponding output mode. All of the overlaps are then averaged, and the argument is taken. The resulting phase proﬁle is then added to the current phase proﬁle of the mask between where the modes have met, a new mask which is used in the following iterations. This processes is repeated to optimise each phase the


Introduction
Multimode optical fibres (MMFs) are a ubiquitous technology in light guiding applications [1]. They form the backbone of the short-range optical links in data centres [2,3], transmit optical signals in spectrometers [4], and deliver high intensity light in laser welding systems [5]. MMFs support thousands of spatial modes-i.e., unique optical field profiles-within the width of a strand of hair, each mode representing an independent communication channel. There has been much interest in unlocking this high spatial information density for imaging and optical communications applications [6]. For instance, recently MMFs have emerged as a promising technology for microendoscopy-enabling imaging with subcellular resolution at the tip of a needle [7,8]. In this case, each spatial mode is capable of conveying a different subset of the spatial information from a scene at the distal (far) end of the fibre. Unfortunately, spatial mode dispersion scrambles this information as it propagates along an MMF. Therefore optical signals must be unscrambled at the proximal (near) end in order to reconstruct images [9][10][11][12].
The current "gold-standard" method to overcome optical scrambling is via measurement of the fibre's transmission matrix (TM): a linear matrix operator that describes how any monochromatic field will have been transformed upon propagating through the fibre [13]. Once measured, the TM can be used to calculate a sequence of prescrambled input fields which are sent from the proximal end to generate a scanning focus at the distal facet. Thus scanning imaging is possible by correlating the return signal with each known focused scan location. This method, known as wavefront shaping [14][15][16], has been used to look through MMFs centimetres deep inside living tissue [7,8]-a task difficult to accomplish in any other such minimally invasive way. A wide variety of imaging modalities have been demonstrated through these MMF-based holographic endoscopes, including fluorescence imaging [7,8,11], chemically and structurally sensitive microscopy [17][18][19], imaging beyond the distal fibre facet [20], and depth ranging using time-of-flight LiDAR [21].
The majority of TM-based imaging techniques demonstrated so far rely on raster-scanning or sequential pattern projection, essentially meaning that light is unscrambled one spatial mode at a time [16]. This currently precludes the delivery of widefield imaging techniques through MMFs. For example, there is no way to conduct super-resolution imaging modalities such as photo-activated localisation microscopy (PALM) [22] or stochastic optical reconstruction microscopy (STORM) [23] at the tip of an MMF. Scanning also places limits on the maximum imaging frame-rate, which is governed by the update rate of the spatial light modulator (SLM) used to shape input optical fields.
Here we propose a new imaging strategy that goes beyond serial wavefront shaping, and simultaneously unscrambles all spatial modes emerging from a fibre in parallel. To achieve this, we describe the design of a passive optical element, which we term an optical inverter -that, in the ideal case, possesses a TM exactly equivalent to the inverse of the MMF's TM (D). Coupled with the MMF, the TM of the entire system collapses to the identity matrix I, since D −1 D = I. Therefore, any coherent, partially-coherent or incoherent field at the distal end of the fibre, within the spatial bandwidth of the MMF and the spectral bandwidth of the combined fibre-inverter system, will be reconstructed in an all-optical manner at the output of the optical inverter. This advance brings closer the vision of channeling any form of widefield microscopy to the tip of an MMF endoscope positioned deep inside living tissue. Figure 1 shows a schematic of our concept.
The prospect of building a device to passively unscramble light that has propagated through an MMF was first considered by Gover et al. as long ago as 1976 [24]. Yet, only recently has our understanding of the inverse design of multimodal photonic systems matured to a level that renders this concept feasible. In 2012, Čižmár and Dholakia took a step forward by demonstrating that a single multiplexed hologram can be used to unscramble up to~100 spatial modes that have propagated through an MMF [11]. However, this approach does not reform images directly, and suffers from exceedingly low conversion efficiency which has restricted its broader application [11,25,26].
In the following, we first describe the working principle of an optical inverter in the case of an ideal MMF, followed by the specifics of our design. We numerically simulate the performance of a system capable of efficiently unscrambling up to~400 spatial modes. Next, we show how this concept can be extended to the more experimentally realistic fibres, leveraging recently discovered fibre memory effects [27,28]. Finally, we describe how our inverter may be designed to dynamically adapt to the changing TM of a moving fibre.

Design
2.1. Architecture of an Optical Inverter. MMFs can be considered a form of opaque scattering media dominated by forward scattering; the vast majority of incident light is transmitted and levels of back-reflection are very low. There-fore, here we focus solely on inversion of the TM of an MMF rather than considering its full scattering matrix. We begin by analysing the structure of the TM of an ideal MMF. Solving the monochromatic wave equation in cylindrical coordinates yields a set of N circularly polarised propagation invariant modes (PIMs, also referred to here as "fibre modes"). These are orthonormal eigenmodes of the fibre, maintaining the same field profile as they propagate along it without coupling to one another or changing polarisation state. Each PIM has a mode-dependent phase velocity β n (where n indexes the mode), and so mode n picks up a global phase delay of θ n = β n L upon reaching the output of a fibre of length L. The dephasing of the PIMs causes their interference pattern at the output to be different from that at the input, resulting in the spatial scrambling of optical signals. Therefore, the TM of an MMF, T fib ∈ ℂ P×P , represented in the real-space basis, is given by T fib = PDP † -see Figure 1(b). Here, D ∈ ℂ N×N is a unitary diagonal matrix capturing the phase delay experienced by all N PIMs supported by the fibre, P ∈ ℂ P×N is a matrix transforming from the N-dimensional PIM-space to the P-dimensional pixelated real-space basis, and † represents the conjugate transpose operator. References [28,29] provide details of how matrix P may be constructed.
To unscramble monochromatic light that has propagated through an ideal MMF, we must undo the modal phase delays picked up during transit, by imparting a mode-dependent phase correction of ϕ n = −θ n = −β n L to each PIM. Thus, in matrix form, the TM of the optical inverter, also represented in real-space, should correspond to such that the TM of the combined fibre-inverter system T com ∈ ℂ P×P is given by T com = T inv T fib = PP † , as shown in Figure 1(b). T com is approximately diagonal, but does not equal the identity matrix because it also captures the spatial filtering due to the fibre itself, i.e., only the components of the field emanating from the object that are supported by the fibre can propagate through the system. However, when transformed to PIM-space, the combined system does indeed collapse to the identity matrix: Optical fields emerging from a fibre are readily addressable in real-space-it is straightforward to impart a spatially dependent phase delay directly to these fields. However, an optical inverter must act nonlocally on each PIM independently-and so physically accessing PIM-space is the key challenge to overcome in our inverter design. We achieve this by showing how to construct optical systems that perform each of the three matrix transformations on the right-handside of Equation (1).
First, light emerging from an MMF is decomposed into fibre modes-this can be accomplished using a passive "fibre mode sorter", the novel design of which we describe below, that redirects and focuses the energy carried by each PIM into a different spatial location across a transverse plane. This 2 Intelligent Computing step is equivalent to the transformation enacted by matrix P † in Equation (1). Next, the light from each sorted PIM passes through a single optical element, which we call the "corrector plane", that imparts mode-dependent phase delays to undo those accumulated during transit through the fibre. This step achieves the diagonal matrix operation D † in Equation (1). Finally, the fibre modes are spatially recombined by propagating through a second fibre mode sorter in the reverse direction, enacting matrix transformation P in Equation (1). Henceforth, we refer to this reverse mode sorter as a "fibre mode combiner". Together, the sorter, the corrector, and the combiner form the fibre inverter, as illustrated in Figure 1(a). The number of PIMs of one circular polarisation supported by an MMF is related to its geometry, and given by N~ðπaNA/λÞ 2 , where a is the radius of the core, NA is the numerical aperture of the fibre, and λ is the wavelength of transmitted light. N also governs the number of independent pixels in images that can be transmitted through an MMF. Therefore, we must design a fibre mode sorter capable of handling as many modes as there are pixels in images we wish to reconstruct.

2.2.
Step-Index Fibre Mode Sorter. High-dimensional spatial mode sorting is a challenging task, with techniques to achieve it is still in their infancy. It is closely related to the inverse design of multimodal photonic systems-an emerging class of devices capable of enacting arbitrary transformations on multiple spatial light modes simultaneously. A variety of methods to achieve this are currently under development, including meshes of waveguides on chip [30,31] or the construction of bespoke scattering structures [32][33][34], but few approaches are able to manipulate the numbers of modes required for imaging applications [35]. Here, we take inspiration from recent advances in multi-plane light conversion [36] and diffractive neural networks [37]. Both of these concepts rely on a cascade of numerically optimised diffractive optical elements (DOEs, also referred to here as "phase masks") separated by free-space, to perform targeted optical transformations such as mode sorting [38][39][40].
It has been shown that N spatial modes can be efficiently sorted in M = 6N + 1 planes using multi-plane light conversion [41]. At first, this appears to prohibit such approaches for imaging applications where N is large. For example, sorting of N = 400 modes would require M = 2401 planes using this scaling-clearly a number far exceeding experimental feasibility. Recently, however, Fontaine et al. demonstrated a special case: a multi-plane light converter (MPLC) capable of sorting hundreds of Hermite-Gaussian (HG) modes with very few (~7) phase masks [39,42]. The design exploits underlying symmetries of the Hermite-Gaussian (HG) basis to efficiently sort HG modes. This HG sorter was then coupled to a cylindrical mode transformer-consisting of two cylindrical lenses-that transforms HG modes to Laguerre-Gaussian (LG) modes [43]. The combined optical system thus creates a high-dimensional LG mode sorter using only a few phase planes. Both LG-modes and fibre PIMs possess rotational symmetry, and this structural similarity hints that it may also be feasible to apply the same principle to sort a large number of step-index PIMs. Here, we explore this possibility to design a high-dimensional step-index fibre mode sorter using a low number of planes.
We consider the design of a device operating on a basis of step-index fibre PIMs that form the full set of propagating eigenmodes, of one handedness of circular polarisation, supported by a step-index MMF. In this work, we simulate an MMF of core radius a = 40:3 μm and NA = 0:1, which supports N = 403 modes per polarisation at a wavelength of λ = 633 nm. We emphasize that the following methods are in no way limited to this specific fibre geometry. Inspired by the LG-mode sorter design as described, we first pass all fibre modes through a cylindrical mode transformer. This 3 Intelligent Computing converts the step-index fibre PIMs to a new basis in which each mode is visually reminiscent of a heavily skewed HG mode. We next optimise the phase profiles of a cascade of M = 15 phase masks, tasking the MPLC to simultaneously sort up to 403 of these "skewed" HG modes. The phase profiles of these DOEs are designed using the wavefront matching method [44], which is a form of adjoint optimisation [45], and closely related to the "backpropagation" algorithms used to train electronic neural networks [46,47]. This algorithm allows fast parallelised calculation of how the phase delay imparted by each pixel on a plane should be adjusted to improve the sorting performance. For more details, see Supplementary Information (SI) §1. Figure 2 shows our step-index fibre mode sorter design. Once the profiles of the phase masks are optimised, we do indeed find that the design inherits some of the properties of the MPLC-based LG mode sorter it is inspired by, and operates with relatively high efficiency and low modal cross-talk. Our design represents the first example of a step-index fibre mode sorter capable of supporting enough modes to make imaging feasible-sorting a number of fibre modes approximately an order of magnitude greater than achieved to date [48].
A unique set of design constraints can be applied to mode sorters used for imaging applications. In our case, we aim to maximise the mode capacity N since it is directly proportional to the pixel resolution of transmitted images. This can be achieved at the expense of a reduction in sorting efficiency, and we can also tolerate some degree of modal cross-talk, which will reduce the contrast and fidelity of transmitted images. Exploiting these trade-offs, we design a device capable of sorting N = 403 fibre modes using 15-planes with a mean sorting efficiency of~0.51, an insertion loss (IL) of 0.49, and a mode dependent loss (MDL) of 0.08. Figure 2(c) shows the efficiency with which each individual mode is sorted. We allow a mean cross-talk of~0.23, i.e., on average,~23% of the power in a given input PIM will be spread amongst the other output channels-see SI §1 for the characteristic coupling matrix and relevant definitions. As shown later, these levels of cross-talk, IL, and MDL only result in a mild reduction in imaging contrast when the full optical inverter is implemented (see also SI §7 for more details). In addition to this 15-plane 403 fibre mode sorter, we also designed an 11-plane 201 mode sorter, and a 7plane 107 mode sorter, which exhibit slightly improved performance due to their marginally more favorable ratios of N/M. SI §1 describes the design and performance of these fibre mode sorters in detail. In each case, the output channels are arranged in a triangular grid (see Figure 2(e)), chosen because this geometry was previously demonstrated to guide the inverse design process to a high-performance solution in the HG mode sorting case that inspired our work [39]. Alternative arrangements are also possible, but in our study, we were not able to find one that resulted in better values of cross-talk and efficiency.

Looking through Ideal MMFs.
Equipped with a design for a high-dimensional fibre mode sorter, next we numerically simulate its integration into an optical inverter to unscramble light that has propagated through an MMF. Figure 2(d) shows an example of a pattern of mode-dependent phase shifts applied at the corrector plane. Our 403-mode inverter requires 2M − 1 = 29 phase masks in total. There are 15 planes in the sorter, 15 in the combiner and 1 in the corrector-this makes up 31 planes, but the last plane of the sorter, the corrector plane, and the first plane of the combiner can be arranged so that they all are in image planes of one another and so can be coalesced into a single plane (see SI §3 for how such an arrangement may be achieved in our proposed experimental setup). The real-space coherent TM of our design, T inv ′ ∈ ℂ P×P (the apostrophe distinguishing from the TM of the ideal inverter T inv ), is given by Here, L ∈ ℂ P×P is the TM of the optical system guiding light from the proximal fibre facet, through the cylindrical mode converter, to the input plane of the MPLC. M ∈ ℂ P×P is the TM of the MPLC. C ∈ ℂ P×P is a diagonal unitary matrix representing the TM of the corrector plane, which applies the phase correction ϕ n to the area that light from the n th sorted PIM passes through, along with additional phase corrections to account for mode dependent phase delays imparted to the light during transit through the sorter and combiner themselves (see SI §1 for details). M Τ represents the propagation through the MPLC in the reverse direction, where Τ is the transpose operator. M is given by where A m ∈ ℂ P×P is a unitary diagonal matrix, whose diagonal elements hold the phase change imparted by each pixel of the m th plane. Here we carry out left matrix multiplication, and H ∈ ℂ P×P is a free-space propagation matrix transforming the field from the output facet immediately after plane m to the input facet immediately before plane m + 1. See SI §2 for more details and a description of how matrices L, C, and H are constructed.
It is important to note that although our inverter design is created by simulating the propagation of spatially coherent fields through the system, the resulting device is capable of performing spatially incoherent imaging-within the spectral bandwidth of the system (which is analysed in §3.3). To simulate incoherent imaging, we calculate the real positive-valued incoherent intensity TM S ∈ ℝ P×P + , linking spatially incoherent fields at either end of the combined fibre-inverter system in the real-space basis [49,50]: S = jT inv ′ T fib j 2 , where we take the element-wise square.
Thus far, MPLCs have been built from a multipass cavity consisting of a mirror placed opposite a reflective liquidcrystal SLM or a lithographically etched reflective DOE [39,51]. Using reflective DOEs in this way leads to a mainly 4 Intelligent Computing forward-scattering device, which matches the forward scattering nature of the MMF itself, and avoids noise associated with multiple reflections between planes that would arise in cascades of transmissive DOEs. Here, we envisage that the fibre mode sorter and combiner are constructed from fixed reflective DOEs to promote high efficiency operation. For example, Fontaine et al. recently implemented a 14-plane MPLC composed of a gold-coated diffraction grating opposite a dielectric mirror, that exhibited a loss of~5 − 6 dB [52,53]. We anticipate implementing the corrector plane with a liquid-crystal SLM, which allows the modal correction phases to be dynamically reconfigured. This way, the same optical inverter can be paired with MMFs of standardised NA and core diameter, but of variable length, by simply modifying the correction phases. The ability to reconfigure these correction phases also provides a number of additional benefits when imaging through imperfect or flexible fibres, as will be discussed later. SI §3 shows schematics of the full optical system of our proposed MMF inverter. A note on polarisation: here, we have focused on inverting the set of PIMs of a single handedness of circular polarisation-taking advantage of the weak circular polarisation coupling exhibited by short lengths of real MMFs [29]. We envisage that a polarisation filter is used to reject light of the opposite polarisation handedness from a scene emitting incoherent light. Our inverter design could potentially be extended to unscramble vector fields at the cost of some increase in complexity by spatially separating the two polarisation states and passing them through separate inverters [54]. We also highlight that our optical inverter design protocol is not sensitive to the exact fibre geometry-the ratio N/M is the most critical performance predictor. For example, we would expect similar inversion performance when designing a new mask set for an MMF supporting an equivalent number of modes, but with a higher NA and lower core radius than the fibre we chose to model here.
Supplementary Movie 1 shows the simulated light field at several stages through the MMF-inverter system as a focused spot is scanned over the distal facet of the fibre. We see that high-contrast spots are reconstructed at the output of the inverter, reproducing the input spots with high fidelity. This can be quantified by measuring the spot power-ratio, which is defined as the ratio of the power within a small disk centred on the target spot position at the output plane of the inverter, to the total power transmitted within the image of the fibre core [28]. SI §4 shows a power-ratio map, indicating the fidelity with which spots are imaged at different regions across the fibre core. Averaging the power-ratio over all output locations yields the mean power-ratio p r . When our inverter is matched with an ideal fibre, the mean power-ratio of reimaged points, p r = 0:74, a value comparable with the typical power-ratio for state-ofthe-art serial wavefront shaping through MMFs [20,21]. Figure 3 shows example images of distal scenes reconstructed at the output of the inverter system. Figures 3(a)  and 3(b) show images of scenes at the distal facet: a spatially coherent speckle pattern (a) and a spatially incoherent scene (b). An advantage of our concept is that, refocussing of the image plane away from the end of the distal fibre facet can be achieved by simply axially repositioning a camera  [20]. It is interesting to note that the objects imaged in (c) and (d) only excite a certain subset of the step-index PIMs, as can be seen at the corrector plane-the missing Gaussian spots here simply indicate the modes which do not contribute to the light field emanating from the object. We quantify the quality of image reconstruction by calculating the overlap integral o (for coherent fields) and correlation c (for incoherent fields) between our inverter output and the ideal output; these values are presented in Figure 3 (and defined in SI §5). We see that the reconstruction quality of our inverter (right-hand column) is close to the theoretical optimum for widefield imaging (2 nd column) in terms of both resolution and contrast. Farfield imaging exhibits the lowest fidelity, due to the greater proportion of high order PIMs excited in this configuration, which are marginally less efficiently sorted (see Figure 2(c)). This could be improved in future designs by more heavily weighting the importance of high-order PIMs in the MPLC design algorithm.

Super-Resolution
Imaging. Our inverter gives direct access to the field at the distal end of a fibre, potentially making a possible host of advanced microscopy techniques beyond widefield imaging. For example, we now test whether the fidelity of the inversion is compatible with super-resolution imaging through an MMF-see Figure 4. Here, we simulate PALM or STORM-which differ only by the fluorophore switching mechanism [22,23]. These techniques are based on the principle that it is possible to locate the centroid of a diffraction limited spot to a better precision than the diffraction limit itself. Multiple widefield fluorescent images are recorded, with fluorophore excitation carefully controlled so that only a random well-separated subset of fluorophores are induced to emit light in any given image. Centroid tracking of the isolated diffraction limited spots then reveals the position of each fluorophore, enabling a super-resolved map of their positions to be compiled.
We simulate a scene consisting of an array of fluorophores arranged in a spiral configuration (Figure 4(a)). The spacing of these fluorophores is below the diffraction limit, so the array is not resolved when imaged directly (Figure 4(b)). However, recording of multiple images of isolated fluorophores (an example shown in Figure 4(c)) yields a super-resolved image of the scene (Figure 4(d)). SI Movie 2 depicts this process. In this case, the mean absolute error in position of a fluorophore is 129 nm, a factor of 24 below the diffraction limit of λ/ð2NAÞ = 3:2 μm (here, λ = 633 nm and NA = 0:1). Note that here, we have assumed ideal conditions in which the detected photons are within the spectral bandwidth of the system (see §3.3), and ignore the effect of

Intelligent Computing
finite signal-to-noise ratio (SNR), which is highly system dependent. However, even if the fibre is not ideal and exhibits modal cross-talk, reducing the fidelity of the inversion, the error still remains well below the diffraction limit (see SI §6 for more details). This is a promising result, which is readily extendable to 3D localisation microscopy [55], and offers the exciting possibility of imaging with molecular specificity at unprecedented resolution deep inside living tissue. Of course in practice, the resolution of such a technique will depend critically on the fidelity of the inversion, the stability of the optical system and the SNR in captured images.

Spectral Bandwidth.
A key characteristic of our proposed fibre-inverter imaging system is the spectral bandwidth over which light can be unscrambled. The bandwidth of the TM of an MMF, Δλ fib , defines the range of wavelengths over which the same TM is valid [21]. Assuming that chromatic dispersion is negligible compared to spatial mode dispersion, Δλ fib can be derived, for a step-index fibre, by considering the optical path length difference between a ray travelling at the critical angle of total internal reflection versus a ray travelling "straight through", parallel to the central fibre axis, yielding where n c is the refractive index of the core (see SI §8 for derivation). This ray picture is equivalent to analysing the rate at which the PIMs with the highest and lowest phase velocities dephase via propagation through the fibre. When the fibre is coupled to an inverter, we quantify the spectral bandwidth of the combined system by observing the rate of reduction in the mean power-ratio of imaged spots as the input wavelength is detuned from the design wavelength. See SI §9 for more detail. Figure 5(a) shows the analytically calculated bandwidth of the fibre itself (purple line, given by Equation (4)), and the numerically simulated bandwidth of the combined MMF-inverter system (green circles) as a function of fibre length L. Both curves follow a similar trend. In the absence of a fibre (i.e., when L = 0), we find that the inverter itself is limited to a bandwidth of Δλ inv ðL = 0Þ~23 nm. When the inverter is coupled to a fibre, the bandwidth of the combined system is limited by the bandwidth of the fibre itself for lengths beyond~1 cm. For example, in the case of a 20 cm long fibre, the spectral bandwidth with which objects at the distal facet can be imaged is about 1 nm.
Despite this narrow bandwidth, there is some tunability in the spectral response of the fibre-inverter system. We find that the wavelength of light that is brought into focus at the output plane can be tuned simply by adjusting the PIMdependent phase delays imparted by the correction plane. The spectral bandwidth over which it is possible to tune the wavelength is determined by the baseline bandwidth of the inverter itself when no optical fibre is present (i.e., 23 nm in our case). This means a form of hyperspectral imaging may be possible: by synchronising the modulation of the corrector plane with a tunable spectral bandpass filter, images across a range of wavelengths could be recorded. See SI §10 for a more in-depth discussion of this possibility.
Furthermore, when the image plane is moved away from the distal facet, the NA of rays emitted from a point on the image plane that are collected by the fibre is reduced below the NA of the fibre. Light emanating from adjacent points in the scene does not interfere in incoherent imaging, and so the reduction in NA for individual points has the effect of increasing the spectral bandwidth of the system.

Intelligent Computing
Therefore the bandwidth becomes a function of the image plane defocus distance z according to 2n c λ 2 z 2 La 2 for z > a NA : See SI §8 for derivation and more detail on under which assumptions Equation (5) holds. Figure 5(b) shows how the spectral bandwidth of the fibre itself (purple line, given by Equation (5)) and the combined MMF-inverter system (green circles, simulated) increase as the depth z to the scene increases. In our case, for depths of z ≲ 4 mm, the fibre itself limits the bandwidth. For greater levels of defocus, we find the inverter limits the spectral bandwidth to~67 nm, which holds all the way to the far-field (green dashed line). This hints at another possibility to overcome the narrow spectral bandwidth of an MMF and enable broadband imaging -by placing a Fourier-transforming lens at the distal end of the fibre (for example, a short section of graded index fibre or a microstructured element). This would transform the system so that it is now broadband at the distal facet and has a narrower bandwidth in the far field.
3.4. Adapting to Imperfect MMFs. Our MMF optical inversion strategy relies on the TM of the MMF being strongly diagonal when represented in the PIM basis. How realistic is this situation likely to be in practice? Power found in off-diagonal elements of the fibre's TM signifies mode coupling-indicating that when the MMF is excited with an individual input PIM, other PIMs are excited upon light reaching the output [56]. Plöschner et al. have shown that with careful measurement and fibre alignment, the TM of short lengths of step-index MMF (up to L~30 cm in length) can hold over 95% of the power on the main diagonal when represented in the PIM basis, i.e., the mode coupling is minimal [29]. Graded index MMFs are less predictable [57]; hence here, we have focused on step-index MMFs, which at present show a greater prospect of being experimentally invertible.
Nonetheless, fibres exhibiting some degree of mode coupling may be encountered in real-world scenarios. To study this, we modify our model to incorporate MMF TMs that have varying levels of mode coupling and loss. We introduce the imperfect fibre TM, T fib ′ ∈ ℂ P×P , and quantify the fraction of power on the main diagonal by p d . T fib ′ is constructed so that power is spread into off-diagonal elements in an experimentally realistic way, as detailed in reference [56]. Figure 6 shows the performance of our optical inverter when coupled with imperfect MMFs. Evidently, as p d decreases, the inverter will less successfully unscramble the light emerging from the fibre. Figure 6(a) shows the reduction in image quality when a fixed inverter-designed for an ideal fibre-is coupled to fibres possessing progressively increasing levels of modal coupling. Once p d ≲ 0:25, the image is no longer discernible. We find that mode coupling is more disruptive to imaging than loss-SI §11 gives details.
We now consider how our inverter may adapt to improve image quality, and first analyse the case where the full TM of an imperfect MMF, T fib ′ , has been measured. One solution would be to redesign every phase mask in an attempt to map the device to a new inverse TM. However, this would mean that all 29 phase masks in our device would need to be reconfigurable. With current technology, this is extremely challenging to achieve while also maintaining a high operating efficiency, as liquid-crystal SLMs are too lossy to be realistically used with tens of reflections. Therefore, we have aimed for a solution where only a single plane-the corrector plane-need be reconfigurable. If T fib ′ is known, the modal phase delays imparted by the corrector plane can be adjusted and set to where ϕ n is held in the n th element of column vector ϕ, and the operator diag ð·Þ creates a vector from the diagonal elements of a square matrix. In Equation (6), T fib ′ has been transformed to PIM-space, and the corrector plane phases are selected to undo the phase delays picked up by the uncoupled component of each PIM, while ignoring any mode coupling. Figure 6(b) shows image formation when adaptively adjusting the correction phases according to Equation (6) as p d reduces. We see the contrast is enhanced compared with the non-adaptive case-thus imaging capabilities reduce gracefully rather than catastrophically failing upon the onset of mode coupling.
3.5. Imaging through Flexible MMFs. We have so far considered the pairing of our inverter with a static MMF whose TM has been accurately measured. However, the TM of an MMF is exceedingly sensitive to fibre configuration. Therefore, if the fibre is free to move and bend, the TM is modified and is no longer accurately known. We now ask, is there a way our inverter can accommodate this scenario?
The structure of our inverter has been designed to leverage fibre memory effects [27,28,56]. Optical memory effects are associated with underlying correlations in a TM, that appear when it is represented in an appropriate basis [28,58]. In the case of an MMF, its approximate cylindrical symmetry results in a rotational memory effect: rotating the input field causes a corresponding rotation in the output field [27]. The diagonal nature of the TM in the PIM basis also leads to a recently uncovered quasi-radial memory effect [28]. Our inverter gives us physical access to the optical fields in the PIM basis, i.e., at the corrector plane, enabling us to adapt the correction phases to take advantage of these memory effects. There are several ways we envisage these memory effects may be employed to invert flexible fibres.
Firstly, Plöschner et al. showed that if the changes in configuration of a short length of step-index MMF can be externally monitored, then the change to its TM can be predicted without needing to be remeasured [29]. Crucially, this study also showed that bending short lengths of fibre mainly altered the phase delay of individual PIMs, without introducing significant levels of modal cross-talk-meaning that the inverse TM of curved MMFs can be well-captured by our inverter simply by updating the phase-delays given to each PIM at the corrector plane with those predicted from fibre curvature.
Secondly, Li et al. recently showed that an estimate of the TM of an MMF can be obtained without optical access to the distal end by deploying a fluorescent particle, which acts as "guide-star" located on the distal fibre facet [28]. In this case, the field at the proximal fibre facet that focuses onto a distal guide-star can be found by optimising the return fluorescent signal intensity-in principle in just a few hundred milliseconds [58,59]. Knowledge of this proximal field, which here we denote by column vector u gs ∈ ℂ P when represented in the real-space basis, reveals the corrector plane phase delays ϕ that bring this point into focus at the output of the inverter where column vector t ∈ ℝ P specifies the location of the guide-star-the element of t corresponding to the guidestar position is set to 1, and all other elements are set to 0. The intuition behind Equation (7) is as follows: the first term on the right-hand-side provides an estimate of the relative phase of the PIMs on the target location at the distal end of the fibre. The second term captures the relative phase of the PIMs at the same lateral location at the proximal end of the fibre. Subtracting these values ensures that the calculated phase corrections only account for phase delays caused by propagation through the MMF itself. Once light emanating from a guide-star at the distal facet of the MMF has been brought into focus at the inverter output, the existence of the rotational and quasi-radial memory effects mean other nearby points are simultaneously corrected. Therefore, it is possible to image clearly through an area around the target point, known as an isoplanatic patch [28]. Figure 6(c) demonstrates image formation using this approach. The size and shape of the isoplanatic patch is governed by the magnitude of p d , if p d = 1, the fibre TM is diagonal, and the isoplanatic patch extends across the entire distal fibre facet ( Figure 6(c), left-most panel). We anticipate that when used with a short section of MMF (e.g., up to L~30 cm long as in reference [29]) we would be in this low mode-coupling regime where a single guide-star can yield an accurate estimate of the TM, and adjustment of the corrector plane phases can restore high fidelity imaging over the whole fibre core. Nonetheless, here, we also show how imaging is impacted as mode coupling increases and p d falls below 1: the isoplanatic patch contracts around the location of the guide-star-see Figure 6(c), and SI movie 3.
Evidently, if the fibre exhibits significant mode coupling-which we expect only in a worst-case-scenariothen multiple guide-stars are required to scan a contracted isoplanatic patch across the whole distal facet. There is a way forward in this case: during calibration the light emanating from several guide-stars simultaneously could be distinguished via emission spectrum or speckle contrast [49,60]. Figure 6(d) shows a series of images as the isoplanatic patch is scanned over 3 separate guide-stars (left 3 panels), and a combined image formed from their weighted average (right hand panel). SI §12 gives more details. This guidestar-based concept performs only slightly worse than the adaptive case ( Figure 6(b)), but has the advantage of potentially being performed in situ, without the need for optical access to the distal fibre facet to remeasure the full fibre TM.
While imaging with the help of multiple guide-stars is no longer single-shot, many spatial modes may be unscrambled in parallel within each isoplanatic patch. This leads to an enhanced frame-rate compared with conventional wavefront shaping [11], with the enhancement dependent upon the level of mode coupling in the MMF. This approach can also be used to boost image contrast without the need for physical guidestars if T fib ′ is known. In this case, the centre of the isoplanatic patch can be arbitrarily relocated (to a position specified by t) using Equation (7) with u gs = T fib ′ t. Furthermore, there have been several alternative proposals to reconstruct the TM of an MMF without access to the distal facet, which could potentially be used in conjunction with our optical inverter [61][62][63].

Intelligent Computing
Lastly, it may be possible to engineer new types of MMF that are less sensitive to bendingso that their TM does not alter significantly when their configuration changes [57]. Although such MMFs have not yet been manufactured, they represent a highly promising prospect if paired with our inverter in the future-enabling high-speed incoherent imaging through flexible MMFs without the need for any adaptive feedback.

Discussion
We now consider our proposal in the context of other approaches that aim to achieve single-shot imaging through MMFs. Kim et al. have demonstrated single-shot computational fluorescence microscopy through rigid MMFs [64]. In this approach, images of distal fluorescent objects are computed directly from the low contrast speckle patterns emanating from the proximal end (such as those shown in Figure 3(b) and 3(c), third column, for example) [65]. Calibration is first required to measure the incoherent intensity TM of the fibre or train a neural network [66]. Machine learning based approaches have also been applied to demonstrate single-shot spatially coherent image transmission through MMFs [67]. These systems capture intensity images of high contrast spatially coherent speckle patterns emerging from the proximal end of a fibre, which are then processed by a pretrained electronic neural network to reconstruct the intensity pattern of the corresponding field at the distal end.
In the context of these computational scattering inversion approaches, our optical inverter can be interpreted as an alloptical diffractive neural network [37,68]-passively performing computational image reconstruction in a few nanoseconds. In this picture, the cascade of diffractive elements mimics the layered architecture of a linear convolutional neural network. For example, adjusting the phase delay imparted by a pixel on one phase plane modifies the diffracted field arriving at the next plane. This is analogous to tuning the weights of the coupling matrix linking the neurons between layers. In our case a linear network (i.e., without any nonlinear activations) perfectly maps onto the linear operator that represents the inverse fibre TM. The network "training" is carried out using fibre modes, which are the optimum basis to represent any possible image transmitted through the MMF. By dividing our diffractive network into three distinct subsections-a mode sorting module, a phase correcting module and a mode combining module-we impose a physically inspired model on the network's structure, in contrast to blindly optimising it.
Intriguingly, our modelling suggests that a physically realised all-optical diffractive neural network is able to outperform its electronic counterpart for light scattering problems. In particular, our design has potential to enable single-shot incoherent imaging though MMFs with minimal loss of spatial resolution-an endeavour that has yet to be achieved. We have also shown how super-resolution imaging may be possible without resorting to use of priors about the nature of the scenes. The root of this enhanced performance stems from the preservation of the phase and spatial coherence information emanating from the scene. In contrast, most electronic neural networks tasked with unscrambling light rely on training images capturing only the intensity of optical fields.
In what situations do we envisage our MMF inversion concept being deployed in practice? The approach will work best with fibres exhibiting very low levels of mode coupling. Since the severity of such mode coupling typically increases with fibre length, this will limit the length of fibres it is possible to invert using these methods. A key application is minimally invasive imaging deep inside the brain. This only requires short stubs of MMF in the range of a one to a few centimetres in length [7], promoting low levels of mode coupling and relatively broad operational bandwidth (see Figure 5(a)). Inversion of longer lengths of fibre is more challenging, but gives access to new scenarios, including imaging through freely moving flexible microendoscopes. Our inverter also has potential for aiding spatial mode multiplexing through short range optical interconnects-in this application a lower modal capacity may be favorable, which could ease the challenge of inverting longer lengths of fibre. Finally, as our optical inverter is a passive element, no measurements of photon state are made until the image is detected at its output. Therefore, our system may also preserve the quantum entanglement of photons sent through it, circumventing the need for active entanglement recovery [69], offering new opportunities in future quantum communication systems or quantum computing schemes.

Conclusion
We have sketched out a new concept-an "optical inverter"-to simultaneously unscramble all light modes that have propagated through an MMF. Our inverter can be understood as a bespoke scattering medium, designed to be complementary to an MMF so as to undo its optical effects. Although our work does not invoke transformation optics [70], the action of the inverter is similar to "cloaking" an MMF within its spectral bandwidth, so that it is possible to see directly through it with no other computational steps necessary in the image reconstruction.
The key advantage of our concept is that it renders possible any form of widefield microscopy at the tip of a hair thin strand of MMF. This includes localisation-based super-resolution imaging [22,23], along with other emerging forms of parallelised super-resolution microscopy [71], structured illumination microscopy [72] and singleobjective light sheet microscopy [73]. Single-shot widefield imaging at any distance beyond the distal end of a short length of MMF also becomes possible. These new imaging modalities come at the cost of a relatively complicated design and currently with a limited spectral bandwidth-at least when imaging the distal fibre facet. However, the reliance of our concept on multi-plane light conversion means it is within the reach of current fabrication technology, and future improvements in design may yield broader bandwidth operation. Furthermore, by introducing only a single reconfigurable phase mask, we have shown how our optical inverter can adapt to a range of fibre inverse TMs: a feature that also represents the first example of a new class of MPLC device structured to actively incorporate optical memory effects. 10 Intelligent Computing The optical inversion strategy we have described here can potentially be extended to unscramble light that has passed through other objects, such as photonic crystal waveguides [74], photonic lanterns [75] or biological tissue [49]. Finally, we anticipate that all-optical inversion of scattered light will find an array of applications beyond optical imaging: benefiting the fields of mode division multiplexing for high capacity optical communications [6,76], as well as quantum cryptography [77] and classical and quantum optical computing [31,78].

Data Availability
The codes and data that support the findings of this study are available from the corresponding author upon request.