Unsupervised Spectrum Anomaly Detection Method for Unauthorized Bands

With the rapid development of wireless communication, spectrum plays increasingly important role in both military and civilian ﬁ elds. Spectrum anomaly detection aims at detecting emerging anomaly signals and spectrum usage behavior in the environment, which is indispensable to secure safety and improve spectrum e ﬃ ciency. However, spectrum anomaly detection faces many di ﬃ culties, especially for unauthorized frequency bands. In unauthorized bands, the composition of spectrum is complex and the anomaly usage patterns are unknown in prior. In this paper, a Variational Autoencoder-(VAE-) based method is proposed for spectrum anomaly detection in unauthorized bands. First of all, we theoretically prove that the anomalies in unauthorized bands will introduce Background Noise Enhancement (BNE) e ﬀ ect and Anomaly Signal Disappearance (ASD) e ﬀ ects after VAE reconstruction. Then, we introduce a novel anomaly metric termed as percentile (PER) score, which focuses on capturing the distribution variation of reconstruction error caused by ASD and BNE. In order to verify the e ﬀ ectiveness of our method, we developed an ISM Anomaly Detection (IAD) dataset. The proposed PER score achieves superior performance against di ﬀ erent type of anomalies. For QPSK type anomaly, our method increases the recall rate from 80% to 93% while keeping a false alarm rate of 5%. The proposed method is bene ﬁ cial to broadband spectrum sensing and massive spectrum data processing. The code will be released at git@github.com:QXSLAB/vae_ism_ano.git.


Introduction
The rapid development of radio technology has resulted in a greatly enlarged demand for spectrum resources. At the same time, the risk of spectrum interference and malicious use has become increasingly serious. For example, unintentional and intentional interference to positioning service such as Global Navigation Satellite System (GNSS) has significantly increased [1,2]. Illegal repeaters frequently affect the mobile operator's cell planning, resulting in poor coverage and dropouts. Reliable spectrum sensing [3,4] and spectrum anomaly detection are essential approaches to secure spectrum safety, which quickly detect the potentially threatening signal so that it can be eliminated in time.
The research of spectrum anomaly detection can be divided into authorized band anomaly detection and unauthorized band anomaly detection. Authorized bands can only be legally accessed by specific systems. The signal should be detected as anomaly spectrum if it is different from these authorized systems. Thus, authorized band anomaly detection can be well solved by signal identification, which has been well studied in recent years [5][6][7]. In contrast, unauthorized band anomaly detection is very challenging. In unauthorized bands, wireless systems with different physical layers coexist, which makes the unauthorized spectrum very complex. As a result, it is hard to manually identify the anomaly spectrum patterns. Besides, due to the sporadic nature of spectrum anomalies, it is hard to collect sufficient anomaly samples to train anomaly detection model. Therefore, a complete automatic feature-extracted learning model is highly required to solve this problem.
In this paper, an unsupervised deep learning method is proposed for unauthorized band spectrum anomaly detection. We convert the spectrum data into a two-dimensional spectrogram, and then use Variational Autoencoder (VAE) [8] to capture the probability distribution of spectrogram samples. Based on the theory of optimization inference, we deduced the influence of anomaly signals on VAE reconstruction. On this basis, we propose an anomaly detection metric termed as percentile (PER) score. Finally, we conduct experiments to verify the proposed method. Experimental results show that our method has superior performance for various types of anomalies. The main contributions of this paper are as follows.
(i) The influence of anomaly signals on VAE output is deduced for the first time. It reveals that anomaly signals will introduce Background Noise Enhancement (BNE) effect and Anomaly Signal Disappearance (ASD) effect. To a certain extent, this solves the problem of insufficient prior knowledge in unlicensed frequency band anomaly detection (ii) A novel anomaly detection metric termed as PER score is proposed. It can effectively capture the distribution variation caused by BNE and ASD and solve the anomaly dilution problem faced by traditional anomaly scores when dealing with burst and narrow-band signals (iii) An ISM Anomaly Detection (IAD) dataset is developed for the study of spectrum anomaly detection. It considers four different kinds of anomalies. We believe it would foster further research on unlicensed frequency band spectrum anomaly detection

Related Work
Numerous researches have been conducted on spectrum anomaly detection, which can be divided into two categories with respect of their pattern extraction methods. The first kind uses manually patterns designed by experts. Liu et al. [9] converted the spectrum anomaly detection problem to a statistical significance testing problem. They leveraged the property that the received signal power (RSS) decays approximately linearly with the logarithmic distance from the source and detected the unauthorized transmitters by making use of the propagation characteristics. Greenstein et al. [10] made use of the property that transmitters at different locations would lead to different spatial distributions of the RSS and detected anomalies by comparing the current pattern with a stored spatial map of the transmitter. Yin et al. [11] obtained the historical pattern by calculating the average of seven days' data. They calculated Mahalanobis distance between measuring spectrum and the historical pattern to detect potential anomalies.
On the other hand, considerable researches leverage data-driven learning methods to extract spectrum patterns. Feng et al. [12] and Rajendran et al. [16] used a deep autoencoder model to perform normal pattern extraction. In their papers, spectrogram was used as the input of the learning model. The anomaly score it used was mean squared error (MSE) between the amplitude across subfrequencies of the true spectrogram and the corresponding reconstructed one. Honghao et al. [13] adopted the idea of classification to detect spectrum anomalies. They used spectrum occupancy sequence as input. Normal and anomaly spectrum patterns were modeled by hidden Markov model (HMM). Therefore, the anomalies can be recognized by computing maximum log-likelihood of the data with respect to each spectrum pattern. O'Shea et al. [14] used a long short-term memory-(LSTM-) based recurrent network to train a time series model and to obtain features. Firstly, it computed the difference between predicted value and true value on training set. Then, it modeled the error vector using a parametric multivariate Gaussian distribution. Finally, it computed the likelihood probability in expected error distribution on test set to distinguish anomaly. Li et al. [15] studied spectrum anomaly detection in LTE band. They built deep neural network (DNN) models to capture spectrum usage patterns and computed root mean squared error (RMSE) between the true amplitude across subfrequencies and the model prediction values.
As summarized in Table 1, most of the current researches on spectrum anomaly detection adopted MSEtype anomaly score. It should be noted that likelihood and MSE are equivalent under the premise of Gaussian distribution. The MSE score pays equal attention to every part of the spectrum, which will dilute the impact of anomalies, especially when the anomaly signal duration is short or the bandwidth is narrow. Instead of using global average error, we argue that the spectrum anomaly detection should pay more attention to the reconstruction error of the worst recovered region in the spectrogram. We will theoretically prove this point in next section.

Principle
In this section, the VAE method is briefly described. Then, we reinterpret VAE training from the view of generation error minimization and analyze the optimal output of VAE for a given codeword. Finally, based on the property of VAE optimal output, a novel anomaly score is designed for unlicensed spectrum anomaly detection.

VAE Method.
The VAE method is illustrated in Figure 1. The input X is first passed through the Encoder to obtain the posterior probability p φ ðZ | XÞ. Then, the codeword Z is sampled from the posterior probability. At last, the output X = d θ ðZÞ is reconstructed by the Decoder.

Optimal Output of VAE.
According to the theory of approximate inference [17], the log-likelihood of X can be Table 1: Deep learning-based spectrum anomaly detection.

Pattern extraction model
Anomaly score Feng et al. [12] Deep autoencoder MSE Honghao et al. [13] Hidden Markov model Likelihood O'Shea et al. [14] Long short-term memory network Likelihood Li et al. [15] Long short term memory network RMSE 2 Space: Science & Technology expressed as Equation (1). λ represents the number of Monte Carlo samples. Y = ½y 1 , y 2 ,⋯,y τ and V = ½v 2 1 , v 2 2 ,⋯, v 2 τ are the expectation and variance of the codeword, where τ represents the dimension of codeword. φ and θ are, respectively, the parameters of Encoder and Decoder.
The training process of VAE aims at maximize log p θ,φ ðXÞ over the training set X. The objective function of VAE is defined as Equation (3).
The first item in Equation (3) can be viewed as a regularization term, and the second term can be simplified as Equation (4). We define g φ,θ ðZÞ as generation error. The physical meaning of generation error can be explained as follows. For certain Encoder output Z, there exist multiple possible inputs X ′ . Each X ′ will introduce a reconstruction error kX ′ − d θ ðZÞk 2 2 . Generation error represents the expectation of these reconstruction errors.
As can be seen in Equation (A.1), for a certain codeword Z l , VAE training is equivalent to generation error minimization. By differentiating g φ,θ ðZÞ with reference to d θ ðZÞ and letting the derivative equal to 0, the optimal VAE output can be derived as Equation (7). It means that the optimal output of VAE for a given codeword Z equals to the expectation of possible inputs of Z.
3.3. Percentile Score. For a normal spectrogram input X N , its codeword Z N should be located near the center of the posterior probability p φ ðZ | X N Þ, while at the edge of posterior Assuming that the prior distribution pðXÞ is approximately uniform, Equation (7) can be approximated as Equation (9). As a result, the VAE output approximates the input origin X N , and the reconstruction error should be small.
For an anomaly spectrogram input X A with anomaly signal A, according to Equation (7), VAE would reconstruct it by weighted summation of normal inputs X ′s that are distinct from X A . This will bring about two effects.
(i) Background Noise Enhancement (BNE) Effect. The noise level in the background region of the spectrogram will be raised, since many signals that are presented X ′ but not in X A would be superimposed on the reconstructed output (ii) Anomaly Signal Disappearance (ASD) Effect. The anomaly signal A would disappear in the reconstruction output, since A is not presented in X ′ The BNE effect will cause large reconstruction errors in certain background regions. It will lead to more large outliers in the reconstruction error. As a result, the probability distribution of reconstruction error will have a longer tail, and the percentile will be lifted. In the same way, the ASD effect will increase the percentile of the reconstruction error in the signal region. On this basis, we propose a percentile (PER) score for anomaly spectrum detection. Different from MSE and MAE scores, PER score focuses on reconstruction errors of badly recovered region in spectrogram. The PER score is shown in Equation (10). We segment the spectrogram X into signal region and background region. The first item represents the ξth percentile of pixel-level reconstruction errors in background region. The second item represents the ηth percentile of pixel-level reconstruction errors in signal region. q ξ ðxÞ measures the value corresponding to the ξ% point in the cumulative distribution of scalar x. r α,γ ðXÞ represent the mask for background region, which is implemented by a max-pooling function with kernel size of γ and a step activation with threshold of α. 1 − r α,γ ðXÞ represent the mask for signal region. f γ represents min-pooling function with kernel size γ and is used to reliably detect the noise floor variation. Symbol ∘ is used to represent Hadamard product between matrices. j·j represents pixel-level absolute value operation. The definition of operator q ξ , f γ , and r α,γ are shown in see the appendix.
In Equation (10), α, β, γ, ξ, and η are hyperparameters and can be tuned according to application scenarios. In this paper, the kernel size γ of max-pooling and min-pooling function is set to 3. In order to balance the difference in the absolute value of the reconstruction error between the signal region and the background, the weight of background error percentile is set to 2. The threshold of step activation is set to 0.05. The percentage parameters ξ and η are, respectively, set as 90 and 99.

Dataset.
In order to verify the effectiveness of our method, we developed a dataset for ISM frequency band anomaly detection, termed as ISM Anomaly Detection (IAD) dataset. Similar to the approaches taken in the literature [12,14,16,18], we collect ISM spectrum data in the laboratory environment as normal spectrogram samples and construct anomaly spectrogram samples by artificially sending specific types of signals to the environment. The experiment setup for data acquisition is shown in Figure 2. Two Universal Software Radio Peripherals (USRPs) are, respectively, used as transmitter and receiver. Both USRPs use the VERT2450 omnidirectional antenna. The receiver USRP is used to collect spectrum data. The center frequency is set to be 2.432 GHz and the bandwidth is set to be 20 MHz. The receiving frequency band includes WiFi and Bluetooth signals. The duration of each acquisition is set to 100 ms. The collected I/Q data is transformed into spectrogram with size of 256 × 256 by Short-Time Fourier Transform (STFT). Then, the spectrogram is down-sampled with a factor of 4. The final size of spectrogram sample is 64 × 64. The transmitter USRP is used to send anomaly signal to the environment. Anomaly signal with four different types of modulations are considered, namely QPSK, 16QAM, GMSK, and CHIRP. The signal bandwidth varies randomly from 1 MHz to 5 MHz, and the transmission duration is 20 ms. The frequency tuning speed of the CHIRP signal is 20 KHz per second. The transmitter USRP is turned on during anomaly spectrogram sample collection, and it is turned off when collecting normal spectrogram samples. Examples of the normal and anomaly spectrogram samples are shown in Figure 3.
The normal spectrogram samples are randomly split into two parts. The first part contains 4388 samples and is used to construct training set X. The second part is used to construct test sets. In order to evaluate the performance of   Figure 4(a). It uses convolution operation to extract hierarchical features from the original spectrogram, then uses dense layers to generate codeword at the bottleneck, and finally uses transposed convolution (ConvT) to gradually restore the spectrogram. Each feature map consists of C planes. Each plane is with shape of H × W and represents the prominence of a certain feature on twodimensional time-frequency plane, where H and W represent the scale of time and frequency dimensions. The second is Fully Connected VAE (FC-VAE) network. Its architecture is shown in Figure 4(b). In FC-VAE network, the 2dimensional (2D) original spectrogram sample is firstly reshaped into 1-dimensional (1D) feature. Then, dense layers are used for feature compression and recovery. At last, the recovered 1D feature is reshaped to generate a spectrogram with the same size as the input. Both VAE networks are implemented in PyTorch and are trained by only normal samples in an unsupervised approach. We use Adam optimizer with a learning rate of 10 −4 . The batch size is set to be 64.

Results.
After training, the input and output of Conv-VAE network on Testset 1 are shown in Figure 3. The results are similar for FC-VAE network and Testset 2, Testset 3, and Testset 4. The first and second rows are the input and output of Conv-VAE network. As can be seen, for anomaly spectrogram samples, QPSK-modulated anomaly signals are disappeared in the reconstructed output. At the same time, the background noise floor of the anomaly spectrogram has been risen. The third row represents the mask used to filter signal region in spectrogram. The fourth and fifth rows are the reconstruction errors in signal region and background. The results confirm that spectrogram anomalies indeed cause ASD and BNE effects. The distribution of pixel reconstruction errors is shown in Figure 5(a). The ASD and BNE effects cause large pixel reconstruction errors. Compared with normal spectrogram, the probability distribution of the pixel reconstruction error for the anomaly spectrogram has a longer tail. As a result, anomaly spectrogram samples have larger reconstruction    Figure 5(b).
The performance of PER score is evaluated by Receiver Operating Characteristic (ROC) test. ROC curve examines the variation of true positive rate (TPR) with false positive rate (FPR). TPR equals the ratio of anomaly samples that are detected as anomaly. It represents the recall rate of all anomaly samples and reflects the level of missed detection. FPR equals the ratio of normal samples that are detected as anomaly, which is the false alarm rate. The Area Under Curve (AUC) metric measures the area under ROC curve and reflects the performance of the anomaly detection. AUC closer to 1 indicates better detection performance.
We compare PER score with traditional anomaly scores in Figure 6. Conv-VAE network and Testset 1 are used here.
MSE and MSE score are defined in Equation (A.9) and Equation (A.8). In addition, we take the codeword from Conv-VAE network as the input feature and use Local Outlier Factor (LOF) [19], One Class SVM (OCS) [20], Elliptic Envelope (EE) [21], and Isolation Forest (IF) [22] methods for spectrogram anomaly detection. These methods are commonly used to detect anomalies in low dimensional data in the field of machine learning [23]. As can be seen, the proposed PER score achieves higher anomaly detection performance under same false alarm rates. Compared with the MSE score, the PER score increases the recall rate from 80% to 93% when the false alarm rate equals 5%.
We further compare the detection performance of PER score against different anomaly signals. The results are shown in Table 3. For all anomaly types, the PER score  Space: Science & Technology achieves the highest detection performance. When the abnormal type is GMSK, the improvement brought by the PER score is relatively small. We guess this is because there is a GMSK-modulated Bluetooth signal in the observation frequency band, which makes it more difficult to detect GMSK anomaly.  Finally, we study the dependence of the detection ability on network architecture. We perform experiments on the FC-VAE network, and the experimental results are shown in Table 4. As can be seen, PER score outperforms all other methods.

Conclusions
In this paper, an unsupervised deep learning method is proposed for unsupervised spectrum anomaly detection in unauthorized bands, which leverages VAE to capture the spectrum data distribution and detects anomalies by selectively comparing the difference between spectrogram and its VAE reconstruction counterparts. We have theoretically proved and experimentally verified that the anomaly spectrum usage will introduce Background Noise Enhancement (BNE) effect and Anomaly Signal Disappearance (ASD) effect. On this basis, the PER score is proposed for spectrum anomaly detection, which focuses on capturing the distribution changes of pixel reconstruction errors caused by ASD and BNE effects. The experimental results show that PER score achieves the superior detection performance against different anomaly signals at ISM band. For QPSK-type anomalies, the PER score increases the recall rate from 80% to 93% while keeping a false alarm rate of 5%. The proposed anomaly detection method is beneficial to guide spectrum sensing system to quickly filter out the high-value information from a large amount of spectrum data and automatically invest more computing and storing resources for time-frequency window containing threatening signals.
Future research should consider how to transfer the anomaly detection model between different scenarios. This will enable the spectrum anomaly detection model to be quickly deployed to unfamiliar scenes. Few-shot learning and transfer learning may play an important role in this topic. At the same time, a multiscenario spectrum anomaly detection dataset should be developed.

ðA:1Þ
The definition of percentile operator q ξ , min-pooling operator f γ , and max-pooling operator r α,γ are shown in Equation (A.4), Equation (A.2), and Equation (A.7). We use ⊙ to denote compound mapping. x i,j represents the pixel value at ith row and jth column in X.