Revolutionizing Future Connectivity: A Contemporary Survey on AI-empowered Satellite-based Non-Terrestrial Networks in 6G

Space-borne platforms, such as satellites, are deployed in space for communication [19]. They move around the Earth in specific orbits with varying angular velocities, relying on gravity to provide the necessary centripetal force to maintain their orbits. The orbital period of a satellite refers to the time required for the satellite to complete one full revolution around the Earth. Due to differences in orbital periods, some satellites may not be visible to ground observers all the time. To characterize this, another term is used to denote the duration of direct visibility for a satellite. This is known as the horizon time, which refers to the maximum duration during which the satellite is within the line of sight of a given ground station or receiver. Depending on their mobility with respect to the Earth, satellites can be classified into two broad categories: Geostationary (GEO/GSO) and Non-Geostationary (NGEO/NGSO) Earth Orbit satellites. We discuss these two types of satellites below and summarize their key features in Table II.

• •

  Geostationary Earth Orbit (GEO or GSO) Satellites: These satellites have an orbital period of 24 hours which is the same as the time required for the Earth to complete a full rotation on its axis. As a result, these satellites appear stationary from the ground and are named Geostationary Earth Orbit (GEO or GSO) Satellites. These satellites orbit on the Earth’s equatorial plane at an altitude of about 35,786 km to maintain this orbital period. Due to this high altitude, it has an extremely large beam footprint (typically the diameter ranges from 200 to 1000 km) covering a pretty wide area. However, it also incurs an extremely long propagation delay (typically around 270 ms) [20] which makes it infeasible for low-latency communications. These satellites have been used in broadcasting services for a very long time, but are not very suitable for low-latency emerging applications.

• •

  Non-Geostationary Earth Orbit (NGEO or NGSO) Satellites: As the name suggests, these satellites orbit around the Earth at a period lower than 24 hours, so they are not stationary with respect to a ground observer. As the orbital period is smaller, their angular velocity is also higher but the altitude is lower compared to GEO satellites. Depending on the heights, they can be divided into two categories: Low Earth Orbit (LEO) and Medium Earth Orbit (MEO) satellites. Typically they are deployed at a height ranging from 200 to 2000 km for LEO and 2000 to 25000 km for MEO satellites. The horizon time is much smaller for NGEO satellites due to smaller orbital periods, for example, the LEO satellites deployed at a height of around 500-600 km with an orbital period of 1.5-2 hours can have a horizon time of 5-10 minutes depending on channel conditions. Due to smaller heights, these satellites have a smaller beam footprint (diameter ranges from 5 to 500 km) with a much smaller propagation delay (typically around 20 ms for LEO satellites and 94 ms for MEO satellites)[20] compared to GEO satellites. With their proximity to Earth and lower cost of launch and maintenance, these satellites, especially the LEO satellites, have gained significant attention in recent years. Their reduced propagation delay and path loss make them an attractive choice for facilitating high-speed data transfer and real-time communication, so as to transform the future 6G connectivity.

High Altitude Platform Systems (HAPS) refer to air-borne platforms that can be used for wireless communication. Airships, balloons, and airplanes are the most prominent types of air-borne platforms in NTN. They are viewed as air-borne counterparts of terrestrial base stations serving as High-altitude International Mobile Base Stations (HIBS) [21]. They usually operate at the stratosphere region with an altitude of around 20 km and a beam footprint size with a diameter of several km. Despite it having a lot smaller propagation delay compared to space-borne platforms, it has some additional challenges related to stabilization on air and refueling.

While both satellites and airborne platforms can be utilized in the development of NTNs, satellites are often considered more critical for discussions related to NTNs. This is due to their global coverage, stable and predictable orbits, high scalability, and the existence of international regulations that govern satellites. As such, satellite networks comprise a significant portion of future NTN-enabled communication networks. Therefore, for the purposes of this article, we will primarily focus on satellite-enabled NTNs in the context of 6G communication technology.

High throughput and extreme coverage requirements, combining both eMBB and mMTC. Examples: Digital twins, pervasive intelligence, global ubiquitous connectivity, etc.

High throughput and low latency requirements, combining both eMBB and URLLC. Examples: HTC, AR, VR, Tactile Internet, multi-sense experiences, etc.

Extreme coverage and low latency requirements, combining both mMTC and URLLC. Example: Vehicle-to-Everything (V2X), intelligent transport and automation, etc.

The principal strength of NTNs lies in their extreme coverage. As discussed in Hexa-X project [22], a flagship for B5G/6G vision and intelligent fabric of technology enablers connecting human, physical, and digital worlds, the vision of enabling 6G networks towards provisioning service everywhere and always through NTN is presented. Due to its extreme coverage, satellites can reach underserved or unserved areas such as islands, remote locations, ships, airplanes, etc. where terrestrial communication is either difficult or impossible to some extent. In times of natural disaster, terrestrial links can be unavailable, in which case users can benefit from the reliable backup of non-terrestrial links. This ensures resilient and robust communication with global connectivity which is considered to be one of the main features of future 6G networks. With the advancements in antenna techniques and miniaturization, high throughput satellites are also deployed in low earth orbits. Consequently, current 5G use cases such as mMTC and eMBB as well as future 6G use cases such as uMBB can be the most important use cases for NTNs. Furthermore, the considerably low latency for LEO satellite systems makes the satellite useful even for low-latency applications. However, 5G URLLC or 6G new use cases with extremely low latency may not be directly applicable for NTN use cases. Nevertheless, NTNs can still be beneficial for these use cases in conjunction with terrestrial networks to improve network efficiencies and reliability. Combining all these, satellites are expected to be one of the major driving forces toward revolutionizing the future 6G applications extensively.

The allowable frequency range of operation is 0.5-100 GHz [24]. Traditionally, six major frequency bands within this range are used in satellite communications, which are listed below:

1. 1.

   L-Band (1-2 GHz): Global Positioning System (GPS) carriers and satellite mobile phones, e.g. Iridium, use this band.

2. 2.

   S-Band (2-4 GHz): This frequency band is used for weather radar, marine radar, etc. satellite communications.

3. 3.

   C-Band (4-8 GHz): Primarily used for satellite television broadcasting.

4. 4.

   X-Band (4-8 GHz): Primarily used in military communications.

5. 5.

   Ku-Band (12-18 GHz): Primarily used for satellite broadcasting services.

6. 6.

   Ka-Band (26-40 GHz): This frequency band is used for high-speed data transmission, including broadband internet access via communication satellites.

However, as per 3GPP, currently, two frequency bands (S and Ka-band) are targeted in particular for integrated TNTN environments considering performance and regulatory concerns [20]. The two target frequency bands are:

• •

  S-Band: The downlink frequency band is 2170-2200 MHz and the uplink frequency band is 1980-2010 MHz.

• •

  Ka-band: The downlink frequency band is 19.7-21.2 GHz and the uplink frequency band is 29.5-30 GHz.

Propagation delay is the time duration taken for a signal to reach its destination. For communication signals, we can calculate the propagation delay for a signal by using the equation: $t=\frac{d}{c}$ where $d$ is the distance between the source and destination and $c=3\times 10^{8}$ m/s is the speed of light. Considering the speed of light as a constant, we observe the propagation delay for a signal is proportional to the propagation distances. Satellites are located very far from the surface of the earth as discussed in Section II-A. Consequently, the propagation delay is going to be extremely large for NTNs. The GEO, MEO, and LEO satellites can have a one-way propagation delay of about 270, 94, and 20 ms respectively as shown in Table II. These values are much larger, especially for GEO and MEO cases compared to conventional terrestrial networks, which generally have a very negligible propagation delay of around a few $\mu s$ [20]. This extended propagation delay has an effect on different network procedures and performances for communication systems.

The propagation loss, or path loss, refers to the reduction in power density that an electromagnetic signal experiences as it travels through space. The most significant component of this path loss is the free space path loss, which is proportional to the distance between the source and destination and the frequency of the signal [25]. For NTNs, this free space path loss is much higher (around 60-120 dB) than it is for terrestrial networks, due to the greater distances between satellites and the use of higher carrier frequencies. In fact, the Ka-band is not suitable for GEO satellites, as it does not meet the minimum link budget for them. The basic path loss component also includes shadow fading [26], as with traditional terrestrial networks.

In addition to that, there is attenuation due to atmospheric gases that depends on frequency, elevation angle, altitude above sea level, and water vapor density[27]. Another important component is attenuation due to rain and fog, which is typically significant for frequencies above $6$ GHz [28]. Additionally, scintillation corresponds to rapid fluctuation in amplitude and phase of the propagating signal in the ionosphere (for Sub-$6$ GHz) and troposphere (for above $6$ GHz) [29]. Depending on different scenarios, either flat fading based on ITU two-state model [30] or fast fading [20] can be considered. The average propagation loss for different types of satellites in different frequency bands is illustrated in Figure 4. This high path loss necessitates the need for efficient power allocation strategies in NTN.

As discussed in Section II-C, for regenerative payloads, satellites can be used as base stations for improved network performances. The terrestrial base stations are located at fixed locations. As the GEO satellites do not change their relative positions with respect to the ground terminal, they appear static in nature with respect to the earth’s surface, so the scenario is similar to terrestrial ones. However, the scenario is very different for NGEO satellites where they need to maintain a lower height and higher angular velocity compared to GEO satellites as discussed in Section II-A, so they do not appear static from the earth’s surface. Due to the dynamic nature of NGEO satellites, they turn into moving base stations in case of regenerative payloads. Due to this high-speed movement of NGEO satellites, different mobility issues arise for NTN platforms.

One of the most important features of the satellites is the large beam footprint associated with them due to their long distances from the earth’s surface. This enables the network coverage of very large areas compared to the coverage area of terrestrial counterparts. It provides us with ubiquitous network coverage including remote, even isolated areas. However, this also creates the necessity for modifications in existing timing and synchronization procedures for conventional terrestrial networks. The cell area is much larger, so the UEs situated at the farthest side of the cells experience a larger delay compared to the UEs situated closer to the satellites [20]. So the timestamps for different network procedures need to be modified according to the distances of the users as we will see in the next subsection.

In wireless communications, Channel State Information (CSI) refers to the information which represents the state of a communication channel between the transmitter(s) and the receiver(s); the process of obtaining this information is known as channel estimation. By having access to CSI, it is possible to adjust transmissions to the current channel conditions, which is essential for achieving reliable communication with high data rates in multi-antenna systems with effective channel resources and interference management. There are numerous advanced approaches, such as Maximum Likelihood estimation and Minimum Mean Square Error (MMSE) estimation, for effective channel estimation in traditional terrestrial cellular networks. Nevertheless, these methods are not well-suited for NTN, particularly for LEO satellites due to the inherent time-variant nature of the satellite communication channels. LEO satellites usually move from horizon to horizon in approximately 5-10 minutes, so a UE remains within the coverage of a specific LEO satellite for a very short time period. Furthermore, the propagation delay for satellite networks, especially in the case of GEO satellites, is considerably larger (250 ms RTT) in comparison to general terrestrial networks. Therefore, the CSI estimated by the LEO satellites can be outdated [31]. Because of these reasons, the CSI estimation in NTN necessitates new efficient techniques in addition to the traditional terrestrial estimation methods.

Since an NGEO satellite operates at a lower altitude, the coverage area of each NGEO satellite is smaller than that of a GEO satellite. Typically around 5-20, NGEO satellites form complex mega-constellations to sustain global coverage across the earth. The NGEO satellite needs to move at a much higher speed than the earth’s rotational speed (can be up to around 7.8 km/s) to get the necessary centripetal force to move around the earth at that low altitude. As a result, these satellites typically orbit around the earth pretty fast (usually within around 2-10 hours) as discussed in Section II-A. This quick orbital motion poses a great challenge for integrating NGEO satellites into traditional wireless communication systems. Due to the smaller orbital period, any specific terrestrial UE can be only visible to an NGEO satellite for a very short span of time, typically several minutes. So the UE needs to undergo multiple handovers within a short span of time interval regardless of its mobility [14]. If the satellite covers an area using multiple spot beams, the scenario is worse because the spot beam is much smaller compared to a total coverage area of an NGEO satellite. So the UEs need to through multiple (beam) handovers within a few minutes, even when they are stationary, for seamless continuation of data sessions. This frequent handover phenomenon in NGEO satellite networks creates a lot of overhead in communication channels, leading to an overall degradation in network performance.

Doppler shift is the shift in the signal frequency due to the motion of the transceivers. In the case of NGEO satellites, satellites are moving at a very high speed under a specific constellation. Due to this relative motion between UEs and satellites, Doppler shift happens in the original signal frequency. Due to frequency offsets, UEs tune to different carrier frequencies than the original carrier frequencies. So the frequency synchronization is lost, and the UEs may interfere with the other users. This is known as Inter-Carrier Interference (ICI) between multiple UEs. Generally, even for the high mobility scenarios in terrestrial networks, the frequency shift is pretty negligible, and so is the Doppler shift. However, the frequency offset is pretty significant in NTN due to the much higher speed of the NGEO satellites. The Doppler shift value mainly depends on the carrier frequency and height of the satellites. For NGEO satellites operating at Ka-Band, the Doppler shift can go from 225 kHz to even 720 kHz [20] depending on the heights. This can cause significant ICI among NTN users which requires efficient strategies for compensation of the Doppler effect.

Spectrum and power are the two fundamental resources for any communication system. In NTN, the allocation of these two resources becomes an even more complex problem due to the high path loss and limited spectrum availability. As discussed in Section II-D, the path loss associated with Non-Terrestrial Networks is much higher compared to terrestrial networks. To correctly decode the transmitted symbols from the received signals, the received signal needs to meet the minimum RSRP requirement. That means the transmitted signal power needs to be much higher (typically at least 10 times the terrestrial transmitted signals) than terrestrial signals. This poses a great obstacle for traditional UEs as they have power limitations. Furthermore, the target frequency bands as discussed in Section II-D for NTN are limited. To support a large number of satellite UEs, this spectrum resource appears to be scarce in NTN systems. So efficient resource (spectrum and power) allocation strategies are needed for integrating NTN into terrestrial networks.

As discussed in Section II-D, the S-Band and Ka-Band are the target bands for NTN. On top of this limited spectrum allocation, we have interference from terrestrial users in these bands. In S-Band, we already have existing terrestrial communication from 4G LTE devices. With the advent of mm-wave technology, terrestrial communication is also using the Ka-band in 5G. So the satellite users will suffer from co-channel interference with the terrestrial users in both bands. To avoid this interference, we have to come up with efficient spectrum-sharing techniques to put the interference below a certain threshold ensuring proper decoding of the received signals.

Timing advances ensure synchronous uplink transmissions for all UEs. The UEs can be located at different distances from the gNB, so there is a differential propagation delay between different UEs. If the uplink reception is not synchronized, the gNB needs to make sure the allocation of resource blocks to a specific UE does not include the resource blocks already in use by other UEs, which is inefficient in terms of resource allocation. Due to the long propagation delay, the TA is much larger than the transmission time slots in NTN compared to NR. Also due to the mobility of LEO satellites, the delay is time-varying and TA needs dynamic updates for proper uplink alignments. The other processes affected by the long propagation delay are Random Access, Hybrid Automatic Repeat Requests (HARQ) procedures, etc [20]. These procedures need to be modified properly to compensate for the long propagation delay.

On top of all these challenges, integration into existing terrestrial networks comes with several open research issues to be addressed. Computational offloading, which involves transferring the computational burden to satellite networks for supporting devices with low computing power, particularly in IoT applications, gets complicated due to extended propagation delay and highly mobile NGEO satellites. Network routing has been studied for a long time, and network slicing has been discussed since the implementation of 5G. However, with the emergence of NTN, the integration of terrestrial networks calls for research in this area with new effective strategies. The ever-changing network topology of mobile NTN platforms makes it challenging to solve these problems in a complex environment.

The Pre-Training Phase includes the choice of learning approach along with the necessary model initialization. The selection and design of the learning approach greatly depend on the nature of the problem for the learning systems. We show in the next few subsections different learning strategies for different problems. Each ML model generally requires some initial set of parameters and initialization that need to be carefully tuned to achieve expected performances.

After setting up the preliminary model with initialization, the most important phase – training begins. The training data is provided as input to the initial model. Typically the raw data collected for a specific problem may not be properly structured to be used for the model. Moreover, these data may contain redundant and unnecessary information which is not beneficial for learning the model. Consequently, data needs to be preprocessed in a suitable manner to have good performance. The features also need to be chosen in such a way that they can capture the correlation for empowering the learning process. The output of the model is fetched for performance evaluation. Based on the feedback from the evaluators, the model is adapted to improve its performance. This whole learning process is known as ‘training’.

After the training, we have a trained ML model based on our provided data. This model can be used to later evaluate in the real environment. Similarly, as training data, testing data can be generated and preprocessed for evaluation. The performance evaluator provides the accuracy of the model using the testing data as inputs. This whole process is known as ‘testing’. In the case of offline learning, the testing phase starts once the training is done. On the other hand, in the case of online training, the testing is generally executed in a parallel manner with training.

In an SL model, a training dataset containing a set of features as inputs and corresponding current outputs is provided to the model. The model with an initial set of parameters is trained through a number of iterations for mapping inputs to outputs. As the output label is clearly defined, the model can improve its performance by comparing its predicted outputs with the actual outputs [40]. SL problems can be broadly classified into two categories depending on the type of output labels: regression and classification problems. Regression [41] is a statistical method that investigates the relationship between a dependent (target) variable to one or more independent (given) variables. In this method, the functional mapping between inputs and outputs is estimated by minimizing the error between the predicted and actual outputs. Here the output label can be continuous. In classification, the output labels correspond to distinct classes arising in computer vision, image classification, etc. Generally, classification problems are solved by using probabilistic classifiers to map output classes from inputs. To train complex SL problems, NNs are used to learn complicated functional mapping between inputs and outputs. We discuss the major ML and DL approaches in the context of SL problems below:

ML Approaches: There are a number of SL algorithms to train the model. Linear regression [42] focuses on regression problems, whereas logistic regression [43] focuses on classification problems. Decision tree is used in classification problems by forming a tree-like structure to learn the best split at every node level based on a statistical measure like information gain [44]. The classification starts at the root node and traverses down along the branches based on intermediate decisions till the leaf nodes which represent the final classification decisions. Naive Bayes Model [45] is a form of a simple probabilistic classifier that uses the Bayesian Theorem to decide the classes under the strong assumption of feature independence. It is very useful, especially in high-dimensional classification problems. Support Vector Machine (SVM) [46] is another important type of classifier that decides the splitting hyperplane between different classes by maximizing the distances between the nearest data point (in both classes) and the hyperplane.

DL Approaches: Different DL approaches are also proposed in the literature to tackle complicated SL problems effectively. Perceptron [47] is one of the first NN architectures that have been proposed. It is a single-layer NN that can do binary classification like logistic regression. The main difference is to introduction of a simple activation function (step function) as a first step to more complex and advanced architectures. The simplest multi-layer NN architecture is the Fully Connected Neural Networks (FCNN) (Figure 8). This is also known as Multi-Layer Perceptron (MLP). It has multiple hidden layers between the input and output layers without any back loops. As the name suggests, all the neurons between two adjacent layers are connected to each other. Extreme Learning Machine (ELM) [48] is a very special type of NNs where the neurons are randomly connected and the training is done one-shot using least square fits. Another different type of NN is the Deep Residual Network (DRN) [49] with extra connections passing input from one layer to a later layer as well as the next layer. There are also Probabilistic Neural Networks (PNN) [50] which can recognize the underlying pattern and generate the probability distribution function for different classes.

Convolutional Neural Networks (CNN) [51] is an important type of NN that can take multidimensional inputs like images and classify them with great accuracy by discovering spatial features (Figure 8). The CNNs are composed of convolutional layers and subsequent pooling layers. The convolutional layers divide the whole input into smaller blocks and scan through them to learn the different features. The idea is to exploit the high correlation among neighboring cells with reduced complexity. A pooling layer is used to simplify this extraction process by getting rid of redundant features. Often CNN is accompanied by an FCNN to take care of nonlinearity and generate final classification results. A counterpart of CNN is the Deconvolutional Network (DN) [52] which takes the classes as inputs and generates CNN input features by comparing them with actual CNN inputs.

Another important type of NN is Recurrent Neural Networks (RNN) [53] with back loops. So the neurons in a layer are not only connected to previous layer neurons but also can be connected to the neurons from the subsequent layers. (Figure 10) This allows it to capture temporal correlation among different layers and can be useful where decisions from past iterations or samples can influence current ones. However, they suffer from vanishing gradient issues due to long-term temporal dependencies [54]. To tackle this issue more sophisticated architectures like Gated recurrent units (GRU) [55] and Long-Term Short Memory (LSTM) [56] with special memory cells and gates are introduced. Reservoir computing (RC) [57] is a low training complexity RNN framework for computation where the inputs are fed into a fixed and non-linear system, known as a reservoir, and then mapped into outputs from the reservoir neurons. Liquid State Machines (LSMs) are examples of RCs where the neurons are randomly connected receiving time-varying inputs. Echo-state networks (ESNs) are also a type of RC that uses a sparsely connected hidden layer (reservoir) with typically 1% connectivity. The connectivity and weights of hidden neurons are fixed and randomly assigned.

Another significant advancement in deep learning architecture, known as transformers, holds immense promise in the development of intelligent systems, particularly in communication environments. The transformer is a sequence-to-sequence neural network model comprising both an encoder and a decoder module, each with an identical architecture [58]. To streamline the input and output sequences, embedding and positional encoding layers are employed. Both the encoder and decoder primarily consist of a self-attention sub-layer and a position-wise sub-layer, with an additional masked attention sub-layer in the decoder. Each sub-layer is complemented by a residual connection and normalization module, facilitating the capture of long-range dependencies within the input data through self-attention.

In UL, a raw unlabeled dataset is provided to discover existing patterns and features [59] using some statistical learning approach. This is very useful when the data is not labeled. The algorithms find the underlying structure of the data and predict the outputs by adapting the model. Here the classes are not explicitly stated, so the classes need to be generated based on the distribution of input features in multi-dimensional spaces. It can be even used for generating labeled data to transform the original problem into an SL problem, which is usually easier to solve. Furthermore, clustering is another important UL problem where the model outputs different clusters based on the inherent pattern of data distribution. Dimensionality reduction can be also classified as a UL problem as it reduces the state space of the feature vectors in a general ML setup.

ML Approaches: There are a number of unsupervised learning algorithms in the literature. Principal Component Analysis (PCA) [60] is primarily used for dimensionality reduction of a high dimensional dataset. It reduces the number of correlated features converting them into a set of uncorrelated features, which are also termed principal components, using orthogonal transformation of basis vectors. Reducing the dimensions of inputs also reduces the number of features to be learned, which later can be leveraged in SL techniques. It is sometimes not considered an UL technique, but rather a preprocessing technique for data analysis with reduced dimensions. In Probabilistic Graph Models (PGMs), the probabilistic relationship between random variables is modeled through a graph [61].

K-means Clustering [62] divides all the data points into K clusters in which each data point belongs to the cluster having the nearest mean. The mean of the data points in a particular cluster defines the center of the cluster. Another variant of K-means Clustering is called K-medoids Clustering where the centralmost data point of a cluster is defined as the center of the cluster [63]. Various mixture models, such as the finite mixture model, Gaussian Mixture Model (GMM) [64], etc. are also used for clustering. Hierarchical clustering can cluster data into a hierarchy of groups without predefining the number of clusters. It also comes with increasing computational costs compared to other clustering approaches. k-Nearest Neighbours (KNN) [65] algorithm determines the k-nearest neighbors for all the data points of an unknown feature vector whose class is to be identified.

DL Approaches: Generally speaking, autoencoders [66] are used to help reduce the noise in data. In an autoencoder, first, we encode a high dimensional input, then decode it to reconstruct the input at the output again (Figure 10). The intermediate hidden layer neurons represent a compressed representation of the inputs getting rid of irrelevant and noisy components. Some other variations of this architecture are variational [67], noisy [68], and sparse [69] autoencoders. In variational autoencoders, the compact representation of data is used to generate new data points sampling from the latent space. In sparse autoencoders, the loss function is also expanded by adding another term called sparsity penalty regularization term for encouraging sparsity in the learned representations. In denoising autoencoders, robust representations are learned from the noisy input data.

Deep Belief Networks (DBNs) [70] is a probabilistic generative graph model composed of hierarchical layers representing feature vectors. Here the top layers create undirected symmetric connections among them forming an associative memory. Greedy layer-wise training can be used for DBNs [71]. Two symmetric DBNs can be extended to the structure of deep autoencoders for efficiently decoding the feature vectors [72]. To use the feature extraction capability CNNs for UL, a combination of CNN and DBN is used in [73]. Hopfield NN [74] is a cyclic recurrent NN architecture where all the nodes are connected to each other. This provides an abstraction of circular shift register memory to form a global energy function and finding clusters without a supervisor. The Boltzmann Machine is another type of recurrent NN that has a stochastic symmetric recurrent architecture [75]. As the convergence rate is generally slow for these NNs, a variant of this, Restricted Boltzmann Machine (RBM) is designed to learn the probability distribution over input data but in a layered manner [76].

In UL, competitive learning approaches, such as Self-Organizing Maps (SOMs) [77], each neuron competes to represent an input subset. Here a single neuron from a group of output neurons is activated while the other neurons adjust their individual values in regard to input data distribution. Generative Adversarial Networks (GAN) [78] consist of any two networks, with one generating data (generative network) and the other judging the generated data (discriminating network). The prediction accuracy of the discriminating network is then used to evaluate the error for the generating network. This creates a form of competition between the discriminator and the generator to get better in their corresponding tasks. We can also use ensemble learning methods [79] comprising multiple learning methods for better performances.

Generative Diffusion Models (GDMs), as introduced in [80], represents a recent breakthrough in UL leveraging DL techniques, drawing inspiration from the principles of thermodynamic diffusion. GDMs have gained widespread recognition due to their remarkable ability to generate high-quality data and simple implementation procedures. In contrast to GANs, GDMs employ a denoising network that iteratively converges to an estimate of the real sample. This model works in two distinct phases: the forward and reverse diffusion processes [81]. In the forward diffusion phase, Gaussian noise is gradually introduced through a series of steps to create the target input for the denoising network. Subsequently, the denoising network is trained to reverse the noise effect for generating the original content.

In RL, an agent learns to behave in a particular environment by performing thousands of actions and getting rewards or penalties based on those actions [82]. This behavior (formally known as policy) is defined by the set of actions the agent learns from its experiences. The environment is defined by some mathematical models, the most common one is the Markov Decision Process (MDP) [83]. Here the feedback is neither provided using explicit labels like in SL nor the model is learned like in UL, but the behavior of an agent is learned through the rewards or penalties based on the set of actions taken by going from one state to another with a transition probability. The goal is to find out the optimum policy so that the total reward can be maximized (or the total penalty can be minimized) over a horizon of future time intervals given the current state of the agent. In Figure 11, we show a generic structure of an RL framework as an MDP model.

ML Approaches: Depending on whether an RL model is explicitly created or not, RL can be fundamentally divided into two major categories: model-based and model-free RL methods [84]. In the model-based methods, the transition probabilities between different states are assumed to be known, whereas, in model-free methods, these probabilities are learned through iterations. Dynamic Programming (DP) [85] is the most popular model-based method used in practice. However, model-free methods like the Monte Carlo (MC) method [86] are most commonly used due to their flexibility and practicality in real systems. Q-learning [87] is one of the most popular model-free methods where ”Q” refers to the expected rewards for an action taken in a given state over the time horizon, known as the value function. Another important counterpart of Q-learning is the State-Action-Reward-State-Action (SARSA) learning method where the agent learns the optimal policy in an online fashion [88]. This Q-learning is extended to the context of stochastic games [89] involving multiple agents in [90], which is also known as the Multi-Agent Reinforcement Learning (MARL) method. Another approach for model-free RL is to learn the policy directly instead of learning the value functions, which is known as the Policy Gradient (PG) method [91]. If the policy gradient is estimated in a deterministic fashion, it is called the Deterministic Policy Gradient (DPG) method [92]. To have the benefits of both approaches, the Actor-Critic (AC) method is proposed in [93] where the critic estimates the value function and the actor updates the policy gradient in the direction suggested by the critic.

DL Approaches: General RL approaches without NNs can work well for small-scale experiments. However, when the action or state space is really large, the computation complexity exponentially increases. This phenomenon is quite common in practical systems like communication networks. To estimate the value functions or policies with RL frameworks with large state or action spaces, DL approaches can be very useful. This learning approach is also known as Deep Reinforcement Learning (DRL). The most popular and simplest DRL approach is the Deep Q-Network (DQN) [94] method. In this approach, instead of an iterative approach for updating Q values in the Q-table, an NN is used to estimate the Q function value approximately. To prevent a large overestimation of action values, another DL framework is introduced on top of DQN for a fair evaluation of policies in Double Deep-Q-Network (DDQN) [95, 96]. In Dueling Deep-Q-Network [97] method, both state and action values are separately estimated. As the expected value function may be overestimated as the expected value does not capture the complete probability distribution of random variables, Distributional Deep-Q-Network [98] is considered to update the Q function value based on its distribution. In the case of continuous action spaces, DL aided DPG method, known as Deep Deterministic Policy Gradient (DDPG) Q-Learning [99] provides better results. To deal with partial observable environments, Deep Recurrent Q-Networks (DRQN) [100] by introducing an LSTM layer in the FCNN architecture of DQN. Similarly, as for RL, MARL approaches can be efficiently solved using DRL architecture for each agent, namely known as a Multi-Agent Deep Reinforcement Learning (MADRL) framework.

FL is a distributed learning technique in which multiple network data owners collaboratively build and train a global DL model, all while ensuring data isolation and privacy, as outlined in [101]. In the FL paradigm, individual data owners are initially provided with a base model by a centralized server, which they then train using their own respective data. Subsequently, these locally trained models are shared back with the central server, allowing it to update and maintain a global model. This iterative process continues until the global model reaches convergence. Consequently, FL enables the development of a globally trained model through distributed efforts, all the while safeguarding the privacy of the data owners.

DcL involves computing nodes conducting local training on their individual DL/ML models and then sharing these models with neighboring computing nodes at each iteration. Global convergence is achieved when all the local models have converged. Notably, this approach ensures that no actual data is exchanged between the computing nodes, but the local models are shared among neighboring nodes. One notable advantage of DcL is the absence of the need for a centralized server, which is a requirement in FL. An illustrative example of DcL can be found in the context of MARL, where agents collaboratively train their models in a distributed manner, as elaborated in [102].

In SpL, instead of sharing model parameters, training occurs across various computing nodes, as described in [103]. Each computing node has the responsibility for training multiple layers of NNs within a DL model. Gradients for backpropagation are exchanged among these nodes to enhance training efficiency. Consequently, it can yield superior privacy performance when compared to FL, as indicated in [104]. Recent endeavors have been made to combine these two approaches, aiming to harness the advantages of both methodologies, as explored in [105].

In NTNs, the complexity of tasks and procedures involved in communication networks is significantly heightened. These tasks encompass a wide range of operations, including resource allocation, channel estimation, modulation, coding, and the intricacies of satellite management control. Attempting to perform these tasks manually is not only challenging but often unfeasible. The complexity involved in optimizing network performance for satellites, in particular, renders manual operations insufficient. Moreover, these tasks require meticulous precision to ensure uninterrupted service and mitigate potential hazards. However, the advent of ML and DL approaches has By harnessing ML and DL, not only can accurate actions be executed, but complex chains of procedures can also be automated seamlessly without the need for human intervention following the general ML framework as discussed in Section III-A.

The deployment of next-generation NTNs is more complex than any other previous-generation cellular network due to its multifaceted architecture. For instance, the integration of satellite networks introduces a significant number of additional parameters to consider for optimum network performance [107]. However, this can result in computationally intractable solutions for practical networks, even if the solutions are computationally tractable, they may be very inefficient. Resource management in TNTN networks is a prime example of this, as resource optimization in TNTN networks often turns into non-convex optimization problems, where only suboptimal or heuristic solutions can be obtained using numerical techniques [108, 109]. Fortunately, DL techniques can approximate complicated functions involving a large number of input variables with the help of NNs, as discussed in Section III-C. As a result, complicated network functionalities can be characterized with NNs and resource management issues can be solved in a tractable manner [110, 111].

Although probabilistic and deterministic models can be used to model NTN functionalities, these models are often derived using very strong assumptions to get the general closed-form expressions, resulting in significant deviations in performances in simulations compared to real networks. In contrast, ML models are obtained based on real data, which means different scenarios are taken into account during training, without the need for making any assumptions. For instance, resource scheduling for users or network slices in a cellular network is typically decided based on the channel condition of the corresponding users or user groups. However, the channel is highly time-variant, so the decision feedback needs to be in real-time to incorporate optimal scheduling decisions for all the users in the network. For NTNs, the scenario is worse due to the extremely time-variant nature due to the high mobility of NGSO satellites and dynamic propagation environments. Various AI models, on the other hand, have shown great promise in dealing with this kind of challenging problem due to their potential to capture real scenarios with more precision than theoretical models with a reasonable amount of computation complexity.

AI algorithms can adapt to changing network conditions and learn from experience. Through ML techniques, AI can continually improve its performance, optimize network operations, and adapt to evolving user demands. By leveraging AI techniques such as RL and predictive modeling, NTNs can adaptively allocate resources, optimize network parameters, and proactively detect and mitigate faults through online learning as discussed in Section III-B. AI enables NTNs to dynamically respond to changing network conditions, enhance operational efficiency, and ensure uninterrupted service delivery. The ability to learn from data and make intelligent decisions without human intervention empowers NTNs to continually improve their performance, optimize resource utilization, and deliver reliable connectivity in complex and evolving environments.

Obtaining optimal algorithms for various challenges in NTNs can be a daunting task. Even if such algorithms are derived for complex systems, their computational complexity often renders them impractical for real-world implementation. This complexity arises from the vast number of variables that govern different network procedures in NTNs. However, data-driven AI techniques offer a promising solution by reducing the dimensions of high-dimensional data through feature learning. Particularly, DL approaches have demonstrated remarkable effectiveness in extracting implicit features from complex systems. As a result, these techniques prove highly valuable in addressing the diverse challenges encountered in NTN environments.

In some cases, traditional methods heavily rely on the exchange of information between various network participants, such as satellites and users. This might lead to a large overhead in communication channels, resulting in a decrease in the overall throughput of the network. AI can be used to reduce the control overhead of NTNs significantly. For example, to calculate the Doppler shift, the UEs must be provided with the latest ephemeris information of the satellites [112]. However, this would cause an immense overhead and a decline in the achievable data rate for the UEs. Alternative DL techniques can be employed to estimate the Doppler shift without requiring any ephemeris information from the satellites [113]. This leads to a significant decrease in transmission overhead over communication channels, resulting in superior network throughput.

Network optimization and management decisions in NTNs usually require real-time implementation, usually in the order of milliseconds to tens of milliseconds. Consequently, complex algorithms cannot be used to obtain these real-time decisions. In most cases, the algorithms become either heuristic or offline. To have an online adaptable approach, AI techniques can be considered as a suitable option. For example, an online DRL-based approach as discussed in Section III-B can be used to obtain resource management decisions in real-time and ensure proper utilization of available resources in NTNs [114]. This is particularly valuable in latency-sensitive decision-making, such as scheduling, handover decisions, etc.

In communication networks, CSI is fed back to the BS from the UE to assist in selecting different schemes - such as modulation, channel coding, etc. - for improved network performance. Leveraging this data, which contains the general state of the channel, different ML approaches can be benefited. For example, RL approaches can use this data to train models. This implies that we do not need to modify the information segments sent from the UE to the BS for deploying these RL schemes, but rather can rely on feedback already existing in the communication networks. This again illustrates the capability of AI to integrate into traditional communication networks without any additional overhead costs. For NTNs, this is more important as the spectrum is more scarce and expensive; utilizing traditional CSI feedback for learning becomes another motivating factor for AI approaches to NTNs.

Channel estimation is an important aspect of NTNs, serving a dual role in encompassing comprehensive network planning and managing interference, similar to other wireless networks. This entails the technique of estimating the impacts of the channel through which a transmitted signal traverses in a wireless environment. Conventionally, the channel effect is encapsulated in an information block termed CSI in modern communication systems. While conventional methods like MMSE or Least Squares are employed for CSI estimation, they often entail high computational costs and may not always align with the demands of real networks. Furthermore, obtaining timely CSI information gets more challenging due to extended propagation delays and fast-changing propagation environments in NTN conditions.

Therefore, ML-based methods are increasingly being adopted by the research community and vendors, as a promising alternative for channel prediction. This channel estimation can potentially be turned into an SL problem by considering channel features such as distance, time delay, received power, azimuth Angle of Arrival (AoA) and Departure (AoD), elevation angle, Root Mean Square (RMS) Delay Spread, and frequency as inputs and CSI as output labels. In [148], the reciprocity property of the downlink and uplink channels in Time Division Duplexing (TDD) systems is considered. So the downlink channel is estimated from uplink CSI using an LSTM-based DL model. In [149], CSI is estimated from historical CSI data using a CNN-LSTM model. However, as channel estimation is a near-real-time process, low-complexity NNs such as ESNs need to be explored for realistic implementations. In [150], a denoising CNN is used to reduce the LS channel estimation error. In [151], a CSI prediction scheme is presented without utilizing any ephemeris information, rather only using past CSI feedback information leveraging GRUs with low prediction error. In [152], an auto-regressive integrated moving average of past CSIs is utilized to predict future CSIs where the order of the past ones is determined by an LSTM network. In [153], graph attention networks are used for cascaded channel estimation for Reflective Intelligent Surface (RIS) assisted satellite networks in IoT communications. In [154], future CSI information is predicted using k-Nearest Neighbour and MLP-based algorithms from past CSI and some correlated network metrics such as latency, terminal velocity, weather, and environment state, etc. which is later used to adapt the modulation and coding scheme for next timestamp. In [155], an RNN-based CSI compression technique is presented especially focusing on future SAGIN networks. An ANN is trained to estimate the fading at 40 GHz band exploiting the knowledge of its previous channel states in [156]. In [157], an LSTM-based CSI prediction framework is discussed to provide in future NTN-integrated 6G networks.

As the LEO satellites move around the Earth typically at a very high speed, both the satellite and ground user transceivers experience a large Doppler effect due to their relative velocity. If the transmitter moves towards (or away from) the receiver, the emitted signal from the transmitter may take less (or more) time to reach the receiver depending on the direction of the movement, hence the frequency of the signal increases (or decreases). This shift in signal frequency due to the motion of the transmitter, the receiver, or both refers to the Doppler shift. If the original frequency is $f_{0}$, the Doppler shift due to the motion of transceivers towards some specific direction with some specific relative velocity can be given by:

For LEO satellites, due to high mobility, This frequency offset is pretty significant (48 kHz with a center frequency of 2GHz [20]). Due to these frequency offsets, UEs tune to some different carrier frequencies from their originally assigned carrier frequencies. This may lead to ICI between multiple UEs as discussed in Section II-E.

There have been significant efforts to characterize the Doppler effect for LEO satellites since the launching of communication satellites. In [161], an equation for Doppler shift is derived for the simple case of LEO satellites with circular orbits in the equatorial plane and ground observing points on the equator. In [162], the Doppler shift is analytically derived assuming the trajectory of the satellite with respect to the earth by a great circle arc and the speed of the satellite as constant. In [112], the Doppler shift is characterized by considering a new orbit generator using different orbital parameters through a rigorous analysis. UEs with Global Navigation Satellite System (GNSS) can get the global positioning of satellites and estimate the amount of Doppler shift needed to be addressed for the next transmission slot [163]. However, this increases the cost and complexity which may not be feasible for ground UEs [31]. Additionally, The GNSS signals are weak, not ubiquitous, and susceptible to interference and spoofing. Recently, there have been also efforts to estimate the Doppler shift in LEO satellite systems using various other approaches, such as stochastic geometry [164], Maximum A Posteriori (MAP) [165], algebraic solutions [166], two-stage estimators consisting of time-varying Burg spectral analyzer and alpha-beta filter [167] etc. In [168], the Doppler shift is estimated using reference signals in more than one frequency position in Orthogonal Frequency Division Multiplexing (OFDM) carrier in a 5G integrated NTN system.

These different theoretical approaches can estimate the Doppler shift with a certain accuracy in different scenarios. However, the methods are generally very cumbersome due to the complexity associated with the orbital mechanics of the satellites. Most of these methods come with simplifying assumptions to keep the approach feasible for practical systems, thereby affecting accuracy. Moreover, due to the constant high-speed movement of LEO satellites, the wireless environment associated with it becomes time-variant. The computation complexity increases more to model these temporal variations using traditional estimation approaches. Additionally, the UEs may need the ephemeris information of the satellites to compute the Doppler shift associated with its motion, which creates large additional overheads in the communication channels. To characterize this Doppler effect, ML-based algorithms seem to appear as potential practical alternatives to the research community.

In wireless communication systems, due to the mobility of the transceivers, the channel between the transceivers changes significantly resulting in received signal power variation and Doppler shift. So, intuitively, the CSI of this channel should contain information about the Doppler shift. This idea has been already explored in terrestrial networks to generate a model using ML[158, 113, 159]. The ground truth values or the labels are usually generated using the ephemeris information. Different channel characteristic variables like Rician K factor, azimuth AoA width, mean azimuth AoA and channel estimation errors are generated randomly, and averaged Power Spectral Density (PSD) is used as inputs with some preprocessing to a multi-layered FCNN to estimate the Doppler shift in [158]. In [113], RSRP values mapped from an ambiguity reducer are used to generate the weights for an MLP. In [159], different time and frequency domain signals with various modulation schemes, delay profiles, and Signal to Noise Ratio (SNR) have been used as inputs to a hybrid CNN-LSTM model to estimate the Doppler shifts. In NTN, the research in this domain is still at the early stage The estimated CSI is used as input to a CNN model to estimate the Doppler shift in [160]. In the future, other potentially efficient SL models can be also explored to generate the real-time accurate Doppler shift in an online manner. In table IV, we summarize the AI approaches for Doppler shift estimation in NTN. Even though the DL techniques are found to be useful in estimating Doppler shift using channel parameters, Doppler shift can be also estimated by analyzing the predictable trajectory of the satellites. Complexity analysis is required to justify the applicability of these DL architectures replacing the state of art methods in real systems.

Due to the new interfaces introduced by satellite-integrated terrestrial architectures, various spoofing and replay attacks can be launched using these interfaces. Spoofing attacks involve an attacker satellite impersonating a legitimate one, while replay attacks involve the retransmission of previously intercepted messages to deceive users. Generally, in terrestrial networks, these kinds of attacks are detected and mitigated by using standard cryptographic techniques, a concept also investigated in satellite communications [169, 170]. However, when it comes to NTN-integrated future 6G networks, these conventional cryptographic methods face several challenges. Firstly, these techniques are computationally intensive and, thereby challenging to implement in satellites due to their limited onboarding capabilities. Secondly, the highly dynamic and massive scale network topology of NTNs, particularly for enabling IoT devices, necessitates significant modifications in network protocol design and introduces overheads that may not be practical to manage with existing architectures Also, these cryptographic techniques often assume that attackers lack the computational resources to break the encryption. However, with ongoing advances in quantum computing research, these assumptions may no longer hold in the future, presenting yet another challenge that needs to be addressed.

Physical layer authentication offers a promising alternative to these conventional techniques. In [171], Wyner introduced the concept of physical layer authentication where the message was encoded in such a way that the mutual information between the legitimate channel and wiretap channel is maximized. This encoding generally captures the unique characteristics of the channel between the user and the legitimate transmitter, serving as a means to verify the transmitter’s identity. This technique has been already explored in terrestrial networks using the CSI information as radio signatures for the transmitter devices [172, 173, 174]. However, the prevalence of Line of Sight (LoS) paths in satellite networks makes radio fingerprinting using channel fading information from CSI impractical. Furthermore, due to the high mobility of NGSO satellites, as discussed in Section II-E, the high Doppler Shift is introduced in received signals, which can be used to verify the identity of the legitimate satellites. In [175], a maximum likelihood estimation and uniform quantizer are used to obtain the secret key bits from the Doppler frequency shifts, which is used in the authentication of legitimate satellites [176] In [177], an orbital information – time difference of arrival-based authentication mechanism is introduced providing low false authentication rates.

In the recent past, various DL techniques have shown lofty promises in the field of extracting features from noisy data, which is also leveraged in this field using different ML models. In [178], CNNs and Autoencoders are used to extract the necessary channel features for physical layer authentication of legitimate satellites. In [179, 180, 181], both the received signal power and Doppler shift are used for radio fingerprinting using SVMs providing improved authentication rates.

In modern satellite-terrestrial integrated networks, the majority of satellite communication systems rely on elementary security threat detection mechanisms. Typically, these mechanisms operate by flagging an anomaly if the received signal frequency deviates from the baseline spectrum by a predetermined threshold. However, this simplistic approach frequently leads to a significant number of false positives. On top of that many anomalies represent unusual behavioral patterns, exhibiting temporal correlations that escape detection by these simple detectors. Consequently, these conventional methods often struggle to effectively identify and respond to sophisticated security threats.

To address these challenges, DL techniques are explored to efficiently detect security threats using various innovative approaches. In the study presented in [182], an ensemble model combining Random Forest (RF) and MLP is developed to improve the performance of security threat detection across diverse datasets for satellite communications. [183] leverages critical feature selection driven by RF to streamline complexity and enhance the relevance of features before the detection phase. These features are then forwarded to different NN architectures, including LSTM, GRU, RF, and ANN enabling robust security threat detection. These models are tested on different datasets where GRU-empowered threat detection models exhibit superior performances by capturing temporal behavioral patterns. In [184], a UL approach using LSTM networks is explored, which can not only detect unforeseen security threats but also does not need any labeled data. In another study as shown in [185], two SL and five UL approaches are considered for threat detection to show the effectiveness of ML techniques. In [186], a DDPG-based DRL framework is considered where the agents decide whether the aerial platform is malicious or not (actions) based on their behavior (states) and the system condition (rewards) for threat detection. Recognizing the computational constraints of satellites and Internet of Things (IoT) devices, federated learning approaches are also investigated as detailed in [187, 188, 189] for threat detection.

Satellites are vulnerable to jamming threats due to their predictable and periodic visibility in NTNs, so anti-jamming approaches are important to tackle this challenge. Conventional spread spectrum techniques are used in anti-jamming for satellite networks. However, they are not very useful in dealing with new smart jamming attacks which can adjust their actions based on the network feedback. Various RL techniques are adopted to tackle these problems in an efficient manner. In [190, 191], a hierarchical anti-jamming Stackelberg game is introduced for routing anti-jamming problems which is later solved by providing fast anti-jamming decisions using a DRL-based routing algorithm for satellites. in [192], a DL-based jamming detection algorithm is proposed for satellite navigation systems. In [193], an anti-jamming coalition game is formed to decrease energy consumption, and suboptimal jamming policies are obtained by RL approaches. In [194], ML-aided cognitive anti-jamming communication is designed, developed, and tested on real satellite-ground links.

Modern communication satellites form multiple beams to support a large number of users over a large area through spatial multiplexing into different NTN cells. Each satellite can effectively reuse the allocated spectrum with very low co-channel interference as well as provide strong signals at the ground user terminals with relatively low transmission power using beamforming techniques. However, due to the high cost and low availability of onboard processing computing resources in satellite systems, mostly simple fixed beam allocation policies are used in traditional satellite communication. These strategies lack the flexibility to adapt to the temporal and spatial variation of traffic demands in real satellite networks. Beam hopping is a technique for allocating beams in a flexible manner so that these changes can be addressed efficiently. It refers to a procedure for activating different beams according to the current demands of an NTN cell covered by those beams, so effectively hopping the set of active beams from one combination to another [195]. In Figure 13, a simple beam-hopping scenario is depicted, where we have different NTN cells with varying demands. We classify the cells into three different categories, e.g., high, medium, and low, based on their traffic demands. In the first scenario, the low-demand NTN cells, e.g. cell $9$, have less number of active beams than high-demand NTN cells, e.g. cell $6$, even lesser than moderate-demand NTN cells, e,g, cell $1,2,5$ or cell $13$. However, due to mobility or change in traffic patterns, the traffic demand in cell $13$ reduces and in cell $5$ increases. As a result, we can see the intensity of the beams also changes accordingly in these two cells at a later time, and a new beam-hopping pattern emerges.

The key question of beam hopping is to find out which beams need to be activated when and for how long while maximizing the network performances given the capacity constraints [202]. This can be effectively formulated as an optimization problem considering different network performance metrics such as system throughput, delay, fairness, etc. as the objective(s) along with power and spectrum constraints. In [203], a convex optimization framework with an objective to match the system capacity to traffic demand along with power allocation constraints is considered. This yields a close-form solution giving insights into resource allocation policies for maximizing network performances from different perspectives. However, from the perspective of real networks, the convex objective function is not very realistic, so the results are not applicable to real networks in a straightforward manner. Assuming the non-convexity of the problem, obtaining a globally optimal solution with efficient algorithms gets difficult. In [109], the steepest gradient descent algorithm is chosen to get the sub-optimal solution using the optimal set of precoding vectors. Some heuristic iterative approaches are also proposed in [108], [204], [205] to tackle these non-convex problems in a practical and feasible manner. Different meta-heuristic approaches like Genetic Algorithm (GA) [206], Simulated Annealing (SA) Algorithm [207], Particle Swarm Optimization (PSO) [208], and combined metaheuristic approaches like GA-SA [209] have been considered to generate suboptimal solutions with a reduced amount of computational complexity.

The main challenge in designing a beam-hopping pattern in an optimization framework lies in the large search space associated with an optimal solution. The size of the search space for finding out an optimal beam hopping pattern scales exponentially with the number of beams in the satellite networks. Modern satellites can have hundreds to thousands of beams depending on their coverage area, so the computational complexity becomes pretty high, and the computation time becomes pretty large to find out the exact solutions. The low-complexity suboptimal solutions using iterative and metaheuristic approaches achieving satisfactory performances in real networks are not very abundant. In this context, the DL approaches turn out to be a suitable alternative for this problem.

In [110], [111], an SL approach is considered by forming labeled datasets with beam hopping patterns as outputs and channel matrix, transmission power, and traffic demand as inputs. First, a mixed integer linear problem formulation for matching the offered capacity to traffic demands is reduced to a simple linear programming problem. A training dataset is generated using conventional optimization algorithms and a DL model is trained on this dataset by considering beam hopping patterns as labels. Furthermore, the optimization framework can be potentially transformed into an RL problem to capture the optimal beam-hopping pattern in a learning environment. In [196],[197] the transmission delay is minimized considering the power and beam allocation constraints using a DRL approach. The state space consists of the average transmission delay and the buffer length with beam hopping pattern as actions and the negative Hadamard product of the current states, the negative of total queuing delay as the reward function. In [198], a combined DRL-metaheuristic approach is considered to optimize both the throughput and delay fairness while at the same time designing different reward functions for the two cases. In [199], [200], a network consisting of real-time and non-real-time traffic is considered. A multi-objective problem minimizing the transmission delay for real-time traffics, maximizing the throughput for non-real-time traffics as well as overall delay fairness is considered. Individual reward functions are designed to capture each of the goals. In [201], a cooperative multi-agent framework is considered to dynamically allocate the power and bandwidth to illuminating beams optimizing throughput and delay fairness using a DDQN. In table V, we summarize the AI approaches for beam-hopping in NTN. As traffic demand changes with time, recursive architectures such as RNN, ESN, etc. should be also explored to design the NN for DL architectures used to address beam-hopping issues. Also, distributed learning architectures can be useful to design efficient beam-hopping schemes.

In traditional communication systems, satellite, and terrestrial cellular networks generally occupy different frequency bands, so they do not interfere with each other. However, the satellites in the new integrated TNTN environment for 6G are expected to use the same S and Ka-Band as discussed in Section II-D. This improves the overall spectral efficiency of the integrated networks as well as provides a better QoE for the users. However, as both the satellite and the ground network use the same frequency band, the signals

produced by them interfere with each other, i.e. cause Co-Channel Interference (CCI) to each other. In Figure 14, a simple spectrum-sharing scenario in the downlink channel in an integrated TNTN network is shown. The satellite user is connected to a satellite and the downlink channel is indicated using the green link. There are three more terrestrial BSs using the same channel as the satellite provide CCI to the satellite user (indicated by red links).

In TNTN, the spectrum-sharing phenomenon needs more attention because we have a hierarchical network scenario consisting of non-terrestrial and terrestrial BSs as shown in Figure 3. To support this complex topology in a single framework, we need to come up with efficient spectrum-sharing strategies causing low interference to the users [217]. In conventional spectrum sharing methods, we use efficient frequency reuse, leveraging directional antennas, adaptive power control, etc. methods to mitigate the effect of CCI. However, traditional four-color frequency reuse can effectively reduce the level of interference at the expense of more spectrum. beamforming approach can reduce the interference greatly, but that too comes at the cost of increasing complexity.

To tackle this situation, a process called spectrum sensing is introduced in cognitive radio networks, where the unlicensed users can sense the occupancy status of the target band using some radio sensing method [218]. The popular spectrum sensing methods are Energy Detection (ED)[219], Cyclo-Stationary Detection (CSD) [220], Eigen Value-based Detection (EVD) [221] etc. However, these methods either are simple with poor performance in low SNR scenarios (ED) or provide good performance but with more computational complexity (CSD and EVD). For these reasons, ML has been adopted for spectrum sharing to capture the correlation with a reduced computational complexity which can be extended to integrated satellite-terrestrial network scenarios [217].

Different intelligent learning approaches are adopted to tackle the spectrum-sharing problem for next-generation TNTN networks [222]. In [210], a spectrum-sharing strategy is developed for LEO satellites from the GEO satellite spectrum historical occupancy data using a CNN-BiLSTM model. Here the LEO satellite users are considered as unlicensed secondary users and the GEO satellite users are considered as the licensed primary users. In [212], a CNN-LSTM-based spectrum sensing method is introduced for satellites to capture the spatial and temporal correlation effectively for spectrum occupancy of satellite systems. In [213], a CNN-LSTM model is introduced to predict the frequency assignment for satellites based on historical data. In [214], a modified Q-Learning algorithm is used in an RL setup for the adaptive selection of access and modulation schemes for NGSO satellites in an NGSO-GEO system. In [215], an SVM model is first used for low complexity spectrum sensing, then a CNN-based spectrum prediction model based on historical data is developed. In [216], a cooperative MADRL framework is considered for bandwidth management in a game-theoretic model minimizing inter-beam interference. In [211], a CNN-based spectrum reconstruction method from incomplete data is discussed for satellite networks. In table VI, we summarize the AI approaches for spectrum sharing in TNTN. However, these spectrum sensing decisions need to be in real-time to increase the overall throughput of the secondary users; this means the conventional LSTM architectures need to be replaced by efficient low-complexity ESN architectures to tackle this in an online manner. Furthermore, spatial spectrum sharing scenarios need to be also considered along with the state of art temporal spectrum sharing scenarios leveraging the benefits of future 3D SAGIN networks.

Power and spectrum are the two fundamental resources for any type of wireless network, and NTN is also not an exception. The spectrum allocation is typically performed by the assignment of carriers with equal width from the allocated spectrum for that service. Hence, the number of assigned carriers and their positions are optimized to achieve good signal quality with the minimum resources. Often, the carrier assignment is achieved by the orthogonal splitting of the spectrum resources, which is also known as frequency reuse. However, the strict orthogonality of the frequency bands cannot be always achieved to achieve better spectral efficiency. In case of lack of orthogonality of spectrum resources used by different transceivers can also introduce CCI. The interfering signal can be effectively suppressed by increasing transmission power for the original signals. However, as power is also a scarce resource, we cannot increase the transmission power indefinitely and increasing transmission power will result in a decrease in energy efficiency. For better resource utilization, a more robust radio resource management needs to be designed by controlling both power and spectrum resources [231].

Generally, an optimization framework can be considered to optimize the system performance with bandwidth and power constraints. In most cases, such optimization problems are non-linear and non-convex due to objective function nonlinearity and complex constraints involving Signal to Interference and Noise Ratio (SINR) [232]. Furthermore, the carrier assignment indicator variables result in a mixed-integer programming problem [232]. Hence, no optimal solution can be determined using the known methods of convex optimization with low computation complexity. Instead, suboptimal and metaheuristic approaches are proposed, which tackle parts of the problem separately and then iteratively tune the parameters [232]. Different suboptimal approaches are adopted to optimize resource allocations [233, 234, 235, 236, 237] for satellite systems. However to reduce computation complexity several heuristic [238] and metaheuristic approaches like GA [239], PSO [240] are explored to reach the suboptimal solutions within a shorter computation time.

To tackle this resource allocation issue in real satellite networks in a practical manner, ML approaches are being started to be adopted by the research community. A DL framework is combined with conventional optimization algorithms to overcome the computation complexity issue of the conventional approach in [230, 229] by reducing the feature space. A model-free DRL framework is adopted for power allocation of high throughput satellites in [226]. A Q-learning-based long-term capacity allocation algorithm in an RL framework is introduced for a heterogeneous satellite network in [223]. In [114], an Actor-Critic and Critic Only based RL framework is considered for optimal resource allocation for LEO satellite networks. Different advanced RL frameworks like DRL [224, 241, 225], Multi-objective DRL [228] and MADRL [227] are also proposed to solve the resource allocation issue for satellites. In table VII, we summarize the AI approaches for resource allocation in TNTN. As both power and spectrum are equally important and scarce resources for NTNs, new DL architectures need to be explored to jointly allocate these resources in an efficient manner for NTNs.

Network slicing refers to the process of virtually partitioning the physical network into different network slices corresponding to different service requirements. The slices are allocated with radio resources as per the demand of the users belonging to the slices. Slicing is useful for wireless networks as each slice can share the same physical network infrastructures while receiving necessary radio resources for guaranteeing a minimum level of service to the users. Also, network slicing provides the flexibility to switch users between slices with different amounts of allocated resources responding to changes in traffic conditions. Integrated TNTN networks appear to be an excellent candidate for applying the concept of network slicing due to their diversified traffic patterns. In fact, different use cases like mMTC and eMBB applications can be extensively benefited through the network slicing in these networks. In Figure 15, a simple network slicing scenario is shown. Here the network consists of a satellite and a terrestrial BS which form 3 slices in a combined manner. Slice 1 is for high-priority users, they share network resources from the satellite and the terrestrial BS (depicted by green links). Slice 2 is for users with low latency requirements, the terrestrial BS provides resources to the users (shown by red links). Slice 3 is for the remote users who can only be served by the satellite (shown by blue links).

In a general network slicing framework, a composite utility function consisting of different network performance characteristics like average throughput and other costs like slice reconfiguration cost, resource reservation cost, etc. is formulated as an objective function that needs to be minimized. The constraints are generally the minimum service level To ensure real-time implementation, simple heuristic approaches are tested on real platforms [242, 243, 244]. requirements depending on the type of services for particular slices. In [242], an extensible 5G network slicing framework in conjunction with satellite networks is discussed to facilitate the integration of satellite services into 5G. In [245], a multi-objective optimization problem comprising latency, computational, and power requirements in an edge-computing scenario is formulated to find suitable slice scheduling strategies based on numerical methods. However, these approaches do not guarantee optimal performance guarantee. To tackle this issue, different AI-based approaches are explored as it is done in the case of traditional 5G terrestrial networks.

In [246], RL-based network slicing frameworks for satellite-integrated future 6G networks are discussed along with experimental results for simple networks. In [247], AI-based network slicing for space-air-ground integrated vehicular networks is discussed from the perspective of slice creation, user association, and resource scheduling. In [248, 249], satellite-terrestrial network slice resource allocation frameworks utilizing network function virtualization are presented which can be leveraged for applying advanced AI-based methods. In [250], FCNNs are used to train a suitable set of network parameters that can produce latency similar to a non-linear optimizer-based network slicer. In [251], a general Radio Access Network (RAN) slicing problem is considered where the objective function is a weighted function of bandwidth and spectrum consumption satisfying QoS and inter-slice isolation constraints. In a simple 2-slice satellite-terrestrial integrated network, different DL architectures are tested. In [252], an ML approach similar to the meta-heuristic ACO approach is considered to realize network slicing in a TNTN environment. An air-ground integrated network is considered in a DRL framework in [253], later solved by the DDPG algorithm. Here both the actor and critic networks are FCNNs consisting of four layers. Distributed learning architectures can be potentially explored in future works for real network implementations.

In order to maintain an orbital path around the Earth at a lower altitude, an LEO satellite needs to move at a much higher velocity (around $7.8$ km/s) compared to a GEO satellite. So these satellites orbit around the Earth typically within $2$ hours [20]. Due to the smaller orbital period, any LEO satellite remains visible to a ground UE for only several minutes which poses a great challenge for integrating these satellites into the traditional terrestrial networks. The UE needs to undergo multiple handovers within a short span of time interval regardless of its mobility status for seamless continuation of the data sessions [14]. This frequent handover phenomenon in LEO satellite networks creates a lot of overhead in communication channels and results in overall degradation in network performances. Moreover, due to lower altitudes, the coverage area of an LEO satellite is much smaller compared to a GEO satellite. Typically a large number of LEO satellites are needed to maintain global coverage across the Earth with complex constellations. In the case of an ultra-dense constellation of LEO satellites (like Starlink), each UE is generally covered by multiple satellites, so the UE can choose the best one from the list of suitable candidate LEO satellites. This problem can be potentially solved in an optimization framework jointly considering different handover decision criteria.

In traditional terrestrial communication networks, a UE chooses to attach to a BS based on periodic signal power and quality measurements, such as Reference Signal Received Power (RSRP), and Reference Signal Received Quality (RSRQ) for the link between the BS and the UE. Moreover, load balancing is also important to ensure no BS gets overloaded or underloaded as a result of initial attachment or handover procedures. However, for LEO satellite networks, choosing a satellite BS merely based on signal measurements and network load information is not enough due to the limited visibility time of these satellites. So the UE also needs to take the potential service time into account before attaching to a satellite. In Figure 16, a simple general handover scenario involving multiple LEO satellites and a single UE is shown. Here initially, the UE is connected to an LEO satellite, indicated as LEO 2, and it needs an immediate handover to some other neighboring satellite covering the UE, either to LEO 1 or LEO 3 as it will soon lose the coverage of LEO 2. As shown in Figure 16, LEO 2 has more network load and bad channel condition, but offers more service time; LEO 3 has less network load, moderate channel condition, but offers less service time. Furthermore, a new satellite, LEO 4 becomes available for providing coverage to the UE with excellent channel conditions, great service time with moderate network load. So even for a simple case involving 4 LEO satellites, the handover decision is not straightforward for a single UE. So finding a suitable handover strategy for a UE jointly considering all handover criteria becomes a complicated problem to be solved.

Different simple greedy strategies, like Maximum Service Time (MST), Maximum Signal Quality (MSQ), or Minimum Network Load (MNL) [262] are adopted to solve the problem in a simple heuristic manner but none of these approaches provide the optimal solution. The satellite handover scenario can be also modeled as a directed graph between different satellites for a single user where the weights can be set by different handover criteria like Quality of Service (QoS), service time, etc. [263, 264]. A bipartite graph matching problem between the satellites and the users [265] is also considered in the literature to provide the optimal handover decision for satellites. In addition, a network flow-based cost minimization approach is considered in [266] by weighting each edge as the QoS perceived by the user. A handover strategy based on a potential game in a bipartite graph is considered in [267]. Different heuristic algorithms are also proposed to solve the problem [268, 269, 270]. A dynamic optimization problem is considered to be solved based on forecasting in [271]. Channel reservation is also associated to design an efficient handover algorithm while balancing the load for satellites in [272].

An RL framework can be naturally adopted for solving this problem considering the handover criteria as states and UEs as agents who act by selecting a suitable LEO satellite and collecting a reward based on the network performances. In [254], only the overall signal quality of the network is maximized using the RL approach without considering any other criteria. In [257, 255], a multi-objective optimization problem considering satellite load and signal quality constraints is solved using the DRL approach. In real networks, we have a large number of UEs; the handover decision for one UE can affect another UE, so the handover problem needs to be solved in a cooperative manner. In [256], a MARL framework is considered where multiple UEs cooperatively optimize the number of handovers in the whole network considering different handover criteria. In [258], using graph matching, a database of optimum handover decisions in satellite networks is produced and later it is used to predict handover decisions using a CNN model. Advanced DL architectures like Auction based DL [259], DDQN [260], Successive DQN [261], etc. are also considered to provide optimal handover decisions. In table VIII, we summarize the AI approaches for handover optimization in NTN involving LEO satellites. However, as all the system models consider the agents located at the UE side, it does not comply with the current standards where the handover decision is generally controlled by the BSs (satellites in this case). Furthermore, the distributed multi-agent learning architectures give rise to stability issues in real implementations. The handover criteria also need to be carefully investigated to provide the agents with the necessary information to learn the mobility behavior of the environment. These issues need to be resolved in an efficient manner for future research work in this domain.

Multiple access is a vital technique that enables multiple users to efficiently share network resources like spectrum and time. In traditional satellite networks, orthogonal multiple access schemes like Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Code Division Multiple Access (CDMA), and Space Division Multiple Access (SDMA) are employed. These schemes allocate distinct time slots, spectra, codes, or spatial divisions to users, ensuring their orthogonality in resource utilization. However, the performance of these conventional methods is constrained by the inherent limitations of these network resources. To meet the extremely high data rate and low latency demands of future 6G networks, innovative and efficient multiple access techniques, such as Non-Orthogonal Multiple Access (NOMA) and Rate Splitting Multiple Access (RSMA), have emerged in satellite network research. These advanced techniques deliver superior performance, achieving higher spectral efficiency and reduced latency, thereby paving the way for the evolution of future 6G networks.

In contrast to other conventional approaches, NOMA allows multiple users to share the same time-frequency resource block by allocating different power levels to users based on their respective channel conditions. The users are assigned the transmit power levels in an inverse manner with respect to their channel conditions, i.e. users with poorer channel conditions are allotted more transmit power whereas users with better channel conditions are allotted lower transmit power. These signals are subsequently encoded, combined using superposition coding, and transmitted to the receiver. Then Successive Interference Cancellation (SIC) technique is employed to decode signals for different users. Starting with the user granted the highest transmission power, SIC successively extracts signals while treating others as interference. This process continues down to the user with the lowest transmission power, efficiently enabling multiple users to share the resource block and enhancing spectral efficiency. However, it’s important to note that SIC’s computational complexity increases with this approach.

The optimum power allocation problem in NOMA for NTNs can be formulated as a non-convex problem which is often difficult to solve using conventional approaches. In[273], a long-term power allocation scheme for NOMA in satellite-IoT networks is solved by deriving the optimal decoding order leveraging DL techniques. A DQN-based DRL approach is investigated in [274] for optimum power allocation in satellite-IoT networks under different channel conditions and delay-QoS requirements of NOMA users. In another study as discussed in [275], the non-convex problem involving integer variables is later reformulated as a mixed-integer convex problem which was later solved by two DL techniques instead of conventional iterative solutions. Some studies also focus on the non-convex user selection problem for given power allocations based on the CSI feedback. Such a study, [276] used DQN to find out the suitable user pairing for delay-limited NOMA-based satellite networks considering the channel conditions and delay constraints as states. k-means UL approach is also considered to find out the pair of terrestrial users to be simultaneously served by space and aerial BSs adopting NOMA in [277]. Q-learning is adopted in [278] to allocate the time slots and communications channels for IoT-satellite terrestrial relay networks.

Another significant multiple access method, RSMA, is also explored in the context of satellite networks to enhance spectral efficiency. In RSMA, user messages are partitioned into two segments: common and private. The common signals are collectively encoded and merged into a unified data stream intended for all users, while each user linearly precodes their private messages. On the receiver side, the common component is extracted while treating the private signals as noise, employing the SIC technique. Subsequently, each user extracts their respective private signals. This provides the users with another way to share the same resource blocks with an increase in spectral efficiency. Generally maximizing the sum rate for both parts is a complicated non-convex problem and can be solved by Weighted MMSE (WMMSE) problem which is difficult to implement in practical hardware. A successive convex approximation as well as KarushKuhnTucker (KKT) conditions are used to calculate the transmit power in RSMA power for different beams in satellite networks in [279]. However, DL techniques can be extremely useful in modeling the solution framework with low complexity as shown in [280, 281]. Here a deep unfolding technique is used to implement the WMMSE algorithm using a deep NN and momentum-accelerated Projection Gradient Descent algorithm. A DRL framework using Proximal Policy Optimization is used to maximize the sum rate in [282]. Here each BS works as an agent, the channel state information i.e. SINR of the private and common messages is used as the states, the action is to find the suitable power allocation whereas the reward is the achieved sum rate.

One of the most important applications of satellite-terrestrial integrated networks is enhancing the computation capabilities of existing terrestrial network architectures leveraging satellites. With traditional terrestrial networks, supporting a diverse set of new applications like AR, VR, etc. with high data processing and extremely low latency requirement can get very challenging. Generally, terrestrial BSs are deployed sparsely due to high infrastructure and maintenance costs. Due to resource constraints, in case of the high demand for data processing for these types of applications, the BSs need to offload the computation tasks to the terrestrial cloud via the satellites [292]. However, due to longer propagation delay, the latency requirements set by the applications are difficult to be met [293]. Nevertheless, due to the emergence of LEO satellites with comparatively low propagation delay, the overall delay is considerably reduced. Also instead of acting as relays, the satellite can now also do the processing works acting as edge-servers. So we can consider a three-level hierarchical architecture comprising of ground UEs connected to terrestrial BSs, LEO satellites, and terrestrial cloud as shown in Figure 17 where the terrestrial BSs can offload the computational tasks to LEO satellites and to terrestrial clouds via the LEO satellites.

The main challenges in task offloading problems lie in meeting the delay constraints for low-latency applications while minimizing the energy consumption for the satellites. So this can be formulated as an optimization problem to come up with an efficient offloading approach for integrated TNTN architecture. Such an optimization problem is solved using different conventional approaches like 3D hypergraph matching [294], game theory [295], stochastic approach [296], efficient algorithms [297] in the existing literature. In [298], a joint optimization framework comprising task offloading and resource allocation is also considered in an integrated satellite-terrestrial environment. Although these algorithms work well in theory for particular scenarios, in real networks, the feasibility of these algorithms is compromised due to different issues. Some of these works do not consider the cooperation among terrestrial cloud and LEO satellite servers which result in sub-optimal approaches [294, 295]. Also, these approaches are some predefined models highly dependent on different network states causing a large overhead in networks. Moreover, they usually converge to the solutions after a large number of iterations causing high computational complexity.

To tackle these issues, different ML approaches are proposed in the literature to solve task offloading problems. In [284], a DRL-based task offloading framework dependent on channel state information is proposed. A similar DRL-based framework is also considered in [299] with additional consideration of the dynamic queue condition in satellites. In [285], both DQN and DDQN are explored to solve the task offloading problem in a decentralized manner. DDPG algorithm is considered to solve the optimization problem in a DQN framework in [283] while taking the potential security issues into account. An LSTM model is used to solve the task offloading problem while considering channel conditions and energy dynamics in [288]. A DL-based caching strategy is considered in satellite edge networks in [289]. As we have multiple satellites in the real networks, to improve the overall system performances, different multi-agent architectures are considered both in a distributed [290] and cooperative environment [286, 287]. Distributed architectures for generating discrete offloading decisions in a supervised manner are also considered in [291]. In table IX, we summarize the AI approaches for task offloading in TNTN. As the delay constraints vary with network traffic types, the offloading decisions need to be derived taking network traffic types into account. Potential research works can show how computational offloading can be done for various network traffics and show superior network performances.

In wireless networks, depending on the traffic and channel conditions, the network traffics are routed to different paths among different network nodes so that the overall network performance can be improved. In any network with static channel and traffic conditions, this routing problem can be transformed into the well-known shortest path problem and solved by Dijkstra’s algorithm [308]. Here the network nodes can be considered as nodes in the graph and the edges can represent the links between different nodes. The weights of the edges can be defined based on the target network performance metrics like delay, jitter, throughput, packet loss, etc. However, the topology of the real satellite-terrestrial integrated networks (shown in Figure 18) are very complex and dynamic due to hierarchical network architecture and uncertain channel and traffic conditions, respectively. So simple Djikstra’s algorithm cannot be directly applied to meet the performance requirements in these networks.

In real satellite networks, Asynchronous Transfer Mode (ATM) routing is introduced in [309]. The well-known Open Shortest Path First (OSPF) [310] protocol-based Internet Protocol (IP) routing is also adapted to the dynamic satellite environment in [311]. However, the dynamics of satellite-terrestrial networks are very different from traditional terrestrial networks due to their highly dynamic network topology, link status, and traffic conditions. Depending on the instantaneous network topology, a static-dynamic combined routing scheme is considered in [312]. An ant-colony-based optimization (ACO) framework is considered in [313]. In [314], a Kalman filter-based Wolf Colony Optimization algorithm is used to solve the local optimal solution issue in [313]. An improved ACO framework is considered to find out the optimal set of links with multiple network constraints in [315]. A Coordinate Graph (CG) model-based network routing approach for three-dimensional TNTN is considered in [316]. In [317], minimum flow maximum residual path-based network flow algorithm is used to find out the optimal network routing path for satellites. In [318], a 3-dimensional network mapping using hyperbolic geometry is considered for integrated satellite-terrestrial networks.

To cope with the dynamic environment in integrated satellite-terrestrial networks, different DL architectures are proposed in the literature. In [303], fuzzy logic is used to evaluate task requirements to improve the CNN output for optimal path allocation. The network routing optimization problem can be put into an RL framework. In [300], a speed-up Q-learning algorithm is used to find out the optimal routing strategy for TNTN. A similar Q-learning-based RL framework is also considered to solve the routing problem for LEO satellites in [301]. To tackle the complexity issue, a DRL framework is used to generate optimal routing strategies in TNTN [307] and LEO satellite networks [301]. FCNN [304], CNN [127], etc. architectures are used to solve the routing problem in a supervised manner. Other ML frameworks like GNN [306] and ELM [305] are also considered to solve the routing problem in NTNs. In table X, we summarize the AI approaches for network routing in TNTN. Recursive NN architectures need to be also explored for capturing the temporal behavior in network routing decisions. Furthermore, the channels are extremely dynamic and time-varying in the case of TNTNs; the channel conditions can be also considered in the learning criteria of RL frameworks.

Traffic prediction is very critical in modern communication systems to ensure high-speed low latency communications. Particularly, in NTNs, accurate traffic prediction is extremely crucial due to the highly dynamic network topology as well as diverse user requirements. At its core, traffic prediction involves forecasting future network traffic based on past usage patterns. Conventional approaches such as Auto Regressive Moving Average (ARMA), Auto Regressive Integrated Moving Average (ARIMA) [319, 320], etc. are typically used for these predictions. However, the DL approaches have emerged as more effective alternatives by providing improved performances in recent times due to their inherent capability to capture spatial and temporal correlations.

In [321], Radial Basis Functions (RBF) neural network-based short-term traffic flow forecasting is proposed. In [322], an LSTM-based architecture is utilized for traffic prediction due to its temporal characteristics handling capability where the attention mechanism is used to balance the effect of inputs on outputs properly. The RNN architectures suffer from gradient explosion issues. To overcome this issue, GRU architectures are explored for traffic prediction in [323, 324, 325]. In [323], the transfer learning approach and particle filter online training algorithm are combined to address the lack of online training data and reduce the training time complexity. In [324], GNNs are used to extract the spatial features of the satellite network traffic from the input network topology, which is later used as an input to a GRU network for traffic prediction. In [325], on top of the attention mechanism and GRU models, PSO is used to obtain the best set of hyperparameters for the network.

Upon examining the contents presented in Section V, it becomes evident that SL and RL approaches take center stage in addressing the diverse array of challenges faced by satellite-based NTNs in future 6G networks, primarily due to the availability of real-world data and feedback mechanisms within existing networks. In the context of SL, having access to well-labeled data is of utmost importance, especially in scenarios involving estimation problems like channel estimation and Doppler Shift estimation. On the contrary, RL shines when dealing with problems lacking clear labels but featuring a notion of reward functions. Furthermore, RL techniques are extremely suitable and efficient for problems where supervision is lacking which is usually the case for many NTN problems such as resource allocation, beam hopping, and network routing as illustrated in Section V of the paper. Meanwhile, RL techniques can be effectively employed across a wide spectrum of problems using general network feedback, such as CSI, acknowledgments, and more. Consequently, a significant portion of research efforts tends to leverage RL frameworks to address their specific challenges.

The emergence of Deep NNs and their effectiveness in addressing intricate challenges in fields like computer vision and natural language processing has piqued the interest of the research community in applying these architectures to network-related issues. Satellite-based NTNs introduce a unique set of challenges, characterized by highly dynamic network conditions and a multitude of variables influencing network performance. Traditional ML) approaches often fall short in comprehensively addressing these complex problems, frequently limited to small-scale issues. As a result, DL techniques have gained significant popularity within the research community, proving to be a more adept choice for tackling the multifaceted challenges encountered in satellite-based NTNs, as elaborated in Section V.

The nature of UL approaches presents a unique set of challenges in the context of highly dynamic and time-varying NTNs. Understanding and capturing the intrinsic behavioral patterns within such networks prove to be particularly hard. However, it is important to note that UL approaches still hold the potential to derive the distribution of crucial network parameters that may not be readily accessible in real networks. These derived parameters can play a pivotal role in addressing various NTN challenges. The distributed learning approaches such as FL can be also beneficial for future satellite-based NTNs as the computing capabilities requirements can be reduced to a minimum enhancing practical feasibility.

Currently, there are some ongoing research efforts focused on developing SDR-based prototypes for NTNs with adaptations to OAI 4G and 5G protocol stacks, as discussed in Section VI-C. However, to fully unlock the potential of AI in NTN for future 6G networks, the integration of the RIC into these implementations is crucial. This integration is particularly important given that the immense benefits of AI in addressing NTN deployment challenges for future 6G networks are demonstrated in Section V but current 5G networks lack a dedicated interface for applying AI algorithms. By enabling the O-RAN framework with RIC, the deployment of AI algorithms in real NTN networks can be efficiently performed, paving the way for advanced capabilities and improved performance.

The cost limitations on onboard computation, the extreme environmental conditions, and the extensive propagation delays form a multifaceted array of challenges when endeavoring to bring AI-driven NTNs to fruition in anticipation of the forthcoming 6G network era, as elaborated in Section VII. These formidable challenges necessitate the development of innovative, resourceful solutions to ensure superior network performance in practical NTN deployments. Notably, three key factors come to the fore when considering the limitations imposed on AI capabilities for satellites and other NTN platforms: power, bandwidth, security, and physical space. Advancements in miniaturization, secured system design, energy-efficient design principles, and the judicious utilization of available bandwidth resources serve as the driving forces enabling AI technologies within satellite-based NTNs.

Section V sheds light on the interconnected nature of the various issues encountered in NTNs. It is crucial to recognize that addressing a singular problem can serve as an initial step toward resolving larger, more complex challenges inherent in TNTNs. However, when transitioning these solutions into real-world networks, it becomes imperative to acknowledge and account for the intricate interdependencies among various aspects. An illustrative example of such interrelations lies in the dynamic nature of network load status following a user’s attachment to a satellite. In this scenario, integrating resource allocation strategies into the handover decisions can yield enhanced network performance. MIMO systems can be also beneficial for the single-user and multi-user cases for NTNs as in LTE-Advanced [356]. By considering the broader context and understanding how different aspects influence one another, we can develop more holistic and effective approaches for real-world NTN implementations.

Presently, the majority of DL algorithms deployed to tackle the time-varying nature of NTNs in beam-hopping, resource allocation, network slicing, etc. rely on feed-forward NNs. While these architectures have proven successful in computer vision applications such as image detection and classification, they may not effectively capture the temporal behavior inherent in these NTN problems. Unlike feed-forward networks, recurrent architectures possess the ability to capture and process temporal dependencies within the problem domain. By leveraging RNNs or other similar architectures, we can effectively model and solve the corresponding NTN challenges in a more comprehensive and accurate manner. In particular, for dynamic spectrum access and sharing approaches low complexity NN architectures such as ESNs can be very useful for NTNs as illustrated in [357, 358].

One major limitation of the current works in the domain is the limited consideration given to online implementation and the associated computational complexity when designing algorithms for various control operations in NTNs. This oversight poses a significant hurdle to the practical application of these algorithms in real NTNs as many control decisions in NTN systems must be made in real-time, and the use of complex deep feed-forward NNs becomes impractical. To address this challenge, exploring alternative options becomes imperative. One such option involves investigating low-complexity architectures such as ESNs and ELMs or combining them with traditional feedforward-NNs. These low-complexity architectures offer a more viable solution for online implementation, enabling the deployment of DL algorithms in real NTN networks in a timely and efficient manner.

In the context of integrated satellite-terrestrial networks, the adoption of distributed learning models can significantly enhance scalability. These models involve distributing the training and inference processes of machine learning algorithms across multiple computing nodes, resulting in accelerated computation and improved efficiency. Various distributed approaches, such as data parallelism, model parallelism, ensemble learning, and federated learning, offer promising solutions to address the diverse challenges faced by NTNs in extended network environments [359]. By leveraging these distributed approaches, NTN systems can effectively harness the power of parallel computing and collaborative learning to overcome constraints and achieve optimal performance.

One of the major motivating factors for implementing feedback-based learning, such as RL methods, in NTNs, is the inherent feedback system of the current cellular networks. CSI information is readily available for the BSs which can be helpful in network optimization approaches. However, with the emergence of NTNs, new challenges arise, necessitating the efficient design of feedback mechanisms to minimize the overall overhead while improving network performance. This consideration is crucial, as AI approaches for addressing various issues may require similar types of feedback. The utilization of combined feedback can prove highly beneficial in optimizing network performance and achieving efficient resource allocation, thus enhancing the overall effectiveness of AI algorithms in NTNs.

The limited availability of computational resources currently poses a challenge to the onboard capability of satellites, especially when deploying AI algorithms. However, the miniaturization of satellite components and equipment has emerged as a solution to this issue. By reducing the weight and size of equipment, miniaturization enables the integration of more powerful processors and larger memory devices within the limited space available on satellites. This advancement in computational resources greatly facilitates the deployment of AI algorithms, unlocking new possibilities for satellite applications. Achieving miniaturization in satellite technology requires innovations in material science, efficient Integrated Circuit (IC) design, advancements in IC fabrication technologies, System-on-Chip (SoC) integration, and Micro-Electro-Mechanical Systems (MEMS) design, among others. The development of miniaturization is particularly crucial for NTNs, as it enhances the onboard capability of satellites and enables the realization of advanced technologies and functionalities in space-based systems.

The launching and maintenance of satellites require substantial power consumption, which imposes limitations on the onboard capability of satellites. Consequently, efficient energy system design becomes a critical criterion for NTNs. To address this, various aspects need to be considered, including lightweight component design, advanced power management techniques, efficient power conversion, optimized propulsion system design, effective energy storage systems, etc. By focusing on these factors, satellite systems can achieve higher energy efficiency, which is essential for the successful deployment of advanced AI algorithms. The performance of these algorithms relies on the availability of computational resources, making energy efficiency a crucial aspect to maximize the satellite’s capabilities within the given power constraints.

As highlighted in Section VII, security concerns in NTNs can be highly significant, introducing new attack vectors and vulnerabilities. NTNs are susceptible to a range of security attacks, including adversarial attacks, data poisoning, DoS attacks, Fuzzy attacks, MiM attacks, and more. These attacks have the potential to severely impact network performance and compromise the integrity and confidentiality of data. To address these challenges, it is essential to design efficient intrusion detection and prevention systems specifically tailored for secure NTNs. By continuously monitoring a set of relevant network parameters and detecting anomalies in the network’s behavioral patterns, mitigation techniques can be promptly deployed to ensure optimal network performance and safeguard against potential degradation caused by security breaches.