Abstracts:Large models have been widely used in the field of neural language processing, information retrieving, etc. With the development of the large models, not only is the parameter scale increased, but the model architecture has also become more complex. For example, the multi-modal transformer-based model mainly has concurrent branches, which we denoted as the concurrent branch model (CBM). Many CBMs have enlarged to tens of billions of parameters, and require distributed resources to train this kind of model. Existing distributed training systems cannot fully handle this type of model architecture because there are interactions between branches. Inspired by the unbalanced resource usage of pipeline parallelism, we prefer to organize different branches with a fine-grained bidirectional pipeline schedule of communication and computation. However, improper coordination between branches leads to idle time for computation and low training efficiency. In this paper, we present Flexpipe, a pipeline engine for c3oncurrent-branch models. We first introduce a branch-aware pipeline parallelism (BAPP) to make full use of the concurrent characteristic of the model architecture. Then, based on a multi-branch pipeline simulator, we propose an adaptive interaction coordinator, which facilitates the low-overhead branch interactions during the distributed model training. We evaluate our approach on popular concurrent branch models combined with modern training systems. Compared with the Chimera, the experiential results show that our method improves the end-to-end training throughput by 20% on average.

Abstracts:Myasthenia Gravis (MG) is an autoimmune neuromuscular disease. Given that extraocular muscle manifestations are the initial and primary symptoms in most patients, ocular muscle assessment is regarded necessary early screening tool. To overcome the limitations of the manual clinical method, an intuitive idea is to collect data via imaging devices, followed by analysis or processing using Deep Learning (DL) techniques (particularly image segmentation approaches) to enable automatic MG evaluation. Unfortunately, their clinical applications in this field have not been thoroughly explored. To bridge this gap, our study prospectively establishes a new DL-based system to promote the diagnosis of MG disease, with a complete workflow including facial data acquisition, eye region localization, and ocular structure segmentation. Experimental results demonstrate that the proposed system achieves superior segmentation performance of ocular structure. Moreover, it markedly improves the diagnostic accuracy of doctors. In the future, this endeavor can offer highly promising MG monitoring tools for healthcare professionals, patients, and regions with limited medical resources.

Abstracts:In neuropathological diseases such as Alzheimer's Disease (AD), neuroimaging and Magnetic Resonance Imaging (MRI) play crucial roles in the realm of Artificial Intelligence of Medical Things (AIoMT) by leveraging edge intelligence resources. However, accurately classifying MRI scans based on neurodegenerative diseases faces challenges due to significant variability across classes and limited intra-class differences. To address this challenge, we propose a novel approach aimed at improving the early detection of AD through MRI imaging. This method integrates a Convolutional Neural Network (CNN) with a Cascade Attention Model (CAM-CNN). The CAM-CNN model outperforms traditional CNNs in AD classification accuracy and processing complexity. In this architecture, the attention mechanism is effectively implemented by utilizing two constraint cost functions and a cross-network with diverse pre-trained parameters for a two-stream architecture. Additionally, two new cost functions, Satisfied Rank Loss (SRL) and Cross-Network Similarity Loss (CNSL), are introduced to enhance collaboration and overall network performance. Finally, a unique entropy addition method is employed in the attention module for network integration, converting intermediate outcomes into the final prediction. These components are designed to work collaboratively and can be sequentially trained for optimal performance, thereby enhancing the effectiveness of AD stage classification and robustness to interference from MR images. Validation using the Kaggle dataset demonstrates the model's accuracy of 99.07% in multiclass classification, ensuring precise classification and early detection of all AD subtypes. Further validation across three feature categories with varying numbers confirms the robustness of the proposed approach, with deviations from the standard criteria of less than 1%. Applied in Alzheimer's patient care, this capability holds promise for enhancing value-based therapy and clinical decision-making. It aids in differentiating Alzheimer's patients from healthy individuals, thereby improving patient care and enabling more targeted therapies.

Abstracts:In Cell-Free (CF) systems, the users are served simultaneously by a large number of low-cost and low-power distributed antennas, taking advantage of spatial diversity. The scarcity of equations that accurately describe the system performance limits optimization techniques to applications of users Quality of Service (QoS) uniformization. Thus, to accurately characterize the performance of such systems, a simplified model for the downlink received signal is proposed and new expressions are derived for the users Outage Probability (OP) and average channel capacity taking into account the channel gain variations characteristics. Different cell-free scenarios are analyzed and several curves are presented for different parameters that characterize the channels. The new theoretical results are corroborated by Monte-Carlo simulations and compared to literature results, which confirm classical cell-free behavior as well as the saturation on channel capacity and OP curves, and reveal that the proposed expressions describe the systems more accurately.

Abstracts:In this article, a finite-time adaptive dynamic surface synchronization tracking controller with guaranteed transient performance is proposed for bilateral teleoperation manipulators. To achieve this objective, we establish a comprehensive model of the teleoperation system incorporating asymmetric time-varying delays, external disturbances, joint frictions, and additive uncertainties. Subsequently, the dynamic surface control approach is introduced to reduce computational complexity by avoiding repeated differentiation of virtual signals in traditional backstepping algorithms. Moreover, this law address the passivity issue associated with time-delayed channels by substituting joint frictions and environmental parameter uncertainties with non-power approximate signals generated using fuzzy logic algorithms. Additionally, through the utilization of the finite-time performance function, assurance is provided for the transient performance of the system. The synchronization errors can converge to a small neighborhood around zero in a finite time which can be arbitrarily set. Theoretically, the Semi-Global Practical Finite-Time Stability (SGPFTS) of the closed-loop signals is derived from the Lyapunov function. The simulation and practical experiment are both performed, and the results verify the effectiveness of the proposed control approach. In the future, the work will consider the teleoperation system where the initial error is not within the constraints of the finite-time performance function, and simplify the adaptive updating law.

Abstracts:Clinical diagnosis for complex disease conditions is a complicated decision process involving systematic inference and differentiation. Artificial Intelligence (AI) models have been a widely established approach to help improve the efficiency of various kinds of clinical decision tasks (e.g., diagnosis, treatment, and prognosis). However, due to the critical requirement of time efficiency, lack of sufficient information, and high probability of comorbid diseases in Outpatient and Emergency Settings (OES), it is still challenging to build clinically feasible AI models using the free text clinical records in OES for complex disease conditions, such as neurosurgery. Here we propose an AI diagnosis model, named LLM4DEU, for neurosurgery disease differentiations by fine-tuning a large language model (i.e., ChatGLM) using the Department of Neurosurgery, the Beijing Tiantan Hospital OES electronic health records. LLM4DEU obtained state-of-the-art performance on clinical diagnosis with a F1 score of 78.53%, which is superior to five well-known baselines (including deep learning models). In addition, we evaluated the actual performance of the model by case studies on the diagnosis of specific neurosurgical diseases (e.g., subdural hematoma, cerebral hemorrhage, and cerebral infarction). The experimental results show that the LLM4DEU model has significant advantages in diagnosing low-incidence disease conditions, and comparative analyses with clinical experts confirm the predictive power of the model in neurosurgical diagnosis.

Abstracts:String validation routines have been widely used in many real-world applications, such as email validation and postcode validation. String test cases are adopted to test these validation routines, to identify potential defects and security risks. Random Testing (RT) is a well-known testing approach to randomly generate string test cases from the input domain (i.e., the set of all possible test inputs), which is simple to implement at a low cost. However, its testing effectiveness may be unsatisfactory for string validation routines. The main reason for this is that RT may have a high probability to generate invalid rather than valid string test cases, due to its randomness property. This research proposes a new RT approach based on the output types (i.e., valid and invalid strings) for string validation routines, namely Output-type-guided Random Testing (RT-O), which attempts to randomly generate both valid and invalid string test cases with a certain probability. This research performed an empirical study involving several real-world string validation routines collected from ten Java open-source projects, to investigate and compare testing performances of RT-O against the previous two widely-used RT methods. The results show that the generated string test cases by RT-O outperform test cases generated by other RT methods.

Abstracts:The rapid growth of marine applications leads to a significant increase in Maritime Devices (MDs). Traditional shore-based maritime communication networks face limitations, such as overloaded and transmission distance to provide network services for MDs. Unmanned Aerial Vehicles (UAVs) act as relays that can expand coverage and enhance the quality of service for offshore communication networks. We consider a multi-UAV-assisted Offshore Internet of Things (mUAV-OloT), and formulate a throughput maximization problem by jointly optimizing channel allocation, Leader MD (LMD) selection, UAV-LMD association, and LMD-MD association. Firstly, we propose the Hypergraph-based Two-Stage Matching (HTSM) algorithm where a Hypergraph-based LMD Selection (HLMDS) strategy is employed to identify the set of LMDs. Secondly, the Kuhn-Munkres algorithm is used to optimize the UAV-LMD association and a Weighted Three-dimensional Hypergraph Matching (WTHM) algorithm is designed to solve the LMD-MD association and channel allocation. Numerical results show that the HTSM algorithm outperforms benchmark algorithms regarding throughput.

Abstracts:Incremental search provides real-time suggestions as users type their queries. However, recent studies demonstrate that its encrypted search traffic can disclose privacy-sensitive data through side channels. Specifically, attackers can derive information about user keystrokes from observable traffic features, like packet sizes, timings, and directions, thereby inferring the victim's entered search query. This vulnerability is known as a remote keystroke inference attack. While various attacks leveraging different traffic features have been developed, accompanied by obfuscation-based countermeasures, there is still a lack of overall and in-depth understanding regarding these attacks and defenses. To fill this gap, we conduct the first comprehensive evaluation of existing remote keystroke inference attacks and defenses. We carry out extensive experiments on five well-known incremental search websites, all listed in Alexa's top 50, to evaluate and compare their real-world performance. The results demonstrate that attacks utilizing multidimensional request features pose the greatest risk to user privacy, and random padding is currently considered the optimal defense balancing both efficacy and resource demands. Our work sheds light on the real-world implications of remote keystroke inference attacks and provides developers with guidelines to enhance privacy protection strategies.

Abstracts:Recent research has shown a burgeoning interest in exploring sparse models for massively Multilingual Neural Machine Translation (MNMT). In this paper, we present a comprehensive survey of this emerging topic. Massively MNMT, when based on sparse models, offers significant improvements in parameter efficiency and reduces interference compared to its dense model counterparts. Various methods have been proposed to leverage sparse models for enhancing translation quality. However, the lack of a thorough survey has hindered the identification and further investigation of the most promising approaches. To address this gap, we provide an exhaustive examination of the current research landscape in massively MNMT, with a special emphasis on sparse models. Initially, we categorize the various sparse model-based approaches into distinct classifications. We then delve into each category in detail, elucidating their fundamental modeling principles, core issues, and the challenges they face. Wherever possible, we conduct comparative analyses to assess the strengths and weaknesses of different methodologies. Moreover, we explore potential future research avenues for MNMT based on sparse models. This survey serves as a valuable resource for both newcomers and established experts in the field of MNMT, particularly those interested in sparse model applications.