Institute of Information Science, Academia Sinica

Abstract

Speculative decoding accelerates large language model (LLM) inference by proposing tokens with a small draft model and verifying them with a large target model. When the two models run sequentially, the draft model sits idle during verification, leaving the draft-side accelerator underutilized. We present a speculative decoding pipeline in which a draft model continuously drafts batches of independent sequences on one GPU while a target model verifies them on another, so that drafting and verification overlap in time. We then maximize the throughput by adjusting two parameters of the pipeline. The first parameter is the length of the draft $k$, i.e., how many tokens the draft model proposes each step. The second parameter is the batch size of the draft model $m$, i.e., how many independent sequences the draft model groups into a single batch for verification to improve efficiency. Trying every possible $ k$ and $ m$ to maximize the token throughput is time consuming. Therefore, we show that throughput changes in a simple and predictable way as a function of $k$ and $m$. We observe that the throughput is a simple function of the draft length with a predictable first order derivative, so we can find good settings from only a few short test runs instead of a full search. We evaluate the pipeline on OPT-125M and OPT-1.3B with an XSum workload. Our quick search finds the same best $(k, m)$ setting as a full search. Over the whole XSum test set, the pipeline at this setting runs $1.22\times$ faster than a batched sequential speculative-decoding baseline on the same two GPUs, with the baseline tuned to the best draft length and draft batch size. Because both systems use the same two GPUs and both use draft-side batching, the speedup comes from overlapping drafting and verification, not from extra hardware or batching alone. The pipeline overlaps drafting and verification so that neither GPU idles, while the batched sequential baseline still executes the two stages sequentially.

Abstract

Distributed deep learning is critical for training convolutional neural network (CNN) models on large-scale image datasets with numerous parameters. Current research primarily focuses on leveraging additional hardware resources and powerful computing units to accelerate the training process. As a result, larger batch sizes are often employed to speed up training. However, training with large batch sizes can lead to lower accuracy due to poor generalization. To address this issue, we propose the dual-batch learning scheme, a distributed training method built on the parameter server framework. This approach maximizes training efficiency by utilizing the largest batch size that the hardware can support while incorporating a smaller batch size to enhance model generalization. By using two different batch sizes simultaneously, this method improves accuracy with minimal additional training time. Additionally, to mitigate the time overhead caused by dual-batch learning, we propose the cyclic progressive learning scheme. This technique repeatedly and gradually increases image resolution from low to high during training, thereby reducing training time. By combining cyclic progressive learning with dual-batch learning, our hybrid approach improves both model generalization and training efficiency for image classification tasks. Experimental results with ResNet-18 demonstrate that, compared to conventional training methods, our approach improves accuracy by 3.3% while reducing training time by 10.1% on CIFAR-100, and further achieves a 34.7% reduction in training time on ImageNet.

Abstract

The exploration-exploitation tradeoff is a fundamental challenge in reinforcement learning. In Monte Carlo Tree Search (MCTS), this tradeoff is balanced explicitly by the pUCT exploration coefficient and implicitly by parallelism when the search is combined with neural network evaluations. However, the exploration coefficient is typically tuned as an algorithmic hyperparameter, while the degree of parallelism is often treated as a system detail. To this end, we characterize how these two factors shape the search-tree structure and jointly affect performance scaling. We prove that, under regularity conditions, the depth of the optimal path grows as $\Theta(\sqrt{n}/\log n)$ for $n$ simulations under the MuZero exploration coefficient formula. For tree-parallel MCTS, we prove that the search depth preserves the same asymptotic order when $T=o(\sqrt{n})$, while sufficiently large parallelism introduces contention and reduces the depth growth rate to $\Theta(n/T)$ for $T$ threads. Our empirical results indicate that, in the Go experiment, each doubling of the thread count reduces the best-performing exploration coefficient by approximately 0.19. Overall, our characterization connects the search-tree structure with the exploration-exploitation behavior in pUCT, providing insight into how exploration coefficients and parallelism can be jointly tuned.

Abstract

Large Language Models (LLMs) have achieved remarkable success across a wide range of complex language tasks. However, deploying these models on resource-constrained platforms remains highly inefficient due to their massive parameter sizes. Although model compression techniques such as pruning and quantization have individually proven effective in reducing model size and inference cost, combining them into a unified method often leads to significant performance degradation, because pruning and quantization impose fundamentally conflicting constraints on the underlying weight distribution. To address this challenge, we propose {\em CARE}, a joint pruning and quantization framework that integrates low-bit weight quantization with semi-structured sparsity while maintaining competitive model accuracy. Our approach leverages second-order Hessian information to compensate for errors arising from compression-induced weight perturbations while strictly preserving the sparsity masks. To further mitigate residual compression errors, we incorporate Low-Rank Adaptation (LoRA) modules to recover accuracy after aggressive compression. In addition, we propose a data-driven sensitivity metric that combines weight magnitudes and gradient information to adaptively assign layer-wise LoRA ranks under a fixed parameter budget. Experimental results on LLaMA-2-7B and LLaMA-3-8B demonstrate that our method reduces the memory footprint by 63.1\% over the uncompressed baseline and outperforms the state-of-the-art SLiM method with 4-bit quantization and 2:4 sparsity.

Abstract

This paper accelerates fast-forward model training that applies pipeline parallelism and activation checkpointing implemented by PyTorch. We prove that it is NP-hard to select checkpoints that minimize the training time under a given memory limit and PyTorch resource usage. We design a dynamic programming algorithm to partition a model into stages that is mathematically proven to minimize the maximum training time among stages. We design a dynamic programming algorithm to select checkpoints that are mathematically proven to minimize the stage training time under a memory limit and PyTorch resource usage. We design an FPTAS for checkpoint selection by rounding the training time of layers. Empirically, our algorithms achieve speedups of up to 2.07x over the state-of-the-art algorithms across models of various sizes and structures. The empirical stage training time corroborates that our algorithms achieve a more balanced stage training time across stages, and our checkpoints have a shorter stage training time for a single stage than the state-of-the-art algorithm.

Abstract

Existing image watermarking methods typically entangle visual content with spatial structure, making them highly sensitive to geometric transformations and alignment discrepancies, especially in the presence of deepfake manipulations. In this paper, we propose a semantic-driven facial watermarking framework for robust identity recovery. The key idea is to decouple identity-related semantic information from spatial layout and encode it into a compact and spatially robust semantic representation. Specifically, we decompose a face into semantic components and aggregate deep features into component-wise latent representations, which are quantized via independent codebooks and converted into a compact bitstream for embedding. After decoding, the embedded semantic information is recovered and used to reconstruct identity-consistent facial content, even under slight geometric distortions and tampering. Experiments on CelebA-HQ and FFHQ demonstrate that our method significantly outperforms existing watermarking approaches in terms of reconstruction quality, identity preservation, and retrieval accuracy under both photometric and geometric attacks, validating the effectiveness of semantic component-wise encoding for reliable face recovery.

Abstract

Our position shifts the target of peer-review reform from reducing uncertainty through evaluator improvement or judgment aggregation to constraining evaluative authority under persistent uncertainty. Scientific peer review aims to assure research quality but also allocates scarce academic opportunities, such as publication in high-impact venues. However, the review system is under growing pressure as submissions increase while reviewer capacity, expertise, scrutiny, and incentives cannot scale accordingly. As peer review operates under persistent evaluative uncertainty, evaluator improvement and judgment aggregation remain structurally limited. We therefore propose authority constrained peer review: before a consequential judgment is released as decision relevant, it should withstand structured scrutiny through critique-authority decoupling, independent secondary evaluation, contestability and reversibility, and accountability for unsupported influence. Our aim is to prevent weakly supported judgments from entering final deliberation with the same authority as judgments that have withstood structured scrutiny, while creating incentives for reviewers to justify consequential claims more carefully. For large-scale computer-science conferences, this position offers an institutional response to uneven review quality and reviewer-assignment dependence; more broadly, it provides a governance principle for any domain where uncertain expert judgments allocate scarce academic opportunities.

Abstract

Low-resource ASR remains difficult because scarce transcripts provide limited supervised evidence for target-side generation. To address this gap, we propose SAMA-ASR, a lightweight adapter mechanism that augments the decoder with semantic anchors from auxiliary translations and an acoustic anchor from speech; in principle, the mechanism can be applied to similar encoder--decoder multitask speech models. Through cross-modal adaptation, SAMA-ASR conditions decoder states on translation-derived semantic embeddings and a speech embedding, combining utterance-level meaning with speech-grounded evidence before token prediction. At evaluation time, these semantic anchors can be generated automatically by an upstream speech-to-text translator rather than supplied as oracle translations. Experiments on two 30-hour datasets covering the low-resource Sinitic varieties Taiwanese Hokkien and Hakka show that SAMA-ASR improves over acoustic, prior prompt-based, and semantic-only translation-guided baselines and remains effective in practical automatic semantic-anchor settings; translator-capacity analyses show that useful semantic anchors can be produced by a compact ST model.

Abstract

Objective: Arteriovenous fistula (AVF) dysfunction is a major cause of morbidity in hemodialysis (HD) patients. Although conventional monitoring tools are clinically effective, they are often limited by cost, invasiveness, and operator dependence. This study proposes a cycle-aware, auscultation-based deep learning framework for objective and scalable assessment of AVF blood flow status. Methods and Procedures: We developed CALM, a physiologically informed learning framework that integrates Hilbert-based cycle segmentation with pretrained audio foundation model (AFM) representations. A clinical auscultation dataset (CHGH-AVF) from 188 HD patients at three standardized vascular access sites was collected and labeled using concurrent blood flow measurements. The proposed approach was evaluated on both three-class flow-severity classification (Low, Medium, and High) and binary flow-quality screening (Ideal vs. Unideal). Model variants incorporating domain-specific low-level descriptors and different feature fusion strategies were also investigated. Results: Across all auscultation sites, CALM consistently outperformed conventional handcrafted-feature baselines, deep learning baselines, and classical machine learning-based classifiers in patient-level flow-severity classification. Ablation studies demonstrated that physiologic cycle segmentation substantially improves discriminability, while integrating complementary low-level descriptors (LLDs) further enhanced performance across anatomically different recording locations. Although classification performance remained highest at the proximal sites, reduced performance at the venous site highlighted the influence of anatomical location and signal quality. Strong performance was also observed for binary flow-quality monitoring, particularly at the proximal auscultation sites. Conclusion: The proposed framework demonstrates the feasibility of AFM–based auscultation for non-invasive AVF flow assessment. By combining physiologic signal modeling with foundation model representations, CALM provides a robust and objective framework for automated assessment of vascular access function.Clinical Impact: This work introduces a multi-site clinical AVF auscultation dataset and establishes automated auscultation analysis as a low-cost and scalable adjunct to existing vascular access monitoring practices in HD care.Clinical and Translational Impact Statement: AFM-based auscultation has the potential to support objective screening of AVF dysfunction and may facilitate future integration into bedside or home monitoring systems, enabling more accessible and scalable vascular access monitoring.

Abstract

Perceptual voice quality assessment plays a vital role in diagnosing and monitoring voice disorders.Traditional methods, such as the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V) and the Grade, Roughness, Breathiness, Asthenia, and Strain (GRBAS) scales, rely on expert raters and are prone to inter-rater variability, emphasizing the need for objective solutions. This study introduces the Voice Quality Assessment Network (VOQANet), a deep learning framework that employs an attention mechanism and Speech Foundation Model (SFM) embeddings to extract high-level features. To further enhance performance, we propose VOQANet+, which integrates self-supervised SFM embeddings with low-level acoustic descriptors—namely jitter, shimmer, and harmonics-to-noise ratio (HNR). Unlike previous approaches that focus solely on vowel-based phonation (PVQD-A), our models are evaluated on both vowel-level and sentence-level speech (PVQD-S) to assess generalizability. Experimental results demonstrate that sentence-based inputs yield higher accuracy, particularly at the patient level. Overall, VOQANet consistently outperforms baseline models in terms of root mean squared error (RMSE) and Pearson correlation coefficient across CAPE-V and GRBAS dimensions, with VOQANet+ achieving even greater performance gains. Additionally, VOQANet+ maintains consistent performance under noisy conditions, suggesting enhanced robustness for real-world and telehealth applications. This work highlights the value of combining SFM embeddings with low-level features for accurate and robust pathological voice assessment.

Abstract

As speech generation models become increasingly realistic and widely accessible, concerns about the misuse, attribution, and governance of synthetic speech continue to grow.Watermarking provides a practical way to make synthesized speech traceable and verifiable. Most existing speech watermarking methods embed watermark information into signal-level representations, such as waveforms or spectrograms. Under sufficiently strong distortions, the embedded watermark may be weakened or destroyed, leading to degraded detectability. In this paper, we propose SSTMark, a training-free speech watermarking framework that operates at the semantic level through text watermarking. Unlike conventional signal-level watermarking methods, SSTMark encodes watermark information into the semantic content conveyed by generated speech, and detects the watermark from the recovered linguistic content. Experiments on AudioMarkBench demonstrate that SSTMark exhibits the strongest average robustness. Compared with the state-of-the-art baselines at a fixed false positive rate of 1%, SSTMark improves the average detection rate by 4.6% and 16.9% on signal-processing edits and compression edits, respectively.

Abstract

Recent studies demonstrate that Federated Learning (FL) is vulnerable to gradient leakage attacks (GLAs). Revisiting key insights in GLAs under FL reveals that (1) Defenses should focus on protecting semantic and fine-grained details of data; (2) GLAs are effective mainly in early rounds; and (3) To avoid semantic leakage, defenses shouldn’t infer the true labels of private images for obfuscation. Building upon these insights, we present a simple, time-adaptive defense strategy that obfuscates the private gradient by employing the gradient from a synthesized image. To this end, a client trains a diffusion model on its own private dataset to generate synthesized images that fit the distribution of private dataset but are distinct in fine-grained details. In addition, a synthesized image is generated conditioned on a non-identical label from the private image to resist semantic leakage. Defense analysis and empirical evaluations demonstrate that our time-adaptive and label agnostic method can better maintain the trade-off between privacy preservation and model utility against GLAs. Our implementation is publicly available at https://github.com/lalakitchen/Diff2Obs.

Abstract

Object detection has revolutionized multiple domains by enabling models to jointly classify and localize targets within data. Yet, its potential in genomic sequence analysis remains largely unexplored. Here, we introduce DNA-DETR, an adaptation of the DETR architecture for one-dimensional genomic object detection. Surprisingly, the direct application of object detection to DNA sequences yielded poor performance, even for elements with simple definitions such as Non-B DNA. We found that the widely used one-hot encoding failed to capture key structural features of several Non-B DNA types. To address this limitation, we systematically investigated how different sequence representations, including one-hot encoding, dot matrix, and their combination, affect detection accuracy and model generalization. Our experiments demonstrate that the choice of representation profoundly affects both localization and classification. Notably, the combined representation consistently outperformed single representations, particularly for complex sequence elements. Our findings suggest that there is no universal “one-representation-fits-all” solution in sequence feature learning. Despite the common perception that end-to-end learning diminishes the importance of representation, our results highlight that thoughtful selection of sequence representation remains critical for model design.

Abstract

Evaluating text-to-music (TTM) systems remains expensive because music impression (MI) and text alignment (TA) scores rely on human mean opinion scores (MOS). Most automatic MOS estimators are trained with point-wise regression or distributional classification. These objectives do not directly optimize rank-based metrics and provide weak geometric constraints for cross-modal coherence. To address these gaps, we propose DeRA-MOS, a decoupled optimization framework for TTM evaluation. For MI, we introduce a batch-aware listwise ranking loss that models relative order within each mini-batch and better aligns with evaluation based on Spearman's rank correlation coefficient (SRCC). For TA, we introduce a score-anchored modality alignment loss that maps human scores to target audio-text similarity and regularizes the latent space before fusion. By effectively mitigating the point-wise training mismatch and modality drift, experiments on MusicEval demonstrate that our decoupled framework yields substantial improvements in both MI and TA ranking metrics, establishing a robust paradigm for large-scale TTM evaluation.

Abstract

Face recognition poses serious privacy risks due to its reliance on sensitive and immutable biometric data. While modern systems mitigate privacy risks by mapping facial images to embeddings (commonly regarded as privacy-preserving), model inversion attacks reveal that identity information can still be recovered, exposing critical vulnerabilities. However, existing attacks are often computationally expensive and lack generalization, especially those requiring target-specific training. Even training-free approaches suer from limited identity controllability, hindering faithful reconstruction of nuanced or unseen identities. In this work, we propose DiMI, the first diusion-driven, training-free model inversion attack. DiMI introduces a novel pipeline combining robust latent code initialization, a ranked adversarial refinement strategy, and a statistically grounded, confidence-aware optimization objective. DiMI applies directly to unseen target identities and face recognition models, oering greater adaptability than trainingdependent approaches while significantly reducing computational overhead. Our method achieves 84.42%–92.87% attack success rates against inversion-resilient systems and outperforms the best prior training-free GAN-based approach by 4.01%–9.82%. The implementation is available at https://github.com/azrealwang/ DiMI.

Abstract

The increasing prevalence of fungal infections represents a growing threat to human health, driven in part by the misuse of antibiotics and the rising incidence of resistance to conventional antifungal agents. Antifungal peptides (AFPs) have emerged as promising alternatives due to their diverse mechanisms of action and their relatively low propensity to develop resistance. To facilitate the systematic discovery of AFPs, we developed AI4AFP. This computational framework integrates curated antifungal peptide resources with advanced machine learning approaches to predict antifungal potential directly from peptide sequences.

Using a comprehensive dataset, we constructed a seven-model ensemble that combines multiple sequence encoding strategies, including ProtBERT-BFD, PC6, and Doc2Vec, with diverse learning algorithms, including random forests, support vector machines, convolutional neural networks, and fine-tuned BERT models. This ensemble demonstrated robust performance on an independent test set, achieving 0.94 in accuracy and 0.89 in Matthews correlation coefficient, outperforming existing AFP prediction methods. Importantly, the predicted AFP score is intended to reflect the general antifungal potential rather than species-specific potency.

Experimental validation against representative fungal pathogens, including Candida albicans, Candida glabrata, and Cryptococcus neoformans, revealed that peptides with high predicted AFP scores exhibited context-dependent antifungal activity. Several candidates displayed pronounced inhibitory effects against specific species, despite limited activity against others, highlighting the inherent species-dependence of antifungal efficacy and supporting the role of AI4AFP as a prioritization tool rather than a species-specific predictor.

To complement antifungal prediction, we further developed a hemolysis classifier that incorporates both peptide sequence and applied concentration as continuous inputs, enabling explicit modeling of the dose-dependent nature of hemolytic toxicity. Experimental determination of the minimum concentration inducing 10% hemolysis (MHC₁₀) provided an empirical safety reference, allowing antifungal activity to be interpreted alongside concentration-dependent toxicity. All models and validation results are implemented on a user-friendly web server, AI4AFP (https://axp.iis.sinica.edu.tw/AI4AFP), providing an accessible platform for the discovery and prioritization of antifungal peptides, with consideration of both efficacy and safety.

Abstract

Deep learning has been successfully applied in various fields, and its impact on deepfake detection is no exception. Deepfakes are fake, yet realistic synthetic content that can be used deceitfully for political impersonation, phishing, slander, or the spread of misinformation. Despite extensive research on unimodal deepfake detection, the identification of complex deepfakes through joint analysis of audio and visual streams remains relatively unexplored. To fill this gap, this survey first provides an overview of audiovisual deepfake generation techniques, applications, and their consequences, and then provides a comprehensive review of state-of-the-art methods that combine audio and visual modalities to increase detection accuracy, summarizing and critically analyzing their strengths and limitations. Furthermore, we discuss existing open source datasets for a deeper understanding, which can contribute to the research community and provide necessary information for beginners who want to analyze deep learning-based audiovisual methods for video forensics. By bridging the gap between unimodal and multimodal approaches, this paper aims to improve the effectiveness of deepfake detection strategies and guide future research on cybersecurity and media integrity.