SSTMark: Robust Training-Free Semantic-Level Speech Watermarking
ACM International Conference on Multimedia (ACM MM), November 2026
Kuan-Lin Chu, Jun-Cheng Chen, Chun-Shien Lu
Abstract
As speech generation models become increasingly realistic and widely accessible, concerns about the misuse, attribution, and governance of synthetic speech continue to grow.Watermarking provides a practical way to make synthesized speech traceable and verifiable. Most existing speech watermarking methods embed watermark information into signal-level representations, such as waveforms or spectrograms. Under sufficiently strong distortions, the embedded watermark may be weakened or destroyed, leading to degraded detectability. In this paper, we propose SSTMark, a training-free speech watermarking framework that operates at the semantic level through text watermarking. Unlike conventional signal-level watermarking methods, SSTMark encodes watermark information into the semantic content conveyed by generated speech, and detects the watermark from the recovered linguistic content. Experiments on AudioMarkBench demonstrate that SSTMark exhibits the strongest average robustness. Compared with the state-of-the-art baselines at a fixed false positive rate of 1%, SSTMark improves the average detection rate by 4.6% and 16.9% on signal-processing edits and compression edits, respectively.
Diffusion to Obfuscation: Time-Adaptive Synthesized Generation Against Gradient Leakage Attacks in Federated Learning
European Conference on Computer Vision (ECCV), September 2026
Farchan Hakim Raswa, Chun-Shien Lu, and Jia-Ching Wang
Abstract
Recent studies demonstrate that Federated Learning (FL) is vulnerable to gradient leakage attacks (GLAs). Revisiting key insights in GLAs under FL reveals that (1) Defenses should focus on protecting semantic and fine-grained details of data; (2) GLAs are effective mainly in early rounds; and (3) To avoid semantic leakage, defenses shouldn’t infer the true labels of private images for obfuscation. Building upon these insights, we present a simple, time-adaptive defense strategy that obfuscates the private gradient by employing the gradient from a synthesized image. To this end, a client trains a diffusion model on its own private dataset to generate synthesized images that fit the distribution of private dataset but are distinct in fine-grained details. In addition, a synthesized image is generated conditioned on a non-identical label from the private image to resist semantic leakage. Defense analysis and empirical evaluations demonstrate that our time-adaptive and label agnostic method can better maintain the trade-off between privacy preservation and model utility against GLAs. Our implementation is publicly available at https://github.com/lalakitchen/Diff2Obs.
TiltDiff: Tilted Weight-Space Diffusion for Neural Network Generation
19th European Conference on Computer Vision (ECCV), September 2026
En-Ni Chuang, Hanjuan Huang, Hao-Jia Song, Hsing-Kuo Pao and Tyng-Luh Liu
Abstract
Generating functional neural-network weights from trained model collections is a central problem in weight-space learning. We introduce TiltDiff, a performance-tilted latent diffusion framework for neural-network generation. TiltDiff tokenizes network weights, encodes them into a compact latent space with a Transformer autoencoder, and uses a U-Net-based diffusion model to synthesize latent representations that decode into functional parameters. To favor stronger models, we weight the denoising loss by validation accuracy, biasing the learned distribution toward higher-performing regions of weight space. Experiments show that TiltDiff improves predictive performance, robustness to random parameter masking, and representational diversity over prior weight-generation methods. We further combine diffusion U-Net connectivity with attribution analysis to identify class-specific decision pathways. These pathways exhibit emergent correspondence across independently generated models despite differing raw parameters, and pathway-level masking verifies their importance for target-class prediction. Our results show that performance-tilted diffusion generates accurate, robust, diverse, and structurally interpretable neural-network weights.
Creat3r: Confidence Reaggregation for Exploration-aware Active 3D Reconstruction
The 43rd International Conference on Machine Learning (ICML), July 2026
Chih-Jung Tsai, Hwann-Tzong Chen and Tyng-Luh Liu
Abstract
We present Creat3r, an iterative next-best-view (NBV) selection framework for efficient, high-quality 3D reconstruction. Starting from a small seed set of image--pose pairs, Creat3r repeatedly selects the most informative next camera pose. After each pose is chosen, the corresponding image is acquired and added to the multi-view set to update a 3DGS reconstruction. To guide selection, Creat3r constructs an intermediate point cloud and estimates reconstruction reliability via a novel \emph{3D confidence field}, which is projected to candidate poses through Gaussian projection to produce \emph{2D confidence} and \emph{exploration maps}. These maps balance exploitation of reliable regions and exploration of uncertain or unseen areas under computational constraints. Experiments with standard 3DGS show that Creat3r consistently outperforms baselines in novel view synthesis and surface reconstruction, achieving higher SSIM and F1 scores with fewer views.
DNA-DETR: sequence representation matters in object detection for functional genomic elements
Briefings in Bioinformatics, June 2026
Tsai, B.S., Wong, J.Y., and Tsai, H.K.*
Abstract
Object detection has revolutionized multiple domains by enabling models to jointly classify and localize targets within data. Yet, its potential in genomic sequence analysis remains largely unexplored. Here, we introduce DNA-DETR, an adaptation of the DETR architecture for one-dimensional genomic object detection. Surprisingly, the direct application of object detection to DNA sequences yielded poor performance, even for elements with simple definitions such as Non-B DNA. We found that the widely used one-hot encoding failed to capture key structural features of several Non-B DNA types. To address this limitation, we systematically investigated how different sequence representations, including one-hot encoding, dot matrix, and their combination, affect detection accuracy and model generalization. Our experiments demonstrate that the choice of representation profoundly affects both localization and classification. Notably, the combined representation consistently outperformed single representations, particularly for complex sequence elements. Our findings suggest that there is no universal “one-representation-fits-all” solution in sequence feature learning. Despite the common perception that end-to-end learning diminishes the importance of representation, our results highlight that thoughtful selection of sequence representation remains critical for model design.
DeRA-MOS: Optimizing Text-to-Music Evaluation via Decoupled Listwise Ranking and Modality Alignment
IEEE Signal Processing Letters, June 2026
Chien-Chun Wang,Hung-Shin Lee,Hsin-Min Wang, and Berlin Chen
Abstract
Evaluating text-to-music (TTM) systems remains expensive because music impression (MI) and text alignment (TA) scores rely on human mean opinion scores (MOS). Most automatic MOS estimators are trained with point-wise regression or distributional classification. These objectives do not directly optimize rank-based metrics and provide weak geometric constraints for cross-modal coherence. To address these gaps, we propose DeRA-MOS, a decoupled optimization framework for TTM evaluation. For MI, we introduce a batch-aware listwise ranking loss that models relative order within each mini-batch and better aligns with evaluation based on Spearman's rank correlation coefficient (SRCC). For TA, we introduce a score-anchored modality alignment loss that maps human scores to target audio-text similarity and regularizes the latent space before fusion. By effectively mitigating the point-wise training mismatch and modality drift, experiments on MusicEval demonstrate that our decoupled framework yields substantial improvements in both MI and TA ranking metrics, establishing a robust paradigm for large-scale TTM evaluation.
Overcoming Copyright Barriers in Corpus Distribution Through Non-Reversible Hashing
The 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), Main Conference, July 2026
Arthur Amalvy, Vincent Labatut, Xavier Bost, and Hen-Hsen Huang
Abstract
While annotated corpora are crucial in the field of natural language processing (NLP), those containing copyrighted material are difficult to exchange among researchers. Yet, such corpora are necessary to fully represent the diversity of data found in the wild in the context of NLP tasks. We tackle this issue by proposing a method to lawfully and publicly share the annotations of copyrighted literary texts. The corpus creator shares the annotations in clear, along with a non-reversible hashed version of the source material. The corpus user must own the source material, and apply the same hash function to their own tokens, in order to match them to the shared annotations. Crucially, our method is robust to reasonable divergences in the version of the copyrighted data owned by the user. As an illustration, we present alignment experiments on different editions of novels. Our results show that our method is able to correctly align 98.7 to 99.79% of tokens depending on the novel, provided the user version is sufficiently close to the corpus creator's version. We publicly release novelshare, a Python implementation of our method.
Rethinking Forgery Attacks on Semantic Watermarks in Black-Box Settings: A Geometric Distortion Perspective
Forty-third International Conference on Machine Learning (ICML), July 2026
Cheng-Yi Lee, Yichi Zhang, Yuchen Yang, and Chun-Shien Lu, and Jun-Cheng Chen
Abstract
Recent studies have shown that semantic watermarks, which embed information into the initial noise of latent diffusion models (LDMs), are vulnerable to black-box forgery attacks. However, existing methods primarily rely on empirical evidence and lack a rigorous theoretical understanding of the conditions under which such attacks succeed or fail. To bridge this gap, we rethink the nature of such attacks through the lens of ratedistortion in the latent space. Our analysis identifies an irreducible distortion floor due to structural mismatches between proxy and target models, which fundamentally limits the fidelity of forged watermarks. We further characterize this distortion as structured geometric deviations on the latent manifold, in the form of global drift and local deformation rather than stochastic noise. Leveraging these insights, we propose a scheme-agnostic detection method that distinguishes forged samples before watermark verification. Extensive experiments demonstrate the effectiveness of our method across diverse black-box scenarios, while preserving robustness to common distortions.
Submodular Optimization for Minimal Augmentation in Robust Language Model Alignment
Forty-third International Conference on Machine Learning (ICML), July 2026
Ching-Chia Kao, Chia-Mu Yu, Chun-Shien Lu, and Chu-Song Chen
Abstract
Safety alignment of large language models is fragile: even small fine-tuning perturbations elastically revert behaviors toward those of the pretraining, with degradation inversely proportional to the size of the alignment set. We ask how to achieve safety alignment with minimal augmentation. To this end, we model augmentation as a set of group actions on sequences and formalize robustness gains as a normalized, monotone submodular function over transformations. We then leverage submodular optimization to select minimal augmentations that provably improve robustness. Experiments confirm that our approach efficiently restores safety alignment while minimizing the overhead of augmentation.
Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement
IEEE Transaction on Audio, Speech and Language Processing, February 2026
Chien-Chun Wang, Hung-Shin Lee, Hsin-Min Wang, and Berlin Chen
Abstract
Pre-trained models for automatic speech recognition (ASR) and speech enhancement (SE) have exhibited remarkable capabilities under matched noise and channel conditions. However, these models often suffer from severe performance degradation when confronted with domain shifts, particularly in the presence of unseen noise and channel distortions. In view of this, we in this paper present URSA-GAN, a unified and domain-aware generative framework specifically designed to mitigate mismatches in both noise and channel conditions. URSA-GAN leverages a dual-embedding architecture that consists of a noise encoder and a channel encoder, each pre-trained with limited in-domain data to capture domain-relevant representations. These embeddings condition a GAN-based speech generator, facilitating the synthesis of speech that is acoustically aligned with the target domain while preserving phonetic content. To enhance generalization further, we propose dynamic stochastic perturbation, a novel regularization technique that introduces controlled variability into the embeddings during generation, promoting robustness to unseen domains. Empirical results demonstrate that URSA-GAN effectively reduces character error rates in ASR and improves perceptual metrics in SE across diverse noisy and mismatched channel scenarios. Notably, evaluations on compound test conditions with both channel and noise degradations confirm the generalization ability of URSA-GAN, yielding relative improvements of 16.16% in ASR performance and 15.58% in SE metrics.
Cross-Attention Reprogramming for ASR: Bridging Discrete Speech Units and Pretrained Language Models
IEEE Access, January 2026
Pei-Jun Liao, Hung-Yi Lee, and Hsin-Min Wang
Abstract
In automatic speech recognition (ASR), an emerging trend involves converting continuous speech features into sequences of discrete speech units (DSUs) via quantization. A key advantage of DSU representations is their compatibility with pretrained language models (PLMs), where DSUs are directly mapped to PLM token indices and the embedding layer is fine-tuned. However, this conventional strategy often relies heavily on large-scale training data to mitigate the inherent modality mismatch. In light of this, we explore a more effective way to exploit the PLM embedding dictionary. Drawing inspiration from Time-LLM, a recent time-series forecasting model, we propose a cross-attention reprogramming mechanism that incorporates codebook information from the DSU quantizer to better align the DSUs with the PLM embeddings. Compared to direct fine-tuning of PLM embeddings, our method consistently achieves improvements on the Discrete Audio and Speech Benchmark (DASB), reaching state-of-the-art performance across most DASB-style settings. We also evaluate our method on LibriSpeech-960, LibriLight-10, and Swedish, Czech, and Hungarian data from Common Voice, and observe similar trends. Notably, the proposed reprogramming method demonstrates significant gains over the fine-tuning baseline, particularly in cross-lingual and low-resource scenarios. This study proposes a new approach to using PLM embedding dictionaries in DSU-based ASR, and lays a foundation for combining speech representations with large language models in other discriminative tasks of speech processing such as speech emotion recognition and spoken question answering.
Can We Formalise Type Theory Intrinsically without Any Compromise? A Case Study in Cubical Agda
Proceedings of the 15th ACM SIGPLAN International Conference on Certified Programs and Proofs (CPP '26), January 2026
Liang-Ting Chen, Fredrik Nordvall Forsberg, Tzu-Chun Tsai
Abstract
We present an intrinsic representation of type theory in the proof assistant Cubical Agda, inspired by Awodey’s natural models of type theory. The initial natural model is defined as quotient inductive-inductive-recursive types, leading us to a syntax accepted by Cubical Agda without using any transports, postulates, or custom rewrite rules. We formalise some meta-properties such as the standard model, normalisation by evaluation for typed terms, and strictification constructions. Since our formalisation is carried out using Cubical Agda's native support for quotient inductive types, all our constructions compute at a reasonable speed. When we try to develop more sophisticated metatheory, however, the 'transport hell' problem reappears. Ultimately, it remains a considerable struggle to develop the metatheory of type theory using an intrinsic representation that lacks strict equations. The effort required is about the same whether or not the notion of natural model is used.